How do I run a Cloud job with custom start URLs?

Use Scrape with custom start URLs when you want to run the same sitemap structure against different starting pages for one Cloud job.

Use custom start URLs

1. Open the sitemap in Web Scraper Cloud.
2. Open the Scrape tab.
3. Enable Scrape with custom start URLs.
4. Enter the URLs as text, one URL per line, or provide them from a file.
5. Review the remaining job settings and start the scrape.

What do custom start URLs change?

Custom start URLs override the sitemap's configured start URL for that job only. They do not permanently change the sitemap.

For example, if the sitemap is built for one product category and the same selector structure works for another category, you can run the second category without editing the sitemap itself.

How many custom start URLs can I use?

Custom start URLs are processed in jobs of up to 20,000 URLs. A single launch can contain up to 200,000 URLs, which Cloud automatically splits into 10 scraping jobs.

To change the start URL permanently

Edit the sitemap itself if future jobs should use the new start URL by default.

See the Run a Cloud scraping job documentation for more information.