What should I do if a website blocks the scraper?

Web Scraper Cloud is designed to automatically handle many temporary access failures. If a page fails or returns no usable data, Cloud automatically retries it, and retries can use a different proxy IP.

This can help with temporary 403, 429, Access denied, CAPTCHA, Cloudflare, Akamai, browser-verification, and similar access pages without requiring you to troubleshoot each failure individually.

If pages still fail after automatic retries

Try changing the proxy configuration:

• If you are using a Datacenter proxy, try a Residential proxy.
• Try a different proxy country or location when the website is region-specific.
• Run a Test scrape after changing the proxy and confirm that the expected page is returned.

Residential proxies are often more suitable for websites that restrict Datacenter traffic or require a specific geographic location.

If access still fails

Some websites use access controls or challenges that cannot be completed reliably by a general scraping workflow. If the issue continues after trying an appropriate proxy configuration, contact Web Scraper Support for help troubleshooting the Cloud job.

See the Website blocks the scraper documentation and Proxy configuration documentation for more information.