Why are pages marked as Failed in Web Scraper Cloud?

A Failed page means Web Scraper could not process that URL successfully because of an HTTP error, access problem, or execution failure.

Inspect the failed page

1. Open the completed scraping job.
2. Select Inspect, then open Failed pages.
3. Open an affected URL and check the failure reason.
4. If a screenshot is available, compare it with the page you expected Web Scraper to receive.

You can also select the linked number under Failed pages in the scraping jobs table to open the affected pages directly.

Common reasons for Failed pages

• 4xx or 5xx response: the target website returned an HTTP error.
• Timeout: the page exceeded Web Scraper Cloud's 10-minute continuous JavaScript execution limit. This is usually caused by long-running scrolling or repeated dynamic pagination clicks.
• Website Log-In / State setup failure: a required input, button, link, or other configured element could not be found.
• CAPTCHA or access challenge: the website returned a challenge or blocked response instead of the expected page.

Use the returned page to identify the cause

A screenshot can reveal an access-denied page, CAPTCHA, consent screen, login page, redirect, or another unexpected page state. Fast driver jobs do not produce screenshots.

Cloud retries Failed pages automatically

Web Scraper Cloud automatically retries Failed pages without using additional URL credits. If pages still fail after the automatic retries, correct the access or sitemap problem before running or continuing the job again.

See the Empty, failed and no-value pages documentation and Troubleshoot Cloud scraping jobs documentation for more information.