Why does pagination stop before all pages are scraped?

If pagination stops early, check the pagination selector on later pages and confirm that each pagination action still loads new records.

Numbered pagination or 'Next' buttons

  • Keep the pagination type set to Auto unless another option is required.
  • Make sure the selector matches the pagination controls you intend to use.
  • Open the second and third pages and confirm that the same selector still matches there.

Load more' buttons

  • Keep the pagination type set to Auto unless another option is required.
  • Confirm that each click adds new records.
  • Check whether the Load more button changes after it is clicked. If its class, text, or structure changes, the original selector may stop matching it.

Infinite scrolling

  • Enable scrolling on the Element selector that matches the repeated records.
  • Only set an Element limit if you intentionally want scraping to stop after a specific number of records.
  • Scroll the page manually and confirm that more records actually load.

Test before running the full scrape

Preview more than one pagination page and run a small test scrape. Confirm that later pages or records are discovered before starting a larger job.

See the Pagination stops early documentation and Pagination, load more and infinite scrolling documentation for more information.

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.

Still need help? Contact Us Contact Us