Why are not all records being scraped?

If a scrape returns some records but not all of them, check whether the record selector matches every item, whether all pages are being reached, and whether different page layouts need to be handled.

Check the repeated record selector

Preview the Element selector that defines each repeated record and confirm that it matches all intended items.

• Check the first and last visible item.
• Check items with different data combinations, such as regular and discounted prices.
• Confirm that advertisements, recommendations, or unrelated elements are not affecting the selector.

Check later pages and dynamically loaded records

If the website uses pagination, test at least the second page and confirm that the same selectors still work there.

If more records appear after scrolling or clicking, confirm that the sitemap performs the required interaction and continues loading additional records.

Check different detail-page layouts

If records come from linked detail pages, preview several different pages. A selector that works on one layout may not match another layout used by the same website.

If this happens in Cloud

Open the scraping job and check whether any pages are listed as Failed, Empty, or No value. These page outcomes can explain why fewer records reached the final dataset.

Run a small test after changes

After the previews look correct, run a limited local scrape or Cloud Test scrape and compare the number of returned records with what you expect from the website.

See the Preview and validate a sitemap documentation and Pagination stops early documentation for more information.