How do I scrape a page with Load more or infinite scroll?

Use a different setup depending on how the website loads additional records.

Load more button

If the page uses a Load more button:

1. Create a Pagination selector.
2. Select the Load more button.
3. Keep the pagination type set to Auto unless the default behavior does not work correctly.
4. Confirm that each click adds new records.
5. Confirm that scraping stops when the button disappears or no new records are added.

Infinite scroll

If new records appear when you scroll down the page:

1. Create an Element selector for the repeated records.
2. Enable scrolling on the Element selector.
3. Set an element limit if you only need a specific number of records.
4. Preview the result and run a small test to confirm that later records load.

If the page uses both pagination and scrolling

Place the scrolling Element selector under the Pagination selector so scrolling runs again on every pagination page.

_root
└── pagination
    └── items (scrolling enabled)
        ├── name
        └── price

See the Pagination, load more and infinite scrolling documentation for the full setup options.