How do I scrape multiple categories?

You can scrape multiple categories by adding them as start URLs or by using Link selectors to discover the category pages.

Use multiple start URLs

Use multiple start URLs when you already know the category pages and they all use the same page structure.

Discover categories with Link selectors

Use a Link selector when the category pages should be discovered from a parent page.

A typical selector tree looks like this:

_root
└── category_link
    └── pagination
        └── item
            ├── name
            └── detail_link

Avoid scraping the same records more than once

Select category links only from the intended navigation area.
Avoid selecting both parent and child categories when they lead to overlapping records.
Use a more specific CSS selector or URL filter if the navigation includes unrelated links.

Include the category name in your data

If you need the category name in each record, create an Element selector around each category. Add a Text selector for the category name and a Link selector for the category URL beneath it.

_root
└── category_wrapper
    ├── category_name
    └── category_url
        └── record_wrapper

See the Scrape multiple categories documentation for more information.