How do I scrape listing and detail pages?

Use a Link selector when some fields are available on the listing page and other fields are only available after opening each item's detail page.

Create the listing record

1. Create an Element selector for the repeated listing items.
2. Add the fields available on the listing page beneath the Element selector.
3. Add a Link selector for the detail-page link beneath the same Element selector.

Add fields from the detail page

1. Open the Link selector in the selector tree.
2. Add the selectors for the fields that should be extracted from the destination page.
3. Preview the fields on several detail pages to confirm they work across different records.

A typical selector tree looks like this:

_root
└── item
    ├── listing_name
    ├── listing_price
    └── detail_link
        ├── sku
        ├── description
        └── availability

Common problems

• Detail-page selectors are attached to the listing wrapper instead of the Link selector.
• The Link selector extracts the URL but is not the parent of the detail-page fields.
• The item wrapper includes more than one detail link.
• Different detail pages use different layouts and need selectors that work across those layouts.

See the Scrape listing and detail pages documentation for more information.