url
, Batch Scrape works by sending an array of URLs under the urls
field.
All URLs in the batch will be processed using the same parameters.
When to Use Batch Scraping
- Extracting titles, authors, and publish dates from a list of blog or news article URLs
- Running
aiExtractRules
across a set of company websites to collect structured data like founding year, services, and contact info - Gathering legal notices or disclaimers from the footer pages of 5,000+ policy URLs
Submit a Batch Scrape Job
Response
Get Job Status & Results
To check the status of your batch job:Example Result
Notes
- Maximum batch size: 10,000 URLs
- All URLs are processed using the same parameters
- Failed URLs do not consume credits
- All outputs are returned as an array of Web Scraping API-style results