invalidURLs lists the addresses that are not public website URLs: they are skipped and the rest runs. A batch that waits for a free place also has status and queuePosition. Read the pages from url as they finish, see Batch results. If you want the URLs found for you, use Crawl.
Fields
The API reference has every field.
Add URLs to a running batch
Send the batchid in appendToId and the new addresses in urls. The batch keeps its own options and webhook, skips URLs it already has, and counts the new ones in total. Added URLs count toward the 10,000 cap.
Retry a submit safely
Send anIdempotency-Key header with any text. The same key and body within 24 hours return the first answer with idempotent-replayed: true. The same key with a different body starts a new batch.
Errors
A batch ends with
status: "failed" when your balance runs out or no page could be read. completed means every page was tried, so check errors. Every code is in Errors.
Next
- Batch results: read, retry, cancel and list batches.
- Scrape: the options every page takes.
- Map: list a site’s URLs to send.