Skip to main content
Use a batch to read many pages in the background with one schema, then turn each page’s fields into a CSV row. You need STOPHY_API_KEY. See the Quickstart.

1. Submit the batch

The json format takes a schema. Use required to ask for fields on every page, and description to fix a format. See JSON. The call answers at once with an id and a url to poll.
invalidURLs lists addresses that are not public pages. They are left out.

2. Read the answers

Read url until status is completed. Each page in data has its object in json and its address in metadata.url. This row has no topics: the field was optional and the page lists none.
A long batch comes back in parts. Follow next until it is missing, and read failed pages from errors. See Batch results.

3. Write the rows

Flatten each json object into a row. Fill a missing field with an empty string, or skip the row if your app needs it. Join topics with ; .

Check it

  • completed plus failed equals total, and your row count equals the pages that returned json.
  • Open three pages and compare each cell. The model fills fields from page text, so a field the page does not state can come back wrong or empty, even when it is required.
  • Dates share one format. If not, tighten the description.
  • Store url and the read time with each row, so you can tell how old it is.
Test the schema on 3 to 5 pages before you send the list. See Pricing. To fill one row from several pages, use Extract.