> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stophy.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Extract

> Get the fields you name from up to 25 pages as JSON. Each value is checked against the page text and left out when the page does not state it.

Extract reads the pages you name and fills your JSON Schema with the values they state.

```bash cURL theme={null}
ID=$(curl -s -X POST https://api.stophy.dev/v1/extract \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://bun.com/docs/runtime/sqlite"],
    "schema": {
      "type": "object",
      "properties": {
        "module": { "type": "string" },
        "pricePerMonth": { "type": "number" }
      }
    }
  }' | jq -r .id)
curl https://api.stophy.dev/v1/extract/$ID -H "Authorization: Bearer $STOPHY_API_KEY"
```

```typescript TypeScript theme={null}
import { Stophy } from "stophy";

const stophy = new Stophy();

const { id } = await stophy.extract({
  urls: ["https://bun.com/docs/runtime/sqlite"],
  schema: {
    type: "object",
    properties: { module: { type: "string" }, pricePerMonth: { type: "number" } },
  },
});
const result = await stophy.extract.result({ id: id! });
console.log(result.data);
```

```python Python theme={null}
from stophy import Stophy

stophy = Stophy()

started = stophy.extract(
    urls=["https://bun.com/docs/runtime/sqlite"],
    schema={
        "type": "object",
        "properties": {"module": {"type": "string"}, "pricePerMonth": {"type": "number"}},
    },
)
result = stophy.extract.result(id=started["id"])
print(result["data"])
```

`POST /v1/extract` answers once the fields are filled, with an `id` and the `invalidURLs` it skipped. Send up to 25 `urls`. Each page is read up to 50,000 characters, and all pages together up to 150,000. The call can take `timeout` (30 seconds by default) plus 45 seconds, so set your client timeout above that.

```json theme={null}
{
  "success": true,
  "status": "completed",
  "data": { "module": "bun:sqlite" },
  "confidence": { "module": 0.9511 },
  "urls": ["https://bun.com/docs/runtime/sqlite"]
}
```

The fields are in `data`. `pricePerMonth` is left out because the page does not state it. `confidence` scores each field from 0 to 1, and `warning` names pages that were skipped or cut. `GET /v1/extract/{id}` is free and returns your result for 24 hours, or 1 hour with `zeroDataRetention`.

## Request fields

Send a `schema`, a `prompt`, or both. With only a `prompt`, the model names up to 20 camelCase fields.

| Field | Default | What it does |
| - | - | - |
| `urls` | none | Up to 25 pages, filled into one answer. An address ending in `/*` reads the pages under it, found through the sitemap. Leave it out and send a `prompt` to search the web. |
| `schema` | none | A JSON Schema with `properties`. See [Schema](#schema). |
| `prompt`, `systemPrompt` | none | What to pull, and standing rules such as units or date format, each up to 10,000 characters. |
| `showSources` | `false` | Add `sources`: the URLs that state each field. |
| `timeout` | `30000` | Milliseconds to read the pages, 1,000 to 300,000. A page not read in time is skipped, named in `warning`, and not charged. |
| `enableWebSearch` | `false` | Also search for your `prompt` and read up to 3 more pages. |
| `allowExternalLinks` | `false` | Also read up to 5 linked pages on other sites. |
| `ignoreSitemap`, `includeSubdomains`, `limit` | `false`, `true`, none | For a `/*` address: find pages by links, read subdomains, read at most `limit` pages. |
| `scrapeOptions` | none | How each page is read, with the options of [Scrape](https://docs.stophy.dev/scrape/overview). |
| `ignoreInvalidURLs` | `true` | `false` refuses the call when an address is not a public page. |
| `zeroDataRetention` | `false` | Keep nothing from the call. See [Zero data retention](https://docs.stophy.dev/production/zero-data-retention.md). |
| `webhook` | none | An address, or an object with `url`, `events`, `headers` and `metadata`, called once with `completed` or `failed`. See [Webhooks](https://docs.stophy.dev/webhooks.md). |

`agent` is accepted and ignored. To click through a site, use [Agent](https://docs.stophy.dev/agent/overview).

## Schema

Pydantic `model_json_schema()` and zod `toJSONSchema()` output work as they are, in draft-07 or 2020-12. A schema holds up to 100 top-level fields, 32,000 bytes and 32 levels of nesting.

| Keyword | Use |
| - | - |
| `type`, `items` | Strings, numbers, booleans, objects and lists. Nested values are checked one by one. |
| `enum`, `const` | A value outside the set is left out. |
| `anyOf`, `oneOf`, `["number", "null"]` | Several shapes, or a nullable value. |
| `$ref` with `$defs` | A shared shape. The schema itself can be a `$ref` to an object. |
| `description` | Tells the model what a field means. |
| `required` | Read, but a field the page does not state is still left out. Parse `data` as partial: `Schema.partial().parse(data)` in zod, defaults on every Pydantic field. |

## Errors

| Code | Cause | Fix |
| - | - | - |
| `400 invalidRequest` | More than 25 `urls`, no `schema` and no `prompt`, no public page in `urls`, no `properties`, or a schema over a limit. | Fix what `error` names. |
| `notFound`, `siteNotRead`, `sourceTimeout` | No page could be read. The answer has an `id`, and its result has `status: "failed"`, `error` and empty `data`. | Check the address, or raise `timeout`. |

A failed call, and one that finds none of your fields, is free. Other codes are on [Errors](https://docs.stophy.dev/api-reference/errors.md).

## Next

* [JSON format](https://docs.stophy.dev/scrape/json.md): the same fields from one page in a scrape call.
* [Agent](https://docs.stophy.dev/agent/overview): navigate a site to find the answer.
* [Batches](https://docs.stophy.dev/batches/overview): scrape a list of pages in one job.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.