> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stophy.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape

> Read one web page, PDF or file as Markdown, HTML, links, a screenshot, a summary or JSON. Pick the formats and choose how the page is read.

Scrape reads one URL and returns it in the formats you ask for.

```bash cURL theme={null}
curl -X POST https://api.stophy.dev/v1/scrape \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://bun.com/docs/runtime/bunfig",
    "formats": ["markdown", "links"]
  }'
```

```typescript TypeScript theme={null}
import { Stophy } from "stophy";

const stophy = new Stophy();

const { data } = await stophy.scrape({
  url: "https://bun.com/docs/runtime/bunfig",
  formats: ["markdown", "links"],
});
console.log(data.markdown);
```

```python Python theme={null}
from stophy import Stophy

stophy = Stophy()

data = stophy.scrape(
    url="https://bun.com/docs/runtime/bunfig",
    formats=["markdown", "links"],
)["data"]
print(data["markdown"])
```

Without `formats` you get the page's main content as Markdown. Send 1 to 20 formats. A read gives up after `timeout`: 60,000 ms by default, at most 300,000. Calls without a key have lower limits and fewer fields, see [Without a key](https://docs.stophy.dev/production/rate-limits.md#without-a-key).

```json theme={null}
{
  "success": true,
  "data": {
    "markdown": "# bunfig.toml\n\nConfigure Bun's behavior using its configuration file bunfig.toml\n\n…",
    "links": ["https://bun.com/docs/runtime/bunfig", "https://bun.com/"],
    "metadata": { "title": "bunfig.toml | Bun Docs", "statusCode": 200, "scrapeId": "170c0c1f-5a19-4f8f-8c32-8fef39c8035e" }
  }
}
```

`data` has one field per format. `data.metadata` describes the page, see [Page metadata](https://docs.stophy.dev/scrape/page-metadata.md). To read the result again, send `GET /v1/scrape/{id}` with `metadata.scrapeId`.

## Formats

A format is a name, or an object with its fields: `{ "type": "highlights", "query": "refund policy" }`.

| Format | What you get | Page |
| - | - | - |
| `markdown`, `html`, `rawHtml`, `rawBase64` | The page text, the cleaned page, the page as sent, or its bytes in Base64 | [Formats](https://docs.stophy.dev/scrape/formats.md) |
| `links`, `images` | Every link or image URL on the page | [Formats](https://docs.stophy.dev/scrape/formats.md) |
| `screenshot`, `network`, `branding` | A link to a JPEG, the JSON the page loaded, or its colors, fonts and logo | [Formats](https://docs.stophy.dev/scrape/formats.md) |
| `audio`, `video` | A YouTube video's audio or video file | [Formats](https://docs.stophy.dev/scrape/formats.md) |
| `summary`, `highlights`, `question` | A summary, the passages that match a query, or one answer | [Summary, highlights and question](https://docs.stophy.dev/scrape/summary.md) |
| `json` with `schema` or `prompt`, `product`, `menu` | An object in the shape you describe, a product, or a menu | [JSON](https://docs.stophy.dev/scrape/json.md) |
| `json` with `rules`, or `attributes` | Values picked by CSS selectors, with no model | [JSON](https://docs.stophy.dev/scrape/json.md) |
| `json` with `parser` | A site's own data | [Pre-built parsers](https://docs.stophy.dev/scrape/prebuilt-parsers) |
| `changeTracking` | What changed since your last read | [Change tracking](https://docs.stophy.dev/scrape/change-tracking.md) |

`summary`, `highlights`, `question`, `product`, `menu` and `json` with `schema` or `prompt` use a model and add a fee, see [Scrape add-ons](https://docs.stophy.dev/account/pricing.md#scrape-add-ons).

## Options

Every option sits next to `url`. The [API reference](https://docs.stophy.dev/api-reference/endpoint/scrape.md) has every field.

| Field | What it does | Page |
| - | - | - |
| `onlyMainContent`, `onlyCleanContent`, `includeTags`, `excludeTags`, `removeBase64Images` | Choose which part of the page is read | [Content filtering](https://docs.stophy.dev/scrape/content-filtering.md) |
| `linkStyle`, `maxChars`, `offset`, `query` | Shorten the Markdown or read it in parts | [Content filtering](https://docs.stophy.dev/scrape/content-filtering.md) |
| `waitFor`, `mobile`, `fastMode`, `blockAds`, `profile`, `location`, `headers`, `skipTlsVerification` | Choose how the page loads and what the site shows you | [Rendering and location](https://docs.stophy.dev/scrape/rendering-and-location.md) |
| `actions` | Click, scroll, type or capture before the read | [Browser actions](https://docs.stophy.dev/scrape/browser-actions.md) |
| `parsers` | PDF modes and layout | [PDFs and documents](https://docs.stophy.dev/scrape/pdfs-and-documents.md) |
| `maxAge`, `minAge`, `lockdown`, `storeInCache` | Accept or refuse stored copies | [Freshness and caching](https://docs.stophy.dev/scrape/freshness-and-caching.md) |
| `checkPromptInjection`, `redactPII` | Scan or clean text before a model reads it | [Safety](https://docs.stophy.dev/scrape/safety.md) |
| `zeroDataRetention` | Keep nothing from the call | [Zero data retention](https://docs.stophy.dev/production/zero-data-retention.md) |

## Errors and warnings

| Code | Cause | Fix |
| - | - | - |
| `504 sourceTimeout` | The read ran past `timeout`. It costs nothing. | Raise `timeout`. A page that needs a browser needs at least 5,000 ms. See [Timeouts](https://docs.stophy.dev/production/timeouts.md). |
| `403 siteNotRead` | Stophy does not read that site as a page | Use a [parser](https://docs.stophy.dev/scrape/prebuilt-parsers) or [Search](https://docs.stophy.dev/search/overview) |
| `400 invalidRequest` | The body has `urls` | Read a list of URLs with [Batches](https://docs.stophy.dev/batches/overview) |
| `400 invalidRequest` | The page is over 8 MB of text, or the file over 20 MB | Read a smaller page or file |

* A page that answers 404 or 410 comes back with that `metadata.statusCode` and a `metadata.error`, at no cost.
* A problem that does not fail the call adds a `warning`: a browser action failed, `maxChars` cut the text, or `onlyCleanContent` or `checkPromptInjection` did not finish.

Every other code is in [Errors](https://docs.stophy.dev/api-reference/errors.md).

## Next

* [Interact](https://docs.stophy.dev/scrape/interact.md): keep working on the page in a browser.
* [Map](https://docs.stophy.dev/map/overview): list a site's URLs.
* [Crawl](https://docs.stophy.dev/crawl/overview): read a whole site.
* [Batches](https://docs.stophy.dev/batches/overview): read a list of URLs.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.