> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stophy.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Content filtering

> Keep the part of a page you need: select elements, set how Markdown links look, cut a long page, read it in parts or return only passages that match a query.

These fields shape the page text that `markdown` and `html` come from, and everything built from the Markdown: `summary`, `highlights`, `question` and `json`. `rawHtml` is the page as sent.

This request keeps the `main` element, drops code blocks, numbers the links and returns the first 600 characters:

```bash theme={null}
curl -X POST https://api.stophy.dev/v1/scrape \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/404",
    "includeTags": ["main"],
    "excludeTags": ["pre"],
    "linkStyle": "numbered",
    "maxChars": 600
  }'
```

```json theme={null}
{
  "success": true,
  "data": {
    "markdown": "# 404 Not Found\n\nThe HTTP **`404 Not Found`** client error response⟨1⟩ status code indicates that the server cannot find the requested resource.\n…\n\n## References\n\n⟨1⟩ https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status#client_error_responses\n",
    "isTruncated": true,
    "nextOffset": 529,
    "warning": "The page text was cut short. Send offset 529 to read on.",
    "metadata": { "costUsd": 0.001 }
  }
}
```

To read on, send the same request with `"offset": 529`. Use the `nextOffset` the answer gives, and repeat until it has none.

| Field | Default | Effect |
| - | - | - |
| `onlyMainContent` | `true` | Keep the main content, without menus, headers, footers and ads. `false` keeps the whole page. |
| `onlyCleanContent` | `false` | Needs a key. A model removes cookie banners, ads, share widgets, breadcrumbs, sign up and comment prompts and related article lists from the Markdown. It removes whole paragraphs and lists only, never rewrites, and keeps code blocks. Not with `zeroDataRetention`. |
| `includeTags` | none | Keep only parts that match these CSS selectors, such as `article` or `.price`. `onlyMainContent` is then not applied. |
| `excludeTags` | none | Remove parts that match these selectors, such as `.comments`. |
| `removeBase64Images` | `true` | Drops `data:image` images. `false` keeps them in Markdown and HTML. |
| `imageUrls` | `false` | An image in the Markdown is its alt text. `true` keeps `![alt](url)`. Ask for `images` to get every image URL. |
| `linkStyle` | `inline` | `inline` keeps `[text](url)`. `numbered` writes `Pricing⟨3⟩` and lists each URL once at the end. `none` keeps the link text only, which makes link heavy pages 10 to 70% shorter. Ask for `links` to get the URLs. |
| `maxChars` | none | Cut the Markdown at this many characters, 500 to 1,000,000. |
| `offset` | `0` | Start this many characters into the page, PDF or file. Not with `query`. |
| `query` | none | What you are looking for, such as `refund window`. A page longer than `maxChars` (8,000 by default) comes back as the matching passages in page order, each with its surrounding text and table header. A shorter page comes back whole. Not with `offset`. |

* A selector list needs at least one selector. If `includeTags` matches nothing, the call returns `400 invalidRequest` naming `includeTags`, and costs nothing. Check the selectors or leave `includeTags` out.
* `maxChars` below 500 returns `400 invalidRequest`. `query` with `offset` does too.
* `onlyCleanContent` is skipped with a `warning` when the page is too long to clean or the model does not answer.
* None of these fields adds a fee. A PDF read with `maxChars`, `offset` or `query` is charged for the share of its pages that comes back. See [Pricing](https://docs.stophy.dev/account/pricing.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.