Skip to main content
POST /v1/crawl/params-preview turns a sentence into crawl options. Use it when you do not know which path patterns or depth to set. No pages are read. Needs an API key.
cURL
url and prompt (1 to 1,000 characters) are required. data holds only the fields it set, from url, includePaths, excludePaths, maxDiscoveryDepth, crawlEntireDomain, allowSubdomains, allowExternalLinks, sitemap, ignoreQueryParameters and limit, described in Crawl. It never sets scrapeOptions. The options are a suggestion, so read them, then send them to POST /v1/crawl. Patterns that are not valid regular expressions are dropped, and at most 20 of each are returned. maxDiscoveryDepth is at most 10. A request with nothing to set returns only url. See Pricing.