Crawl a small set of web pages synchronously.
Endpoint
/v1/web/crawl
Query
URLs to crawl. Repeat the `url` query parameter for multiple pages (max 5).
When true, scroll the page to load dynamically appended content (infinite scroll). Default false.
Wait for a CSS selector before extraction. Must be prefixed with "css:" (e.g. css:main). JavaScript wait conditions are not supported.
const response = await fetch( "https://api.socialfetch.dev/v1/web/crawl?url=https://www.socialfetch.dev", { headers: { "x-api-key": "YOUR_API_KEY" } } ); const data = await response.json(); console.log(data);