Try Luni

POST /v1/scrape

Web Scraping API

A public URL in. Clean content and its source out. Choose the representation your application needs without building a page parser for every site.

Start buildingRead the docs
curl -X POST https://api.luni.gg/v1/scrape \
  -H "Authorization: Bearer $LUNI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: example-scrape-b4351ed7d7f4" \
  -d '{
  "url": "https://example.com",
  "formats": [
    "markdown",
    "links"
  ]
}'

Choose the content you keep

Request markdown for retrieval and reading, html for processed markup, rawHtml for the fetched source, or links for further discovery. You can combine supported formats in one request. Add template: article or template: product for deterministic structured fields with source evidence and explicit missing fields. These templates do not generate or infer missing facts.

Main-content extraction is enabled by default. Set onlyMainContent to false when the surrounding page content matters to your workflow. Inspect a few representative pages before applying the same settings to a whole website.

  • Supported format names: markdown, html, rawHtml and links.
  • Use timeout in milliseconds, up to 30000, to bound the fetch.
  • Keep API keys on your server, never in a public browser bundle.

A page has a clear cost

A successful page costs one credit. Requests that return an error, a challenge page or no useful content consume zero page credits. Use a stable Idempotency-Key to retrieve the same completed result on retry without another charge. A changed request body needs a new key.

Check the HTTP status and the returned operation rather than assuming every response contains a usable document. You can inspect credit usage and saved results in the workspace.

Know the extraction boundary

Retrieval uses public HTTP. Clients do not choose a render mode. Arbitrary page actions, account login and screenshots are outside the public API. A page whose content appears only after client-side JavaScript may still not return useful text.

Choose a representative sample that includes long articles, short pages and pages with repeated navigation. Evaluate the returned text for your own application instead of treating an example response as a quality benchmark.

Questions, answered.