Web data for AI agents

Give an agent focused web tools. Keep credentials, page budgets and decisions about which sources to read under your application's control.

Start building Read the docs

curl -X POST https://api.luni.gg/v1/scrape \
  -H "Authorization: Bearer $LUNI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: example-scrape-c954d38d0c27" \
  -d '{
  "url": "https://example.com",
  "formats": [
    "markdown"
  ]
}'

Expose a narrow tool contract

Wrap the scrape or search endpoint in a server-side function with explicit input validation. Decide which hosts, page counts and formats your agent is allowed to request. Keep the Luni key in the server environment, and give tools that only read results or usage a read-only key.

Treat fetched page text as untrusted data. A page can contain instructions written for an automated reader; those instructions should not change your agent's permissions, reveal secrets or trigger unrelated actions.

Return context with its limits

Pass the extracted text and source URL back to the model together. Include an explicit error state when a fetch fails rather than giving the model an empty document that looks like evidence. Set a maximum number of search results and fetched pages per task.

The current API supports public-page HTTP extraction. It does not click through websites, operate logged-in sessions or fill forms. This page describes integrating Luni's HTTP API into your agent; it does not claim an official agent plugin or CLI. MCP clients can use the hosted Luni MCP server instead.

Account for retries

Create one idempotency key for each logical tool operation and reuse it for a retry with the same body. Store the operation ID alongside your agent trace. A completed replay retrieves the saved result without another charge.

Search is available in every workspace, but a query or page fetch can still fail. Handle failed or unavailable work as a recoverable state in your application.