Scrape with Python
Fetch a public page with a small Python script. This walkthrough uses the HTTP API directly and keeps your key in an environment variable.
Create a key for the script
Create an API key in your workspace with full access; a read-only key cannot start requests. Set an expiry appropriate for the task and store the key as LUNI_API_KEY in your server or shell environment. The example reads the variable rather than embedding a credential in the file.
Use a recent Python environment and install requests with python -m pip install requests. Do not commit a key or print it in application logs. If a key is exposed, revoke it and create a replacement before running the workflow again.
Send one bounded request
The script below sends one public URL and asks for Markdown. The HTTP client has a 60-second timeout; Luni's own page-fetch timeout is separately bounded by the API. raise_for_status() keeps an HTTP error from being treated as an extracted page.
The response is a JSON envelope. Read data.markdown only after the request succeeds. When adapting the script, keep the URL in the JSON body and the credential in the Authorization header.
import os
import requests
response = requests.post(
"https://api.luni.gg/v1/scrape",
headers={
"Authorization": "Bearer " + os.environ["LUNI_API_KEY"],
"Idempotency-Key": "python-first-page-001",
},
json={"url": "https://example.com", "formats": ["markdown"]},
timeout=60,
)
response.raise_for_status()
result = response.json()
print(result["data"]["markdown"])Retry the operation, not the charge
python-first-page-001 is an example logical request ID. Reuse it when retrying this exact request with the same body. A completed replay returns the saved result without charging again. Generate a new key for a different URL, output format or genuinely new observation.
In a queue, store the logical request ID with the work item before making the HTTP call. That allows another worker to resume the same operation after a process failure. A changed body with a reused idempotency key returns 409 rather than silently starting different work.
Inspect and store the result
Review the text and source metadata before building a larger ingestion run. A successful page costs one credit; error or challenge pages consume zero page credits. The workspace records actual usage, so use it to check the behavior of your sample.
Save results in your own storage when you need them beyond the seven-day retention window. If a source is JavaScript-only or requires authentication, handle the missing content explicitly; this API does not provide an interactive browser session.