Webhooks#

Crawl, Company and Monitor submissions accept webhook with a public url and a secret of 16–200 characters. Keep the secret private. Crawl and Company notifications identify the operation; they do not contain its result payload. Monitor events include change status and a text diff when content changed.

{
  "webhook": {
    "url": "https://your-app.example/luni-events",
    "secret": "<your-signing-secret>"
  }
}

X-Luni-Event-ID: <event-id>
X-Luni-Signature: t=<unix-timestamp>,v1=<hex-signature>

{
  "id": "<event-id>",
  "type": "crawl.completed",
  "operationId": "<operation-id>",
  "status": "completed"
}

The signature is HMAC-SHA256 over the timestamp, a dot and the exact raw body bytes. Verify before parsing or processing the event. Check timestamp freshness and deduplicate by event ID in persistent storage before acknowledging.

import hashlib
import hmac
import time

def verify_event(raw_body: bytes, signature_header: str, secret: str):
    parts = dict(part.split("=", 1) for part in signature_header.split(","))
    timestamp = parts["t"]
    if abs(time.time() - int(timestamp)) > 300:
        raise ValueError("Stale webhook")
    expected = hmac.new(
        secret.encode(), timestamp.encode() + b"." + raw_body, hashlib.sha256
    ).hexdigest()
    if not hmac.compare_digest(expected, parts["v1"]):
        raise ValueError("Invalid webhook signature")

Delivery is at least once, with bounded retries for up to 24 hours. Any 2xx response acknowledges a delivery. Crawl and Company events are sent when an operation ends: completed, failed, cancelled or expired, as in crawl.cancelled. Polling remains the authority for operation state.