> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aireserve.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Trace export

> Push one metadata event per request to your own observability stack — an OpenTelemetry collector, a signed webhook, or Datadog Logs. Never prompts or completions.

Every request that goes through the gateway settles as one usage record: the model
asked for and the model that served it, the provider, tokens including the cache split,
fair-market and charged cost, latency, time to first token, status and error attribution,
and the key, user, team and customer tags that made the call. Trace export pushes that
record — and only that record — to destinations your organization controls, as it settles.

Exports are **customer-configured and metadata-only**. The event never contains prompt or
completion text, message content, system prompts, tool arguments, headers, or provider
error messages (upstream-worded messages can echo request text, so they are excluded too).
The exporter's payload type is checked at compile time and in the test suite for any
content-shaped field; adding one fails the build.

Destinations are managed by organization admins in the portal under **Observe › Trace
Export**. There is no public API endpoint for destinations in v1.

## The event

One event per billable request, emitted after the response has been sent and the usage
record written. Schema `aireserve.trace.v1`.

```json theme={"dark"}
{
  "schema": "aireserve.trace.v1",
  "event_id": "4f7b8e5a-1c2d-4e3f-9a0b-6c7d8e9f0a1b",
  "request_id": "req_01HZX9K3Q4M2N8P7R6S5T4V3W2",
  "occurred_at": "2026-11-05T09:30:12.418Z",
  "status": "success",
  "model": {
    "requested": "claude-sonnet-5",
    "served": "claude-sonnet-5",
    "provider": "anthropic",
    "leg": "claude-sonnet-5",
    "route_reason": "direct",
    "billed_as_leg": null,
    "byok": false
  },
  "usage": {
    "prompt_tokens": 1842,
    "completion_tokens": 310,
    "total_tokens": 2152,
    "cached_input_tokens": 1536,
    "cache_creation_input_tokens": 0,
    "cache_creation_1h_input_tokens": 0,
    "prompt_includes_cache": false,
    "media_kind": null,
    "media_count": null
  },
  "cost": { "charged_usd": 0.006312, "fmv_usd": 0.006312, "currency": "USD" },
  "timing": { "latency_ms": 2410, "ttft_ms": 388, "ttft_source": "firstChunk" },
  "error": null,
  "harness": "claude-code",
  "harness_agent_id": null,
  "attribution": {
    "client_id": "org_8f2c1a",
    "api_key_id": "key_3b9e7d",
    "token_id": null,
    "user_id": "usr_51aa0c",
    "team_id": "team_platform",
    "conversation_id": null,
    "session_id": "sess_c0ffee",
    "end_customer_id": "acme-eu",
    "end_customer_org_id": null,
    "feature": "support-copilot"
  },
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"
}
```

### Fields

| Field | Type | Meaning |
| - | - | - |
| `schema` | string | Always `aireserve.trace.v1`. |
| `event_id` | string | The usage record id. Stable across redeliveries — dedupe on it. |
| `request_id` | string | The gateway's id for the request. |
| `occurred_at` | string | ISO-8601 instant the request settled. |
| `status` | `success` \| `error` | Outcome of the request. |
| `model.requested` | string \| null | What the caller asked for, alias-resolved; `auto` for the auto-router. |
| `model.served` | string | The catalog id that served the request. |
| `model.provider` | string \| null | Serving provider label (`openai`, `anthropic`, `bedrock`, `fireworks`, …). |
| `model.leg` | string \| null | Internal serving leg when it differs from `served` (a pinned route or pool member), else the served id. |
| `model.route_reason` | `direct` \| `fallback` \| `bypass` \| null | Why this leg served: the primary, a [fallback](https://docs.aireserve.com/capabilities/fallbacks.md), or a bypass of the routing stack. |
| `model.billed_as_leg` | string \| null | Set when the request was priced at another leg's card than the served one. |
| `model.byok` | boolean | The request rode your organization's own provider key ([BYOK](https://docs.aireserve.com/capabilities/byok.md)). |
| `usage.prompt_tokens` | number | Input tokens. |
| `usage.completion_tokens` | number | Output tokens. |
| `usage.total_tokens` | number | Input plus output. |
| `usage.cached_input_tokens` | number | Input tokens read from the prompt cache. |
| `usage.cache_creation_input_tokens` | number | Input tokens written to the provider's prompt cache, all tiers. 0 for providers without cache-write pricing. |
| `usage.cache_creation_1h_input_tokens` | number | The portion of `cache_creation_input_tokens` written with Anthropic's 1-hour TTL (billed at a higher multiplier than the 5-minute default). |
| `usage.prompt_includes_cache` | boolean \| null | Whether `prompt_tokens` already includes cache reads and writes (OpenAI-compatible surfaces) or excludes them (Anthropic-native). Null when unknown. |
| `usage.media_kind` | string \| null | `image`, `video`, `audio` or `voice` for media requests; null for token requests. |
| `usage.media_count` | number \| null | Number of media items produced (for example images returned). Null for token requests. |
| `cost.charged_usd` | number \| null | What your organization was charged, in USD. Null when the record carried no cost (some error records). |
| `cost.fmv_usd` | number \| null | Fair-market (provider list price) value, in USD. |
| `cost.currency` | `USD` | Always USD. |
| `timing.latency_ms` | number \| null | Wall-clock time from request receipt to the last byte sent. |
| `timing.ttft_ms` | number \| null | Time to first token for streamed requests. |
| `timing.ttft_source` | `firstChunk` \| `headers` \| null | Whether TTFT was measured at the first content chunk or at response headers. |
| `error.fault` | `client` \| `provider` \| `platform` \| null | Who has to act. Only present on error events (`error` is null on success). |
| `error.reason` | string \| null | A closed snake\_case reason (`model_not_found`, `provider_timeout`, `budget_cap`, …). |
| `error.origin` | string \| null | Where the upstream error was stamped (`provider`, `router`), when known. |
| `harness` | string \| null | The tool or surface that made the call (`cursor`, `claude-code`, `copilot`, `api`, …). |
| `harness_agent_id` | string \| null | Subagent id when the harness reported one. |
| `attribution.client_id` | string | Your organization id. |
| `attribution.api_key_id` | string \| null | The AI Reserve API key that made the call. |
| `attribution.token_id` | string \| null | The gateway token id behind that key, when one applies. |
| `attribution.user_id` | string \| null | The member the key belongs to. |
| `attribution.team_id` | string \| null | The member's team. |
| `attribution.conversation_id` | string \| null | Web chat conversation, when the call came from chat. |
| `attribution.session_id` | string \| null | Harness session id (a Claude Code session, a Codex thread). |
| `attribution.end_customer_id` | string \| null | Your own customer tag ([customer attribution](https://docs.aireserve.com/capabilities/customer-attribution.md)). |
| `attribution.end_customer_org_id` | string \| null | Your customer's organization tag, when you send one. |
| `attribution.feature` | string \| null | Your own `x-aireserve-feature` tag. |
| `trace_id` | string \| null | The W3C trace id the caller propagated in `traceparent`, when any. |

**Versioning.** Additive changes (a new nullable field) keep `aireserve.trace.v1`. A
removed or re-typed field ships as `aireserve.trace.v2` with a new schema string, so a
consumer that pins on `schema` is never surprised. Parse unknown fields leniently.

**Test events.** The portal's **Send test event** button pushes a synthetic event through the
destination's real sender so you can confirm wiring without waiting for traffic. It is
recognizable and filterable: `model.served` is `aireserve-test-model`, `request_id` starts
with `test_`, `event_id` is the zero UUID, `attribution.feature` is `trace-export-test`, and
cost is `0`. Nothing bills or reconciles against it.

## Destinations

An organization can have up to 10 destinations of any mix. Each destination gets every
event for the organization; filtering is done on your side.

### OpenTelemetry collector (OTLP/HTTP)

Point it at any OTLP/HTTP endpoint — the OpenTelemetry Collector, Honeycomb, Grafana
Tempo, Datadog APM through the Datadog Agent, and so on. The gateway appends `/v1/traces`
to the endpoint you give it (or uses it unchanged if it already ends in `/v1/traces`) and
sends OTLP JSON built by the OpenTelemetry SDK, so the wire format is exactly what any
OTLP-compliant backend expects.

| Setting | Value |
| - | - |
| Endpoint | `https://` base URL, for example `https://api.honeycomb.io` or `https://otlp.example.com:4318`. Plain `http://` is accepted for `localhost` only. |
| Secret (optional) | Sent as `Authorization: Bearer <secret>` by default. |
| Auth header (optional) | A header name to carry the secret raw instead, for backends that want their own header: `x-honeycomb-team`, or `Authorization` with a pre-built `Basic …` value for Grafana Cloud. |

One **client** span per event:

* **Name:** `<media kind or text> <served model>`, the GenAI semantic-convention
  `{operation} {model}` shape — `text claude-sonnet-5`, `image gpt-image-2`.
* **Trace id:** the caller's own trace id when they sent a `traceparent` header, so the
  span lands inside your application's trace. Otherwise derived deterministically from the
  request id.
* **Span id:** derived from `event_id`, so a redelivered batch re-exports the *same* span
  and collectors dedupe it.
* **Start / end:** `occurred_at - latency_ms` to `occurred_at`.
* **Status:** `OK` on success; `ERROR` with `error.reason` as the message otherwise.
* **Resource:** `service.name = aireserve-gateway`. **Scope:** `aireserve-trace-export`.
* **Attributes:** every non-null leaf of the event flattened under `aireserve.` plus the
  GenAI semantic-convention aliases LLM views key on.

```json theme={"dark"}
{
  "name": "text claude-sonnet-5",
  "kind": 3,
  "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
  "spanId": "2c1f0a9d8e7b6a5f",
  "startTimeUnixNano": "1793950210008000000",
  "endTimeUnixNano": "1793950212418000000",
  "status": { "code": 1 },
  "attributes": [
    { "key": "aireserve.schema", "value": { "stringValue": "aireserve.trace.v1" } },
    { "key": "aireserve.event_id", "value": { "stringValue": "4f7b8e5a-1c2d-4e3f-9a0b-6c7d8e9f0a1b" } },
    { "key": "aireserve.request_id", "value": { "stringValue": "req_01HZX9K3Q4M2N8P7R6S5T4V3W2" } },
    { "key": "aireserve.status", "value": { "stringValue": "success" } },
    { "key": "aireserve.model.served", "value": { "stringValue": "claude-sonnet-5" } },
    { "key": "aireserve.model.provider", "value": { "stringValue": "anthropic" } },
    { "key": "aireserve.usage.prompt_tokens", "value": { "intValue": 1842 } },
    { "key": "aireserve.usage.completion_tokens", "value": { "intValue": 310 } },
    { "key": "aireserve.usage.cached_input_tokens", "value": { "intValue": 1536 } },
    { "key": "aireserve.cost.charged_usd", "value": { "doubleValue": 0.006312 } },
    { "key": "aireserve.timing.latency_ms", "value": { "intValue": 2410 } },
    { "key": "aireserve.timing.ttft_ms", "value": { "intValue": 388 } },
    { "key": "aireserve.harness", "value": { "stringValue": "claude-code" } },
    { "key": "aireserve.attribution.user_id", "value": { "stringValue": "usr_51aa0c" } },
    { "key": "aireserve.attribution.end_customer_id", "value": { "stringValue": "acme-eu" } },
    { "key": "gen_ai.request.model", "value": { "stringValue": "claude-sonnet-5" } },
    { "key": "gen_ai.response.model", "value": { "stringValue": "claude-sonnet-5" } },
    { "key": "gen_ai.provider.name", "value": { "stringValue": "anthropic" } },
    { "key": "gen_ai.usage.input_tokens", "value": { "intValue": 1842 } },
    { "key": "gen_ai.usage.output_tokens", "value": { "intValue": 310 } }
  ]
}
```

The span above is abbreviated — every field in the table arrives as `aireserve.<path>`
(null fields are omitted). Error events add `error.type = <error.reason>`.

A minimal Collector receiver that accepts it:

```yaml collector-config.yaml theme={"dark"}
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
        # Require the destination's secret as a bearer token.
        auth:
          authenticator: bearertokenauth

extensions:
  bearertokenauth:
    token: ${env:AIRESERVE_EXPORT_TOKEN}

service:
  extensions: [bearertokenauth]
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp/your-backend]
```

### Webhook

Any HTTPS endpoint you run. The gateway POSTs a JSON batch and signs it with the
destination's secret so your receiver can prove the batch came from AI Reserve and was not
replayed.

| Setting | Value |
| - | - |
| URL | `https://` only (plain `http://` is accepted for `localhost`). Redirects are not followed. |
| Secret | At least 16 characters. Generate it yourself (for example `openssl rand -hex 32`) and paste it in the portal; it is shown again only as its last 4 characters. |

The request:

```http theme={"dark"}
POST /aireserve/traces HTTP/1.1
Host: hooks.example.com
Content-Type: application/json
User-Agent: AIReserve-TraceExport/1
X-AIReserve-Event: aireserve.trace.v1
X-AIReserve-Delivery: 7c0d2a1e-5b3f-4a98-9e6d-0f1a2b3c4d5e
X-AIReserve-Signature: t=1793950213,v1=5f1d0c7e9a2b4c6d8e0f1a3b5c7d9e1f2a4b6c8d0e2f4a6b8c0d2e4f6a8b0c2d

{
  "schema": "aireserve.trace.v1",
  "batch_id": "7c0d2a1e-5b3f-4a98-9e6d-0f1a2b3c4d5e",
  "sent_at": "2026-11-05T09:30:13.002Z",
  "events": [ { "schema": "aireserve.trace.v1", "event_id": "…", "...": "…" } ]
}
```

* `events` holds up to 100 events in the order they settled on that gateway instance.
* `batch_id` (also in `X-AIReserve-Delivery`) is stable across retries of the same batch,
  and the body is byte-identical across retries — only the signature's `t` changes. Dedupe
  on `batch_id`, or on each event's `event_id` if you flatten batches.
* Respond with any `2xx` as fast as you can and process asynchronously. A send that takes
  more than 10 seconds counts as a failure and is retried.

#### Verify the signature

`X-AIReserve-Signature` is `t=<unix seconds>,v1=<hex HMAC-SHA256>` where the MAC is
computed over the string `<t>.<raw request body>` with the destination's secret. Verify
against the **raw bytes** you received (before any JSON parsing), compare in constant time,
and reject timestamps more than five minutes from your clock to defeat replays. During a
secret rotation the header can carry more than one `v1=` entry; accept the batch if any of
them matches.

```typescript Node.js theme={"dark"}
import { createHmac, timingSafeEqual } from "node:crypto";

export function verifyAIReserveSignature(
  secret: string,
  rawBody: string | Buffer,
  header: string | undefined,
  toleranceSeconds = 300,
): boolean {
  if (!header) return false;
  let timestamp: number | null = null;
  const signatures: string[] = [];
  for (const part of header.split(",")) {
    const [key, value] = part.trim().split("=", 2);
    if (key === "t" && value && /^\d+$/.test(value)) timestamp = Number(value);
    else if (key === "v1" && value && /^[0-9a-f]{64}$/i.test(value)) signatures.push(value.toLowerCase());
  }
  if (timestamp === null || signatures.length === 0) return false;
  if (Math.abs(Math.floor(Date.now() / 1000) - timestamp) > toleranceSeconds) return false;

  const expected = createHmac("sha256", secret).update(`${timestamp}.`).update(rawBody).digest();
  return signatures.some((candidate) => {
    const given = Buffer.from(candidate, "hex");
    return given.length === expected.length && timingSafeEqual(given, expected);
  });
}

// Express: keep the raw body — express.json() would re-serialize it.
app.post("/aireserve/traces", express.raw({ type: "application/json" }), (req, res) => {
  if (!verifyAIReserveSignature(process.env.AIRESERVE_WEBHOOK_SECRET!, req.body, req.header("X-AIReserve-Signature"))) {
    return res.status(401).end();
  }
  const batch = JSON.parse(req.body.toString("utf8"));
  queue.push(batch); // ack first, process later
  res.status(202).end();
});
```

```python Python theme={"dark"}
import hashlib
import hmac
import re
import time


def verify_aireserve_signature(secret: str, raw_body: bytes, header: str | None, tolerance_s: int = 300) -> bool:
    if not header:
        return False
    timestamp = None
    signatures = []
    for part in header.split(","):
        key, _, value = part.strip().partition("=")
        if key == "t" and value.isdigit():
            timestamp = int(value)
        elif key == "v1" and re.fullmatch(r"[0-9a-fA-F]{64}", value):
            signatures.append(value.lower())
    if timestamp is None or not signatures:
        return False
    if abs(int(time.time()) - timestamp) > tolerance_s:
        return False
    expected = hmac.new(secret.encode(), f"{timestamp}.".encode() + raw_body, hashlib.sha256).hexdigest()
    return any(hmac.compare_digest(candidate, expected) for candidate in signatures)


# Flask
@app.post("/aireserve/traces")
def aireserve_traces():
    if not verify_aireserve_signature(WEBHOOK_SECRET, request.get_data(), request.headers.get("X-AIReserve-Signature")):
        return "", 401
    queue.put(request.get_json(force=True))
    return "", 202
```

### Datadog Logs

Sends straight to your Datadog account's log intake on **your** API key — no agent or
pipeline needed. One log entry per event, with the whole event nested under `aireserve` so
facets such as `@aireserve.model.served` and `@aireserve.cost.charged_usd` work immediately.

| Setting | Value |
| - | - |
| Site | Your Datadog site: `datadoghq.com`, `us3.datadoghq.com`, `us5.datadoghq.com`, `datadoghq.eu`, `ap1.datadoghq.com` or `ddog-gov.com`. |
| API key | A Datadog **API** key (not an application key). |
| Service (optional) | The `service` on each entry. Defaults to `aireserve`. |
| Tags (optional) | Up to 20 extra `key:value` tags appended to `ddtags`, for example `env:prod,team:platform`. |

The request:

```http theme={"dark"}
POST /api/v2/logs HTTP/1.1
Host: http-intake.logs.datadoghq.com
Content-Type: application/json
DD-API-KEY: <your API key>
User-Agent: AIReserve-TraceExport/1

[
  {
    "ddsource": "aireserve",
    "ddtags": "source:aireserve,aireserve.model:claude-sonnet-5,aireserve.provider:anthropic,aireserve.status:success,aireserve.harness:claude-code,env:prod,team:platform",
    "service": "aireserve",
    "status": "info",
    "timestamp": 1793950212418,
    "message": "aireserve claude-sonnet-5 ok 2152 tokens",
    "aireserve": { "schema": "aireserve.trace.v1", "event_id": "…", "...": "…" }
  }
]
```

* `status` is `info` for successful requests and `error` for failed ones; error entries
  read `aireserve <model> error <reason> (<fault>)`.
* `timestamp` is the settle instant in epoch milliseconds, so entries land at the time the
  request happened rather than when the batch was delivered.
* Useful starting queries: `source:aireserve @aireserve.status:error`,
  `source:aireserve @aireserve.attribution.end_customer_id:acme-eu`, or a measure over
  `@aireserve.cost.charged_usd` grouped by `@aireserve.model.served`.

## Delivery

Export happens **after** the response. Each gateway instance pushes the settled event onto
an in-memory queue and returns; everything else — batching, sending, retrying, pausing —
runs in the background, so a slow or failing destination never slows or fails a request.

| Behavior | Detail |
| - | - |
| Batching | Up to 100 events per send, flushed at least every 2 seconds. |
| Backlog | 5,000 events per gateway instance. When a destination falls behind, the oldest events are dropped and the drop is counted. |
| Retries | Up to 5 attempts per batch with exponential backoff and full jitter (1 s, 2 s, 4 s, 8 s, capped at 15 s) while the failure is retryable: a network error, a timeout, or HTTP 408, 425, 429 or 5xx. |
| Dead letters | Any other 4xx (bad credentials, wrong URL, rejected payload) is not retried; the batch is dead-lettered at once and counted. |
| Circuit breaker | After 5 consecutive failed batches the destination is paused for 5 minutes and its events are dropped and counted. When the pause lapses one batch goes through as a probe: success resumes delivery, failure pauses it again. |
| Send budget | 10 seconds per attempt. |

Delivery is **at least once**: a batch can arrive twice if your endpoint accepted it but the
acknowledgement was lost. Ordering is per gateway instance, not global. Dedupe on
`event_id` (or the OTLP span id, which derives from it).

### Health

The portal shows live health for every destination: last successful delivery, last failure
and its reason (an HTTP status or a short transport reason, never a response body),
failures in the last 24 hours, events dropped, and events delivered. A destination with a
tripped breaker shows when it will retry. **Send test event** bypasses the queue and the
breaker so you can check a fix immediately.

## Security

* Secrets (bearer tokens, webhook secrets, Datadog API keys) are encrypted at rest with
  AES-256-GCM in the same envelope scheme that protects BYOK provider keys, decrypted only
  inside the gateway at the moment of sending, and shown in the portal only as their last 4
  characters. They are never written to logs, error records or audit entries.
* Destination URLs must be HTTPS (loopback `http://` is allowed for a local collector during
  development). Redirects are not followed, and response bodies are never read.
* Destinations must be public hosts. Private, loopback, link-local and cloud-metadata
  addresses, and internal-only names (`localhost`, `*.internal`, `*.local`), are refused when
  you save, and the host is resolved again on every send so a name that later points at a
  private address is refused too and shows as a failed delivery.
* Adding, editing, pausing, testing and removing destinations are admin-only actions,
  recorded in the internal audit log and on your organization's Activity page.
* Events carry ids and measurements only. The payload type is guarded against content-shaped
  fields (`prompt`, `messages`, `content`, `tool_calls`, `headers`, `error_message`, …) by a
  compile-time check and a unit test, and the event builder reads a fixed allow-list of usage
  metadata rather than copying it.

## Set it up

**1. Open Trace Export**

In the [portal](https://portal.aireserve.com), go to **Observe › Trace Export** (admins
only). From the platform admin console the same page is under the client's
**Observability** section.

**2. Add a destination**

Choose **OpenTelemetry collector**, **Webhook** or **Datadog Logs**, give it a name, and
paste the endpoint and secret. Settings are validated before they are saved.

**3. Send a test event**

Use **Send test event** on the new destination and look for model `aireserve-test-model`
on the receiving side. A refusal is reported with the HTTP status so you can fix the URL
or secret in place.

**4. Watch it deliver**

Real traffic starts flowing within about a minute. The row shows last success, failures
in the last 24 hours, dropped and delivered counts. Pause a destination with the toggle
at any time; events during a pause are not queued.

> [!NOTE]
>
> v1 manages destinations in the portal only. A public API for destinations, per-destination
> filters (by model, key or team) and an S3/GCS batch sink are on the roadmap; tell us which
> you need first.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.