Skip to main content
A gateway earns its place by adding capabilities without adding noticeable latency. We measure ours the hard way: the same request, to the same model at the same provider, sent through each gateway and to the provider directly, interleaved from one client, and we publish the paired difference. The method, the numbers and the raw data are below.

​
What is measured

  • Two surfaces, one model each. Anthropic Messages with Claude Sonnet 4.6 served by Anthropic, and OpenAI Responses with GPT-5.5 served by OpenAI: the two wire formats every gateway in the comparison serves, so the provider side is identical for every target.
  • Pinned providers. Each gateway is asked for the same upstream provider by its own routing controls (for example OpenRouter’s provider.order with allow_fallbacks: false), so no target wins by routing to a different host.
  • Two numbers per target. For streaming requests, time to the first text token (TTFT). For non-streaming requests, total time to the complete response. Both from the client’s side, from the moment the request is sent.
  • Paired overhead, with its uncertainty. In every round each target is called once in random order. A gateway’s overhead is the median, over all rounds, of its time minus the direct provider call’s time in the same round, which cancels the provider’s own minute-to-minute drift. Each median is published with a 95% confidence interval (1.58 × IQR ÷ √rounds, the notched box-plot rule). A median that does not clear its own interval is labelled within noise: the run cannot tell that gateway apart from calling the provider directly, in either direction.
  • Small output, by design. max_tokens: 32 and the prompt “Reply with exactly one word: pong.” keep generation time short, so the numbers measure the path, not the model.
  • One client. A GitHub-hosted runner in the US East region, warm-up request per target excluded. The number of rounds is stated with each run (60 per mode for runs from 2026-10-18 on; the first published run used 20). More rounds narrow the intervals and steady the medians.

​
Results

Run of 2026-10-18 (UTC), 60 rounds per mode, max_tokens: 32, targets: Anthropic directly, OpenAI directly, AI Reserve, OpenRouter, Vercel AI Gateway. Anthropic Messages, Claude Sonnet 4.6 at Anthropic Overhead added on top of calling the provider directly (median of paired rounds, ± its 95% confidence interval): “Within noise” means the median does not clear its own interval: the run cannot tell that gateway apart from calling the provider directly, in either direction. Absolute latency from the runner, p50 (p95), and the fastest single round: Model ids as sent: Anthropic directly claude-sonnet-4-6, AI Reserve claude-sonnet-4-6, OpenRouter anthropic/claude-sonnet-4.6, Vercel AI Gateway anthropic/claude-sonnet-4.6. OpenAI Responses, GPT-5.5 at OpenAI Overhead added on top of calling the provider directly (median of paired rounds, ± its 95% confidence interval): Absolute latency from the runner, p50 (p95), and the fastest single round: Model ids as sent: OpenAI directly gpt-5.5, AI Reserve gpt-5.5, OpenRouter openai/gpt-5.5, Vercel AI Gateway openai/gpt-5.5.

​
Reading the numbers

  • Overhead is what the gateway adds; the p50 and p95 columns include the provider’s own latency, which dominates both.
  • A negative median is not, by itself, the gateway beating the provider. A request through a gateway travels one more hop than a direct call, and the best rounds in the table show roughly what that hop costs: a gateway’s fastest round is usually tens of milliseconds behind the provider’s fastest. The sign of a median can come from the tail instead. A provider’s own response time varies by a second or more from one request to the next, the pairing does not cancel that, and a run in which a few more slow responses happen to land on the direct calls tips the median negative. The interval says whether that happened. A negative median inside its own interval is within noise, and the honest reading is that the gateway added nothing the run could measure. One that clears its interval is a real but small effect, pooled already-open connections to the provider and a network path that happens to be shorter than the client’s, and it is worth tens of milliseconds, not hundreds.
  • p95 shows tail behavior over a short run and moves between runs; the medians and their intervals are the stable comparison.
  • The runner sits in one US region. A client in Europe or Asia sees different absolute numbers for every target, but the gateway overheads move together.

​
What is not in this table

  • Other gateways. The comparison covers the gateways that serve both surfaces above with provider pinning. We add a gateway when we can run it under the same rules, from the same client, with a production key.
  • Other models. The two surfaces above are the ones every target serves identically. Per-model latency across our own catalog is a different question, answered per model in the catalog and in your own usage analytics.

​
Reproduce it

Everything a run needs is on this page: the prompt, max_tokens, the model id sent to each target, the provider-pinning rule, the interleaving and the paired-median definition. Send the same request to each target from one client, alternate the order every round, and take the median of the per-round differences together with its interval (1.58 × IQR ÷ √rounds). We keep the raw per-request samples of every published run (round, target, mode, status, time to first token, total time) and will share them, or run the same harness against your own region or model mix, on request in your shared Slack channel.