> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aireserve.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Reliability

> How the gateway stays up: provider failover, traffic splitting, transport recovery, a multi-zone platform with highly available data stores, deploy safety, and the numbers we measure ourselves against.

An AI gateway can fail you in two ways: the provider behind your model can go down, or the
gateway itself can. This page covers what is in place for both, in plain terms, including
the parts that are not redundant yet. The companion page,
[Handling provider downtime](/reliability/business-continuity), covers how you would know
about an incident, how we keep you running, and what we commit to.

## The short version

| Layer | What is in place |
| - | - |
| Provider failover | [Fallback chains](/capabilities/fallbacks) per request or per organization, platform provider failover for models served by more than one provider, and a fleet-wide health breaker that routes around a failing provider and probes it back in. |
| Traffic splitting | Weighted pools across providers that serve identical weights, with a canary that trips a degraded leg out of the split automatically. Every response discloses which model and provider answered. |
| Transport recovery | Connection retries, a one-time re-send of a stream that drops before its first byte, first-token budgets and end-to-end request deadlines, so a hung provider never hangs you. |
| Platform | A multi-zone Google Cloud region, independently autoscaled services with warm minimums, a regional high-availability PostgreSQL with point-in-time recovery, high-availability Redis, a global load balancer. |
| Change safety | Every change ships to a full staging environment first, production deploys are approved and configuration-guarded, a post-deploy error canary watches the cutover, and rollback is one command. |
| Monitoring | External uptime checks every 60 seconds, a platform-health sweep every 5 minutes, a provider status feed, and alerting to the team around the clock. |

## Provider failover

**Fallback chains.** Name the models you accept, in order, on the request (`models` or
`fallbacks`) or set a default chain per model in the portal under Inference → Router. When
the primary fails on an upstream error (provider 5xx, rate limit, timeout, connection error)
after the gateway's own retries, the same request is sent to the next model in your list.
The response carries `x-aireserve-served-model` whenever something other than the primary
answered. Details, limits and edge cases: [Fallback models](/capabilities/fallbacks).

**Platform provider failover.** Several models are served by more than one provider with
identical weights, for example the Claude models on Anthropic and on Amazon Bedrock. Once
failover is on for your organization, a failure at the primary provider rolls the same
request to the sibling leg automatically: same model, same request, nothing to configure per
model and nothing for your code to do. Your ledger records both the model you asked for and
the leg that served, so billing and reporting follow what actually ran.

**Who turns it on.** Failover is off for a new organization, so a provider's real error
reaches you instead of a serve on a model nobody chose. Your administrators switch it on in
the portal: configuring any fallback chain under Inference → Router enables both layers for
the organization (provider failover for the models we have paired across providers, plus
the chains you wrote for specific models), and a single request opts in on its own by
carrying `models`.
To keep a chained model on the same weights first, list its sibling as the first rung
(`claude-sonnet-4-6-bedrock` ahead of a different model, for example). We can also set a
chain up for you; ask in your shared Slack channel.

**Health breaker.** Provider legs share one health record across the whole gateway fleet.
Three slow failures (hangs or 5xx) on a leg inside a rolling two-minute window trip it open:
new requests from organizations with a chain are routed to the next healthy rung instead of
each one rediscovering the failure and paying the full timeout. A leg closes again only after
consecutive successful synthetic probes, never on a timer, so a provider that is still sick
does not get traffic back. Rate limits (429) deliberately do not trip the breaker; they are
handled per request by retries and the chain walk, which costs milliseconds rather than a
leg.

**What never fails over.** Errors that are yours to fix (a malformed request, an unknown
model, a budget or spend cap you have exhausted) are returned honestly with a
[stable error code](/api-reference/errors) and are never retried on another model.
Requests on your own provider key ([BYOK](/capabilities/byok)) never fall back to a
different provider.

## Traffic splitting and routing control

**Serving pools.** For a model served on more than one provider with identical weights, we
can split an organization's traffic across those providers by weight, for example 70/30
between Anthropic and Bedrock, including a minimum-prompt-size rule so only long prompts take
a given leg. A canary compares each leg with the source leg over consecutive five-minute
windows on p95 latency, cache-hit ratio, rate-limit rate and upstream error rate, and trips a
degraded leg out of the split automatically; its share redistributes to the healthy legs and
a human re-arms it after the cause is understood. Ask us to set a pool up for your
organization.

**Routing variants and pins.** `:nitro` and `:floor` suffixes ask for the fastest or the
cheapest provider of a model per request, and an organization can pin a model to one provider
for data-residency or contractual reasons. See
[Provider routing variants](/models#provider-routing-variants).

**Disclosure.** Whenever a request is answered by something other than exactly what you asked
for, the response says so: `x-aireserve-served-model` names the model that answered and
`x-aireserve-served-provider` the provider, and the usage ledger keeps the requested and
served identities side by side on every row. No header means the request was served as
requested.

## Transport recovery

* **Stale connections.** A connection to the inference engine or a provider that was closed
  on the far side while idle is retried once at connect time, before anything is sent.
* **Streams that die before the first byte.** A streamed request whose provider connection
  drops before any content has been written is re-sent once, automatically. Nothing has
  reached you and nothing has been billed, so the retry is invisible.
* **First-token and silence budgets.** A stream that produces no first token inside its
  budget is cut with a 504 instead of hanging your client, and the leg that hung is counted
  against its health. Once content is flowing, only silence can end a stream, never a wall
  clock, so long generations are safe.
* **Deadlines.** The request budget is minted at the edge and travels with the request to
  every hop, so a slow upstream is given up on at the budget, not after a chain of nested
  timeouts.
* **Retry-After.** Every 503 the gateway answers for itself carries `Retry-After`, and the
  SDKs' default retry policies honor it. See [Retry policy](/api-reference/errors#retry-policy-bedrock-standard-mode-analog).

## The platform

| Component | Posture |
| - | - |
| Region | Google Cloud `us-east4` (Northern Virginia). Every service runs across the region's zones; a single zone failure is absorbed. |
| Edge | A global external Application Load Balancer with Google-managed TLS in front of the API, with no CDN in the path so streams are never buffered. A Cloud Armor policy sits at the edge. |
| Serving tier | Four independently scaled services (API gateway, inference proxy, inference engine, portal) on Cloud Run, each with warm minimum instances, startup CPU boost and no CPU throttling, scaling to dozens of instances under load. |
| Database | Cloud SQL for PostgreSQL in regional high-availability mode (synchronous standby in a second zone, automatic failover), point-in-time recovery, and seven daily backups. |
| Cache and coordination | Memorystore for Redis in standard high-availability mode with an automatic failover replica. The request-path features that use it (rate limiting, the health breaker) fail open or fall back to a per-instance copy if Redis is unavailable, rather than failing requests. |
| Secrets | Google Secret Manager, mounted at deploy time; provider keys never live in images or code. |

> [!WARNING]
>
> **One region today.** Multi-zone protects against a zone or instance failure, not against a region-wide Google Cloud
> event, which would take the API down until the region recovers. There is no automatic cross-region failover yet; a
> second region in active-passive mode is the next infrastructure milestone, and [Handling provider
> downtime](/reliability/business-continuity) covers what you do in the meantime.

## Change safety

* **Staging first, every time.** Every merge deploys to a full staging environment (same
  services, same configuration shape) and runs the smoke suite before anything reaches
  production.
* **Approved, pinned production deploys.** Production deploys are dispatched explicitly,
  reviewer-approved and pinned to a commit. A configuration guard refuses a deploy that would
  silently change a strict serving flag.
* **Post-deploy error canary.** After traffic cuts over, a canary compares each serving
  service's error rate against the window before the deploy and raises an alarm on a
  regression.
* **Rollback is one command.** Previous revisions are retained; a rollback is a traffic switch
  to the last known-good revision, done by a person within minutes.
* **Smoke test on a real path.** The suite exercises the production path customers use
  (gateway, proxy, engine, providers, voice) rather than an internal shortcut.

## Monitoring and alerting

* External uptime checks from Google's global checker fleet probe the API and the portal
  every 60 seconds over the same DNS, certificate and load-balancer path customers use.
* A platform-health sweep runs every 5 minutes with detectors for success rate, latency and
  timeouts per model and provider, provider-origin and router-origin rate limiting,
  settlement lag, traffic flat-lines and per-customer latency; each fires once per crossing,
  with a cooldown, and posts a recovery line when it clears.
* Provider status pages are aggregated at [aireserve.com/status](https://aireserve.com/status)
  so you can see at a glance whether an incident is upstream.
* Alerts reach the team in Slack and by email at any hour; billing and provider-account
  canaries run every 15 minutes so a dead provider key is found before you find it.

## What we measure

Figures for the 30 days to October 14, 2026, measured at the API edge (every request to
`api.aireserve.com`, all customers, all endpoints):

| Measure | Result |
| - | - |
| Requests | 7.5 million |
| Requests answered without a 5xx, provider-caused failures counted against us | 99.90% |
| Outage windows (five-minute windows where most requests failed) | None |
| Degraded windows (five-minute windows with more than 10% 5xx) | Five short periods, about 40 minutes in total. The majority were caused by our own components, not by providers; each has a fix in place or in progress. |

We publish these numbers rather than a service-level agreement. No uptime guarantee or
service credits are in force today; an SLA follows once cross-region failover is in place and
has been exercised. If you need the same figures for your own traffic, your usage analytics
carry status and latency per request, and your account team can pull them for a review.

Questions, or a questionnaire to fill in? Write to [security@aireserve.com](mailto:security@aireserve.com)
or ask in your shared Slack channel.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.