Skip to main content
Name the models you accept, in order of preference, on the request itself. If the first one fails, the gateway retries the same request on the next one, so a provider incident on one model does not take your feature down. No portal configuration is needed.
models is shorthand. The first entry is the primary model; the rest become fallbacks. The two spellings below are identical:
Keep model a single string. Official OpenAI SDKs type it that way, and every coding tool that replays model verbatim depends on it. List up to four models: one primary plus three fallbacks.

Which model answered

When a fallback served the request, the response carries the x-aireserve-served-model header naming it. Read the header, not the body’s model field. The header is absent when the primary answered. Your usage ledger records both the requested and the served model on every row, so reporting and billing follow what actually ran.

How entries are checked

Every entry is held to the same policy as the primary model. An entry is dropped from the chain, never served silently, when it is:
  • not a model in the catalog (after alias resolution, so fable-5-1 and claude-fable-5-1 are both fine);
  • outside your organization’s model allowlist or inside its blocklist;
  • behind an organization consent your organization has not accepted (China-hosted serving, training use, and so on), or behind a restricted-access list you are not on;
  • the primary itself, or a duplicate of an earlier entry.
If nothing survives, the request runs exactly as if you had sent no list.

Portal defaults and the request-level list

Organization admins can configure a default fallback chain per model in the portal under Inference → Router. A request-level list replaces that default for the one request that carries it. Fallbacks are opt-in per organization or per request. A new organization has no failover of any kind, so a provider’s real error reaches you. Configuring at least one fallback chain in the portal switches failover on for the whole organization: the chains you wrote apply to the models they name, and the models we have paired across providers with identical weights (the Claude models on Anthropic and Amazon Bedrock, for example) fail over to their sibling provider automatically, same model, same request, with nothing to configure per model. To keep a chained model on the same weights first, list the sibling as its first rung (claude-sonnet-4-6-bedrock, then a different model). Ask support to change either behaviour. Two edge cases:
  • "fallbacks": [] opts a single request out of the organization’s default chain. models never produces an empty list, so it cannot opt out by accident.
  • Requests on your organization’s own provider key (BYOK) never fall back to a different provider.

Where it applies

Failover engages on upstream failures (provider errors, rate limits, timeouts, connection errors) after the gateway’s own retries on the primary are exhausted. The same request, unchanged, is then sent to the next model in your list.