models is shorthand. The first entry is the primary model; the rest become
fallbacks. The two spellings below are identical:
model a single string. Official OpenAI SDKs type it that way, and every
coding tool that replays model verbatim depends on it. List up to four models:
one primary plus three fallbacks.
Which model answered
When a fallback served the request, the response carries thex-aireserve-served-model header naming it. Read the header, not the body’s
model field. The header is absent when the primary answered. Your usage
ledger records both the requested and the served model on every row, so
reporting and billing follow what actually ran.
How entries are checked
Every entry is held to the same policy as the primary model. An entry is dropped from the chain, never served silently, when it is:- not a model in the catalog (after alias resolution, so
fable-5-1andclaude-fable-5-1are both fine); - outside your organization’s model allowlist or inside its blocklist;
- behind an organization consent your organization has not accepted (China-hosted serving, training use, and so on), or behind a restricted-access list you are not on;
- the primary itself, or a duplicate of an earlier entry.
Portal defaults and the request-level list
Organization admins can configure a default fallback chain per model in the portal under Inference → Router. A request-level list replaces that default for the one request that carries it. Fallbacks are opt-in per organization or per request. A new organization has no failover of any kind, so a provider’s real error reaches you. Configuring at least one fallback chain in the portal switches failover on for the whole organization: the chains you wrote apply to the models they name, and the models we have paired across providers with identical weights (the Claude models on Anthropic and Amazon Bedrock, for example) fail over to their sibling provider automatically, same model, same request, with nothing to configure per model. To keep a chained model on the same weights first, list the sibling as its first rung (claude-sonnet-4-6-bedrock, then a
different model). Ask support to change either behaviour. Two edge cases:
"fallbacks": []opts a single request out of the organization’s default chain.modelsnever produces an empty list, so it cannot opt out by accident.- Requests on your organization’s own provider key (BYOK) never fall back to a different provider.
Where it applies
Failover engages on upstream failures (provider errors, rate limits, timeouts,
connection errors) after the gateway’s own retries on the primary are
exhausted. The same request, unchanged, is then sent to the next model in your
list.