gpt-5.4-mini, gpt-5.5,
claude-opus-4-8). Message shapes, tool configs, streaming loops, and error
handling carry over verbatim.
Java note.
converseStream in the AI Reserve SDK is synchronous
(a blocking push handler or a pull iterator) rather than the AWS async
CompletableFuture publisher — for most codebases this removes boilerplate rather
than adding it.Bedrock-routed models
Some teams prefer a model’s serving behavior via AWS Bedrock over the provider’s direct API. AI Reserve offers both paths behind the same endpoint: append-bedrock to a model ID and the gateway routes that request through AWS
Bedrock instead of the provider directly. Everything else — SDK, key, shapes,
streaming, errors — is identical, so you can A/B the two paths by changing only the
model string.
Models that Bedrock does not serve (OpenAI GPT, Gemini, Grok, Perplexity Sonar, Moonshot)
have no
-bedrock variant — the direct IDs remain the only path for those.
All variants route through AWS Bedrock except claude-fable-5-bedrock,
claude-opus-4-8-bedrock, and claude-opus-4-7-bedrock, which
temporarily serve via Anthropic’s direct API while Bedrock access for those models is
being enabled on our AWS account — the IDs are stable and will be re-pointed
gateway-side with no client change.
Latency
Compared to calling a provider (or Bedrock) directly from your own code, the AI Reserve gateway adds a small, fixed toll — authentication, rate limiting, quota checks, and request translation:
Rule of thumb: ~60–200 ms added to time-to-first-token, and effectively
zero between tokens — streaming responses are piped through without
re-buffering, so inter-token latency is determined entirely by the model provider.
Against typical generation times of seconds to tens of seconds, the overhead is low
single-digit percent. The toll does not grow with context size or generation length.
Note that provider-side differences (for example, Bedrock vs. direct-API
time-to-first-token for the same Claude model, which varies by region and load) are
independent of the gateway and typically larger than the gateway’s own overhead. The
-bedrock variants make measuring both paths for your workload a
one-string A/B test.
Already on boto3 or the AWS JS SDK and want to keep them? See
AWS SDK (boto3) — the gateway serves the Bedrock Converse API at the
same paths, so unmodified clients work by overriding the endpoint.