> ## Documentation Index > Fetch the complete documentation index at: https://docs.aireserve.com/llms.txt > Use this file to discover all available pages before exploring further. # Prompt caching > Cache long, stable prompt prefixes server-side — ~90% off cache reads on Anthropic models, automatic on OpenAI and Gemini. When consecutive requests share a long prefix — a large system prompt, a pasted document, a growing conversation — providers can cache that prefix server-side. Cached input tokens bill at a steep discount (**\~90% off** cache reads on Anthropic models; the first request pays a one-time **\~25% premium** on cache writes) and time-to-first-token improves because the provider skips re-processing the prefix. | Models | Caching behavior | What you do | | ------------------------------------------- | -------------------------------------- | ---------------------------- | | OpenAI (`gpt-*`) | Automatic for prompts of 1,024+ tokens | Nothing | | Gemini (`gemini-*`) | Automatic (implicit caching) | Nothing | | Claude (`claude-*` and `-bedrock` variants) | Opt-in per request | Add a `cache_control` marker | ## Marking a prefix on Claude models The gateway accepts the Anthropic-style marker in OpenAI-compatible format: write the message `content` as an array of parts and put `"cache_control": {"type": "ephemeral"}` on the content block where the cacheable prefix ends — everything up to and including that block is cached. The gateway forwards the marker to Anthropic (or AWS Bedrock for `-bedrock` variants) unchanged. A message may carry the marker on its **last** content block only, and a request may contain at most four markers. ```bash theme={"dark"} curl https://api.aireserve.com/v1/chat/completions \ -H "Authorization: Bearer $AUDACITY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "claude-sonnet-4-6", "messages": [ { "role": "system", "content": [ { "type": "text", "text": "You are a contracts analyst. ", "cache_control": {"type": "ephemeral"} } ] }, {"role": "user", "content": "Summarize the termination clauses."} ], "max_tokens": 1024 }' ``` * **Minimum size.** Each model has a minimum cacheable prefix (see the table below); shorter prefixes are processed normally with **no error and no cache entry** — the marker is silently ignored. * **Lifetime.** Cache entries live for **5 minutes** by default, refreshed on each hit — steady traffic keeps the prefix warm. * **Exact-prefix match.** Reuse requires a byte-identical prefix up to the marker; put stable content (system prompt, documents) first and variable content after it. ## Minimum cacheable prefix per model Minimums differ by model *and by route*. In particular, AWS Bedrock enforces a higher **4,096-token** minimum for Opus and Haiku than Anthropic's direct API does — a prefix between 1,024 and 4,095 tokens that caches fine on `claude-opus-4-6` will get zero cache activity on `claude-opus-4-6-bedrock`. This is an AWS-side constraint (verified against Bedrock directly), not gateway behavior. | Model | Minimum prefix | Notes | | ------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Claude Sonnet (direct: `claude-sonnet-4-6`, `claude-sonnet-4-5`) | 1,024 tokens | Anthropic documented minimum | | Claude Sonnet on Bedrock (`claude-sonnet-4-6-bedrock`, `claude-sonnet-4-5-bedrock`) | 1,024 tokens | Same as direct | | Claude Opus (direct: `claude-fable-5`, `claude-opus-4-8`, `claude-opus-4-7`, `claude-opus-4-6`) | 1,024 tokens | Anthropic documented minimum | | Claude Haiku (direct: `claude-haiku-4-5-20251001`) | 2,048 tokens | Anthropic documented minimum | | **Claude Opus / Haiku on Bedrock** (`claude-opus-4-6-bedrock`, `claude-haiku-4-5-bedrock`, and other `-bedrock` Opus variants) | **4,096 tokens** | AWS Bedrock enforces a higher floor; sub-4,096 prefixes are accepted but never cached (no error, full input billing) | | OpenAI (`gpt-*`) | 1,024 tokens | Automatic — no marker needed | | Grok (`grok-*`) | No published minimum | Automatic — no marker needed; cache reads surface in `usage` when xAI serves them | | Gemini (`gemini-2.5-*`, `gemini-3-*`) | 1,024 (Flash) / 4,096 (Pro) tokens | Implicit caching is automatic and best-effort: cache hits depend on Google-side capacity and recency, so identical prompts may report `cached_tokens` only on some responses | **Placeholder Bedrock variants.** `claude-fable-5-bedrock`, `claude-opus-4-8-bedrock`, and `claude-opus-4-7-bedrock` temporarily serve via Anthropic's direct API (see [Bedrock-routed models](/migrate/from-bedrock#bedrock-routed-models)), so today their effective caching minimum is the direct-API **1,024 tokens**. Treat 4,096 as the stable planning number — it becomes exact when those IDs are re-pointed to Bedrock, with no client change. If your prefix is below the model's minimum, requests still succeed — you simply pay the full input rate. Sizing the stable prefix above the minimum (for Bedrock Opus/Haiku, above 4,096 tokens) is what unlocks the cache discount. ## Using the typed SDKs (`cachePoint`) As of **SDK v0.2.0**, all five SDKs support prompt caching natively via a Bedrock-style `cachePoint` content block. Place a cache point after the stable prefix you want cached — in the `system` list, in message `content`, or both. The SDK translates each cache point into the `cache_control` wire marker on the preceding content part; the block itself is never sent. The same limits apply: at most four cache points per request, and a cache point with nothing before it in the same message is silently ignored. ```python Python theme={"dark"} response = client.converse( modelId="claude-sonnet-4-6", system=[ {"text": long_system_prompt}, {"cachePoint": {"type": "default"}}, ], messages=[{ "role": "user", "content": [ {"text": big_reference_document}, {"cachePoint": {"type": "default"}}, {"text": "Summarise the key risks."}, ], }], ) # Cache activity is reported in usage (Bedrock names): print(response["usage"]["cacheReadInputTokens"]) # tokens served from cache print(response["usage"]["cacheWriteInputTokens"]) # tokens written to cache ``` ```typescript TypeScript theme={"dark"} const response = await client.send( new ConverseCommand({ modelId: "claude-sonnet-4-6", system: [ { text: longSystemPrompt }, { cachePoint: { type: "default" } }, ], messages: [ { role: "user", content: [ { text: bigReferenceDocument }, { cachePoint: { type: "default" } }, { text: "Summarise the key risks." }, ], }, ], }) ); // Cache activity is reported in usage (Bedrock names): console.log(response.usage?.cacheReadInputTokens); // tokens served from cache console.log(response.usage?.cacheWriteInputTokens); // tokens written to cache ``` ```go Go theme={"dark"} resp, err := client.Converse(ctx, &audacityruntime.ConverseInput{ ModelId: audacity.String("claude-sonnet-4-6"), System: []types.SystemContentBlock{ {Text: longSystemPrompt}, {CachePoint: &types.CachePointBlock{Type: types.CachePointTypeDefault}}, }, Messages: []types.Message{{ Role: types.ConversationRoleUser, Content: []types.ContentBlock{ &types.ContentBlockMemberText{Value: bigReferenceDocument}, &types.ContentBlockMemberCachePoint{ Value: types.CachePointBlock{Type: types.CachePointTypeDefault}, }, &types.ContentBlockMemberText{Value: "Summarise the key risks."}, }, }}, }) // Cache activity is reported in usage (Bedrock names): fmt.Println(resp.Usage.CacheReadInputTokens) // tokens served from cache fmt.Println(resp.Usage.CacheWriteInputTokens) // tokens written to cache ``` ```java Java theme={"dark"} ConverseResponse resp = client.converse(request -> request .modelId("claude-sonnet-4-6") .system( SystemContentBlock.fromText(longSystemPrompt), SystemContentBlock.fromCachePoint(CachePointBlock.defaultType())) .messages(Message.builder() .role(ConversationRole.USER) .content( ContentBlock.fromText(bigReferenceDocument), ContentBlock.fromCachePoint(CachePointBlock.defaultType()), ContentBlock.fromText("Summarise the key risks.")) .build())); // Cache activity is reported in usage (Bedrock names): System.out.println(resp.usage().cacheReadInputTokens()); // tokens served from cache System.out.println(resp.usage().cacheWriteInputTokens()); // tokens written to cache ``` ```rust Rust theme={"dark"} use audacity_sdk::{CachePointBlock, ContentBlock, SystemContentBlock}; let response = client.converse() .model_id("claude-sonnet-4-6") .system(SystemContentBlock::text(long_system_prompt)) .system(SystemContentBlock::cache_point()) .messages( Message::builder() .role(ConversationRole::User) .content(ContentBlock::Text(big_reference_document)) .content(ContentBlock::CachePoint(CachePointBlock::new())) .content(ContentBlock::Text("Summarise the key risks.".into())) .build()? ) .send() .await?; // Cache activity is reported in usage (Bedrock names): println!("{}", response.usage().cache_read_input_tokens); // tokens served from cache println!("{}", response.usage().cache_write_input_tokens); // tokens written to cache ``` ## Verifying and billing Cache activity is reported in the response `usage` object. On the wire the fields are `cache_creation_input_tokens` (tokens written to the cache) and `cache_read_input_tokens` (tokens served from it); the SDKs surface them under the Bedrock names `cacheWriteInputTokens` and `cacheReadInputTokens`. Both are metered per request in your usage reporting and billed at the premium write / discounted read rates — a non-zero cache-read count confirms the cache is doing its job. **SDK note.** Typed `cachePoint` support requires **SDK v0.2.0 or later** in every language. On OpenAI and Gemini models caching is automatic — the marker is ignored there, so the same code runs unchanged across providers.