JSON response healing

Models that are asked for JSON still wrap it in a markdown fence now and then, leave a trailing comma, or run out of tokens mid-object. When your request sets response_format to json_object or json_schema and the model’s answer does not parse, the gateway makes one deterministic repair pass before it returns the response, then checks the result with a strict JSON parser. If the repair works you receive parseable JSON and a header that says so. If it does not, you receive the model’s output exactly as it came. Nothing to configure, on by default.

{
  "model": "claude-fable-5-1",
  "messages": [{ "role": "user", "content": "Return the invoice as JSON." }],
  "response_format": { "type": "json_object" }
}

What gets repaired

Each repair is a fixed, rule-based rewrite. No model is involved and the same input always produces the same output. The three classes run in this order on one pass.

Fences and surrounding prose

Everything before the first { or [ and everything after its matching closer is dropped: markdown fences, “Here is the JSON:” preambles, closing remarks.

Before

Here is the invoice:

```json
{"total": 42, "currency": "USD"}
```

After

{"total": 42, "currency": "USD"}

Trailing commas

A comma directly before a } or ] is removed. Commas inside string values are left alone.

Before

{"items": ["a", "b",], "total": 2,}

After

{"items": ["a", "b"], "total": 2}

Cut-off output

When the model stopped early (finish_reason: "length" is the usual sign), the open string is terminated and every open array and object is closed, in the right order. A key that never received its value is dropped, since there is nothing to put in it.

Before

{"customer": "Acme", "lines": [{"sku": "A1", "qty": 2}, {"sku": "B2", "qty

After

{"customer": "Acme", "lines": [{"sku": "A1", "qty": 2}, {"sku": "B2"}]}

A healed cut-off response is valid JSON, not a complete answer. Check finish_reason and raise max_tokens if you see this often.

Knowing when it happened

The response carries x-aireserve-healed: json only when a repair was applied and the result parses. The header is absent when the output was already valid and when the repair did not succeed, so its presence is the one signal you need to log or branch on.

curl -i https://api.aireserve.com/v1/chat/completions \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-fable-5-1",
    "messages": [{ "role": "user", "content": "Return {\"ok\": true} as JSON." }],
    "response_format": { "type": "json_object" }
  }'
# look for "x-aireserve-healed: json" among the response headers

from openai import OpenAI

client = OpenAI(base_url="https://api.aireserve.com/v1", api_key=os.environ["AIRESERVE_API_KEY"])

res = client.chat.completions.with_raw_response.create(
    model="claude-fable-5-1",
    messages=[{"role": "user", "content": "Return the invoice as JSON."}],
    response_format={"type": "json_object"},
)
healed = res.headers.get("x-aireserve-healed") == "json"  # False unless a repair was applied
invoice = json.loads(res.parse().choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.aireserve.com/v1", apiKey: process.env.AIRESERVE_API_KEY });

const { data, response } = await client.chat.completions
  .create({
    model: "claude-fable-5-1",
    messages: [{ role: "user", content: "Return the invoice as JSON." }],
    response_format: { type: "json_object" },
  })
  .withResponse();

const healed = response.headers.get("x-aireserve-healed") === "json"; // false unless a repair was applied
const invoice = JSON.parse(data.choices[0].message.content ?? "null");

Turning it off

Send healing: false in the request’s provider block to receive the model’s output untouched for that request. The block is consumed by the gateway and never forwarded, and healing is honored on its own, whether or not you use the block’s routing fields.

{
  "model": "claude-fable-5-1",
  "messages": [{ "role": "user", "content": "Return the invoice as JSON." }],
  "response_format": { "type": "json_object" },
  "provider": { "healing": false }
}

Guarantees

Valid output is never altered. Output that already parses is returned byte for byte, whitespace included. Healing only ever runs on output that fails to parse.
One pass, or nothing. If the repaired text still does not parse, you get the original, unchanged, with no header.
Parseable, not schema-checked. Healing makes the text valid JSON. It does not add missing fields or check the result against a json_schema; validate the parsed object as you would any model output.
Billed as served. Tokens and cost are the provider’s figures for what the model generated. Healing changes the text you receive, never the usage.

Where it applies

Endpoint	Healing
`POST /v1/chat/completions`	Non-streaming requests with `response_format` of `json_object` or `json_schema`.
Streaming requests	Not applied. Tokens reach you as they are generated, so there is no finished body to repair. Parse the assembled text client-side instead.
`POST /v1/responses`	Not applied; the request passes through unchanged.
`POST /v1/messages`	Not applied; the Anthropic dialect has no `response_format`.

Requests without a JSON response_format are never touched, whatever the model returns.

Getting started

Connect your tool

GPU Infrastructure

AWS Hub

Capabilities

Migrate

Performance

Data & privacy

Reliability

What gets repaired

Fences and surrounding prose

Trailing commas

Cut-off output

Knowing when it happened

Turning it off

Guarantees

Where it applies

​What gets repaired

​Fences and surrounding prose

​Trailing commas

​Cut-off output

​Knowing when it happened

​Turning it off

​Guarantees

​Where it applies

What gets repaired

Fences and surrounding prose

Trailing commas

Cut-off output

Knowing when it happened

Turning it off

Guarantees

Where it applies