Serve GLM-5.3

This playbook serves the canonical native-FP8 zai-org/GLM-5.3 checkpoint with vLLM on one 8×H200 node. It is the simplest vendor-documented topology that leaves useful runtime headroom.

Vendor-verified · AI Reserve qualification pending. The model and vLLM recipes are upstream-supported, but this exact AI Reserve launch has not completed the acceptance run. Pin an image digest after qualification; a floating image or version tag is not a production release artifact.

Recipe contract

Item	Pinned value
Artifact	`zai-org/GLM-5.3`
Revision reviewed	`ba32f8c19c8738e2c57c71798c601d477a7bf844`
Weights	Native block FP8; about 756 GB downloaded and about 893 GB of runtime budget before KV/cache headroom
Architecture	About 753B total / 39B active, text-only MoE
Context	1,048,576 architectural maximum; qualify smaller production limits before admitting 1M requests
Primary engine	vLLM 0.29.0+ reference path
Reference image	`vllm/vllm-openai:v0.29.0`; replace with its qualified digest
Hardware	8×`H200_141GB`
Persistent storage	1,200 GB minimum for this checkpoint, cache metadata, and operational headroom
License	GLM-5.3 custom license; MaaS businesses above the license’s revenue threshold require Z.AI security review
Last reviewed	2026-10-14

The full 1M window is a separate capacity target. Upstream documents 8×B200 with FP8 KV for that operating point; on H200, admit only the maximum context and concurrency your load test proves.

1. Set up the API client

Every command on this page is copy-paste. Paste the shell block once per terminal and replace only the API key; every later step reads these variables. The Python and TypeScript tabs read the same variables from the environment.

export AIRESERVE_API_KEY="aireserve_api_..."   # the only value you type: a key with Compute: Manage access
export API="https://api.aireserve.com"
export VLLM_API_KEY="$(openssl rand -hex 32)"  # generated for you; it protects your model server (see below)
echo "VLLM_API_KEY=$VLLM_API_KEY"               # save it: every request to the endpoint needs it

import json, os, time, uuid
from urllib.error import HTTPError
from urllib.request import Request, urlopen

API = "https://api.aireserve.com"
KEY = os.environ["AIRESERVE_API_KEY"]
VLLM_KEY = os.environ["VLLM_API_KEY"]

def api(path, method="GET", body=None, idempotency_key=None):
    headers = {"Authorization": f"Bearer {KEY}"}
    if body is not None:
        headers["Content-Type"] = "application/json"
    if idempotency_key:
        headers["Idempotency-Key"] = idempotency_key
    request = Request(
        f"{API}{path}",
        data=json.dumps(body).encode() if body is not None else None,
        headers=headers,
        method=method,
    )
    try:
        with urlopen(request) as response:
            return json.load(response)
    except HTTPError as error:
        raise RuntimeError(error.read().decode()) from error

def wait(operation_id):
    while True:
        operation = api(f"/v1/gpu/operations/{operation_id}")
        if operation["status"] == "succeeded":
            return operation
        if operation["status"] in {"failed", "cancelled"}:
            failure = operation.get("failure") or {}
            raise RuntimeError(f"{failure.get('actor', 'unknown')}: {failure.get('message', operation.get('errorMessage'))} {failure.get('next', '')}")
        time.sleep(5)

const API = "https://api.aireserve.com";
const KEY = process.env.AIRESERVE_API_KEY!;
const VLLM_KEY = process.env.VLLM_API_KEY!;

async function api(path: string, options: RequestInit = {}) {
  const response = await fetch(`${API}${path}`, {
    ...options,
    headers: {
      Authorization: `Bearer ${KEY}`,
      ...(options.body ? { "Content-Type": "application/json" } : {}),
      ...options.headers,
    },
  });
  if (!response.ok) throw new Error(await response.text());
  return response.json();
}

async function wait(operationId: string) {
  while (true) {
    const operation = await api(`/v1/gpu/operations/${operationId}`);
    if (operation.status === "succeeded") return operation;
    if (["failed", "cancelled"].includes(operation.status)) {
      const failure = operation.failure ?? {};
      throw new Error(`${failure.actor ?? "unknown"}: ${failure.message ?? operation.errorMessage} ${failure.next ?? ""}`);
    }
    await new Promise((resolve) => setTimeout(resolve, 5_000));
  }
}

Two keys, two jobs. AIRESERVE_API_KEY authenticates you to AI Reserve and creates the volume, quote, and instance. VLLM_API_KEY is a password you invent for the vLLM server you are about to start: step 5 passes it as vLLM’s --api-key, so the public HTTP port rejects anyone who does not send it. No vendor issues it and AI Reserve never sees it.

The shell tabs need curl and jq.

Check stock first

A quote is free and creates nothing. Run one before creating storage, because the volume in step 2 is placed next to this hardware and fails the same way when none is in stock:

curl -sS -X POST "$API/v1/gpu/instances/quote" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"accelerator": "H200_141GB", "count": 8, "region": "united_states", "image": "vllm/vllm-openai:v0.29.0", "diskGb": 100}' \
  | jq '{availability, price, error}'

A response with availability and price means you can continue. An error with code: no_capacity means this shape is not in stock in that region right now: try another region here and in steps 2 and 3, or wait, and read no_capacity reasons.

2. Create the weights volume

Available now. The accelerator hint places the volume where an H200 can mount it. Storage bills until you delete it, even after compute is terminated.

VOLUME_ACCEPTED=$(curl -sS -X POST "$API/v1/gpu/volumes" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: glm-5-3-weights-v1" \
  -d '{
    "name": "glm-5-3-weights",
    "region": "united_states",
    "sizeGb": 1200,
    "acceleratorHint": "H200_141GB",
    "storageBillingAcknowledged": true
  }')
export VOLUME_ID=$(jq -r .resourceId <<<"$VOLUME_ACCEPTED")
export OPERATION_ID=$(jq -r .operationId <<<"$VOLUME_ACCEPTED")
jq -e .resourceId <<<"$VOLUME_ACCEPTED" >/dev/null \
  && echo "accepted: volume $VOLUME_ID (operation $OPERATION_ID)" \
  || { echo "The volume was not accepted:"; jq . <<<"$VOLUME_ACCEPTED"; }

accepted = api(
    "/v1/gpu/volumes",
    method="POST",
    idempotency_key="glm-5-3-weights-v1",
    body={
        "name": "glm-5-3-weights",
        "region": "united_states",
        "sizeGb": 1200,
        "acceleratorHint": "H200_141GB",
        "storageBillingAcknowledged": True,
    },
)
wait(accepted["operationId"])
volume_id = accepted["resourceId"]

const accepted = await api("/v1/gpu/volumes", {
  method: "POST",
  headers: { "Idempotency-Key": "glm-5-3-weights-v1" },
  body: JSON.stringify({
    name: "glm-5-3-weights",
    region: "united_states",
    sizeGb: 1200,
    acceleratorHint: "H200_141GB",
    storageBillingAcknowledged: true,
  }),
});
await wait(accepted.operationId);
const volumeId = accepted.resourceId;

In the shell, wait for the operation before quoting compute (the Python and TypeScript tabs already call wait):

while :; do
  OPERATION=$(curl -sS "$API/v1/gpu/operations/$OPERATION_ID" \
    -H "Authorization: Bearer $AIRESERVE_API_KEY")
  STATUS=$(jq -r .status <<<"$OPERATION")
  echo "operation $OPERATION_ID: $STATUS"
  case "$STATUS" in
    succeeded) break ;;
    failed|cancelled)
      jq -r '.failure // empty | "whose: \(.actor)\nwhy:   \(.message)\nnext:  \(.next)"' <<<"$OPERATION"
      break ;;
  esac
  sleep 10
done

3. Quote the serving node

Available now. The quote creates nothing, expires after 120 seconds, and binds the hardware, image, disk, and volume.

QUOTE=$(curl -sS -X POST "$API/v1/gpu/instances/quote" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"accelerator\": \"H200_141GB\",
    \"count\": 8,
    \"region\": \"united_states\",
    \"image\": \"vllm/vllm-openai:v0.29.0\",
    \"diskGb\": 100,
    \"volumeId\": \"$VOLUME_ID\",
    \"volumeMountPath\": \"/models\"
  }")
export QUOTE_ID=$(jq -r .quoteId <<<"$QUOTE")
jq '{quoteId, expiresAt, price, availability, storage, error}' <<<"$QUOTE"

quote = api("/v1/gpu/instances/quote", method="POST", body={
    "accelerator": "H200_141GB",
    "count": 8,
    "region": "united_states",
    "image": "vllm/vllm-openai:v0.29.0",
    "diskGb": 100,
    "volumeId": volume_id,
    "volumeMountPath": "/models",
})
quote_id = quote["quoteId"]
print(quote["price"], quote["availability"], quote["expiresAt"])

const quote = await api("/v1/gpu/instances/quote", {
  method: "POST",
  body: JSON.stringify({
    accelerator: "H200_141GB",
    count: 8,
    region: "united_states",
    image: "vllm/vllm-openai:v0.29.0",
    diskGb: 100,
    volumeId,
    volumeMountPath: "/models",
  }),
});
const quoteId = quote.quoteId;
console.log(quote.price, quote.availability, quote.expiresAt);

4. Pin and download the model

Manual today. AI Reserve does not yet have a model-preload resource. vLLM downloads the pinned revision into HF_HOME during first launch. To preload it separately, run this in a qualified loader image with the same volume mounted. The verification command requires the current hf CLI from huggingface_hub 1.18 or later, not the deprecated huggingface-cli:

hf download zai-org/GLM-5.3 \
  --revision ba32f8c19c8738e2c57c71798c601d477a7bf844 \
  --local-dir /models/GLM-5.3

hf cache verify zai-org/GLM-5.3 \
  --revision ba32f8c19c8738e2c57c71798c601d477a7bf844 \
  --local-dir /models/GLM-5.3 \
  --fail-on-missing-files

Do not substitute zai-org/GLM-5.3-BF16: its runtime budget is about 1.786 TB and it requires a different topology. Treat third-party NVFP4 builds as separate artifacts that need their own quality run.

5. Launch vLLM

Available now, runtime qualification pending. The image entrypoint receives the model and engine arguments below. HF_HOME keeps the resumable download on the persistent volume. The raw vLLM port is public, so --api-key is mandatory.

LAUNCH=$(jq -n \
  --arg quoteId "$QUOTE_ID" \
  --arg apiKey "$VLLM_API_KEY" \
  '{
    quoteId: $quoteId,
    name: "glm-5-3-vllm",
    env: {
      HF_HOME: "/models/hf",
      VLLM_USE_RUST_FRONTEND: "1"
    },
    ports: [{port: 8000, protocol: "http"}],
    publicPortsAcknowledged: true,
    dockerStartCmd: [
      "--model", "zai-org/GLM-5.3",
      "--revision", "ba32f8c19c8738e2c57c71798c601d477a7bf844",
      "--served-model-name", "glm-5.3",
      "--host", "0.0.0.0", "--port", "8000",
      "--api-key", $apiKey,
      "--kv-cache-dtype", "fp8_e4m3",
      "--tensor-parallel-size", "8",
      "--speculative-config", "{\"method\":\"mtp\",\"num_speculative_tokens\":5}",
      "--max-num-seqs", "32",
      "--tool-call-parser", "glm47",
      "--reasoning-parser", "glm47",
      "--enable-auto-tool-choice"
    ]
  }' | curl -sS -X POST "$API/v1/gpu/instances" \
    -H "Authorization: Bearer $AIRESERVE_API_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: glm-5-3-serve-v1" \
    --data-binary @-)
export INSTANCE_ID=$(jq -r .resourceId <<<"$LAUNCH")
export OPERATION_ID=$(jq -r .operationId <<<"$LAUNCH")
jq -e .resourceId <<<"$LAUNCH" >/dev/null \
  && echo "accepted: instance $INSTANCE_ID (operation $OPERATION_ID)" \
  || { echo "The launch was not accepted:"; jq . <<<"$LAUNCH"; }

accepted = api(
    "/v1/gpu/instances",
    method="POST",
    idempotency_key="glm-5-3-serve-v1",
    body={
        "quoteId": quote_id,
        "name": "glm-5-3-vllm",
        "env": {"HF_HOME": "/models/hf", "VLLM_USE_RUST_FRONTEND": "1"},
        "ports": [{"port": 8000, "protocol": "http"}],
        "publicPortsAcknowledged": True,
        "dockerStartCmd": [
            "--model", "zai-org/GLM-5.3",
            "--revision", "ba32f8c19c8738e2c57c71798c601d477a7bf844",
            "--served-model-name", "glm-5.3",
            "--host", "0.0.0.0", "--port", "8000",
            "--api-key", VLLM_KEY,
            "--kv-cache-dtype", "fp8_e4m3",
            "--tensor-parallel-size", "8",
            "--speculative-config", '{"method":"mtp","num_speculative_tokens":5}',
            "--max-num-seqs", "32",
            "--tool-call-parser", "glm47",
            "--reasoning-parser", "glm47",
            "--enable-auto-tool-choice",
        ],
    },
)
wait(accepted["operationId"])
instance_id = accepted["resourceId"]

const launched = await api("/v1/gpu/instances", {
  method: "POST",
  headers: { "Idempotency-Key": "glm-5-3-serve-v1" },
  body: JSON.stringify({
    quoteId,
    name: "glm-5-3-vllm",
    env: { HF_HOME: "/models/hf", VLLM_USE_RUST_FRONTEND: "1" },
    ports: [{ port: 8000, protocol: "http" }],
    publicPortsAcknowledged: true,
    dockerStartCmd: [
      "--model", "zai-org/GLM-5.3",
      "--revision", "ba32f8c19c8738e2c57c71798c601d477a7bf844",
      "--served-model-name", "glm-5.3",
      "--host", "0.0.0.0", "--port", "8000",
      "--api-key", VLLM_KEY,
      "--kv-cache-dtype", "fp8_e4m3",
      "--tensor-parallel-size", "8",
      "--speculative-config", '{"method":"mtp","num_speculative_tokens":5}',
      "--max-num-seqs", "32",
      "--tool-call-parser", "glm47",
      "--reasoning-parser", "glm47",
      "--enable-auto-tool-choice",
    ],
  }),
});
await wait(launched.operationId);
const instanceId = launched.resourceId;

In the shell, wait for the launch operation (the Python and TypeScript tabs already call wait):

while :; do
  OPERATION=$(curl -sS "$API/v1/gpu/operations/$OPERATION_ID" \
    -H "Authorization: Bearer $AIRESERVE_API_KEY")
  STATUS=$(jq -r .status <<<"$OPERATION")
  echo "operation $OPERATION_ID: $STATUS"
  case "$STATUS" in
    succeeded) break ;;
    failed|cancelled)
      jq -r '.failure // empty | "whose: \(.actor)\nwhy:   \(.message)\nnext:  \(.next)"' <<<"$OPERATION"
      break ;;
  esac
  sleep 10
done

A running instance is not a ready model. vLLM still has to load about 756 GB of weights (downloading them first unless step 4 staged them) and shard them across eight GPUs, which takes tens of minutes. Step 6 waits for /v1/models before the first request. To watch progress:

curl -N "$API/v1/gpu/instances/$INSTANCE_ID/logs?tail=200" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY"

The upstream recipe body still mentions 0.28.0, but its current compatibility header requires 0.29.0+. This playbook deliberately requires 0.29.0 and remains qualification-pending until one immutable image digest passes.

6. Discover and verify the endpoint

Retrieve the HTTP URL only after the operation succeeds. The shell block sets ENDPOINT from the port 8000 connection, then waits for the first successful /v1/models, which is the readiness signal.

CONNECTIONS=$(curl -sS "$API/v1/gpu/instances/$INSTANCE_ID/connections" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY")
export ENDPOINT=$(jq -er '.http[] | select(.port == 8000) | .url' <<<"$CONNECTIONS")
[ -n "$ENDPOINT" ] && echo "ENDPOINT=$ENDPOINT" || jq . <<<"$CONNECTIONS"

until curl -fsS "$ENDPOINT/v1/models" -H "Authorization: Bearer $VLLM_API_KEY" >/dev/null 2>&1; do
  echo "model still loading..."; sleep 30
done
curl -sS "$ENDPOINT/v1/models" -H "Authorization: Bearer $VLLM_API_KEY"
curl -N "$ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer $VLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3",
    "messages": [{"role": "user", "content": "Reply with exactly: ready"}],
    "stream": true,
    "max_tokens": 256,
    "chat_template_kwargs": {"reasoning_effort": "low", "clear_thinking": true}
  }'

connections = api(f"/v1/gpu/instances/{instance_id}/connections")
endpoint = next(item["url"] for item in connections["http"] if item["port"] == 8000)

request = Request(
    f"{endpoint}/v1/chat/completions",
    data=json.dumps({
        "model": "glm-5.3",
        "messages": [{"role": "user", "content": "Reply with exactly: ready"}],
        "max_tokens": 256,
        "chat_template_kwargs": {"reasoning_effort": "low", "clear_thinking": True},
    }).encode(),
    headers={"Authorization": f"Bearer {VLLM_KEY}", "Content-Type": "application/json"},
)
with urlopen(request, timeout=300) as response:
    print(json.load(response))

const connections = await api(`/v1/gpu/instances/${instanceId}/connections`);
const connection = connections.http.find((item: { port: number }) => item.port === 8000);
if (!connection) throw new Error("port 8000 is not available in instance connections");
const endpoint = connection.url;

const response = await fetch(`${endpoint}/v1/chat/completions`, {
  method: "POST",
  headers: { Authorization: `Bearer ${VLLM_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "glm-5.3",
    messages: [{ role: "user", content: "Reply with exactly: ready" }],
    max_tokens: 256,
    chat_template_kwargs: { reasoning_effort: "low", clear_thinking: true },
  }),
});
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());

clear_thinking=true is required for ordinary multi-turn chat; otherwise the template preserves prior reasoning. Run the same test at low, high, and max, then add a tool schema and verify that tool_calls is valid JSON and that arguments satisfy the schema.

7. Qualify throughput and context

Run the benchmark client from another machine so it does not contend with the server. Start with prefix caching disabled for a clean baseline, then enable it and measure the production request mix.

OPENAI_API_KEY="$VLLM_API_KEY" vllm bench serve \
  --backend openai-chat \
  --base-url "$ENDPOINT" \
  --endpoint /v1/chat/completions \
  --model glm-5.3 \
  --dataset-name random \
  --random-input-len 8000 \
  --random-output-len 1024 \
  --request-rate 1 \
  --num-prompts 64

Sweep 4K, 32K, 128K, then the largest context you intend to admit. Repeat at increasing request rates and concurrency until p95 TTFT or TPOT misses your objective. Synthetic prompts understate MTP acceptance; repeat with real agent traces and report mean accepted draft length. Before promotion, verify:

no OOM, worker restart, or malformed tool call at the admitted limits;
p50/p95/p99 TTFT, TPOT, end-to-end latency, queue time, and error rate;
input/output throughput, KV and prefix-cache hit rate, HBM and host memory;
MTP enabled and disabled on identical prompts;
rolling restart and rollback to the prior image digest.

AI Reserve does not currently provide managed replicas, load balancing, draining, durable workload logs, customer GPU metrics, or autoscaling for raw instances. Those are platform gaps, not optional production checks. Put an authenticated load balancer and monitoring layer in front before accepting customer traffic.

8. Clean up

Terminate compute first. Delete the volume only after confirming the checkpoint cache is no longer needed.

curl -sS -X DELETE "$API/v1/gpu/instances/$INSTANCE_ID" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Idempotency-Key: glm-5-3-terminate-v1"

curl -sS -X DELETE "$API/v1/gpu/volumes/$VOLUME_ID" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Idempotency-Key: glm-5-3-delete-weights-v1"

api(
    f"/v1/gpu/instances/{instance_id}",
    method="DELETE",
    idempotency_key="glm-5-3-terminate-v1",
)
api(
    f"/v1/gpu/volumes/{volume_id}",
    method="DELETE",
    idempotency_key="glm-5-3-delete-weights-v1",
)

await api(`/v1/gpu/instances/${instanceId}`, {
  method: "DELETE",
  headers: { "Idempotency-Key": "glm-5-3-terminate-v1" },
});
await api(`/v1/gpu/volumes/${volumeId}`, {
  method: "DELETE",
  headers: { "Idempotency-Key": "glm-5-3-delete-weights-v1" },
});

Getting started

Connect your tool

GPU Infrastructure

Containers & orchestration

Capabilities

Migrate

Performance

Data & privacy

Reliability

Recipe contract

1. Set up the API client

Check stock first

2. Create the weights volume

3. Quote the serving node

4. Pin and download the model

5. Launch vLLM

6. Discover and verify the endpoint

7. Qualify throughput and context

8. Clean up

Primary sources

​Recipe contract

​1. Set up the API client

​Check stock first

​2. Create the weights volume

​3. Quote the serving node

​4. Pin and download the model

​5. Launch vLLM

​6. Discover and verify the endpoint

​7. Qualify throughput and context

​8. Clean up

​Primary sources

Recipe contract

1. Set up the API client

Check stock first

2. Create the weights volume

3. Quote the serving node

4. Pin and download the model

5. Launch vLLM

6. Discover and verify the endpoint

7. Qualify throughput and context

8. Clean up

Primary sources