Serve DeepSeek V4 Pro 0813

This playbook serves the released deepseek-ai/DeepSeek-V4-Pro-0813 checkpoint on one 8×H200 node. It does not use the older preview weights and does not treat the mutable hosted API ID deepseek-v4-pro as an artifact pin.

Vendor-verified · AI Reserve qualification pending. Pin a qualified vLLM image digest before production. The upstream recipe documents 800K context on H200 to preserve KV headroom; that is a configured ceiling, not a promise of useful concurrency at 800K.

Recipe contract

Item	Pinned value
Artifact	`deepseek-ai/DeepSeek-V4-Pro-0813`
Revision	`72e1d3230f6c080a530b0a1d46f8eb4602340597`
Weights	Native mixed MXFP4 experts + FP8 linears; about 892.8 GB downloaded and about 960 GB runtime budget
Architecture	1.65T total / 49B active, text-only MoE with attached DSpark drafter
Context	1,048,576 architectural maximum; 800,000 in this H200 recipe
Primary engine	vLLM 0.25.0+
Reference image	`vllm/vllm-openai:v0.25.0`; replace with the qualified digest
Hardware	8×`H200_141GB`
Persistent storage	1,200 GB minimum; use 2,000 GB when retaining conversion or rollback artifacts
License	MIT, ungated
Last reviewed	2026-10-14

DeepSeek’s repository has no Jinja chat template. Production serving depends on the engine’s deepseek_v4 tokenizer, reasoning, and tool parsers. Raw Transformers loading is not an equivalent serving contract.

0. Set up your shell

Every command on this page is copy-paste. Paste this block once per terminal and replace only the API key; every later step reads these variables.

export AIRESERVE_API_KEY="aireserve_api_..."   # the only value you type: a key with Compute: Manage access
export API="https://api.aireserve.com"
export VLLM_API_KEY="$(openssl rand -hex 32)"  # generated for you; it protects your model server (see below)
echo "VLLM_API_KEY=$VLLM_API_KEY"               # save it: every request to the endpoint needs it

Two keys, two jobs. AIRESERVE_API_KEY authenticates you to AI Reserve and creates the volume, quote, and instance. VLLM_API_KEY is a password you invent for the vLLM server you are about to start: step 4 passes it as vLLM’s --api-key, so the public HTTP port rejects anyone who does not send it. No vendor issues it and AI Reserve never sees it.

The Python and TypeScript tabs read the same three variables from the environment. The shell tabs need curl and jq; the Python tabs need requests (python -m pip install requests).

Check stock first

A quote is free and creates nothing. Run one before creating storage, because the volume in step 1 is placed next to this hardware and fails the same way when none is in stock:

curl -sS -X POST "$API/v1/gpu/instances/quote" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"accelerator": "H200_141GB", "count": 8, "region": "united_states", "image": "vllm/vllm-openai:v0.25.0", "diskGb": 100}' \
  | jq '{availability, price, error}'

A response with availability and price means you can continue. An error with code: no_capacity means this shape is not in stock in that region right now: try another region here and in steps 1 and 2, or wait, and read no_capacity reasons.

1. Allocate persistent storage

Available now. The accelerator hint places the volume in a data center where 8×H200 has stock today, so the instance in step 4 can mount it. If step 0 reported no stock, this call fails the same way.

ACCEPTED=$(curl -sS -X POST "$API/v1/gpu/volumes" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: deepseek-v4-pro-0813-weights-v1" \
  -d '{
    "name": "deepseek-v4-pro-0813-weights",
    "region": "united_states",
    "sizeGb": 1200,
    "acceleratorHint": "H200_141GB",
    "storageBillingAcknowledged": true
  }')
export VOLUME_ID=$(jq -r .resourceId <<<"$ACCEPTED")
export OPERATION_ID=$(jq -r .operationId <<<"$ACCEPTED")
jq -e .resourceId <<<"$ACCEPTED" >/dev/null \
  && echo "accepted: volume $VOLUME_ID (operation $OPERATION_ID)" \
  || { echo "The volume was not accepted:"; jq . <<<"$ACCEPTED"; }

import os, requests, time

API = "https://api.aireserve.com"
AUTH = {"Authorization": f"Bearer {os.environ['AIRESERVE_API_KEY']}"}

response = requests.post(
    f"{API}/v1/gpu/volumes",
    headers={**AUTH, "Idempotency-Key": "deepseek-v4-pro-0813-weights-v1"},
    json={
        "name": "deepseek-v4-pro-0813-weights",
        "region": "united_states",
        "sizeGb": 1200,
        "acceleratorHint": "H200_141GB",
        "storageBillingAcknowledged": True,
    },
    timeout=60,
)
response.raise_for_status()
accepted = response.json()
volume_id = accepted["resourceId"]
operation_id = accepted["operationId"]

const API = "https://api.aireserve.com";
const auth = { Authorization: `Bearer ${process.env.AIRESERVE_API_KEY}` };

const response = await fetch(`${API}/v1/gpu/volumes`, {
  method: "POST",
  headers: {
    ...auth,
    "Content-Type": "application/json",
    "Idempotency-Key": "deepseek-v4-pro-0813-weights-v1",
  },
  body: JSON.stringify({
    name: "deepseek-v4-pro-0813-weights",
    region: "united_states",
    sizeGb: 1200,
    acceleratorHint: "H200_141GB",
    storageBillingAcknowledged: true,
  }),
});
if (!response.ok) throw new Error(await response.text());
const accepted = await response.json();
const volumeId = accepted.resourceId;
const operationId = accepted.operationId;

Wait for the operation to finish before you quote compute. Persistent storage bills independently of compute and is not deleted when the instance terminates.

while :; do
  OPERATION=$(curl -sS "$API/v1/gpu/operations/$OPERATION_ID" \
    -H "Authorization: Bearer $AIRESERVE_API_KEY")
  STATUS=$(jq -r .status <<<"$OPERATION")
  echo "operation $OPERATION_ID: $STATUS"
  case "$STATUS" in
    succeeded) break ;;
    failed|cancelled)
      jq -r '.failure // empty | "whose: \(.actor)\nwhy:   \(.message)\nnext:  \(.next)"' <<<"$OPERATION"
      break ;;
  esac
  sleep 10
done

def wait_for(operation_id):
    while True:
        operation = requests.get(f"{API}/v1/gpu/operations/{operation_id}", headers=AUTH, timeout=60).json()
        print("operation", operation_id, operation.get("status"))
        if operation.get("status") == "succeeded":
            return operation
        if operation.get("status") in {"failed", "cancelled"}:
            failure = operation.get("failure") or {}
            raise RuntimeError(f"{failure.get('actor', 'unknown')}: {failure.get('message', operation.get('errorMessage'))} {failure.get('next', '')}")
        time.sleep(10)

wait_for(operation_id)

async function waitFor(operationId: string) {
  while (true) {
    const response = await fetch(`${API}/v1/gpu/operations/${operationId}`, { headers: auth });
    const operation = await response.json();
    console.log("operation", operationId, operation.status);
    if (operation.status === "succeeded") return operation;
    if (["failed", "cancelled"].includes(operation.status)) {
      const failure = operation.failure ?? {};
      throw new Error(`${failure.actor ?? "unknown"}: ${failure.message ?? operation.errorMessage} ${failure.next ?? ""}`);
    }
    await new Promise((resolve) => setTimeout(resolve, 10_000));
  }
}

await waitFor(operationId);

2. Quote the H200 node

Available now. Inspect the all-in hourly rate before launching. A quote expires after 120 seconds, so run step 4 right after it; if it has expired, run this step again.

QUOTE=$(curl -sS -X POST "$API/v1/gpu/instances/quote" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"accelerator\": \"H200_141GB\",
    \"count\": 8,
    \"region\": \"united_states\",
    \"image\": \"vllm/vllm-openai:v0.25.0\",
    \"diskGb\": 100,
    \"volumeId\": \"$VOLUME_ID\",
    \"volumeMountPath\": \"/models\"
  }")
export QUOTE_ID=$(jq -r .quoteId <<<"$QUOTE")
jq '{quoteId, expiresAt, availability, price, storage, error}' <<<"$QUOTE"

response = requests.post(
    f"{API}/v1/gpu/instances/quote",
    headers=AUTH,
    json={
        "accelerator": "H200_141GB",
        "count": 8,
        "region": "united_states",
        "image": "vllm/vllm-openai:v0.25.0",
        "diskGb": 100,
        "volumeId": volume_id,
        "volumeMountPath": "/models",
    },
    timeout=60,
)
response.raise_for_status()
quote = response.json()
quote_id = quote["quoteId"]
print(quote["price"], quote["availability"], quote["expiresAt"])

const quoteResponse = await fetch(`${API}/v1/gpu/instances/quote`, {
  method: "POST",
  headers: { ...auth, "Content-Type": "application/json" },
  body: JSON.stringify({
    accelerator: "H200_141GB",
    count: 8,
    region: "united_states",
    image: "vllm/vllm-openai:v0.25.0",
    diskGb: 100,
    volumeId,
    volumeMountPath: "/models",
  }),
});
if (!quoteResponse.ok) throw new Error(await quoteResponse.text());
const quote = await quoteResponse.json();
const quoteId = quote.quoteId;
console.log(quote.price, quote.availability, quote.expiresAt);

3. Download and verify the checkpoint

Manual today. AI Reserve has no managed preload job. Use the exact revision on a loader with the same persistent volume, or allow vLLM to populate HF_HOME on first launch. The verification command requires the current hf CLI from huggingface_hub 1.18 or later, not the deprecated huggingface-cli:

hf download deepseek-ai/DeepSeek-V4-Pro-0813 \
  --revision 72e1d3230f6c080a530b0a1d46f8eb4602340597 \
  --local-dir /models/DeepSeek-V4-Pro-0813

hf cache verify deepseek-ai/DeepSeek-V4-Pro-0813 \
  --revision 72e1d3230f6c080a530b0a1d46f8eb4602340597 \
  --local-dir /models/DeepSeek-V4-Pro-0813 \
  --fail-on-missing-files

Do not replace this checkpoint with DeepSeek-V4-Pro preview, DeepSeek-V4-Pro-DSpark, or an NVIDIA NVFP4 conversion. They are separate artifacts with different draft modules and qualification requirements.

4. Launch vLLM

Available now, runtime qualification pending. DSpark is attached inside the 0813 checkpoint. Probabilistic draft sampling is the upstream recommendation for sampled chat; benchmark it against target-only serving before promotion.

LAUNCH=$(jq -n \
  --arg quoteId "$QUOTE_ID" \
  --arg apiKey "$VLLM_API_KEY" \
  '{
    quoteId: $quoteId,
    name: "deepseek-v4-pro-0813-vllm",
    env: {HF_HOME: "/models/hf", VLLM_USE_RUST_FRONTEND: "1"},
    ports: [{port: 8000, protocol: "http"}],
    publicPortsAcknowledged: true,
    dockerStartCmd: [
      "--model=deepseek-ai/DeepSeek-V4-Pro-0813",
      "--revision=72e1d3230f6c080a530b0a1d46f8eb4602340597",
      "--served-model-name=deepseek-v4-pro-0813",
      "--host=0.0.0.0", "--port=8000",
      "--api-key=\($apiKey)",
      "--trust-remote-code",
      "--kv-cache-dtype=fp8",
      "--block-size=256",
      "--enable-expert-parallel",
      "--tensor-parallel-size=8",
      "--max-model-len=800000",
      "--gpu-memory-utilization=0.95",
      "--max-num-seqs=512",
      "--no-enable-flashinfer-autotune",
      "--compilation-config={\"mode\":0,\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}",
      "--tokenizer-mode=deepseek_v4",
      "--tool-call-parser=deepseek_v4",
      "--enable-auto-tool-choice",
      "--reasoning-parser=deepseek_v4",
      "--speculative-config={\"method\":\"dspark\",\"num_speculative_tokens\":7,\"draft_sample_method\":\"probabilistic\"}"
    ]
  }' | curl -sS -X POST "$API/v1/gpu/instances" \
    -H "Authorization: Bearer $AIRESERVE_API_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: deepseek-v4-pro-0813-serve-v1" \
    --data-binary @-)
export INSTANCE_ID=$(jq -r .resourceId <<<"$LAUNCH")
export OPERATION_ID=$(jq -r .operationId <<<"$LAUNCH")
jq -e .resourceId <<<"$LAUNCH" >/dev/null \
  && echo "accepted: instance $INSTANCE_ID (operation $OPERATION_ID)" \
  || { echo "The launch was not accepted:"; jq . <<<"$LAUNCH"; }

command = [
    "--model=deepseek-ai/DeepSeek-V4-Pro-0813",
    "--revision=72e1d3230f6c080a530b0a1d46f8eb4602340597",
    "--served-model-name=deepseek-v4-pro-0813",
    "--host=0.0.0.0", "--port=8000",
    f"--api-key={os.environ['VLLM_API_KEY']}",
    "--trust-remote-code",
    "--kv-cache-dtype=fp8",
    "--block-size=256",
    "--enable-expert-parallel",
    "--tensor-parallel-size=8",
    "--max-model-len=800000",
    "--gpu-memory-utilization=0.95",
    "--max-num-seqs=512",
    "--no-enable-flashinfer-autotune",
    '--compilation-config={"mode":0,"cudagraph_mode":"FULL_DECODE_ONLY"}',
    "--tokenizer-mode=deepseek_v4",
    "--tool-call-parser=deepseek_v4",
    "--enable-auto-tool-choice",
    "--reasoning-parser=deepseek_v4",
    '--speculative-config={"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}',
]
response = requests.post(
    f"{API}/v1/gpu/instances",
    headers={**AUTH, "Idempotency-Key": "deepseek-v4-pro-0813-serve-v1"},
    json={
        "quoteId": quote_id,
        "name": "deepseek-v4-pro-0813-vllm",
        "env": {"HF_HOME": "/models/hf", "VLLM_USE_RUST_FRONTEND": "1"},
        "ports": [{"port": 8000, "protocol": "http"}],
        "publicPortsAcknowledged": True,
        "dockerStartCmd": command,
    },
    timeout=60,
)
response.raise_for_status()
launched = response.json()
instance_id = launched["resourceId"]
operation_id = launched["operationId"]
wait_for(operation_id)

const command = [
  "--model=deepseek-ai/DeepSeek-V4-Pro-0813",
  "--revision=72e1d3230f6c080a530b0a1d46f8eb4602340597",
  "--served-model-name=deepseek-v4-pro-0813",
  "--host=0.0.0.0", "--port=8000",
  `--api-key=${process.env.VLLM_API_KEY!}`,
  "--trust-remote-code",
  "--kv-cache-dtype=fp8",
  "--block-size=256",
  "--enable-expert-parallel",
  "--tensor-parallel-size=8",
  "--max-model-len=800000",
  "--gpu-memory-utilization=0.95",
  "--max-num-seqs=512",
  "--no-enable-flashinfer-autotune",
  '--compilation-config={"mode":0,"cudagraph_mode":"FULL_DECODE_ONLY"}',
  "--tokenizer-mode=deepseek_v4",
  "--tool-call-parser=deepseek_v4",
  "--enable-auto-tool-choice",
  "--reasoning-parser=deepseek_v4",
  '--speculative-config={"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}',
];
const launchResponse = await fetch(`${API}/v1/gpu/instances`, {
  method: "POST",
  headers: {
    ...auth,
    "Content-Type": "application/json",
    "Idempotency-Key": "deepseek-v4-pro-0813-serve-v1",
  },
  body: JSON.stringify({
    quoteId,
    name: "deepseek-v4-pro-0813-vllm",
    env: { HF_HOME: "/models/hf", VLLM_USE_RUST_FRONTEND: "1" },
    ports: [{ port: 8000, protocol: "http" }],
    publicPortsAcknowledged: true,
    dockerStartCmd: command,
  }),
});
if (!launchResponse.ok) throw new Error(await launchResponse.text());
const launched = await launchResponse.json();
const instanceId = launched.resourceId;
const operationId = launched.operationId;
await waitFor(operationId);

Wait for the launch operation the same way as in step 1 (wait_for and waitFor are already defined there):

while :; do
  OPERATION=$(curl -sS "$API/v1/gpu/operations/$OPERATION_ID" \
    -H "Authorization: Bearer $AIRESERVE_API_KEY")
  STATUS=$(jq -r .status <<<"$OPERATION")
  echo "operation $OPERATION_ID: $STATUS"
  case "$STATUS" in
    succeeded) break ;;
    failed|cancelled)
      jq -r '.failure // empty | "whose: \(.actor)\nwhy:   \(.message)\nnext:  \(.next)"' <<<"$OPERATION"
      break ;;
  esac
  sleep 10
done

A running instance is not a ready model. vLLM still has to load about 893 GB of weights (downloading them first unless step 3 staged them) and shard them across eight GPUs, which takes tens of minutes. Step 5 waits for /v1/models before the first request. To watch progress:

curl -N "$API/v1/gpu/instances/$INSTANCE_ID/logs?tail=200" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY"

5. Verify reasoning and tools

Set ENDPOINT from the HTTP connection for port 8000. Start with a small output cap only in non-thinking mode; high and max reasoning may legitimately need very large output budgets.

CONNECTIONS=$(curl -sS "$API/v1/gpu/instances/$INSTANCE_ID/connections" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY")
export ENDPOINT=$(jq -er '.http[] | select(.port == 8000) | .url' <<<"$CONNECTIONS")
[ -n "$ENDPOINT" ] && echo "ENDPOINT=$ENDPOINT" || jq . <<<"$CONNECTIONS"

response = requests.get(
    f"{API}/v1/gpu/instances/{instance_id}/connections",
    headers=AUTH,
    timeout=60,
)
response.raise_for_status()
connection = next(item for item in response.json()["http"] if item["port"] == 8000)
endpoint = connection["url"]

const connectionResponse = await fetch(`${API}/v1/gpu/instances/${instanceId}/connections`, {
  headers: auth,
});
if (!connectionResponse.ok) throw new Error(await connectionResponse.text());
const connections = await connectionResponse.json();
const connection = connections.http.find((item: { port: number }) => item.port === 8000);
if (!connection) throw new Error("port 8000 is not available in instance connections");
const endpoint = connection.url;

Wait until the model is loaded. The first successful /v1/models is the readiness signal; until then the port returns connection errors or 5xx:

until curl -fsS "$ENDPOINT/v1/models" -H "Authorization: Bearer $VLLM_API_KEY" >/dev/null 2>&1; do
  echo "model still loading..."; sleep 30
done

curl -sS "$ENDPOINT/v1/models" \
  -H "Authorization: Bearer $VLLM_API_KEY"

curl -sS "$ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer $VLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-pro-0813",
    "messages": [{"role": "user", "content": "What is 17*19? Return only the final integer."}],
    "temperature": 1,
    "top_p": 0.95,
    "max_tokens": 4096,
    "chat_template_kwargs": {"thinking": true, "reasoning_effort": "high"}
  }'

response = requests.post(
    f"{endpoint}/v1/chat/completions",
    headers={"Authorization": f"Bearer {os.environ['VLLM_API_KEY']}"},
    json={
        "model": "deepseek-v4-pro-0813",
        "messages": [{"role": "user", "content": "What is 17*19? Return only the final integer."}],
        "temperature": 1,
        "top_p": 0.95,
        "max_tokens": 4096,
        "chat_template_kwargs": {"thinking": True, "reasoning_effort": "high"},
    },
    timeout=300,
)
response.raise_for_status()
print(response.json())

const response = await fetch(`${endpoint}/v1/chat/completions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.VLLM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "deepseek-v4-pro-0813",
    messages: [{ role: "user", content: "What is 17*19? Return only the final integer." }],
    temperature: 1,
    top_p: 0.95,
    max_tokens: 4096,
    chat_template_kwargs: { thinking: true, reasoning_effort: "high" },
  }),
});
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());

Run low, high, and max; streaming; multi-turn; and tool-call tests. DeepSeek wraps tool calls in its tagged DeepSeek markup language (DSML), which the engine parser converts to OpenAI-shaped tool_calls. With tools, preserve every prior reasoning_content value. Reject malformed tool markup at the application boundary and retry with a bounded policy rather than executing partially parsed arguments.

6. Qualify performance

Run target-only and DSpark servers from clean restarts, using identical model revision, prompts, and scheduler settings:

OPENAI_API_KEY="$VLLM_API_KEY" vllm bench serve \
  --backend openai-chat \
  --base-url "$ENDPOINT" \
  --endpoint /v1/chat/completions \
  --model deepseek-v4-pro-0813 \
  --dataset-name random \
  --random-input-len 8192 \
  --random-output-len 1024 \
  --request-rate 1 \
  --num-prompts 64

Repeat with real 4K, 32K, 128K, 384K, and 800K traces. Sweep concurrency until p95 TTFT or TPOT misses the production objective. Capture HBM, KV capacity, queue time, cache hit rate, OOMs, errors, DSpark accepted length, and tool-call validity. Never use synthetic-acceptance modes as evidence.

Raw GPU instances have no managed replica set, load balancer, autoscaler, health policy, traffic drain, durable log archive, or customer metrics API. Those are required production layers. Keep the previous model revision and image digest deployable until the replacement passes a drained canary.

7. Clean up

curl -sS -X DELETE "$API/v1/gpu/instances/$INSTANCE_ID" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Idempotency-Key: deepseek-v4-pro-0813-terminate-v1"
curl -sS -X DELETE "$API/v1/gpu/volumes/$VOLUME_ID" \
  -H "Authorization: Bearer $AIRESERVE_API_KEY" \
  -H "Idempotency-Key: deepseek-v4-pro-0813-delete-weights-v1"

for path, key in [
    (f"/v1/gpu/instances/{instance_id}", "deepseek-v4-pro-0813-terminate-v1"),
    (f"/v1/gpu/volumes/{volume_id}", "deepseek-v4-pro-0813-delete-weights-v1"),
]:
    response = requests.delete(f"{API}{path}", headers={**AUTH, "Idempotency-Key": key}, timeout=60)
    response.raise_for_status()

for (const [path, key] of [
  [`/v1/gpu/instances/${instanceId}`, "deepseek-v4-pro-0813-terminate-v1"],
  [`/v1/gpu/volumes/${volumeId}`, "deepseek-v4-pro-0813-delete-weights-v1"],
] as const) {
  const response = await fetch(`${API}${path}`, {
    method: "DELETE",
    headers: { ...auth, "Idempotency-Key": key },
  });
  if (!response.ok) throw new Error(await response.text());
}

Getting started

Connect your tool

GPU Infrastructure

Containers & orchestration

Capabilities

Migrate

Performance

Data & privacy

Reliability

Recipe contract

0. Set up your shell

Check stock first

1. Allocate persistent storage

2. Quote the H200 node

3. Download and verify the checkpoint

4. Launch vLLM

5. Verify reasoning and tools

6. Qualify performance

7. Clean up

Primary sources

​Recipe contract

​0. Set up your shell

​Check stock first

​1. Allocate persistent storage

​2. Quote the H200 node

​3. Download and verify the checkpoint

​4. Launch vLLM

​5. Verify reasoning and tools

​6. Qualify performance

​7. Clean up

​Primary sources

Recipe contract

0. Set up your shell

Check stock first

1. Allocate persistent storage

2. Quote the H200 node

3. Download and verify the checkpoint

4. Launch vLLM

5. Verify reasoning and tools

6. Qualify performance

7. Clean up

Primary sources