moonshotai/Kimi-K3 with SGLang on one 8×B300 node.
That is the smallest topology whose unified low-latency and balanced paths are
marked verified by the SGLang maintainers.
Recipe contract
Kimi K3 always reasons. It also requires the complete prior assistant message—
including
reasoning_content and tool_calls—on every multi-turn or tool
follow-up. Dropping those fields changes the conversation contract.
0. Set up your shell
Every command on this page is copy-paste. Paste this block once per terminal and replace only the API key; every later step reads these variables.Two keys, two jobs.
AIRESERVE_API_KEY authenticates you to AI Reserve
and creates the volume, quote, and instance. SGLANG_API_KEY is a password you
invent for the SGLang server you are about to start: step 4 passes it as
SGLang’s --api-key, so the public HTTP port rejects anyone who does not
send it. No vendor issues it and AI Reserve never sees it.curl and jq; the Python tabs need
requests (python -m pip install requests).
Check stock first
A quote is free and creates nothing. Run one before creating storage, because the volume in step 1 is placed next to this hardware and fails the same way when none is in stock:availability and price means you can continue. An error
with code: no_capacity means this shape is not in stock in that region right
now: try another region here and in steps 1 and 2, or wait, and read
no_capacity reasons.
1. Create the weights volume
Available now. The accelerator hint places the volume in a data center where 8×B300 has stock today, so the instance in step 4 can mount it. If step 0 reported no stock, this call fails the same way. Storage continues billing after the instance is terminated.2. Quote 8×B300
Available now. A quote does not reserve scarce B300 capacity. Launch it before its 120-second expiry.3. Pin and download the model
Manual today. The serving process can download intoHF_HOME, but a
separate preload makes startup and rollback predictable. The verification
command requires the current hf CLI from huggingface_hub 1.18 or later,
not the deprecated huggingface-cli:
4. Launch SGLang
Available now, runtime qualification pending. This is the conservative 65K B300 profile published by SGLang. It keeps the vision path enabled, uses TP8, and avoids speculative decoding until target-only quality and latency are known. The API key protects the public HTTP port.wait_for and
waitFor are already defined there):
/v1/models before the first request. To watch progress:
--mem-fraction-static or context until that evidence is captured.
5. Verify text, vision, and thinking history
Retrieve the port-30000 URL from the instance’sconnections response and set
it as ENDPOINT.
/v1/models is the
readiness signal; until then the port returns connection errors or 5xx:
Vision smoke test
IMAGE_URL is any image your server can fetch; a public sample is set here so
the test runs as written. Exercise the vision encoder separately from text:
assistant_message exactly as returned. Verify all
three reasoning levels, streaming, tool schemas, malformed-tool recovery, and a
real image request. Do not publish self-hosted video support: the current open
checkpoint metadata and serving processor establish text and image input only.
6. Qualify the operating point
SGLang’s B300 figures used 8,192 input and 1,024 output tokens, 64 warm-up requests, and cache flushes. Reproduce that baseline, then replace synthetic traffic with your traces:SGLANG_SIMULATE_ACC_LEN in a production or accuracy run: upstream used it for
synthetic throughput projection, not measured acceptance. Compare DSpark and
target-only serving with identical prompts and retain tool/vision quality
tests.