Run Kimi K3 locally — hardware requirements and self-hosting reality

​

What is available now?

​

Kimi K3 at a glance

​

Can you run Kimi K3 locally?

​

Kimi K3 hardware requirements

​

Why the architecture matters for self-hosting

• Kimi Delta Attention (KDA) — a hybrid linear-attention design that makes million-token prompts cheaper to prefill. It needs matching kernels and cache semantics: Moonshot says conventional prefix caching does not map cleanly to KDA and that its vLLM prefill-cache contribution ships with the weights. A runtime that merely loads the weights is not automatically K3-compatible.
• Attention Residuals (AttnRes) — layers selectively retrieve useful representations from earlier depth instead of adding every layer’s output with fixed weight. Moonshot’s AttnRes paper reports more stable information flow in deeper models.
• Stable LatentMoE — 16 of 896 experts route per token. Sparse activation cuts per-token compute, but it does not shrink the stored checkpoint: every expert must stay resident across the serving cluster, which makes fast interconnects a first-order requirement.
• Native MXFP4 — quantization-aware training ran from the supervised fine-tuning stage onward, targeting MXFP4 weights and MXFP8 activations. This is the intended serving format, not a community quant — so the 4-bit figure above is a floor, not a compromise.

​

K3 API behavior to preserve

• K3 always reasons. Use the top-level reasoning_effort: "max"; do not send K2.x’s thinking parameter.
• Streaming responses separate reasoning into reasoning_content and the final answer into content.
• For multi-turn conversations and tool loops, replay the complete assistant message, including preserved reasoning and tool-call fields. Returning only content can destabilize later turns.
• max_completion_tokens defaults to 131,072 and can be set as high as 1,048,576.
• The launch API fixes sampling values. Moonshot recommends that you omit temperature, top_p, n, and penalty parameters rather than override them.
• Public image URLs are not accepted. Send base64 data or an ms://<file-id> reference.

​

Connect K3 with Tokios after the weights arrive

​

Call it from anywhere

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kimi-k3-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Audit this repository and propose a phased refactor."}]
  }'

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model kimi-k3-tunnel

​

Launch caveats

• The weights, model card, license, technical report, checksum manifest, and self-hosting commands are still pending.
• K3 benchmark results are Moonshot-reported. Independent reproduction must wait for the weights and evaluation setup.
• Moonshot warns that losing preserved reasoning history, or switching another model’s existing session to K3, can make output unstable.
• K3 is trained for long, difficult tasks and can act too proactively when instructions are ambiguous. Put explicit boundaries in your system prompt or AGENTS.md.
• A 1M-token limit is a maximum, not a target. Long contexts increase prefill time, cache storage, and inference cost.

​

FAQ