Confirmed now versus still pending
This separation matters. K3 may be described as an open model, but a future weight-release commitment is not the same thing as weights you can download today.
What changed from Kimi K2
K3 increases scale from K2’s 1T parameters to 2.8T, expands the context from 256K to 1M tokens, and changes the architecture in four important ways.Kimi Delta Attention handles long sequences
Kimi Delta Attention (KDA) is a linear-attention mechanism in the Kimi Linear family. Hybrid linear-attention architectures avoid applying full quadratic attention at every layer. That matters most during long prefill, when the server reads a large prompt before it generates the first token. KDA is not just a speed flag. It needs matching kernels and cache semantics. Moonshot says conventional prefix caching does not map cleanly to KDA and that its vLLM prefill-cache contribution will ship with the weights. A runtime that can load generic transformer weights is therefore not automatically K3-compatible.Attention Residuals route information across depth
Standard PreNorm residual connections repeatedly add layer outputs with fixed unit weights. Attention Residuals let a layer attend over preceding representations and select what to reuse. The Attention Residuals paper describes a block variant that reduces memory and pipeline-communication overhead while keeping most of the gain. This is depth-wise routing, separate from token attention. KDA decides how token positions interact. AttnRes decides which earlier layer representations remain useful.Stable LatentMoE raises sparsity
K3 uses Stable LatentMoE and routes 16 of 896 experts. Sparse activation cuts per-token computation relative to a dense 2.8T model. It does not cut the stored checkpoint to the active-expert size: all experts still need to be resident across the cluster. At this scale, expert placement and load balance affect throughput. Moonshot also names Quantile Balancing and a fully balanced expert-parallel training method, but the implementation details are pending in the technical report.QAT starts before deployment
Moonshot says it applied quantization-aware training from the supervised fine-tuning stage onward, using MXFP4 weights and MXFP8 activations. This is not a community quant applied after release. The model was trained to tolerate the precision used for serving. MXFP4 stores values at 4-bit precision plus shared block-scale information. The 4-bit calculation below is therefore a lower bound, not an exact checkpoint prediction.The DGX Spark memory calculation
NVIDIA lists 128 GB of LPDDR5x unified memory per DGX Spark. The GPU and CPU share that pool. It is not 128 GB for weights plus separate system RAM. At exactly four bits per parameter:- MXFP4 block scales and tensor metadata
- runtime allocations and communication buffers
- activations and temporary workspaces
- the KV or recurrent state required by the selected context
- vision encoder state for multimodal requests
- memory used by DGX OS and the serving processes
Spark count table
Why aggregate memory is not enough
Even a hypothetical 16-Spark cluster would face three practical problems.The supported topology stops at four
NVIDIA’s current multi-node guidance supports up to four DGX Spark nodes. It positions that 512 GB cluster for models up to about 700B parameters. K3 is four times that parameter count. NVIDIA publishes two-node, three-node ring, and four-node switched examples, not a 14- or 16-node K3 recipe.LLM inference synchronizes constantly
Each Spark has 273 GB/s local memory bandwidth and a 200 GbE ConnectX-7 network interface. A 200-gigabit link carries at most 25 GB/s before protocol overhead, far below local memory bandwidth. Distributed LLM inference executes layer by layer and repeatedly exchanges partial results. NVIDIA explicitly warns that this fine-grained communication makes scaling less efficient as nodes are added. More boxes solve capacity but can make each generated token wait on a larger communication fabric.K3 targets a supernode
Moonshot recommends a 64-plus-accelerator supernode so expert parallelism can operate inside a large, high-bandwidth communication domain. That recommendation is about throughput and topology, not just whether the file barely fits. A Spark remains useful for developing clients, prompts, tool schemas, and smaller local models. It is not the target K3 inference platform.How to read Moonshot’s benchmark table
Moonshot reports strong launch results across coding, agentic work, reasoning, and vision. A few representative K3 (max) scores are:
These are vendor-reported results, not independent validation. Moonshot’s own footnotes also show that models were not always evaluated with the same agent harness: K3 may use Kimi Code, another model may use Claude Code, and GPT models may use Codex. Harness quality, tool policy, context compaction, and fallback behavior can materially affect agent benchmarks.
The practical reading is: K3 appears competitive enough to test seriously, but the launch table does not prove that it will reproduce those results in your runtime. Wait for the weights, prompts, harness details, and third-party evaluations.
API behavior is not drop-in K2
K3 is live askimi-k3 on Moonshot’s OpenAI-compatible Chat Completions API, but it is not a drop-in K2 configuration: thinking is always on, reasoning_effort replaces the thinking parameter, streaming splits reasoning_content from content, and multi-turn replay must preserve the complete assistant message. The Kimi K3 model page keeps the full list of protocol details to preserve when you test the API or validate a self-hosted runtime later.
Known launch limitations
Moonshot names three limitations directly:- History sensitivity. K3 expects preserved thinking history. Broken replay or mid-session model switching can destabilize output.
- Excessive proactiveness. The model may make unexpected decisions when a long task contains ambiguity. Use explicit permissions, stop conditions, and edit boundaries.
- A remaining experience gap. Moonshot says K3 still trails the strongest proprietary models in overall user experience even where individual benchmarks are competitive.
AGENTS.md, require approval for destructive or external actions, and test session compaction before you trust a long autonomous run.
What to do until the weights ship
Wait for artifacts linked by Moonshot itself — not similarly named community repositories — then recalculate from the real checkpoint manifest before buying anything. The Kimi K3 model page carries the step-by-step readiness plan: verifying the official artifacts, serving on a supported cluster, and connecting the deployment through Tokios.Primary sources
- Kimi K3: Open Frontier Intelligence — launch, architecture, benchmarks, availability, and limitations
- Kimi K3 API guide — current model id, request behavior, vision, tools, and context limits
- Attention Residuals paper — depth-routing design and scaling experiments
- DGX Spark hardware overview — 128 GB unified memory, 273 GB/s bandwidth, and ConnectX-7
- NVIDIA’s four-Spark scaling guidance — supported node count, 700B positioning, and communication limits
Kimi K3
Track K3 availability, hardware requirements, and the Tokios path once official weights ship.
Kimi models
Compare K3 with the downloadable K2.6 and K2.7-Code you can self-host today.
DGX Spark
Run models that fit the Spark and give them a private Tokios endpoint.
Model serving and scaling
Tensor, pipeline, and expert parallelism — the vocabulary behind the supernode requirement.
FAQ
How much VRAM does Kimi K3 need?
How much VRAM does Kimi K3 need?
About 1.4 TB at the native MXFP4 precision (4 bits per parameter, 2.8T parameters), before block scales, runtime buffers, activations, and KV cache. In practice, a serving cluster needs significantly more — Moonshot recommends 64+ accelerators in a supernode configuration.
How many DGX Sparks do you need for Kimi K3?
How many DGX Sparks do you need for Kimi K3?
At minimum, 11 Sparks would match the raw 1.4 TB weight floor (11 x 128 GB = 1,408 GB), but that leaves no room for OS, CUDA overhead, KV cache, or inference state. A realistic estimate is 14-16 Sparks. However, NVIDIA only documents Spark clusters up to 4 nodes (512 GB), and there is no supported K3-on-Spark topology.
Why can't MoE models like K3 fit in less memory if only 16 experts activate?
Why can't MoE models like K3 fit in less memory if only 16 experts activate?
MoE sparse activation cuts per-token compute, not stored checkpoint size. All 896 expert weights must remain resident across the serving cluster. The ~50B active parameters determine inference speed (FLOPs per token); the 2.8T total parameters determine memory requirements.
What hardware can actually run Kimi K3?
What hardware can actually run Kimi K3?
Moonshot recommends a supernode with 64 or more accelerators — the kind of high-bandwidth interconnect domain where expert parallelism operates efficiently. Consumer GPUs (24 GB), prosumer unified memory (128-192 GB), and even 4-node Spark clusters (512 GB) are all far below K3’s requirements.
When will Kimi K3 weights be available?
When will Kimi K3 weights be available?
Moonshot has promised downloadable weights by July 27, 2026. The license, model card, checkpoint manifest, and self-hosting instructions are not yet published. vLLM has confirmed it received a KDA prefix caching contribution from Moonshot, expected to ship day 0 with the official release.