메인 콘텐츠로 건너뛰기
Last verified: 2026-10-14
Kimi K3 is the model you came for, but it does not fit on hardware you own. Its 2.8T parameters need about 1.4 TB at the native 4-bit floor — even 8 DGX Sparks (1 TB pooled) fall 376 GB short, and Moonshot targets K3 at a supernode with 64 or more accelerators. See Run Kimi K3 locally for the availability picture and the DGX Spark memory math for the full calculation. So the useful question is not “how do I run K3” — it’s “what is the closest model that actually runs on my machine, today?” This page answers that by hardware tier.
Already running one of the models below? Register it with Tokios to reach it as https://api.tokios.com from any OpenAI- or Anthropic-compatible client — see Connect your pick with Tokios at the bottom.

​
The short answer by hardware

Every model above is one you can download and serve today. The rows are ordered by memory, not by capability — pick your hardware, then read that tier below. One caveat on the big-RAM row: Kimi K2.6 and K2.7-Code hit interactive speed on a multi-GPU server, but on a RAM-offload-only workstation they run at single-digit tokens per second — fine for batch work, slow for coding agents.

Found your tier? Connect it next

Pair the Tokios connector and register a deployment name in about 5 minutes, then call your model from anywhere.

​
The closest thing to K3 itself: Kimi K2.6 and K2.7-Code

If you want K3 specifically because it is Kimi, the honest substitute is the rest of the Kimi family. Kimi K2.6 (general flagship) and Kimi K2.7-Code (coding-specialized, the default in Moonshot’s Kimi Code CLI) are 1T-parameter open-weight MoEs — one full hardware tier below K3. They ship at native INT4 (~595 GB on disk), so they need a multi-GPU server or a big-RAM workstation, not a desktop, but they are downloadable now. This is the strongest “run instead” answer on the page because it is not a substitute at all — it is the same family, with the same agentic tool-calling and multimodal input, at a size that real hardware can hold. See Kimi models for the vLLM/SGLang serving path and quant options.
Kimi’s kimi-* tags on Ollama are :cloud-only — they run on Ollama’s infrastructure, not yours, so there is nothing local for a connector to reach. Self-host the weights with vLLM or SGLang if you want Kimi on hardware you control.

​
Strix Halo (128 GB unified)

An AMD Ryzen AI Max+ 395 exposes roughly 96 GB to the GPU after BIOS and driver allocation. MoE models in the 35B–122B range are the sweet spot — they run 3–14× faster than a dense model of the same parameter count.
  • Coding / agents: Qwen3-Coder-30B at Q4_K_M (~19 GB) is the community’s go-to for Cline, Aider, and opencode, at roughly 10–15 tok/s.
  • General / reasoning: GLM-4.5-Air (106B MoE, ~60 GB) is the quality pick; Qwen3.5-122B-A10B (~70 GB) is the maximum-capability option. Both land around 10–15 tok/s at Q4_K_M.
Full fit tables, quant formats, and the Vulkan-vs-ROCm story are in Strix Halo model compatibility and Serve a model on Strix Halo.

​
1 DGX Spark (128 GB unified)

A single Spark has ~115–120 GB usable after system overhead. It runs dense models up to ~70B at Q4 and MoE models up to ~120B total at NVFP4 — NVFP4 is the recommended default on the SM121 GPU.
  • Coding / agents: Qwen3.5-35B-A3B at NVFP4 (~18 GB) sustains 47–50 tok/s; the newer Qwen3.6-35B-A3B reaches up to ~120 tok/s with the Atlas engine. Qwen3-Coder-30B is the coder-tuned alternative.
  • General / reasoning: GLM-4.5-Air (106B MoE, ~53 GB) is the widely reported best-quality pick; Qwen3.5-122B-A10B (~61 GB) is the largest MoE that fits, at ~30 tok/s.
See DGX Spark model compatibility for the full model list and vLLM on DGX Spark for the launch commands.

​
2 DGX Sparks (256 GB combined)

A second Spark does not unlock a “bigger Kimi.” It does unlock DeepSeek-V4-Flash-0731: 2 nodes provide 256 GB of aggregate unified memory for its roughly 162–167 GB weight floor. The route is community-tested rather than vendor-verified for 0731, so start at 32K context and revalidate model load, tool calls, and sustained generation. A 405B dense model at FP8 needs roughly 405 GB for weights alone and does not fit in 2 Sparks. If your goal is a large coding model rather than more concurrency, skip to 4 Sparks — Qwen3-Coder-480B needs a ~250 GB+ memory floor and only gets real KV-cache headroom at 512 GB. See One vs two DGX Sparks for the decision framework and worked throughput numbers.

​
4 DGX Sparks (512 GB combined)

4 nodes is NVIDIA’s documented maximum Spark cluster, positioned for models up to about 700B parameters. This is the tier that makes the K3 verdict concrete: even doubling past 4 Sparks to 8 still falls 376 GB short of K3 — so 4 Sparks is where you run the largest models that do fit.
  • Coding / agents: Qwen3-Coder-480B (large MoE coder, ~250 GB+ memory floor) runs here with the KV-cache headroom it lacked at 256 GB.
  • General / reasoning: Start with DeepSeek-V4-Flash-0731, the current official release with tool calling and 1M context. DeepSeek-R1 671B at Q4_K_M (~404 GB) also fits when you specifically need the older R1 family.
The repo does not publish measured tok/s at this cluster size, so treat any throughput figure here as an estimate until you benchmark your own nodes. See DGX Spark model compatibility and the Kimi K3 hardware deep dive for the cluster math.

​
Big-RAM or multi-GPU server (256 GB+)

This is the tier where the truest K3 alternatives live, because it is where the rest of the Kimi family runs. On an 8×80 GB datacenter box, serve Kimi K2.6 or K2.7-Code with vLLM at --tensor-parallel-size 8 for interactive speed. On a 256 GB+ RAM workstation with GPU offload, a community GGUF runs a low-bit quant slowly — single-digit tokens per second, not interactive.
  • Coding / agents: Kimi K2.7-Code — Moonshot’s coding-specialized 1T MoE.
  • General / reasoning: Kimi K2.6 for mixed agentic work; MiniMax M3 (428B / 23B active) or GLM-5.2 are strong non-Kimi options at this tier.
Deployment commands and quant options are on Kimi models; Homelab GPU server covers building the rig, and Which local model for which task? helps you choose.

​
Single consumer GPU (8–24 GB)

A single 24 GB card is a different class of tool from K3 — set expectations accordingly. It is excellent for fast, local coding help and general chat, not for frontier reasoning.
  • Coding / agents: Qwen3-Coder-30B-A3B at Q4_K_M (19 GB) is a comfortable fit on a 24 GB card.
  • General / reasoning: gpt-oss-20b at native MXFP4 (14 GB) fits comfortably and exposes low/medium/high reasoning effort.
Check any other model against your exact card with the model-fit calculator, and see RTX workstation for the full setup.

​
Connect your pick with Tokios

Once your chosen model is running on a local runtime — Ollama, llama.cpp, vLLM, or SGLang exposing an OpenAI-compatible /v1 — the Tokios connector dials out and gives you one authenticated endpoint at https://api.tokios.com, with no inbound ports and no VPN. Each model page above carries the full connect walkthrough; the short version:
1

Pair the connector

Sign in at tokios.com/console, open Setup, click Start pairing, and run the installer on the machine serving your model with the one-time TKS-XXXX-XXXX claim code.
Windows
macOS
Linux
Then approve the device on the Setup or Connectors tab.
2

Register a deployment

Point the connector at your runtime’s /v1 base URL and register the upstream model id under a public deployment name — see register a model.
3

Create an API key

In the Keys tab, click Create key and copy the sk-tok-… key now — it’s shown only once.

​
Call it from anywhere

​
FAQ

It depends on your hardware. If you want Kimi specifically, run Kimi K2.6 (general) or Kimi K2.7-Code (coding) on a multi-GPU server — same family, one tier down. On a single DGX Spark, the best all-round pick is GLM-4.5-Air for quality or Qwen3.5-35B-A3B for speed. On Strix Halo, Qwen3-Coder-30B for coding and GLM-4.5-Air for general use.
Qwen3.5-35B-A3B at NVFP4 — 47–50 tok/s on 1 Spark (the newer Qwen3.6-35B-A3B reaches up to ~120 tok/s with the Atlas engine) — or the coder-tuned Qwen3-Coder-30B. For a large coding MoE you need a 4-Spark cluster (Qwen3-Coder-480B). See DGX Spark model compatibility.
Not K3 itself — its floor is ~1.4 TB. But 128 GB (1 DGX Spark or a Strix Halo box) comfortably runs 100B-class MoE models like GLM-4.5-Air that deliver strong agentic and reasoning quality. That is the realistic frontier-adjacent tier for a single 128 GB machine.
Yes — Kimi K2.6 and Kimi K2.7-Code are open-weight 1T MoEs at native INT4 (~595 GB), so they run on a multi-GPU server (interactive) or a big-RAM workstation with offload (slowly — single-digit tokens per second). Only K3 is out of reach on self-owned hardware. See Kimi models.
The weights are out (released July 27, 2026 — see Run Kimi K3 locally), but K3’s ~1.4 TB floor rules out every consumer and prosumer setup, so the models on this page remain your realistic local options.

Run Kimi K3 locally

The availability picture and why no rig you own holds K3 today.

Kimi models

Self-host K2.6 and K2.7-Code — the closest downloadable Kimi models.

DGX Spark model compatibility

The full fit tables, quant formats, and recommended models per use case.

What size model fits your GPU?

Check any model against your exact hardware before you commit.