Skip to main content
Last verified: 2026-10-14
Kimi is Moonshot AI’s frontier-scale model family. The current lineup spans three models: Kimi K3, the new 2.8-trillion-parameter flagship (weights released July 27, 2026), and the downloadable Kimi K2.6 and Kimi K2.7-Code — both 1-trillion-parameter open-weight MoEs you can self-host today on serious hardware. This page is the honest guide to which Kimi you can actually run yourself, and how to reach it from anywhere through Tokios.

​
The Kimi lineup at a glance

Wondering whether you can run Kimi K3 locally? The weights are out (released July 27, 2026), but the floor is roughly 1.4 TB of memory. The rest of this page covers the Kimi models you can download today.
If you just want Kimi’s answers and don’t care where it runs, Moonshot’s hosted API (platform.kimi.com) is the easy path. Tokios is for the case where you want the weights on your hardware — a serious multi-GPU or high-RAM rig — and a private, authenticated endpoint in front of it.

​
Can you run Kimi locally?

Honestly: not on a normal desktop. Kimi K2.6 ships already at native INT4 — that’s the small format — and it’s still about 595 GB across 64 checkpoint shards. There is no consumer GPU, and no single datacenter GPU, that holds it. The realistic options: Community re-quantizations trade quality for size but don’t change the conclusion: bartowski/moonshotai_Kimi-K2.6-GGUF ranges from an extreme IQ1_S at ~207 GB up to Q4_0 at ~584 GB, and Unsloth’s dynamic 2-bit build (~340–350 GB) still wants a 350 GB+ combined VRAM/RAM budget. As a rule of thumb, plan for RAM + VRAM roughly equal to the quant size. For most people, Kimi is a cloud/API model — and that’s a fine answer. Run it locally only if you have the rig for it.
Already serving Kimi on your own hardware? Register it with Tokios to reach it from any client. Not sure your rig can hold it? See what to run instead of Kimi K3.

​
”Kimi on Ollama” is cloud, not local

Kimi K2.x models are in the Ollama library — kimi-k2.6, kimi-k2.7-code, kimi-k2.5 — but every one of them exposes only a :cloud tag. Ollama’s cloud models don’t download; requests are proxied to Ollama’s hosted infrastructure while you stay signed in. So ollama run kimi-k2.6:cloud is convenient, but it is not self-hosting — the weights never touch your machine, and there’s nothing local for a connector to point at. If your goal is Kimi on hardware you control, skip the :cloud tag and self-host the weights with a server runtime, below. Tokios then fronts your deployment, not someone else’s cloud.

​
Run it on your own hardware

Moonshot’s deployment docs target production-grade runtimes — vLLM, SGLang, and KTransformers — not single-GPU stacks. On a multi-GPU server, serve the native INT4 weights and expose an OpenAI-compatible endpoint:
Tensor-parallel across every GPU in the box. vLLM serves an OpenAI-compatible API on http://127.0.0.1:8000/v1 — that’s what the Tokios connector points at. Check the model card for the current recommended flags and any tool-call parser before serving.
Kimi K2.6 is distributed at native INT4 already. Don’t try to “upgrade” it to BF16 to run it — a BF16 upcast is roughly 2.1 TB and buys you nothing. The INT4 weights are the intended format.

​
Variants & specs

Every value below is from the Moonshot model cards on Hugging Face and Moonshot’s platform docs. Verify against the model card before you standardize on one — Moonshot ships new Kimi releases roughly monthly. Sources: moonshotai/Kimi-K2.6, moonshotai/Kimi-K2.7-Code, platform.kimi.com/docs/models, ollama.com/library/kimi-k2.6.
Kimi’s Modified MIT license is permissive for most uses, but adds an attribution clause for very large commercial deployments (displaying “Kimi K2.6” in your product’s UI above a high MAU/revenue threshold). Read the LICENSE file on the model card before you build a commercial product on it.

​
Capabilities

  • Agentic tool calling — K2.x is trained for tool use and multi-step agent loops. When you serve it with vLLM or SGLang, enable the tool-call parser the model card recommends so tools requests work through your endpoint.
  • Native multimodal input — text, image, and video understanding across the K2.x line.
  • Reasoning — K2.x exposes thinking through Moonshot’s thinking API parameter. Kimi K3 replaces it with a top-level reasoning_effort — don’t mix the two conventions; see the Kimi K3 page for K3’s protocol details.
  • 256K context — long enough for repository-scale coding sessions, but budget KV-cache memory on top of the weights when you size a long-context deployment.

​
Which Kimi should you run?

  • K2.6 is Moonshot’s general-purpose downloadable flagship — reach for it for writing, analysis, conversation, and mixed agentic work.
  • K2.7-Code is scoped to coding and agentic tasks — see the dedicated section below.
  • K2.5 is the prior flagship, still available if you’ve already standardized on it.
  • K3 is the frontier flagship (weights released July 27, 2026), but it moves a full hardware tier upward — see Kimi K3 for hardware requirements, or what to run instead if it won’t fit your hardware.

​
Kimi K2.7-Code for coding agents

Kimi K2.7-Code (June 2026) is the coding-specialized sibling of K2.6: same 1T-MoE architecture, size, and hardware requirements, but tuned for agentic engineering. It cuts reasoning-token usage roughly 30% versus K2.6 and is the default model in Moonshot’s Kimi Code CLI. If coding is the job, it’s the Kimi to serve. It shares K2.6’s deployment story exactly — the same INT4 weights size, the same vLLM/SGLang serving path (swap the model id for moonshotai/Kimi-K2.7-Code), and the same :cloud-only situation on Ollama. Once it’s behind your Tokios endpoint, it pairs naturally with coding agents: see Use Claude Code and Use OpenAI Codex for client setup.

​
Connect it with Tokios

Once Kimi is running on your own server — vLLM or SGLang exposing an OpenAI-compatible /v1 — pair the Tokios connector so you can reach it as https://api.tokios.com from Claude Code, Cline, or any other compatible client, instead of only from inside your LAN.
1

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing to get a one-time claim code (TKS-XXXX-XXXX). Install and run the connector on the server hosting Kimi:
Windows
macOS
Linux
Then approve the device on the Setup or Connectors tab.
2

Point the connector at your runtime's /v1 API

vLLM and SGLang expose an OpenAI-compatible surface — typically http://127.0.0.1:8000/v1. Set that as the upstream in tokios-connector.json — see Connector config.
3

Register a deployment

In the Models tab, register the model and choose a public deployment name, for example kimi-tunnel. This name — not the upstream model id — is what clients send in the model field.
4

Create an API key

In the Keys tab, click Create key. Copy the sk-tok-… key now — it’s shown only once.

​
Call it from anywhere

The model field is always your registered deployment name (kimi-tunnel above), never the upstream model id. See Use Claude Code and Use OpenAI Codex for full client setup — Kimi K2.7-Code pairs naturally with a coding agent.

​
Caveats

  • Kimi K2.x is a 1T-parameter MoE — all trillion parameters must fit in memory even though only ~32B are active per token. This is a multi-GPU-server or big-RAM-workstation model, not a single-GPU one.
  • Ollama’s kimi-* tags are :cloud-only — they run on Ollama’s infrastructure, not yours. There’s nothing local for a connector to reach; self-host with vLLM/SGLang if you want Kimi on your own hardware.
  • Kimi ships at native INT4; don’t upcast to BF16 to run it.
  • Kimi K3 weights are available (released July 27, 2026) — the Kimi K3 page covers hardware requirements and what to verify before self-hosting it.
  • Moonshot releases Kimi versions frequently and deprecates older previews — confirm the current model id on platform.kimi.com/docs/models before you standardize.

Kimi K3

The 2.8T frontier flagship: what’s available now and the hardware reality.

What to run instead of K3

Can’t fit K3? The best model for your exact hardware, with the deploy guide.

What size model fits your GPU?

The sizing math behind why a 1T MoE needs a server, not a desktop.

Homelab GPU server

Building the multi-GPU rig a model like Kimi actually needs.

Which local model for which task?

Lighter models that run on hardware you already own.