Run DeepSeek locally and use it from anywhere

DeepSeek-R1 is DeepSeek’s reasoning model: it thinks through a problem in a visible <think>…</think> block before answering, and it’s the model DeepSeek is best known for. The way almost everyone runs it locally is a distill — a smaller Qwen or Llama model fine-tuned on R1’s reasoning traces, so the reasoning behavior runs on a laptop-class GPU instead of a multi-GPU server. This page leads with those distills, then covers the full 671B R1 and its siblings (V3.1, V3) for when you have the hardware to run DeepSeek’s frontier weights yourself. Pair Tokios with whatever you land on to reach it from any machine, teammate, or hosted agent through https://api.tokios.com — no inbound ports, no VPN.

Run it locally with Ollama

The deepseek-r1 tag on Ollama ships the distills, not the full model. Pull and run the default (currently the 7b Qwen distill):

ollama run deepseek-r1

For a specific size, name the tag directly — for example ollama run deepseek-r1:14b. See the Ollama library page for the full tag list.

Ollama’s default context window is 2048 tokens, far below what these models support — and R1-style reasoning burns through context fast on its way to an answer. Raise num_ctx (via /set parameter num_ctx <value> in the CLI, or in your Modelfile/API request) toward the context size listed in the table below.

Variants & tags

Every value below is from the Ollama library and each model’s Hugging Face config. Tags, sizes, and defaults change as new quantizations ship — confirm against ollama.com/library/deepseek-r1 before you standardize on one.

Distill	Base model	Ollama tag (`Q4_K_M`)	Download size	Context
1.5B	Qwen2.5-Math-1.5B	`deepseek-r1:1.5b`	~1.1 GB	128K
7B	Qwen2.5-Math-7B	`deepseek-r1:7b` (default)	~4.7 GB	128K
8B	Llama-3.1-8B	`deepseek-r1:8b`	~4.9 GB	128K
8B (0528, Qwen3 base)	Qwen3-8B	`deepseek-r1:8b-0528-qwen3-q4_K_M`	~5.2 GB	128K
14B	Qwen2.5-14B	`deepseek-r1:14b`	~9.0 GB	128K
32B	Qwen2.5-32B	`deepseek-r1:32b`	~20 GB	128K
70B	Llama-3.3-70B-Instruct	`deepseek-r1:70b`	~43 GB	128K
671B (full R1)	—	`deepseek-r1:671b`	~404 GB	160K
671B (R1-0528, full)	—	`deepseek-r1:671b-0528-q4_K_M`	~404 GB	160K

Sources: ollama.com/library/deepseek-r1/tags, deepseek-ai/DeepSeek-R1 README.

The 671b tags are the full DeepSeek-R1 (and R1-0528), not distills — Ollama just publishes them under the same deepseek-r1 name. Everything from 1.5b through 70b is a distill: a Qwen or Llama model fine-tuned on R1’s reasoning traces, not R1 itself shrunk down. See how inference works for what that distinction means for output quality.

Quantization & formats

Ollama’s default tags are GGUF Q4_K_M. Every distill size and the full 671b also publish -q8_0 and -fp16 tags on the library page for less lossy (and much larger) downloads. See Quantization formats and the hardware they need for what runs on your GPU, and what quantization is for how to read the Q4_K_M-style names.

Hardware fit

Comfort at roughly Q4 for a modest context length. Treat every entry as an estimate — the ground truth is the actual GGUF size plus the KV cache for your context. See what size model fits your GPU? for the sizing math.

Target	1.5B–8B (~1–5 GB)	14B (~9 GB)	32B (~20 GB)	70B (~43 GB)	671B (~404 GB)
Laptop / 8 GB GPU	Comfortable	Tight	Won’t fit	Won’t fit	Won’t fit
RTX 3090 / 4090 (24 GB)	Comfortable	Comfortable	Comfortable	Won’t fit	Won’t fit
RTX 5090 (32 GB)	Comfortable	Comfortable	Comfortable, longer context	Won’t fit	Won’t fit
RTX 6000 Ada / A6000 (48 GB)	Comfortable	Comfortable	Comfortable, long context	Tight, short context	Won’t fit
Apple Silicon / Strix Halo (96–128 GB unified)	Comfortable	Comfortable	Comfortable	Comfortable	Won’t fit
Multi-GPU server (8× 80 GB class)	Comfortable	Comfortable	Comfortable	Comfortable	Comfortable at `Q4`

The full 671b needs roughly 404 GB just for weights at Q4_K_M — that’s multiple datacenter-class GPUs or an equivalent unified-memory cluster, not a single workstation. Everything 70b and under is realistic on the hardware hobbyists actually own.

Run it — beginner to advanced

Ollama (easiest)
LM Studio (GUI)
vLLM (advanced)
SGLang (advanced)

ollama run deepseek-r1:14b

Leave Ollama listening on the default http://127.0.0.1:11434. Raise num_ctx to use more of the model’s context window.

Search for DeepSeek R1 in LM Studio’s model catalog, pick a distill size that fits your VRAM (start with 14b or 32b at Q4_K_M), and start LM Studio’s local server. It exposes an OpenAI-compatible API on http://127.0.0.1:1234/v1.

For a distill:

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --tensor-parallel-size 2 \
  --max-model-len 32768

For the full model, shard across enough GPUs to hold ~404 GB (Q4) or more at higher precision. Flags are from the model card’s vLLM section.

python3 -m sglang.launch_server \
  --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --trust-remote-code \
  --tp 2

Swap the model id for deepseek-ai/DeepSeek-R1 and raise --tp to run the full model. Flags are from the model card’s SGLang section.

Capabilities

Reasoning: R1 and every distill emit a <think>…</think> block before the final answer. DeepSeek’s own model card recommends a temperature of 0.5–0.7 (0.6 suggested), no system prompt, and — for math — explicitly asking the model to “reason step by step” and box its final answer.
Native tool-calling: Not reliable on the distills. Ollama’s chat template for the R1 distills doesn’t support tool/function-calling the way Llama 3.1/3.3 or Qwen3 do — see ollama/ollama#8517 (closed, unresolved for the distill template). If your workload needs an agent that calls tools, pick a model built for it — see Choose a model by task.
Context: 128K for every distill (1.5b through 70b); 160K for the full 671b R1 and R1-0528.
Vision: None of the models on this page are multimodal — DeepSeek-R1 and its distills are text-only.

Beyond R1: V3.2 and the V4 frontier

R1 is a reasoning-tuned checkpoint built on DeepSeek’s V3 architecture. Its successors — DeepSeek-V3.2 and the current-frontier DeepSeek-V4 (Flash and Pro) — are also open-weight and MIT-licensed, but they’re multi-GPU or cloud territory, not a single-workstation pull:

Model	Ollama tag	Parameters	Context	Notes
DeepSeek-V3	`deepseek-v3:671b`	671B (37B active, MoE)	160K	The non-reasoning base line R1 was built from
DeepSeek-V3.1	`deepseek-v3.1:671b` (incl. `terminus` tags)	671B (37B active, MoE)	160K	Successor to V3, adds a `terminus` refinement
DeepSeek-V3.2	Not on Ollama	671B-class MoE	160K	Hugging Face only (`deepseek-ai/DeepSeek-V3.2`), MIT, published December 2025
DeepSeek-V4-Flash	Not on Ollama	284B total / 13B active (MoE)	1,048,576 (1M)	Hugging Face only (`deepseek-ai/DeepSeek-V4-Flash`), MIT, published June 2026
DeepSeek-V4-Pro	Not on Ollama	1.6T total / 49B active (MoE)	1,048,576 (1M)	Hugging Face only (`deepseek-ai/DeepSeek-V4-Pro`), MIT, published June 2026

Sources: ollama.com/library/deepseek-v3.1/tags, ollama.com/library/deepseek-v3/tags, Hugging Face API/config.json for deepseek-ai/DeepSeek-V3.2, DeepSeek-V4-Flash, and DeepSeek-V4-Pro.

V3.2 and the V4 line aren’t a one-command Ollama pull yet — as of this writing they’re Hugging Face weights meant for vLLM/SGLang on multi-GPU hardware, or a hosted API. If you want DeepSeek’s current frontier model and don’t have that hardware, a cloud endpoint is the practical option; for local-first use, the R1 distills or the 671b R1/V3/V3.1 tags on Ollama are what you can actually pull today.

Connect it with Tokios

Once a DeepSeek model is running locally, pair the Tokios connector so you can reach it as https://api.tokios.com from Claude Code, Cline, or any other compatible client — instead of only from localhost:11434.

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing to get a one-time claim code (TKS-XXXX-XXXX). Install and run the connector on the machine running Ollama:

Windows

irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'

macOS

curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX

Linux

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Then approve the device on the Setup or Connectors tab.

Point the connector at Ollama's /v1 API

Ollama exposes an OpenAI-compatible surface at /v1. For a single-model setup, the console’s Setup wizard fills this in automatically as http://127.0.0.1:11434/v1. For multiple models, set it explicitly in tokios-connector.json — see Connector config.

In the Models tab, register the model and choose a public deployment name, for example deepseek-r1-tunnel. This name — not Ollama’s local model id — is what clients send in the model field.

Create an API key

In the Keys tab, click Create key. Copy the sk-tok-… key now — it’s shown only once.

Call it from anywhere

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Explain the halting problem."}]
  }'

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model deepseek-r1-tunnel

The model field is always your registered deployment name (deepseek-r1-tunnel above), never Ollama’s local model id. Tokios turns whatever’s running on localhost into a stable, authenticated endpoint — it doesn’t change what the model can do. See Use Claude Code and Use Cline for full client setup.

Caveats

Tool-calling is unreliable on the R1 distills. If you’re wiring up an agent, CLI, or IDE plugin that needs the model to call tools, use a model with confirmed native tool support instead — see Choose a model by task.
Raise the context window. Ollama’s 2048-token default cuts off R1-style reasoning output quickly; increase num_ctx for real use, up to 128K (distills) or 160K (full 671b).
A distill is not the full model. The 1.5b–70b tags are Qwen/Llama checkpoints fine-tuned on R1’s reasoning traces — they reproduce the reasoning behavior on light hardware, not R1’s full capability. Benchmark quality drops as size drops; see the DeepSeek-R1 model card for the distill evaluation table.
Licensing differs by distill. The full R1/R1-0528 weights and the Qwen-based distills (1.5b, 7b, 14b, 32b) are MIT. The Llama-based distills (8b, 70b) carry Meta’s Llama 3.1/3.3 license instead, inherited from their base model — check that license before commercial use of those two specific tags.
671B needs real multi-GPU hardware — roughly 404 GB at Q4_K_M alone — not a typical single-workstation setup. V3.2 and V4-Flash/Pro aren’t on Ollama yet and require the same scale via vLLM/SGLang or a hosted API.
Tokios passes tool-calling requests straight through to whatever the upstream model supports — it doesn’t add tool-calling capability the underlying model lacks.

Quickstart

Pair a connector and register your first model end to end.

Choose a model by task

Pick a model with reliable tool-calling if you’re building an agent.

Match a model to your GPU

Compare VRAM classes across distill sizes before you pick a tag.

Quantization & hardware

Which formats — GGUF, FP16, and beyond — run on your hardware.

Get Started

Models

Concepts

Reference

Run DeepSeek locally and use it from anywhere — R1, V3.2, and V4

Run it locally with Ollama

Variants & tags

Quantization & formats

Hardware fit

Run it — beginner to advanced

Capabilities

Beyond R1: V3.2 and the V4 frontier

Connect it with Tokios

Call it from anywhere

Caveats

Quickstart

Choose a model by task

Match a model to your GPU

Quantization & hardware

​Run it locally with Ollama

​Variants & tags

​Quantization & formats

​Hardware fit

​Run it — beginner to advanced

​Capabilities

​Beyond R1: V3.2 and the V4 frontier

​Connect it with Tokios

​Call it from anywhere

​Caveats

Quickstart

Choose a model by task

Match a model to your GPU

Quantization & hardware

Run it locally with Ollama

Variants & tags

Quantization & formats

Hardware fit

Run it — beginner to advanced

Capabilities

Beyond R1: V3.2 and the V4 frontier

Connect it with Tokios

Call it from anywhere

Caveats