Run Phi locally and use it from anywhere

Phi-4 is Microsoft’s small-model family: dense models trained on curated, reasoning-heavy synthetic data so they punch above their parameter count on math and logic. The lineup spans a 14B desktop model, a 3.8B mini model light enough for a laptop or edge box, and a 5.6B multimodal model that adds image and audio input. Run whichever fits your hardware locally, then pair the Tokios connector to call it from any machine, teammate, or hosted agent through https://api.tokios.com — no inbound ports, no VPN.

Run it locally with Ollama

Phi-4 (14B) is in the Ollama library. Pull and run it:

ollama run phi4

For the smaller mini model, run ollama run phi4-mini. See the Ollama library page for the full tag list — this page covers connecting it, not every runtime detail.

Ollama’s default context window is small (2048 tokens) — well below what Phi-4 supports natively. Raise num_ctx in your Modelfile or client request to use more context, especially on the mini model’s 128K window.

Variants & tags

Every value below is from the model cards on Hugging Face and the Ollama library. Tags and download sizes change as new quantizations ship — confirm against ollama.com/library/phi4 and ollama.com/library/phi4-mini before you standardize on one.

Property	`phi4` (14B)	`phi4-mini` (3.8B)	Phi-4-multimodal (5.6B)
Ollama tag	`phi4:14b` (alias `:latest`)	`phi4-mini:3.8b` (alias `:latest`)	Not on Ollama — Hugging Face / Transformers only
Download size (`Q4_K_M`)	~9.1 GB	~2.5 GB	—
Parameters	14B dense	3.8B dense	5.6B (Phi-4-mini backbone + vision/speech encoders)
Context window	16,384 tokens	131,072 tokens	131,072 tokens
Modality	Text	Text	Text + image + audio
Tool/function calling	No (chat template has no tool role)	Yes — `<\|tool\|>` system-prompt format	Yes — `<\|tool\|>` system-prompt format
License	MIT	MIT	MIT
Hugging Face	`microsoft/phi-4`	`microsoft/Phi-4-mini-instruct`	`microsoft/Phi-4-multimodal-instruct`

Sources: ollama.com/library/phi4, ollama.com/library/phi4-mini, microsoft/phi-4 model card, microsoft/Phi-4-mini-instruct model card, microsoft/Phi-4-multimodal-instruct model card.

There are also phi4-reasoning (14B, <think>-style chain-of-thought, 32K context) and phi4-mini-reasoning (3.8B, 128K context) tags in the Ollama library, fine-tuned from the base models for explicit reasoning traces. They’re separate pulls (ollama run phi4-reasoning), not a flag on the base models — see ollama.com/library/phi4-reasoning if you want the reasoning-tuned variant instead of the instruct one covered on this page.

Quantization & formats

Ollama’s default tags are GGUF Q4_K_M. The library also publishes -q8_0 and -fp16 for both phi4 and phi4-mini. Which format you can run depends on your hardware — see Quantization formats and the hardware they need for the full matrix, and what quantization is for how to read the Q4_K_M-style names.

Hardware fit

Comfort at roughly Q4 for a modest context length. Treat every entry as an estimate — the ground truth is the actual GGUF size plus the KV cache for your context. See what size model fits your GPU? for the sizing math.

Target	`phi4-mini` (~2.5 GB)	`phi4` 14B (~9.1 GB)
Laptop / integrated GPU, CPU-only	Comfortable	Slow but workable
RTX 3060 / 4060 (8–12 GB)	Comfortable, long context	Comfortable
RTX 3090 / 4090 (24 GB)	Comfortable, full context	Comfortable, long context
RTX 5090 (32 GB, Blackwell)	Comfortable	Comfortable, full 16K+
Apple Silicon (16 GB+ unified)	Comfortable	Comfortable
AMD Strix Halo (ROCm/Vulkan)	Comfortable	Comfortable

Run it — beginner to advanced

Ollama (easiest)
LM Studio (GUI)
Transformers (advanced)
vLLM (advanced)

ollama run phi4

Leave Ollama listening on the default http://127.0.0.1:11434. Raise num_ctx to use more context, and swap in phi4-mini for a lighter footprint.

Search for Phi-4 in LM Studio’s model catalog, download a GGUF quant that fits your VRAM (start with Q4_K_M), then start LM Studio’s local server. It exposes an OpenAI-compatible API on http://127.0.0.1:1234/v1.

Phi-4-mini and Phi-4-multimodal are the variants worth self-hosting outside Ollama — multimodal needs trust_remote_code for its custom vision/audio modules:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-4-mini-instruct",
    device_map="cuda",
    torch_dtype="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-4-mini-instruct")

For Phi-4-multimodal, load with AutoModelForCausalLM.from_pretrained("microsoft/Phi-4-multimodal-instruct", trust_remote_code=True) and use the model card’s processor for image/audio inputs — it isn’t a plain chat pipeline. Flags are from each model card’s usage section.

Phi-4-multimodal’s model card documents serving it with vLLM, including its speech/vision LoRA adapters:

vllm serve microsoft/Phi-4-multimodal-instruct \
  --dtype auto \
  --trust-remote-code \
  --max-model-len 131072 \
  --enable-lora \
  --max-lora-rank 320 \
  --limit-mm-per-prompt audio=3,image=3 \
  --max-loras 2

Add --tensor-parallel-size <N> to shard across GPUs. Flags are from the model card’s vLLM section.

Capabilities

Reasoning: The instruct models covered here answer directly — no <think> block. Microsoft ships separate phi4-reasoning and phi4-mini-reasoning checkpoints (see the tip above) that emit explicit <think>…</think> chain-of-thought for math-heavy tasks; pull those instead if you want that behavior.
Tool-calling: Phi-4 (14B) has no tool role in its chat template — don’t rely on structured function calls from it. Phi-4-mini and Phi-4-multimodal support it: wrap your tool definitions in <|tool|>...<|/tool|> inside the system message, per their model cards.
Vision and audio: Only Phi-4-multimodal. It accepts text, image, and audio in the same prompt (<|image_1|>, <|audio_1|> placeholders) and is not available as an Ollama tag — run it via Transformers or vLLM.
Long context: Phi-4 (14B) is limited to 16,384 tokens. Phi-4-mini and Phi-4-multimodal both support 131,072 tokens.

Connect it with Tokios

Once Phi-4 is running locally, pair the Tokios connector so you can reach it as https://api.tokios.com from Claude Code, Cline, or any other compatible client — instead of only from localhost:11434.

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing to get a one-time claim code (TKS-XXXX-XXXX). Install and run the connector on the machine running Ollama:

Windows

irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'

macOS

curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX

Linux

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Then approve the device on the Setup or Connectors tab.

Point the connector at Ollama's /v1 API

Ollama exposes an OpenAI-compatible surface at /v1. For a single-model setup, the console’s Setup wizard fills this in automatically as http://127.0.0.1:11434/v1. For multiple models, set it explicitly in tokios-connector.json — see Connector config.

In the Models tab, register the model and choose a public deployment name, for example phi-tunnel. This name — not Ollama’s local model id — is what clients send in the model field.

Create an API key

In the Keys tab, click Create key. Copy the sk-tok-… key now — it’s shown only once.

Call it from anywhere

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "phi-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Walk through solving this step by step: what is the derivative of x^3 + 2x?"}]
  }'

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model phi-tunnel

The model field is always your registered deployment name (phi-tunnel above), never Ollama’s local model id. See Use Claude Code and Use Cline for full client setup.

Caveats

Phi-4 (14B) tops out at a 16,384-token context window — much shorter than most current open models. If you need long context on a Phi model, use phi4-mini (131,072 tokens) instead.
Phi-4 (14B) has no tool-calling support in its chat template. Reach for phi4-mini if your workflow needs function calls.
Phi-4-multimodal isn’t on Ollama — you’ll need Transformers or vLLM (with trust_remote_code) to serve its vision/audio capabilities locally.
Reasoning-tuned variants (phi4-reasoning, phi4-mini-reasoning) are separate model pulls, not a mode switch on the instruct models covered here.

Quickstart

Pair a connector and register your first model end to end.

What size model fits your GPU?

Match Phi-4 to your available VRAM before you pull it.

Quantization & hardware

Which of Phi-4’s GGUF formats runs on your hardware.

Which local model for which task?

See how Phi-4 compares to other local models by task.

Get Started

Models

Concepts

Reference

Run Phi locally and use it from anywhere

Run it locally with Ollama

Variants & tags

Quantization & formats

Hardware fit

Run it — beginner to advanced

Capabilities

Connect it with Tokios

Call it from anywhere

Caveats

Quickstart

What size model fits your GPU?

Quantization & hardware

Which local model for which task?

​Run it locally with Ollama

​Variants & tags

​Quantization & formats

​Hardware fit

​Run it — beginner to advanced

​Capabilities

​Connect it with Tokios

​Call it from anywhere

​Caveats

Quickstart

What size model fits your GPU?

Quantization & hardware

Which local model for which task?

Run it locally with Ollama

Variants & tags

Quantization & formats

Hardware fit

Run it — beginner to advanced

Capabilities

Connect it with Tokios

Call it from anywhere

Caveats