Run Qwen3-VL locally and use it from anywhere

Qwen3-VL is Qwen’s dedicated vision-language family — it reads text, images, and video, supports long context on the local-friendly sizes, and ships in both dense and mixture-of-experts (MoE) sizes with separate Instruct and reasoning-enhanced Thinking editions. It supersedes the older Qwen2.5-VL; if you’re picking a vision model to run today, start here — it is the line with verified Ollama tags, while the newer Qwen3.8-27B also takes image and video input but runs only through llama.cpp, LM Studio, or a server runtime for now. Run whichever size fits your hardware locally, then pair the Tokios connector to call it from any machine, teammate, or hosted agent through https://api.tokios.com — no inbound ports, no VPN.

Run it locally with Ollama

Qwen3-VL is in the Ollama library under the slug qwen3-vl. Pull and run the default tag (the 8B dense Instruct model):

ollama run qwen3-vl

For other sizes, add a tag — for example ollama run qwen3-vl:30b-a3b for the MoE model, or ollama run qwen3-vl:8b-thinking for the reasoning edition. See the Ollama library page for the full tag list — this page covers connecting it, not every runtime detail.

Ollama’s default context window is small (2048 tokens) — far below Qwen3-VL’s native 256K. Raise num_ctx in your Modelfile or client request to use more of its context.

Variants & tags

Every value below is from the Hugging Face model cards and the Ollama library. Tags and download sizes change as new quantizations ship — confirm against ollama.com/library/qwen3-vl/tags before you standardize on one.

Property	`8B` (dense)	`30B-A3B` (MoE)
Ollama tag	`qwen3-vl:8b` (alias `:latest`)	`qwen3-vl:30b-a3b`
Download size (default quant)	~6.1 GB	~20 GB
Download size (`bf16`)	~18 GB	~62 GB
Parameters	8B dense	30B total / 3B active (128 experts, 8 routed)
Context window	262,144 native	262,144 native
Modality	Text + image + video	Text + image + video
Editions	Instruct and Thinking	Instruct and Thinking
License	Apache-2.0	Apache-2.0
Hugging Face	`Qwen/Qwen3-VL-8B-Instruct`	`Qwen/Qwen3-VL-30B-A3B-Instruct`

Sources: ollama.com/library/qwen3-vl/tags, Qwen/Qwen3-VL-8B-Instruct config, and Qwen/Qwen3-VL-30B-A3B-Instruct config. The Ollama library also lists 2B, 4B, 32B, and 235B-A22B sizes; this table focuses on the verified local-friendly sizes.

The 30B-A3B model is mixture-of-experts — only 3B parameters are active per token, so it decodes faster than a dense model of the same total size. But all parameters still have to fit in memory. Don’t size your GPU off the active-parameter count alone. See how inference works for why.

Quantization & formats

Ollama’s tags cover a default quant plus -q8, and separate -bf16 tags for every size, each available in both -instruct and -thinking editions. Which format you can run depends on your hardware — see Quantization formats and the hardware they need for the full matrix, and what quantization is for how to read quant names like Q4_K_M.

Hardware fit

Comfort at roughly Q4 for a modest context length. Treat every entry as an estimate — the ground truth is the actual GGUF size plus the KV cache for your context. See what size model fits your GPU? for the sizing math.

Target	`8B` (~6 GB)	`30B-A3B` MoE (~20 GB)
RTX 3090 / 4090 (24 GB)	Comfortable, long context	Comfortable
RTX 5090 (32 GB, Blackwell)	Comfortable, long context	Comfortable
RTX 6000 Ada / A6000 (48 GB)	Comfortable, long context	Comfortable, long context
RTX 5080 / 4080 (16 GB)	Comfortable	Tight — reduce context
Apple Silicon (36 GB+ unified)	Comfortable	Comfortable
AMD Strix Halo (ROCm/Vulkan, ≤96 GB)	Comfortable	Comfortable

Run it — beginner to advanced

Ollama (easiest)
LM Studio (GUI)
vLLM (advanced)
SGLang (advanced)

ollama run qwen3-vl

Leave Ollama listening on the default http://127.0.0.1:11434. Raise num_ctx to use more context, and send images in your chat request the way you would with any other Ollama vision model.

Search for Qwen3-VL in LM Studio’s model catalog, download a GGUF quant that fits your VRAM (start with Q4_K_M), then start LM Studio’s local server. It exposes an OpenAI-compatible API on http://127.0.0.1:1234/v1, including vision input.

Qwen recommends vLLM (vllm>=0.11.0) for fast Qwen3-VL serving. The official example targets a large multi-GPU box; scale the flags down for your hardware:

vllm serve Qwen/Qwen3-VL-8B-Instruct \
  --max-model-len 262144 \
  --media-io-kwargs '{"video": {"num_frames": -1}}'

Add --tensor-parallel-size <N> to shard across GPUs. Qwen’s own README does not publish a --tool-call-parser or --reasoning-parser flag for Qwen3-VL — check the vLLM multimodal serving docs and the model card before relying on structured tool calls or thinking-tag parsing in production.

python -m sglang.launch_server \
  --model-path Qwen/Qwen3-VL-8B-Instruct \
  --host 0.0.0.0 \
  --port 22002

Add --tp <N> for tensor parallelism. Flags are from the Qwen3-VL GitHub README.

Capabilities

Vision: Multimodal — accepts text, images, and video in the same request, with native support for long-document OCR, GUI/agentic screen understanding, and spatial grounding.
Long context: 262,144 tokens natively (confirmed for the 8B and 30B-A3B configs); Qwen’s model card describes it as expandable further with YaRN scaling on vLLM.
Reasoning: Ships as two separate editions per size — Instruct (direct answers) and Thinking (reasoning-enhanced, e.g. qwen3-vl:8b-thinking on Ollama). Qwen’s public deployment examples do not publish a dedicated reasoning-parser flag for Qwen3-VL on vLLM or SGLang.
Tool-calling: The chat template on Hugging Face defines a <tool_call> JSON format, and Ollama applies it automatically. Qwen’s official vLLM/SGLang deployment examples for Qwen3-VL don’t show a --tool-call-parser flag — test function-calling on your runtime before shipping it.

Connect it with Tokios

Once Qwen3-VL is running locally, pair the Tokios connector so you can reach it as https://api.tokios.com from Claude Code, Cline, or any other compatible client — instead of only from localhost:11434.

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing to get a one-time claim code (TKS-XXXX-XXXX). Install and run the connector on the machine running Ollama:

Windows

irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'

macOS

curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX

Linux

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Then approve the device on the Setup or Connectors tab.

Point the connector at Ollama's /v1 API

Ollama exposes an OpenAI-compatible surface at /v1. For a single-model setup, the console’s Setup wizard fills this in automatically as http://127.0.0.1:11434/v1. For multiple models, set it explicitly in tokios-connector.json — see Connector config.

In the Models tab, register the model and choose a public deployment name, for example qwen3-vl-tunnel. This name — not Ollama’s local model id — is what clients send in the model field.

Create an API key

In the Keys tab, click Create key. Copy the sk-tok-… key now — it’s shown only once.

Call it from anywhere

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-vl-tunnel",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "What is in this image?"},
          {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}}
        ]
      }
    ]
  }'

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3-vl-tunnel

The model field is always your registered deployment name (qwen3-vl-tunnel above), never Ollama’s local model id. See Use Claude Code and Use Cline for full client setup.

Caveats

Qwen3-VL ships Instruct and Thinking as separate downloads rather than one model with a toggle — pick the tag you need (qwen3-vl:8b vs. qwen3-vl:8b-thinking) up front.
Qwen’s official README and model cards don’t publish explicit --tool-call-parser or --reasoning-parser flags for Qwen3-VL on vLLM/SGLang, unlike some other Qwen3 releases. Test tool-calling and thinking-tag behavior on your runtime before relying on it in production.
The 30B-A3B MoE model needs all of its total parameters in memory even though only 3B are active per token — size your hardware off the full download, not the active count.
The 32B and 235B-A22B variants need more hardware than most local Tokios users will have. Treat them as advanced pulls and confirm their exact files, context settings, and runtime support before planning around them.

Quickstart

Pair a connector and register your first model end to end.

What size model fits your GPU?

Match Qwen3-VL to your available VRAM before you pull it.

Quantization & hardware

Which of Qwen3-VL’s formats — GGUF, bf16 — runs on your hardware.

Which local model for which task?

See how Qwen3-VL compares to other local vision models by task.

Get Started

Models

Concepts

Reference

Run Qwen3-VL locally and use it from anywhere

Run it locally with Ollama

Variants & tags

Quantization & formats

Hardware fit

Run it — beginner to advanced

Capabilities

Connect it with Tokios

Call it from anywhere

Caveats

Quickstart

What size model fits your GPU?

Quantization & hardware

Which local model for which task?

​Run it locally with Ollama

​Variants & tags

​Quantization & formats

​Hardware fit

​Run it — beginner to advanced

​Capabilities

​Connect it with Tokios

​Call it from anywhere

​Caveats

Quickstart

What size model fits your GPU?

Quantization & hardware

Which local model for which task?

Run it locally with Ollama

Variants & tags

Quantization & formats

Hardware fit

Run it — beginner to advanced

Capabilities

Connect it with Tokios

Call it from anywhere

Caveats