Run Nemotron locally and use it from anywhere

​

Run it locally with Ollama

• nemotron3 is the Nano Omni multimodal build (text + image input). ollama run nemotron3 pulls the default nemotron3:33b (~28 GB), served at 128K context.
• nemotron-3-nano is the text-only Nano reasoning model, served at its full 1M context. ollama run nemotron-3-nano pulls the 30B-A3B MoE (~24 GB at q4_K_M, only ~3.5B active per token); ollama run nemotron-3-nano:4b is a ~2.8 GB dense model for laptops and edge boxes.

# Multimodal Nano Omni (text + image, 128K context)
ollama run nemotron3

# Text-only Nano, full 1M context — 30B MoE, ~3.5B active per token
ollama run nemotron-3-nano

​

Variants & tags

​

Quantization & formats

​

Hardware fit

​

Run it — beginner to advanced

ollama run nemotron3

wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/resolve/main/nano_v3_reasoning_parser.py

vllm serve nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 \
  --served-model-name model \
  --max-num-seqs 8 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser-plugin nano_v3_reasoning_parser.py \
  --reasoning-parser nano_v3

python3 -m sglang.launch_server \
  --model-path nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 \
  --trust-remote-code \
  --tp 1 \
  --attention-backend flashinfer \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nano_v3

​

Capabilities

• Reasoning: Nemotron 3 models are unified reasoning/non-reasoning models — they emit a reasoning trace before the final answer by default, and it’s on across the whole line (Nano, Super, and Ultra all default to enable_thinking=True). Turn it off with enable_thinking=False in the chat template if you want a direct response at a slight accuracy cost on harder prompts. Budget control (reasoning_budget) lets you cap how many tokens the reasoning trace can use.
• Native tool-calling: Yes — the model cards document tool-calling support with the qwen3_coder tool-call parser on both vLLM and SGLang, trained in via synthetic tool-calling data during post-training. On vLLM/SGLang you must set the parser flags shown above, or tool calls won’t parse; when combining tools with reasoning, also set "chat_template_kwargs": {"enable_thinking": true, "force_nonempty_content": true} in the request.
• Vision: The plain-text Nano/Super/Ultra checkpoints are text-only. The Ollama nemotron3 tag is Nano Omni and exposes text + image input. Broader video/audio/image/text behavior belongs to the upstream Omni model card, not to the simple Ollama tag surface documented here.
• Long context: Up to 262,144 tokens natively on Nano/Super, extensible to 1M with VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. The hybrid Mamba-2/attention architecture is what makes that context length practical — a pure-attention model this size would need far more memory for the KV cache. Ollama’s nemotron3 tag currently exposes only 128K of that range.

​

Connect it with Tokios

irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'

curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

​

Call it from anywhere

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nemotron-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Walk through this incident report and flag the root cause."}]
  }'

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model nemotron-tunnel

​

Caveats

• Nemotron emits a reasoning trace by default across the family. If a downstream client shows the raw thinking or you want lower latency, disable it with enable_thinking=False or set a reasoning_budget.
• The Ollama nemotron3 tag is Nano Omni (multimodal, 128K as served). For the text-only Nano with the full 1M context, pull nemotron-3-nano instead — or serve the Hugging Face checkpoint on vLLM/SGLang for the documented tool-call and reasoning parsers.
• Tool calling requires the qwen3_coder parser flags on vLLM/SGLang, plus force_nonempty_content when combining tools with reasoning — without them, tool calls won’t parse correctly.
• Every Nemotron 3 tier (Nano, Super, Ultra) is a mixture-of-experts model — size your hardware off the total parameter count, not the active-parameter count. Super and Ultra are multi-GPU/datacenter tiers; don’t plan to run them on a single consumer card.
• Nemotron 3 uses the NVIDIA Nemotron Open Model License, not Apache or MIT — commercial use is allowed under its terms, but check the exact checkpoint license before redistribution or fine-tuning.