Run Qwen3.6-35B-A3B on DGX Spark

Qwen3.6-35B-A3B is the sweet-spot model for DGX Spark. It’s a 35B-parameter mixture-of-experts model with only 3B active parameters per token, giving you large-model capability with small-model speed. On a single DGX Spark with NVFP4 quantization, you get 47-50 tok/s sustained throughput — or up to 120 tok/s with the Atlas engine. This guide walks you through downloading the model, serving it with vLLM, and exposing it through Tokios for team access. Last verified: 2026-10-24

What you’ll get

Model: Qwen3.6-35B-A3B (35B total, 3B active per token)
Hardware: Single NVIDIA DGX Spark (128 GB unified memory)
Throughput: 47-50 tok/s with vLLM, up to 120 tok/s with Atlas
Context: 32K+ tokens
Memory usage: ~18 GB for weights, leaving 100+ GB for KV cache
Use case: High-speed coding assistant, chat, JSON extraction, agentic workflows

Prerequisites

NVIDIA DGX Spark with DGX OS installed
Internet connection for downloading model weights (~18 GB)
Tokios account (free tier works)
15 minutes

Step 1: Install vLLM

DGX Spark ships with CUDA and Python. Install vLLM v0.31.0 or later:

pip install --upgrade vllm

Verify the installation:

vllm --version

Step 2: Download the model

The NVFP4 quantization is natively supported by the Blackwell GPU’s SM121 architecture and gives 75% size reduction with minimal quality loss.

huggingface-cli download Qwen/Qwen3.6-35B-A3B-NVFP4

The download is ~18 GB. On DGX Spark’s 273 GB/s memory bandwidth, this takes about 40 seconds.

If you prefer GGUF format for llama.cpp or Ollama, use Qwen/Qwen3.6-35B-A3B-GGUF instead. However, NVFP4 is recommended for DGX Spark because it’s natively supported by the hardware and works with both vLLM and Atlas.

Step 3: Serve the model with vLLM

Start the vLLM server with the NVFP4 model:

vllm serve Qwen/Qwen3.6-35B-A3B-NVFP4 \
  --host 127.0.0.1 \
  --port 8000 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90

The server starts in ~30 seconds. You’ll see:

INFO:     Started server process [12345]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000

Test the endpoint:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3.6-35B-A3B-NVFP4",
    "messages": [{"role": "user", "content": "Write a Python hello world"}],
    "max_tokens": 100
  }'

You should get a response in ~2 seconds (time-to-first-token ~0.12s with vLLM).

Step 4: Benchmark performance

Run a quick benchmark to verify throughput:

python -m vllm.entrypoints.benchmark_serving \
  --backend vllm \
  --model Qwen/Qwen3.6-35B-A3B-NVFP4 \
  --dataset-name sonnet \
  --dataset-path sonnet.txt \
  --num-prompts 10 \
  --request-rate 1

Expected results on DGX Spark:

Metric	Value
Throughput (single user)	47-50 tok/s
Time to first token	~0.12s
Memory usage	~18 GB weights + ~2-4 GB KV cache

For maximum throughput (up to 120 tok/s), use the Atlas engine instead of vLLM. Atlas is a pure Rust/CUDA engine optimized for Blackwell GPUs. Install with pip install atlas-inference and serve with atlas serve Qwen/Qwen3.6-35B-A3B-NVFP4.

Step 5: Connect with Tokios

Your model is running on 127.0.0.1:8000. To give your team remote access with per-user API keys, install the Tokios connector.

Install the connector

Get your claim code from the Tokios dashboard Setup tab, then run:

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Claim the connector

In the dashboard, approve the device. Then configure the connector to point at vLLM:

{
  "upstream": "http://127.0.0.1:8000/v1"
}

Register the deployment

On the Models tab, map the upstream model id to a public deployment name:

Upstream model id: Qwen/Qwen3.6-35B-A3B-NVFP4
Deployment name: qwen36-35b

Create API keys

On the Keys tab, create a key for each teammate. Scope each key to the qwen36-35b deployment.

Step 6: Use it from any client

Your team can now call the model from anywhere using the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="qwen36-35b",
    messages=[{"role": "user", "content": "Write a Python hello world"}],
)
print(response.choices[0].message.content)

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen36-35b",
    "messages": [{"role": "user", "content": "Write a Python hello world"}]
  }'

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.tokios.com/v1',
  apiKey: 'sk-tok-YOUR_KEY',
});

const response = await client.chat.completions.create({
  model: 'qwen36-35b',
  messages: [{ role: 'user', content: 'Write a Python hello world' }],
});
console.log(response.choices[0].message.content);

What else fits on DGX Spark?

The 128 GB unified memory can hold multiple models. See the full DGX Spark model compatibility matrix for other options:

Qwen3.5-122B-A10B — 122B MoE, ~30 tok/s, ~61 GB weights
GLM-4.5-Air — 106B MoE, community-reported “best model for Spark”, ~53 GB weights
Qwen3-32B — 32B dense, 9-44 tok/s, ~18 GB weights (wide performance spread)
DeepSeek-V4.1-Flash — 763B MoE, requires 2 Sparks (see V4.1 Flash on 2 DGX Sparks)

Troubleshooting

vLLM fails to start with “CUDA out of memory”

Reduce --gpu-memory-utilization from 0.90 to 0.85. DGX Spark’s unified memory layout means the GPU and CPU share the same pool, and vLLM’s profiling can overestimate GPU-only usage.

Slow time-to-first-token (>1s)

Make sure you’re using NVFP4 quantization, not GGUF. NVFP4 is natively supported by the SM121 architecture and has much faster loading. Also verify you’re using vLLM v0.31.0 or later.

Gemma-4-26B-A4B-it fails to load

This model has a known issue with vLLM’s gpu_memory_utilization parameter on unified memory (SM121). Use NVFP4 quantization explicitly or switch to a different model. Track vLLM issue #46307 for updates.

Next steps

DGX Spark model compatibility — full matrix of what fits
vLLM on DGX Spark — advanced vLLM configuration
Team access to local models — give your team API access
What is a local LLM? — complete guide to running LLMs locally

Getting started with local LLMs

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

Run Qwen3.6-35B-A3B on DGX Spark

What you’ll get

Prerequisites

Step 1: Install vLLM

Step 2: Download the model

Step 3: Serve the model with vLLM

Step 4: Benchmark performance

Step 5: Connect with Tokios

Install the connector

Claim the connector

Register the deployment

Create API keys

Step 6: Use it from any client

What else fits on DGX Spark?

Troubleshooting

vLLM fails to start with “CUDA out of memory”

Slow time-to-first-token (>1s)

Gemma-4-26B-A4B-it fails to load

Next steps

​What you’ll get

​Prerequisites

​Step 1: Install vLLM

​Step 2: Download the model

​Step 3: Serve the model with vLLM

​Step 4: Benchmark performance

​Step 5: Connect with Tokios

​Install the connector

​Claim the connector

​Register the deployment

​Create API keys

​Step 6: Use it from any client

​What else fits on DGX Spark?

​Troubleshooting

​vLLM fails to start with “CUDA out of memory”

​Slow time-to-first-token (>1s)

​Gemma-4-26B-A4B-it fails to load

​Next steps

What you’ll get

Prerequisites

Step 1: Install vLLM

Step 2: Download the model

Step 3: Serve the model with vLLM

Step 4: Benchmark performance

Step 5: Connect with Tokios

Install the connector

Claim the connector

Register the deployment

Create API keys

Step 6: Use it from any client

What else fits on DGX Spark?

Troubleshooting

vLLM fails to start with “CUDA out of memory”

Slow time-to-first-token (>1s)

Gemma-4-26B-A4B-it fails to load

Next steps