Skip to main content
DGX Spark ships with 128GB of unified memory shared between the Grace CPU and the Blackwell GPU. That pool is large enough for dense models up to ~70B at 4-bit quantization and MoE models up to ~120B total parameters at NVFP4 — but not every model at every quantization fits, and not every runtime respects the unified memory layout the same way. This page tells you what works, what works with caveats, and what doesn’t fit at all.

​
Quick verdict

One DGX Spark runs dense models up to ~70B at Q4 and MoE models up to ~120B total parameters at NVFP4. MoE models are the better fit because only a fraction of their parameters activate per token, giving you higher throughput for the same memory footprint.

​
How the 128GB gets used

DGX Spark does not have discrete VRAM. The Grace CPU and Blackwell GPU share a single 128GB memory pool over a 273 GB/s interconnect. When you load a model, 3 things consume that pool:
  1. Model weights — the largest fixed cost, determined by parameter count and quantization.
  2. KV cache — grows with context length and concurrent requests. See DGX Spark memory tuning for sizing math.
  3. System and CUDA overhead — typically 4-8 GB for DGX OS, CUDA runtime, and vLLM’s CUDA graphs.
That leaves a practical usable budget of roughly 115-120 GB for weights and KV cache after system overhead.
DeepSeek-V4-Flash-0731 does not fit on 1 Spark. 2 Sparks provide enough aggregate memory to shard its published 162–167 GB weight floor, but published dual-Spark evidence is community-tested and needs 0731 revalidation. Start at 32K–128K context and read DeepSeek V4 Flash local hardware before configuring a cluster.DeepSeek-V4.1-Flash (552B backbone + 196B Engram → ~763B total) also does not fit on 1 Spark. On 2 Sparks, it fits with mixed-precision serving (FP8 dense + MXFP4 experts, ~79–105 GiB per node). Community benchmarks show 38–73 tok/s single-stream decode and up to 1M context. See the V4.1 Flash on 2 DGX Sparks guide for configuration and gotchas.

​
Dense model compatibility

Dense models activate every parameter for every token. That means weight memory and compute scale linearly with parameter count — a 70B dense model needs roughly twice the memory and twice the FLOPs of a 35B dense model. On DGX Spark, dense models work up to 70B at 4-bit quantization, but concurrency is limited by KV cache pressure.
Qwen3-32B shows a wide performance spread (9 vs 44 tok/s) across community benchmarks on the same hardware. The difference traces to flash attention settings and quantization backend. Test your specific configuration before committing to a serving choice.
A 70B dense model at NVFP4 uses ~35 GB for weights, but each concurrent user adds KV cache on top. Community reports cap multi-user concurrency at 5-10 users before the Spark becomes unresponsive. For multi-user 70B workloads, consider two DGX Sparks.

​
Dense model caveats on unified memory

  • ChatGPT-OSS-20B on Ollama reports 89% GPU usage but unacceptable latency. Community recommendation: switch to vLLM or SGLang for this model.
  • Qwen3.6-27B at NVFP4 is estimated to achieve 40–55 tok/s for a single user, based on similar dense models in the 27–32B range. The wide spread reflects differences in quantization backend and flash attention settings. With ~14–15 GB for weights, you have ample room for long context on a single Spark.
  • 70B dense models work but are the practical ceiling for dense architectures on a single Spark. The KV cache per concurrent request is the bottleneck, not weight memory.
  • For dense models above 70B, you need a two-Spark cluster. See One vs two DGX Sparks.

​
MoE model compatibility

MoE models are the sweet spot for DGX Spark. All expert weights load into memory, but only a subset activates per token — so you get a large model’s capability with a small model’s compute cost.
Gemma-4-26B-A4B-it fails on DGX Spark because vLLM’s gpu_memory_utilization parameter is not respected during startup profiling on unified memory (SM121). The system becomes unresponsive. The workaround is to use NVFP4 quantization explicitly, but community reports indicate the fix is still incomplete as of mid-2026. Track vLLM issue #46307 for updates.

​
MoE performance notes

  • Qwen3.5-35B-A3B at NVFP4 achieves 47-50 tok/s sustained for a single user. Time-to-first-token drops from 3 seconds (Ollama) to 0.12 seconds with vLLM.
  • Qwen3.6-35B-A3B achieves 47–50 tok/s sustained with stock vLLM for a single user. With the Atlas engine (pure Rust/CUDA), it reaches up to ~120 tok/s — 2.8x faster than stock vLLM — and supports up to 1M token context.
  • Qwen3.5-122B-A10B sustains ~30 tok/s for JSON-KIE tasks with 100% MTP acceptance rate, but breaks under 100K concurrent load.
  • GLM-4.5-Air at 106B MoE with 12B active parameters is widely reported by the community as the best model for Spark. It fits the memory budget well.
  • DeepSeek-V4.1-Flash at 763B total (8B/16B active) requires 2+ Sparks with mixed-precision serving. Community benchmarks on 2 Sparks (TP=2, FP8 dense + MXFP4 experts) show 38–73 tok/s single-stream decode, scaling to 135–270 tok/s aggregate at 16 concurrent users. 1M context is verified. The full configuration and gotchas are in the V4.1 Flash on 2 DGX Sparks guide.
GLM-4.6 (357B total / 32B active) does not fit on a single DGX Spark. At Q4-class quantization it needs ~178 GB — well over the 128 GB unified memory pool. For 128 GB hardware, use GLM-4.5-Air instead. See GLM family for the full variant comparison.

​
Quantization format comparison on SM121

The Blackwell GPU in DGX Spark uses the SM121 architecture. Not every quantization format is natively supported.
NVFP4 is the recommended default for DGX Spark. It is natively supported by the SM121 hardware, gives a 75% size reduction with low quality impact, and works with both vLLM and the Atlas engine. Use GGUF Q4_K_M only if you run llama.cpp or Ollama.

​
Context length limits

Context length directly affects how much memory the KV cache consumes, which reduces the budget available for model weights. On DGX Spark’s unified memory, every byte of KV cache comes from the same 128GB pool that holds weights, the CUDA runtime, and the OS. Using FP8 KV cache (--kv-cache-dtype fp8) halves these numbers with less than 2% quality impact, effectively doubling the context you can serve. For detailed KV cache sizing math and tuning parameters, see DGX Spark memory tuning.

​
Kimi K3 on DGX Spark

Kimi K3 does not fit on any practical DGX Spark cluster. The model’s 2.8T total parameters (~50B active per token through Stable LatentMoE) need about 1.4 TB at the native MXFP4 precision. Eight DGX Sparks pool 1 TB and still fall 376 GB short. NVIDIA documents Spark clusters up to 4 nodes (512 GB), and Moonshot recommends a supernode with 64 or more accelerators for K3. For the full calculation, see the Kimi K3 hardware deep dive. For Spark today, the models that fit are in the tables above — up to ~70B dense at Q4 and ~120B MoE at NVFP4 on a single node. See what to run instead of Kimi K3 for the best pick at each Spark count.

​
Qwen3.8 feasibility

The Qwen3.8 flagship shipped in August 2026 as Qwen3.8-2.4T-A95B: a Mixture-of-Experts model with 2.4 trillion total parameters and 95B active per token, text-only, released under Qwen’s custom qwen3.8-max license. The generation also includes Qwen3.8-27B (released 2026-08-14, Apache-2.0), a dense vision-language model that does fit a single Spark — see the verdict below the cluster math.

​
Memory requirements by quantization

Weight-only memory calculations (before KV cache, CUDA graphs, and system overhead): Even at the most aggressive quantization, a single DGX Spark cannot run Qwen3.8-2.4T-A95B. This is a multi-node model that requires at least 10 Sparks at Q4 — and realistically 20+ Sparks at FP8 for reasonable KV cache headroom.

​
Multi-node cluster sizing

MoE models load all expert weights into memory even though only a subset activates per token. Qwen3.8-2.4T-A95B activates 95B of its 2.4T, but the 1.2 TB at Q4 is a hard floor regardless. The MoE advantage is in compute (fewer FLOPs per token), not in memory.

​
Runtime support status

Qwen publishes serving recipes for vLLM, SGLang, and TokenSpeed alongside the 2.4T weights. llama.cpp, Ollama, and LM Studio support is not yet documented — the model’s hybrid Gated DeltaNet + Gated Attention architecture needs dedicated kernel work before GGUF serving is realistic.

​
The Qwen3.8 that fits one Spark

Qwen3.8-27B is the single-machine member of the generation: a dense 27B vision-language model released 2026-08-14 under Apache-2.0. Every official precision fits one Spark — 55.6 GB BF16 outright, and the official 30.9 GB FP8 build leaves roughly 90 GB of the 128 GB pool for KV cache. Serve it through vLLM or SGLang; no GGUF or NVFP4 build exists yet. Full sizing is on the Qwen3.8 model page; tested Spark configurations will be added here once verified.

​
GLM-5.2 feasibility

​
Do not size MoE models by active parameters

Active parameters affect per-token compute, not the resident weight floor. GLM-5.2 stores 753B parameters (Hugging Face reports 753,329,940,480 across 78 layers, 8 of 256 routed experts active per token plus 1 shared), so the transparent lower bound at exactly 4 bits per weight is:
That is roughly 3× a single DGX Spark’s 128 GB, so 1 Spark is not close and 2 Sparks at 256 GB still fall short of the weights alone. 3 Sparks clear the floor on paper at 384 GB but leave about 7 GB across the whole cluster for format metadata, the operating system, runtime buffers, and KV cache — unusable in practice. 4 Sparks at 512 GB is the first configuration with real headroom. FP8 would require 753 GB for weights alone, which is 6 Sparks before any overhead.
Do not claim GLM-5.2 fits on 1 or 2 Sparks at INT4 or FP8 — the published 753B checkpoint size rules it out. Do not promise a 1M context on a 4-Spark cluster from parameter counts alone. Verify the exact downloadable files, runtime support, and measured KV-cache use first.

Picked a model? Connect it

Give your DGX Spark deployment one authenticated endpoint at https://api.tokios.com — no inbound ports, about 5 minutes.

​
Choosing between speed and model size

The fundamental trade-off on DGX Spark is between model capability and throughput:
  • If speed matters most (coding agents, interactive chat): Pick a 35B MoE model at NVFP4. You get 47-50 tok/s (or 120 tok/s with Atlas), which feels instant.
  • If quality matters most (complex reasoning, research): Pick the largest model that fits — Qwen3.5-122B-A10B or GLM-4.5-Air — and accept 25-30 tok/s.
  • If concurrency matters most (team serving): Stay at 35B MoE and use FP8 KV cache to maximize simultaneous users. See DGX Spark memory tuning for the math.

​
Verify compatibility on your Spark

Community benchmarks give a strong signal, but your specific DGX OS version, CUDA driver, and vLLM build may behave differently. Run these checks before committing to a model:
  1. Check memory before launch:
  2. Start the model with conservative settings:
  3. Increase max-model-len incrementally until you hit your target context or memory pressure.
  4. Monitor during inference:
  5. Benchmark throughput:
If the system becomes unresponsive at any step, power cycle and reduce gpu_memory_utilization by 0.05.

​
Serve your model through Tokios

Once you have a model running on your DGX Spark with vLLM, Ollama, or llama.cpp, expose it through a single authenticated endpoint without opening inbound ports. The Tokios connector dials out from the Spark and gives you an OpenAI- and Anthropic-compatible API at https://api.tokios.com. See Turn your DGX Spark into a private endpoint for the full setup, or jump to register a model to map your local model id to a deployment name.

vLLM on DGX Spark

Step-by-step vLLM installation with NVFP4 settings for each model size.

DGX Spark memory tuning

KV cache sizing, batch trade-offs, and tuning parameters for 128GB.

One vs two DGX Sparks

When to scale to a second Spark for distributed inference and models that exceed 1 node.

Quantization concepts

Understand NVFP4, AWQ, FP8, and GGUF before choosing a format.