One vs two DGX Sparks — when to scale and how to set it up

One DGX Spark handles models that fit within its 128 GB unified-memory pool. Two Sparks connected over a 200Gb/s QSFP link can run selected distributed models that do not fit on 1 node. The added network hop also adds coordination overhead, so measure throughput and latency on your exact runtime instead of assuming a 2× gain.

Quick verdict

Model size	1 Spark	2 Sparks	Notes
≤35B MoE	Yes	Overkill	Runs comfortably at 47-120 tok/s
70B dense	Yes (tight)	Better	Single Spark caps at 5-10 concurrent users
120B MoE	Yes	Better	Fits at NVFP4 but memory is tight
DeepSeek-V4-Flash-0731	No	Community-tested	~162–167 GB weights before KV cache; revalidate 0731
405B dense at FP8	No	No	~405 GB weights before runtime overhead
460B dense-equivalent weight count at 4-bit	No	Tight	~230 GB raw weights before format, runtime, or KV-cache overhead
700B at 4-bit	No	No	~350 GB raw weights before overhead

When 1 Spark is enough

A single DGX Spark at 128GB is the right choice when:

Your model fits at a reasonable quantization. Dense models up to 70B at NVFP4 (~35 GB) or MoE models up to 120B at NVFP4 (~60 GB) load on a single node with room for KV cache.
You serve 1-5 concurrent users. Single-Spark inference handles small team workloads at 35B MoE. For 70B dense, plan for 5-10 users maximum.
Latency matters more than raw model size. A single Spark avoids the inter-node communication overhead that tensor parallelism introduces. For a 35B MoE model, single-node serving delivers 47-50 tok/s — fast enough for real-time coding agents and chat.
You want simple operations. 1 machine, 1 runtime, 1 point of failure. No network configuration, no distributed launch, no NCCL debugging.

See the DGX Spark model compatibility matrix for a full table of what fits.

When you need 2 Sparks

Add a second Spark when any of these apply:

Your quantized model exceeds 128 GB but fits comfortably below 256 GB after runtime and KV-cache headroom. DeepSeek-V4-Flash-0731 is an example, with a published 162–167 GB weight floor.
You need more KV-cache capacity or concurrency on a model that is already validated for distributed serving. Tensor parallelism can distribute memory pressure, but it does not guarantee proportional throughput.
You are prepared to benchmark the networked runtime. Communication overhead and the model architecture determine the result.

A 405B dense model at FP8 needs roughly 405 GB for weights alone. It does not fit in 2 Sparks’ 256 GB combined memory. For a tested 2-Spark DeepSeek route and context guidance, see DeepSeek V4 Flash local hardware.

200 Gbps QSFP link configuration

Two DGX Sparks connect directly via a 200Gb/s QSFP cable — no switch required. NVIDIA publishes an official “Connect Two Sparks Playbook” for this configuration.

Connect the QSFP cable

Plug the 200Gb/s QSFP cable into the QSFP port on each DGX Spark. The link is a direct point-to-point connection — it does not go through your network switch.

Verify the link

On both Sparks, confirm the link is up and running at full speed:

ip link show

Look for the QSFP interface showing UP state. Verify the speed:

ethtool <qsfp-interface> | grep Speed

You should see 200000Mb/s (200 Gb/s).

Configure IP addresses

Assign static IPs on the QSFP link subnet. On Spark A:

sudo ip addr add 10.0.0.1/24 dev <qsfp-interface>

On Spark B:

sudo ip addr add 10.0.0.2/24 dev <qsfp-interface>

Test connectivity:

ping 10.0.0.2   # from Spark A
ping 10.0.0.1   # from Spark B

Set NCCL environment variables

NCCL (NVIDIA Collective Communications Library) handles inter-node GPU communication. Set these on both Sparks before launching vLLM:

export NCCL_SOCKET_IFNAME=<qsfp-interface>
export NCCL_IB_DISABLE=0
export NCCL_NET_GDR_LEVEL=2

These variables tell NCCL to use the QSFP link for communication and enable GPU Direct RDMA for the lowest latency.

Tensor parallelism across 2 Sparks

vLLM supports distributed tensor parallelism, where each Spark holds a shard of the model weights and they communicate over the QSFP link during inference.

Launch a distributed model

On Spark A (the head node), start vLLM in distributed mode:

vllm serve <your-distributed-model> \
  --tensor-parallel-size 2 \
  --quantization fp8 \
  --gpu-memory-utilization 0.85 \
  --max-model-len 8192 \
  --distributed-executor-backend ray

vLLM uses Ray to coordinate the two Sparks. Spark A loads half the model shards, Spark B loads the other half, and they synchronize through NCCL over the QSFP link.

Tensor parallelism is the right parallelism strategy for two Sparks. Pipeline parallelism (splitting layers sequentially) introduces pipeline bubbles that waste compute on small clusters. Tensor parallelism splits each layer’s computation across both GPUs, giving better utilization for 2-node setups.

Model examples for 2 Sparks

Model	Quantization	Weight floor	Configuration	What to verify
DeepSeek-V4-Flash-0731	Published fused/GGUF weights	~162–167 GB	Community-tested route; revalidate 0731	KV cache, tool calls, and end-to-end throughput
70B dense	NVFP4	~35 GB	`--tensor-parallel-size 2`	Compare throughput and concurrency with 1 Spark
120B MoE	NVFP4	~60 GB	`--tensor-parallel-size 2`	KV-cache headroom and network overhead

Pipeline parallelism (when to consider)

Pipeline parallelism assigns different layers to each Spark. It is simpler to configure but less efficient for 2 nodes because the second Spark sits idle while the first processes each token’s early layers. Use pipeline parallelism only when:

The model architecture does not support tensor parallelism.
You are chaining more than 2 Sparks and want to reduce communication overhead per hop.

vllm serve <model> \
  --pipeline-parallel-size 2 \
  --tensor-parallel-size 1

Network requirements between Sparks

The QSFP link is purpose-built for this configuration, but the network path matters for performance:

Requirement	Specification	Why
Link speed	200 Gb/s minimum	Tensor parallelism exchanges activation tensors every layer — bandwidth is the bottleneck
Latency	Sub-microsecond (direct QSFP)	Each token requires multiple all-reduce operations; latency compounds
Topology	Direct point-to-point	Avoid switches between Sparks; each hop adds latency
Protocol	RoCE or native QSFP	NCCL uses RDMA when available for zero-copy GPU-to-GPU transfers

Do not use a standard Ethernet switch between two Sparks for tensor parallelism. A 10GbE or even 100GbE switch adds latency that bottlenecks the decode phase. The 200Gb/s direct QSFP link is the supported and tested configuration.

Cost-benefit analysis

Factor	1 Spark	2 Sparks
Hardware cost	1x DGX Spark	2x DGX Spark + QSFP cable
Power	~240W	~480W
Setup complexity	Low (single host)	Medium (network + distributed vLLM)
Max model size	Limited by 128 GB memory	Limited by 256 GB combined memory, runtime overhead, and interconnect
Max concurrency (70B)	5-10 users	10-20 users
Throughput	Measure on your runtime	Measure against a 1-Spark baseline
Failure domain	Single point of failure	2 points; need both for distributed models

When the second Spark pays off

You need a distributed model with a measured weight and KV-cache plan. For example, DeepSeek-V4-Flash-0731 needs 2 nodes and starts as a community-tested configuration.
You serve a team on a model whose distributed runtime increases your measured throughput or concurrency enough to justify the second unit.
You run latency-sensitive workloads on 120B models. The extra KV cache headroom from two Sparks lets you use longer context or serve more users without hitting the memory wall.

When it doesn’t

Your model is ≤70B and your team is small. One Spark at 47-50 tok/s on a 35B MoE model is already faster than most people type. A second Spark adds cost and complexity for no user-visible improvement.
You run MoE models that already fit. GLM-4.5-Air, Qwen3.5-122B, and Nemotron-3-Super-120B all fit on one Spark at NVFP4. Splitting them across two nodes introduces communication overhead that may not recover the parallelism gain.

Set up a 2-Spark cluster

Connect and verify the QSFP link

Follow the 200Gb/s link configuration steps above. Confirm 200Gb/s link speed and bidirectional ping on both Sparks.

Install vLLM on both Sparks

Both nodes need identical vLLM installations. Follow the vLLM on DGX Spark guide on each unit.

pip install vllm

Install Ray on both Sparks

vLLM uses Ray for distributed coordination:

pip install ray

Start the Ray head node on Spark A:

ray start --head --port=6379

Join Spark B to the Ray cluster:

ray start --address=10.0.0.1:6379

Verify both nodes appear:

ray status

Set NCCL environment variables on both Sparks

export NCCL_SOCKET_IFNAME=<qsfp-interface>
export NCCL_IB_DISABLE=0
export NCCL_NET_GDR_LEVEL=2

Launch vLLM in distributed mode on Spark A

vllm serve <your-model> \
  --tensor-parallel-size 2 \
  --quantization fp8 \
  --gpu-memory-utilization 0.85 \
  --max-model-len 8192 \
  --distributed-executor-backend ray

vLLM distributes the model shards across both Sparks automatically.

Verify distributed serving

Test the endpoint from either Spark:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<your-model>",
    "messages": [{"role": "user", "content": "Hello from two Sparks"}],
    "max_tokens": 64
  }'

Serve both Sparks through Tokios

You can aggregate multiple Sparks behind a single Tokios endpoint. Install the connector on each Spark, register each as a separate deployment, and use model routing to direct traffic. For a 2-Spark tensor-parallel setup where vLLM runs on Spark A and coordinates with Spark B internally, you only need the connector on Spark A — vLLM’s /v1 endpoint already represents the combined cluster.

# On Spark A (the vLLM head node)
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Register the distributed deployment in the Tokios dashboard, and clients reach both Sparks through https://api.tokios.com without any awareness of the underlying topology.

vLLM on DGX Spark

Installation and launch commands for single-node vLLM.

Model compatibility

Check which models fit on 1 vs 2 Sparks.

Memory tuning

KV cache math and tuning parameters for either configuration.

DGX Spark use case

Full Tokios connector setup for DGX Spark.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

One vs two DGX Sparks — when to scale and how to set it up

Quick verdict

When 1 Spark is enough

When you need 2 Sparks

200 Gbps QSFP link configuration

Tensor parallelism across 2 Sparks

Launch a distributed model

Model examples for 2 Sparks

Pipeline parallelism (when to consider)

Network requirements between Sparks

Cost-benefit analysis

When the second Spark pays off

When it doesn’t

Set up a 2-Spark cluster

Serve both Sparks through Tokios

vLLM on DGX Spark

Model compatibility

Memory tuning

DGX Spark use case

​Quick verdict

​When 1 Spark is enough

​When you need 2 Sparks

​200 Gbps QSFP link configuration

​Tensor parallelism across 2 Sparks

​Launch a distributed model

​Model examples for 2 Sparks

​Pipeline parallelism (when to consider)

​Network requirements between Sparks

​Cost-benefit analysis

​When the second Spark pays off

​When it doesn’t

​Set up a 2-Spark cluster

​Serve both Sparks through Tokios

vLLM on DGX Spark

Model compatibility

Memory tuning

DGX Spark use case

Quick verdict

When 1 Spark is enough

When you need 2 Sparks

200 Gbps QSFP link configuration

Tensor parallelism across 2 Sparks

Launch a distributed model

Model examples for 2 Sparks

Pipeline parallelism (when to consider)

Network requirements between Sparks

Cost-benefit analysis

When the second Spark pays off

When it doesn’t

Set up a 2-Spark cluster

Serve both Sparks through Tokios