Run DeepSeek-V4-Flash-0731 locally: hardware requirements

You can self-host deepseek-ai/DeepSeek-V4-Flash-0731, the official release that supersedes the V4 Flash preview. The answer depends on more than aggregate memory. The 284B target model activates 13B parameters per token. Its bundled DSpark draft head raises the full checkpoint count to about 304B parameters. The published serving formats still need roughly 162–167 GB of resident weights before KV cache, temporary buffers, and the operating system. The 3 configurations below can meet that floor. They do not have the same software maturity. The practical recommendation is simple: start at 32K–128K context, prove stable generation and tool calls, then measure cache capacity. Treat 384K and 1M as separate engineering targets. The model advertises a 1M-token context window; that does not mean every local configuration can reserve a usable 1M-token cache.

What fits, and how confident should you be?

Configuration	Aggregate accelerator or unified memory	Interconnect	Practical starting context	Maturity	Verdict
2× RTX PRO 6000 Blackwell	192 GB VRAM	PCIe; host topology varies by workstation	32K–128K after measurement	Community validation required	Fits the weight floor, but 0731 has no official TP=2 validation
2× DGX Spark	256 GB unified memory	ConnectX-7 cluster link	32K–128K after measurement	Community-tested	The most documented prosumer cluster path; preview-checkpoint results need 0731 revalidation
2× Strix Halo	Up to 256 GB unified memory	llama.cpp RPC over your network	32K at first, if stable	Experimental	Capacity-feasible with GGUF, but the distributed runtime is proof-of-concept
4× GB300	Vendor reference configuration	Datacenter-scale platform	Follow the model card	Verified	Official 0731 reference configuration

Verified means the named software or model source documents the exact path. Community validation required means the capacity and a closely related runtime path are documented, but this exact checkpoint and hardware combination has not been reproduced publicly. Community-tested means public users have reported the approach, but you must reproduce it on your versions. Experimental means the upstream project itself warns that the route is not production ready.

Start with the memory math

The official model card describes the 284B target model, 13B active parameters per token, and the attached DSpark speculative-decoding module. SGLang’s current hardware matrix counts about 304B parameters for the complete 0731 checkpoint. The active count helps explain compute cost. It does not tell you how much model state must be resident:

304B checkpoint parameters × the mixed bytes per stored weight + format metadata
  + KV cache + runtime workspaces + communication buffers
  + operating-system and serving-process memory

For a concrete downloadable format, Unsloth lists a lossless GGUF release at about 162 GB. Other serving formats are closer to 167 GB once their fused weights and metadata are counted. That is the weight floor, not a deployment recommendation.

Item	Plan for it
Model files	At least 200 GB free local disk for a 162–167 GB checkpoint plus a safe download/conversion margin
Weights	162–167 GB before cache and runtime overhead
KV cache	Measure it at your target context, concurrency, and cache dtype
Runtime buffers	Leave room for kernels, NCCL/RPC communication, tokenization, and model-server allocations
System memory	Unified-memory systems share the same pool with the OS and other processes

Do not treat 192 GB as 192 GB available to V4 Flash. Two 96 GB GPUs clear the weight floor, but leave limited room for KV cache and runtime allocations. A successful model load is not yet a usable deployment.

2× RTX PRO 6000 Blackwell

NVIDIA lists the RTX PRO 6000 Blackwell with 96 GB of GDDR7 memory and 1,792 GB/s memory bandwidth per card in its product specifications. Two cards give 192 GB of VRAM, enough to hold the published weight formats in principle. SGLang documents tensor parallelism of 2 for the earlier Flash checkpoint in its DeepSeek V4 guide. Its 0731 validation list covers larger 4–8 GPU configurations, not this exact 2-GPU setup. Call this path Community validation required, not verified.

Suggested sequence

Confirm both GPUs are visible to your selected runtime.
Start with a 32K context limit and 1 request at a time.
Raise toward 128K only after measuring VRAM usage, first-token latency, and stable completion.
If you need 384K or 1M, size the KV cache from measured usage instead of subtracting weights from total VRAM by hand.

Community-tested — earlier Flash TP=2 baseline. Adapt the exact model id and flags to the version shown in the SGLang guide. This is a starting pattern, not an official 0731 validation:

sglang serve \
  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --tp 2 \
  --context-length 32768

Before using the command, compare its argument names and supported quantization against your installed SGLang release. The checkpoint, container, drivers, and workstation topology all affect whether TP=2 succeeds.

2× DGX Spark

Each DGX Spark has 128 GB of LPDDR5x unified memory and 273 GB/s memory bandwidth. NVIDIA documents a ConnectX-7 clustering path for 2 systems in its Spark stacking guide, and lists the hardware details in its DGX Spark overview. This gives 256 GB of aggregate memory, which is meaningfully less tight than 192 GB for weight plus cache capacity. This is still a networked distributed runtime. NVIDIA documents the cluster fabric, not an official 0731 DeepSeek recipe. Community reports for 2 Sparks target the prior Flash checkpoint. Revalidate model load, generation, tool calls, and sustained requests after moving to 0731.

Suggested topology

Spark 1 (head node)                         Spark 2 (worker)
  model-server coordinator  <--- CX7 --->    worker rank
  127.0.0.1:<port>/v1
  tokios-connector

Only the head node needs the Tokios connector. It must point to the head node’s local OpenAI-compatible /v1 endpoint, not directly to the worker or the cluster fabric. Community-tested — dual-Spark starting pattern. First complete NVIDIA’s 2-Spark networking setup. Then use a distributed runtime recipe that explicitly supports your installed version and begin at 32K context. Do not present a preview-checkpoint command as an official 0731 recipe.

NVIDIA calls the high-speed connection a ConnectX-7 cluster link. It is not a substitute for a single-card NVLink-style memory pool: the runtime still shards work and communicates between nodes. Measure throughput and latency with your prompt sizes.

2× Strix Halo

AMD lists the Ryzen AI Max+ 395 with up to 128 GB of unified memory in its processor specifications. Two fully provisioned systems have enough aggregate capacity for the 162 GB GGUF weight release. The problem is the distributed serving layer. The available route is llama.cpp RPC. Its upstream RPC README describes RPC as proof-of-concept and warns that it is fragile and insecure. That makes this an Experimental capacity demonstration, not a production architecture. Experimental — llama.cpp RPC shape. Run the RPC worker only on a private, trusted network. The exact binary names and flags change across llama.cpp builds, so use the upstream README for the commands that match your revision:

head node: llama.cpp server with a split GGUF model and an RPC worker endpoint
worker:    llama.cpp RPC server reachable only from the head node
Tokios:    1 connector on the head node, pointed at the head node's /v1 endpoint

Do not put an RPC listener on the public internet. Do not call the RPC route production-ready just because the model loads. For a stable production service, use a runtime with a documented distributed serving model instead.

Official baselines you can compare against

Use vendor and runtime examples to set your expectations, not to infer that smaller hardware has the same support level.

Baseline	What it establishes
Verified: 4× GB300 with vLLM	The 0731 model card publishes the exact reference command with DSpark enabled.
Verified: 4× GB300 with SGLang	The model card and SGLang guide publish the 0731 launch path with the bundled DSpark head.
Historical: 1× MI325X, 4K context	The vLLM recipe validates the earlier preview checkpoint on a 256 GB MI325X. It is useful historical evidence, not 0731 validation.

Verified — current 0731 vLLM reference. The official model card uses 4× GB300, expert parallelism, FP8 KV cache, and the bundled DSpark module:

vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --data-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'

For agentic workloads, DeepSeek recommends temperature = 1.0 and top_p = 0.95. Keep those sampling settings separate from server startup flags and validate them with your agent harness.

Validate in this order

Download the exact checkpoint and record its revision.
Load it with 32K context and 1 concurrent request.
Send a short chat request, a long prompt, and a tool-call request if your workload needs tools.
Record memory consumption, first-token latency, generation rate, and errors.
Raise context toward 128K. Repeat the same workload.
Only then test 384K or 1M, with a KV-cache budget and rollback plan.

This order separates weight-fit failures from cache pressure, distributed communication problems, and tool-format issues. It also prevents an advertised maximum context from becoming an untested production promise.

Expose the head node through Tokios

Tokios needs 1 stable OpenAI-compatible upstream model endpoint. On a cluster, that is the head node, not every worker.

Keep the model server on the head node private

Confirm the serving process responds on the head node’s loopback /v1 endpoint, such as http://127.0.0.1:8000/v1. Keep worker-to-worker traffic on the cluster network.

Pair 1 connector on the head node

In the Tokios dashboard, open Setup and select Start pairing. Install and run tokios-connector on the head node. The connector dials out; it does not require an inbound port.

Configure the head-node upstream

Set the connector’s BaseUrl to the head node’s local /v1 endpoint. See Connector config for the complete configuration format.

In Models, register a deployment such as deepseek-v4-flash. Map it to the upstream model id your head-node server exposes. Clients send the deployment name, not the Hugging Face id.

Create a scoped API key

Create an sk-tok-… key in Keys, optionally scoped to deepseek-v4-flash. Clients then call https://api.tokios.com/v1.

Verified — Tokios API request. This checks Tokios’s documented OpenAI-compatible endpoint after the local cluster is healthy:

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{"role": "user", "content": "Summarize this deployment's health checks."}]
  }'

The command does not validate that a specific DeepSeek cluster is healthy; run the local checks first.

DeepSeek family overview

Compare V4 Flash with R1 distills and the V3 line.

vLLM remote access

Keep a vLLM /v1 endpoint private and reach it through Tokios.

DGX Spark

Set up and use a DGX Spark with a connector.

RTX workstation

Choose and configure an RTX-based local inference machine.

​What fits, and how confident should you be?

​Start with the memory math

​2× RTX PRO 6000 Blackwell

​Suggested sequence

​2× DGX Spark

​Suggested topology

​2× Strix Halo

​Official baselines you can compare against

​Validate in this order

​Expose the head node through Tokios

DeepSeek family overview

vLLM remote access

DGX Spark

RTX workstation

What fits, and how confident should you be?

Start with the memory math

2× RTX PRO 6000 Blackwell

Suggested sequence

2× DGX Spark

Suggested topology

2× Strix Halo

Official baselines you can compare against

Validate in this order

Expose the head node through Tokios