Skip to main content
Not recommended hardware. Tokios does not recommend AMD Strix Halo (Ryzen AI Max+ 395) as a primary inference platform. The hardware is community-explored — driver support is inconsistent, ROCm stability depends on exact kernel/firmware versions, and Vulkan is the only reliable backend. For production or daily-driver inference, we recommend a Mac Studio, DGX Spark, or RTX workstation instead. This page exists for users who already own Strix Halo hardware and want to make the most of it.
The AMD Ryzen AI Max+ 395 (“Strix Halo”) ships with up to 128GB of unified memory shared between the CPU and the Radeon 8060S iGPU. That pool is large enough to load models that would need a $2,000+ discrete GPU — but the 215 GB/s memory bandwidth and the MoE-vs-dense distinction decide whether you get 2 tok/s or 30 tok/s. This page tells you exactly what fits and what to expect.

​
Quick verdict: MoE models dominate

MoE (mixture-of-experts) models are 3–14× faster than dense models of the same total parameter count on Strix Halo. The reason is bandwidth: only the active parameters in an MoE model are read from memory per token, so a 122B MoE model with 10B active parameters generates tokens at nearly the speed of a 10B dense model — not a 122B one. If you are choosing a model for daily use, start with a MoE model in the 35B–122B range.

​
128GB unified memory explained

Strix Halo does not have discrete VRAM. The iGPU draws from the same DDR5 pool as the CPU. Out of the box, the BIOS and OS reserve some of that 128GB:

​
Configure BIOS memory on Linux

On Linux, the kernel manages the split automatically. You typically see 95–110GB available for GPU workloads after boot. No manual BIOS change is needed in most cases.

​
Configure memory on Windows

On Windows, you must set 2 things:
  1. BIOS → UMA Frame Buffer — set to the largest option available (typically “Auto” or a fixed value).
  2. AMD Adrenalin → VRAM allocation — open AMD Software: Adrenalin Edition, go to Settings → Graphics → Advanced, and move the VRAM slider to 96GB.
Without the Adrenalin VRAM allocation on Windows, your iGPU may only see 15.5GB — far too little for large models. Set both the BIOS and the Adrenalin slider before attempting to load anything over 30B parameters.

​
Dense model compatibility

Dense models activate every parameter on every token, so the full weight tensor must be read from memory each step. At 215 GB/s, that creates a hard ceiling on token speed.
Qwen3-Coder-30B at Q4_K_M is the community’s go-to dense model for coding agents (Cline, Aider, opencode) on Strix Halo. It fits easily and delivers 10–15 tok/s.

​
MoE model compatibility

MoE models keep the total parameter count high but only activate a fraction per token. That means far less memory bandwidth per step, and dramatically faster generation. MoE models with 3–12B active parameters are the sweet spot. They give you frontier-class quality at speeds that feel responsive in a chat or coding session.

​
ROCm vs Vulkan backend comparison

Strix Halo’s Radeon 8060S iGPU (gfx1151) can run inference through 2 backends. Each has tradeoffs. Use Vulkan unless you have a specific reason to need ROCm (for example, BF16 precision for a fine-tune evaluation).
See vLLM and alternatives on Strix Halo for runtime options including llama.cpp Vulkan and Ollama.

​
Bandwidth constraints

Strix Halo’s DDR5 memory delivers 215 GB/s theoretical bandwidth. That is the single most important number for understanding performance:
  • Token generation is memory-bandwidth-bound. Each token requires reading the model’s active weights from memory.
  • A 70B dense model at Q4 (~40GB) needs 40GB ÷ 215 GB/s ≈ 186ms per token → ~5 tok/s.
  • A 122B MoE model with 10B active at Q4 (~7GB active weights) needs 7GB ÷ 215 GB/s ≈ 33ms per token → ~30 tok/s.
Prompt processing (prefill) is compute-bound and runs faster relative to token generation. You will notice that long prompts process quickly, then generation settles into the bandwidth-limited rate.

​
Performance expectations by use case

​
2-node clustering over RDMA

If 128GB is not enough, 2 Strix Halo nodes can be clustered over RDMA RoCE using Intel E810 NICs, giving a combined 256GB unified memory pool. The community has demonstrated this with llama.cpp RPC and models like MiniMax-M2 and GLM-4.6.
Network overhead is significant — 10GbE minimum, 25GbE recommended. Only pursue clustering when a single node cannot hold your model.

​
Check your memory usage

Before loading a model, verify how much GPU memory is available:
Subtract OS overhead (~4GB) and leave 2–4GB for KV cache at your target context length. A 70GB model in Q4 leaves roughly 20GB for KV cache on a 96GB allocation — enough for 8K–16K context on most models.

​
Next steps

Strix Halo use case

Set up the Tokios connector on your Strix Halo box for remote access.

vLLM and alternatives on Strix Halo

Runtime options including llama.cpp Vulkan and Ollama.

Quantization guide

Understand GGUF quantization levels and quality tradeoffs.