Skip to main content
Not recommended hardware. Tokios does not recommend AMD Strix Halo (Ryzen AI Max+ 395) as a primary inference platform. The hardware is community-explored — driver support is inconsistent, ROCm stability depends on exact kernel/firmware versions, and Vulkan is the only reliable backend. For production or daily-driver inference, we recommend a Mac Studio, DGX Spark, or RTX workstation instead. This page exists for users who already own Strix Halo hardware and want to make the most of it.
The AMD Ryzen AI Max+ 395 (“Strix Halo”) ships with up to 128GB of unified memory shared between the CPU and the Radeon 8060S iGPU. That pool is large enough to load models that would need a $2,000+ discrete GPU — but the 215 GB/s memory bandwidth and the MoE-vs-dense distinction decide whether you get 2 tok/s or 30 tok/s. This page tells you exactly what fits and what to expect.

​
Quick verdict: MoE models dominate

MoE (mixture-of-experts) models are 3–14× faster than dense models of the same total parameter count on Strix Halo. The reason is bandwidth: only the active parameters in an MoE model are read from memory per token, so a 122B MoE model with 10B active parameters generates tokens at nearly the speed of a 10B dense model — not a 122B one. If you are choosing a model for daily use, start with a MoE model in the 35B–122B range.

​
128GB unified memory explained

Strix Halo does not have discrete VRAM. The iGPU draws from the same DDR5 pool as the CPU. Out of the box, the BIOS and OS reserve some of that 128GB:

​
Configure BIOS memory on Linux

On Linux, the kernel manages the split automatically. You typically see 95–110GB available for GPU workloads after boot. No manual BIOS change is needed in most cases.

​
Configure memory on Windows

On Windows, you must set 2 things:
  1. BIOS → UMA Frame Buffer — set to the largest option available (typically “Auto” or a fixed value).
  2. AMD Adrenalin → VRAM allocation — open AMD Software: Adrenalin Edition, go to Settings → Graphics → Advanced, and move the VRAM slider to 96GB.
Without the Adrenalin VRAM allocation on Windows, your iGPU may only see 15.5GB — far too little for large models. Set both the BIOS and the Adrenalin slider before attempting to load anything over 30B parameters.

​
Dense model compatibility

Dense models activate every parameter on every token, so the full weight tensor must be read from memory each step. At 215 GB/s, that creates a hard ceiling on token speed.
Qwen3-Coder-30B at Q4_K_M is the community’s go-to dense model for coding agents (Cline, Aider, opencode) on Strix Halo. It fits easily and delivers 10–15 tok/s.

​
MoE model compatibility

MoE models keep the total parameter count high but only activate a fraction per token. That means far less memory bandwidth per step, and dramatically faster generation. MoE models with 3–12B active parameters are the sweet spot. They give you frontier-class quality at speeds that feel responsive in a chat or coding session.
GLM-4.6 (357B total / 32B active) does not fit on Strix Halo. At Q4_K_M quantization it needs ~178 GB — well over the 128 GB unified memory pool, even with 2-node RDMA clustering (256 GB). For Strix Halo, use GLM-4.5-Air instead. See GLM family for the full variant comparison.

​
Kimi K3 on Strix Halo

Kimi K3 does not fit Strix Halo. The model’s 2.8T total parameters (~50B active per token through Stable LatentMoE) need about 1.4 TB at the native MXFP4 precision. Strix Halo’s full 128 GB pool — roughly 90–110 GB after OS overhead — is about 1.27 TB short of the raw weight floor. Moonshot recommends serving K3 on a supernode with 64 or more accelerators. For the full memory calculation, see the Kimi K3 hardware deep dive. For Strix Halo today, the models that fit are in the tables above — up to ~122B MoE at Q4 on a single node. See what to run instead of Kimi K3 for the best pick at 128 GB.

​
Qwen3.8 on Strix Halo

The Qwen3.8 flagship shipped in August 2026 as Qwen3.8-2.4T-A95B: a Mixture-of-Experts model with 2.4 trillion total parameters and 95B active per token, text-only, released under Qwen’s custom qwen3.8-max license. The generation also includes Qwen3.8-27B (released 2026-08-14, Apache-2.0), a dense vision-language model that does fit Strix Halo — see below.

​
Weight-only memory at each quantization level

Even at the most aggressive quantization, Strix Halo cannot run Qwen3.8-2.4T-A95B. This is a multi-node cluster model that requires at least 5 DGX Sparks (or equivalent) at Q2 — and realistically 10+ nodes at Q4 for reasonable KV cache headroom. For full cluster sizing, see the Qwen3.8 model page.
MoE models load all expert weights into memory even though only a subset activates per token. Qwen3.8-2.4T-A95B activates 95B of its 2.4T, but the 1.2 TB at Q4 is a hard floor regardless. The MoE advantage is in compute (fewer FLOPs per token), not in memory.

​
The Qwen3.8 that fits Strix Halo

Qwen3.8-27B is the single-machine member of the generation: a dense 27B vision-language model released 2026-08-14 under Apache-2.0. The official 30.9 GB FP8 build fits a 96 GB GPU allocation with room to spare, and even the 55.6 GB BF16 release fits — though a Q4_K_M GGUF (~17 GB, not yet published) will be the practical llama.cpp/Vulkan path on this hardware. Full sizing is on the Qwen3.8 model page; tested configurations will be added here once verified.

​
ROCm vs Vulkan backend comparison

Strix Halo’s Radeon 8060S iGPU (gfx1151) can run inference through 2 backends. Each has tradeoffs. Use Vulkan unless you have a specific reason to need ROCm (for example, BF16 precision for a fine-tune evaluation).
See vLLM and alternatives on Strix Halo for runtime options including llama.cpp Vulkan and Ollama.

​
Bandwidth constraints

Strix Halo’s DDR5 memory delivers 215 GB/s theoretical bandwidth. That is the single most important number for understanding performance:
  • Token generation is memory-bandwidth-bound. Each token requires reading the model’s active weights from memory.
  • A 70B dense model at Q4 (~40GB) needs 40GB ÷ 215 GB/s ≈ 186ms per token → ~5 tok/s.
  • A 122B MoE model with 10B active at Q4 (~7GB active weights) needs 7GB ÷ 215 GB/s ≈ 33ms per token → ~30 tok/s.
Prompt processing (prefill) is compute-bound and runs faster relative to token generation. You will notice that long prompts process quickly, then generation settles into the bandwidth-limited rate.

​
Performance expectations by use case

​
Experimental 2-node clustering over RDMA

2 Strix Halo nodes can provide 256 GB of aggregate unified memory through llama.cpp RPC, but this is a proof-of-concept path, not a production configuration.
llama.cpp describes RPC as fragile, insecure, and proof-of-concept. Do not expose an RPC worker to an untrusted network, and do not rely on it for a production endpoint. Network setup, model support, context capacity, and performance all require local validation.
Only pursue this route for controlled experiments when a single node cannot hold the model. DeepSeek V4 Flash local hardware explains why its lossless GGUF can fit in the aggregate pool but remains an experimental Strix Halo deployment.

​
Check your memory usage

Before loading a model, verify how much GPU memory is available:
Subtract OS overhead (~4GB) and leave 2–4GB for KV cache at your target context length. A 70GB model in Q4 leaves roughly 20GB for KV cache on a 96GB allocation — enough for 8K–16K context on most models.

​
Next steps

Strix Halo use case

Set up the Tokios connector on your Strix Halo box for remote access.

vLLM and alternatives on Strix Halo

Runtime options including llama.cpp Vulkan and Ollama.

Quantization guide

Understand GGUF quantization levels and quality tradeoffs.