AMD Strix Halo model compatibility: what fits on 128GB unified

​

Hardware overview

​

What fits (and what doesn’t)

​

128 GB configuration (Ryzen AI Max+ 395)

• Qwen3.5-122B-A10B at Q4_K_M (~70 GB, 10-18 tok/s)
• GPT-OSS 120B at Q4 (~59 GB, 35-51 tok/s)
• GLM-4.5-Air at Q6_K_XL (~60 GB, 10-15 tok/s)
• Qwen3.6-35B-A3B at Q4 (~20 GB, 54+ tok/s)
• Qwen3 70B at Q4 (~40 GB, 12-20 tok/s)

• DeepSeek V4.1 Flash (763B) — smallest GGUF is ~233 GB, nearly 2x the 128GB
• DeepSeek V4-Flash-0731 (284B) — needs ~155 GB, exceeds 128GB
• GLM-5.2 (753B) — needs ~377 GB at Q4, far beyond 128GB
• Qwen3.5-397B at Q4 (~224 GB) — exceeds 128GB

​

192 GB configuration (Ryzen AI Max+ Pro 495)

• Everything that fits on 128GB, plus:
• Qwen3 235B at Q3_K_M (~130 GB, 8-12 tok/s)
• DeepSeek V4-Flash-0731 (284B) at Q3 (~130 GB, 10-15 tok/s) — tight fit
• Multiple 70B models simultaneously

• DeepSeek V4.1 Flash (763B) — still too large
• GLM-5.2 (753B) — still too large
• Qwen3.5-397B at Q4 (~224 GB) — exceeds 192GB

​

Backend decision: Vulkan vs ROCm

​

Vulkan (recommended for most users)

• Pros: Works out of the box, no special drivers needed, broad model support via llama.cpp
• Cons: 10-20% slower than ROCm (when ROCm works), no native AMD tensor core acceleration
• Performance: ~10-18 tok/s on 120B MoE models
• Setup: Use llama.cpp with Vulkan backend

# Install llama.cpp with Vulkan support
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make GGML_VULKAN=1

# Run a model
./llama-cli -m qwen3.5-122b-a10b-q4_k_m.gguf \
  -p "Explain quantum computing" \
  -n 128 \
  --gpu-layers 999

• You want a stable, supported path
• You’re running GGUF quantization models
• You don’t want to deal with ROCm driver issues
• You’re okay with 10-20% lower performance

​

ROCm 7.2 (for advanced users)

• Pros: 20-30% faster than Vulkan (when it works), native RDNA 3.5 acceleration
• Cons: Requires HSA_OVERRIDE_GFX_VERSION=11.5.1 environment variable, limited model support, driver issues
• Performance: ~15-25 tok/s on 120B MoE models (when working)
• Setup: Use vLLM or llama.cpp with ROCm backend

# Install ROCm 7.2
# (Follow AMD's official ROCm installation guide for your Linux distro)

# Set the override for Strix Halo
export HSA_OVERRIDE_GFX_VERSION=11.5.1

# Run with vLLM
vllm serve Qwen/Qwen3.5-122B-A10B \
  --quantization awq \
  --max-model-len 8192

• You need maximum performance and accept the tinkering
• You’re running AWQ or FP8 quantization (better supported in ROCm)
• You’re comfortable debugging driver issues
• You’re on Linux (ROCm has limited Windows support)

​

Model recommendations by use case

​

Individual developer (128 GB config)

​

Research / power user (192 GB config)

​

Maximum performance (ROCm path)

​

System vendors

​

Mini-PCs

​

Desktops

​

Laptops

​

Performance benchmarks

​

Single-user inference (Vulkan backend)

​

Comparison to other platforms

​

Setup guide

​

Linux (recommended)

1. Install Ubuntu 22.04 or 24.04
   • Use a USB installer (Rufus or balenaEtcher)
   • Select “Install alongside Windows” for dual-boot
2. Install prerequisites
   sudo apt update
   sudo apt install build-essential git cmake wget
   
3. Build llama.cpp with Vulkan support
   git clone https://github.com/ggerganov/llama.cpp
   cd llama.cpp
   make GGML_VULKAN=1
   
4. Download a model
   # Example: Qwen3.5-122B-A10B at Q4_K_M
   wget https://huggingface.co/Qwen/Qwen3.5-122B-A10B-GGUF/resolve/main/qwen3.5-122b-a10b-q4_k_m.gguf
   
5. Run inference
   ./llama-cli -m qwen3.5-122b-a10b-q4_k_m.gguf \
     -p "Explain quantum computing in simple terms" \
     -n 128 \
     --gpu-layers 999
   

​

Windows

1. Install Ollama
   • Download from ollama.com
   • Ollama automatically detects Vulkan-capable GPUs
2. Pull a model
   ollama pull qwen3.5:122b-a10b-q4
   
3. Run inference
   ollama run qwen3.5:122b-a10b-q4
   

​

ROCm setup (advanced)

1. Install ROCm 7.2
   # Follow AMD's official guide: https://rocm.docs.amd.com/
   wget https://repo.radeon.com/rocm/rocm-7.2.0/rocm-7.2.0.tar.gz
   tar -xzf rocm-7.2.0.tar.gz
   cd rocm-7.2.0
   sudo ./install.sh
   
2. Set environment variable
   echo 'export HSA_OVERRIDE_GFX_VERSION=11.5.1' >> ~/.bashrc
   source ~/.bashrc
   
3. Build llama.cpp with ROCm support
   git clone https://github.com/ggerganov/llama.cpp
   cd llama.cpp
   make GGML_HIP=1
   
4. Run inference
   ./llama-cli -m qwen3.5-122b-a10b-q4_k_m.gguf \
     -p "Explain quantum computing" \
     -n 128 \
     --gpu-layers 999
   

​

Common issues and troubleshooting

​

Issue: Vulkan not detected

# Install Vulkan drivers
sudo apt install mesa-vulkan-drivers

# Rebuild llama.cpp with Vulkan
cd llama.cpp
make clean
make GGML_VULKAN=1

# Verify Vulkan is working
./llama-cli --list-devices
# Should show your AMD GPU

​

Issue: ROCm crashes on certain models

• Switch to Vulkan backend (recommended)
• Try a different model architecture (some work better than others)
• Update to latest ROCm version
• Check community forums for model-specific workarounds

​

Issue: Slow performance on 120B models

• Use smaller models (70B instead of 120B)
• Use MoE models with fewer active parameters (Qwen3.6-35B-A3B has 3B active)
• Accept the performance — Strix Halo is not designed for maximum throughput
• Upgrade to DGX Spark or Mac Studio for 2-4x faster inference

​

Issue: Out of memory on large models

• Use lower quantization (Q4 instead of Q8)
• Reduce --ctx-size to limit KV cache
• Use a smaller model
• Upgrade to 192GB Pro 495 configuration

​

Is Strix Halo right for you?

• You need 128GB unified memory in a compact form factor
• You want a quiet, low-power system (15-54W TDP)
• You’re running 70B-120B models at Q4-Q6
• You prefer Linux or are comfortable with Vulkan backend
• Budget is $2,500-4,300

• You need to run frontier models (V4.1 Flash, GLM-5.2) — they don’t fit
• You need maximum throughput — DGX Spark is 2x faster
• You want plug-and-play simplicity — Mac Studio is easier
• You need CUDA support — Strix Halo is AMD-only
• You’re on a tight budget — used RTX 3090 (24GB) is $700-900

​

Next steps