Mac Studio M5 Ultra buying guide: 256GB vs 512GB for local LLM

​

Mac Studio M5 Ultra configurations

​

What fits on 256GB vs 512GB?

​

256 GB configuration

• Qwen3.5-122B-A10B at Q8 (~122 GB, ~55 tok/s)
• Qwen3 235B at Q4 (~130 GB, ~41 tok/s)
• GLM-4.5-Air (106B MoE) at Q8 (~106 GB, ~60 tok/s)
• DeepSeek V4-Flash-0731 (284B) at native precision (~155 GB, ~25 tok/s)

• DeepSeek V4.1 Flash (763B total) — smallest usable quant is ~246 GiB at Q2_K, no KV headroom
• GLM-5.2 (753B) — needs ~377 GB at Q4, exceeds 256 GB
• Qwen3.5-397B at Q4 (~224 GB) — tight fit, minimal context

​

512 GB configuration

• DeepSeek V4.1 Flash (763B) at Q2_K (~246 GiB, ~10-20 tok/s)
• DeepSeek V4.1 Flash at Q4 (~402 GiB, ~10-15 tok/s)
• GLM-5.2 (753B) at Q4 (~418 GB, ~26 tok/s)
• Qwen3.5-397B at Q4 (~224 GB, ~35 tok/s)
• Qwen3.5-397B at Q8 (~448 GB, ~25 tok/s)
• Multiple 120B models simultaneously

• GLM-5.2 at FP8 (~753 GB) — exceeds 512 GB
• Kimi K3 (2.8T params) — needs ~1.4 TB at Q4, far beyond 512 GB

​

Apple MLX distributed inference

​

What’s possible today

• Runs Muse Spark Open (816B params) at 36.7 tok/s
• Fits DeepSeek V4.1 Flash at FP8 (~511 GB, ~40 tok/s)
• Fits GLM-5.2 at FP8 (~753 GB, ~30 tok/s)

• Runs DeepSeek V4.1 Flash at Q4 (~402 GB, ~20 tok/s)
• Runs GLM-5.2 at Q4 (~418 GB, ~18 tok/s)
• Fits Qwen3.5-397B at FP8 (~448 GB, ~30 tok/s)

​

How it works

• 2+ Mac Studio M5 Ultra (or Mac Pro M5 Ultra)
• Thunderbolt 5 cables (direct connection) or Thunderbolt 5 network switch
• macOS 15.2+ with MLX 0.22+
• Models must support tensor parallelism (most MLX-compatible models do)

# On each Mac Studio
pip install mlx-lm --upgrade

# Launch distributed inference (example: 2x M5 Ultra)
mlx.launch --hosts host1,host2 \
  mlx_lm.generate \
  --model deepseek-ai/DeepSeek-V4.1-Flash-MLX \
  --quantization q4 \
  --tensor-parallel-size 2 \
  --prompt "Explain quantum computing"

​

Comparison to DGX Spark clustering

• You need 512+ GB unified memory in a desktop form factor
• You want silent operation (Mac Studio is near-silent)
• You’re already in the Apple ecosystem (macOS, Xcode, etc.)
• You need to run V4.1 Flash at FP8 (only 4x M5 Ultra fits this)

• You need maximum throughput (4x DGX Spark at 87-125 tok/s vs 2x M5 Ultra at 20 tok/s)
• You want lower cost per GB ($12,000 for 512 GB vs $22,600 for 1 TB)
• You prefer Linux-based serving (vLLM, SGLang)
• You need ConnectX-7 RoCE networking for low-latency clustering

​

Software backend comparison

​

MLX (mlx-lm)

• Pros: Native Apple Silicon optimization, fastest single-user performance, supports latest models first
• Cons: Single-request at a time, no concurrent batching, macOS-only
• Performance: ~50-60 tok/s on 120B models, ~10-20 tok/s on frontier models
• Setup: pip install mlx-lm

mlx_lm.generate \
  --model Qwen/Qwen3.5-122B-A10B-MLX \
  --quantization q8 \
  --prompt "Write a poem about local LLMs"

​

Ollama (with MLX backend)

• Pros: Easiest setup, GUI available, supports MLX backend since v0.19
• Cons: One request at a time, no concurrent batching, slower than raw MLX
• Performance: ~40-50 tok/s on 120B models (10-20% slower than raw MLX)
• Setup: Download from ollama.com, select MLX backend in settings

ollama run qwen3.5:122b-a10b-q8

​

vLLM-MLX

• Pros: Concurrent batching, OpenAI-compatible API, production-grade
• Cons: Newer project, less stable than raw MLX, requires more setup
• Performance: ~35-45 tok/s per concurrent user (with batching overhead)
• Setup: pip install vllm-mlx

vllm serve Qwen/Qwen3.5-122B-A10B-MLX \
  --quantization q8 \
  --max-num-seqs 10 \
  --api-server-host 0.0.0.0

​

gmlx

• Pros: 1.7x faster prompt processing than llama.cpp, optimized for Apple Silicon
• Cons: Newer project, smaller community, limited model support
• Performance: ~60-70 tok/s on 120B models (fastest backend for prompt-heavy workloads)
• Setup: Build from source (see gmlx GitHub)

​

Model recommendations by use case

​

Individual developer (256 GB config)

​

Research / frontier models (512 GB config)

​

Team serving (4x M5 Ultra cluster)

​

Price-performance analysis

​

Cost per GB of unified memory

​

Cost per tok/s (inference throughput)

​

Setup guide

​

Initial setup

1. Unbox and connect
   • Connect Thunderbolt 5 cable (if using distributed inference)
   • Connect power, display, keyboard, mouse
   • Boot into macOS
2. Install prerequisites
   # Install Homebrew (if not already installed)
   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
   
   # Install Python 3.11+
   brew install [email protected]
   
   # Install Xcode command line tools
   xcode-select --install
   
3. Install MLX backend
   pip3 install mlx-lm --upgrade
   
4. Download a model
   # Example: Qwen3.5-122B-A10B at Q8
   huggingface-cli download Qwen/Qwen3.5-122B-A10B-MLX \
     --local-dir ~/models/qwen3.5-122b-q8
   
5. Run inference
   mlx_lm.generate \
     --model ~/models/qwen3.5-122b-q8 \
     --prompt "Explain quantum computing in simple terms"
   

​

Distributed inference setup (2+ Mac Studios)

1. Connect Thunderbolt 5
   • Use direct Thunderbolt 5 cable (up to 2m) or Thunderbolt 5 network switch
   • Verify connection: system_profiler SPThunderboltDataType
2. Configure hosts
   # On each Mac Studio, note the hostname
   hostname
   # Output: MacStudio-1.local, MacStudio-2.local, etc.
   
3. Launch distributed inference
   # On primary Mac Studio
   mlx.launch --hosts MacStudio-1.local,MacStudio-2.local \
     mlx_lm.generate \
     --model deepseek-ai/DeepSeek-V4.1-Flash-MLX \
     --quantization q4 \
     --tensor-parallel-size 2 \
     --prompt "Write a research paper on local LLM inference"
   
4. Monitor performance
   # On each Mac Studio, monitor resource usage
   top -l 1 | grep "CPU usage"
   sudo powermetrics --samplers gpu_power -i 1000
   

​

Next steps