> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Mac Studio M5 Ultra buying guide: 256GB vs 512GB for local LLM

> Should you buy the 256GB or 512GB Mac Studio M5 Ultra for local LLM inference? What models fit, Apple MLX distributed inference, and comparison to DGX Spark.

The Mac Studio M5 Ultra is the largest unified memory desktop available — up to 512 GB shared between CPU and GPU. The 256GB vs 512GB decision is a \$3,000+ upgrade that determines whether you can run frontier models like DeepSeek V4.1 Flash (763B params). This guide breaks down what fits, what doesn't, and when to choose Mac Studio over DGX Spark or RTX PRO.

> [!NOTE]
>
> Last verified: 2026-10-21. Apple Silicon performance and model compatibility evolve rapidly with each MLX release. Check [Apple MLX documentation](https://github.com/ml-explore/mlx) for the latest backend updates.

## Mac Studio M5 Ultra configurations

| Config | Memory | Bandwidth | Price (MSRP) | Best for |
| - | - | - | - | - |
| **M5 Ultra 256 GB** | 256 GB unified | 1,200 GB/s | \~\$8,500 | 120B models at Q8, 200B models at Q4 |
| **M5 Ultra 512 GB** | 512 GB unified | 1,200 GB/s | \~\$11,300+ | Frontier models (V4.1 Flash, GLM-5.2) at Q2-Q4 |

Both configurations share the same M5 Ultra chip — the only difference is memory capacity. The 512GB variant costs \~\$2,800 more, which buys you 256 GB of additional unified memory.

> [!TIP]
>
> **Memory is soldered — you cannot upgrade later.** Buy the configuration you need at purchase. There is no SO-DIMM slot or expansion option.

## What fits on 256GB vs 512GB?

The 256GB vs 512GB decision comes down to whether you need to run frontier-scale models (200B+ params) or whether 120B models at high precision are sufficient.

### 256 GB configuration

**Fits comfortably:**

* Qwen3.5-122B-A10B at Q8 (\~122 GB, \~55 tok/s)
* Qwen3 235B at Q4 (\~130 GB, \~41 tok/s)
* GLM-4.5-Air (106B MoE) at Q8 (\~106 GB, \~60 tok/s)
* DeepSeek V4-Flash-0731 (284B) at native precision (\~155 GB, \~25 tok/s)

**Does NOT fit:**

* DeepSeek V4.1 Flash (763B total) — smallest usable quant is \~246 GiB at Q2\_K, no KV headroom
* GLM-5.2 (753B) — needs \~377 GB at Q4, exceeds 256 GB
* Qwen3.5-397B at Q4 (\~224 GB) — tight fit, minimal context

**Best use case:** 120B models at Q8 (highest quality) or 200B+ models at Q4 (good quality). Ideal for individual developers running large but not frontier-scale models.

### 512 GB configuration

**Fits comfortably:**

* DeepSeek V4.1 Flash (763B) at Q2\_K (\~246 GiB, \~10-20 tok/s)
* DeepSeek V4.1 Flash at Q4 (\~402 GiB, \~10-15 tok/s)
* GLM-5.2 (753B) at Q4 (\~418 GB, \~26 tok/s)
* Qwen3.5-397B at Q4 (\~224 GB, \~35 tok/s)
* Qwen3.5-397B at Q8 (\~448 GB, \~25 tok/s)
* Multiple 120B models simultaneously

**Does NOT fit:**

* GLM-5.2 at FP8 (\~753 GB) — exceeds 512 GB
* Kimi K3 (2.8T params) — needs \~1.4 TB at Q4, far beyond 512 GB

**Best use case:** Frontier models (V4.1 Flash, GLM-5.2) at practical quantization. The 512GB config unlocks the largest open-weight models that fit in a desktop form factor.

> [!WARNING]
>
> **Frontier models are slow on 512GB.** DeepSeek V4.1 Flash at Q4 runs \~10-15 tok/s on a single M5 Ultra — usable for interactive chat but not high-throughput serving. For faster inference, you need multi-GPU (2x DGX Spark at 38-73 tok/s) or multi-Mac Studio (Apple MLX distributed inference).

## Apple MLX distributed inference

As of October 2026, Apple MLX supports distributed inference over Thunderbolt 5 RDMA, allowing multiple Mac Studios to pool their unified memory.

### What's possible today

**4x Mac Studio M5 Ultra (512 GB each) = 2 TB total memory**

* Runs Muse Spark Open (816B params) at 36.7 tok/s
* Fits DeepSeek V4.1 Flash at FP8 (\~511 GB, \~40 tok/s)
* Fits GLM-5.2 at FP8 (\~753 GB, \~30 tok/s)

**2x Mac Studio M5 Ultra (512 GB each) = 1 TB total memory**

* Runs DeepSeek V4.1 Flash at Q4 (\~402 GB, \~20 tok/s)
* Runs GLM-5.2 at Q4 (\~418 GB, \~18 tok/s)
* Fits Qwen3.5-397B at FP8 (\~448 GB, \~30 tok/s)

### How it works

Apple MLX distributed inference uses Thunderbolt 5's 128 Gbps (16 GB/s) bidirectional bandwidth with RDMA (Remote Direct Memory Access) for low-latency tensor parallelism.

**Requirements:**

* 2+ Mac Studio M5 Ultra (or Mac Pro M5 Ultra)
* Thunderbolt 5 cables (direct connection) or Thunderbolt 5 network switch
* macOS 15.2+ with MLX 0.22+
* Models must support tensor parallelism (most MLX-compatible models do)

**Setup:**

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
# On each Mac Studio
pip install mlx-lm --upgrade

# Launch distributed inference (example: 2x M5 Ultra)
mlx.launch --hosts host1,host2 \
  mlx_lm.generate \
  --model deepseek-ai/DeepSeek-V4.1-Flash-MLX \
  --quantization q4 \
  --tensor-parallel-size 2 \
  --prompt "Explain quantum computing"
```

> [!TIP]
>
> **Thunderbolt 5 RDMA is production-ready.** Community benchmarks from Context Studios and presenc.ai show stable 30-40 tok/s on 4x M5 Ultra clusters. This is no longer experimental — it's a viable path to 1-2 TB unified memory for frontier models.

### Comparison to DGX Spark clustering

| Feature | 2x Mac Studio M5 Ultra (512 GB) | 2x DGX Spark (256 GB) | 4x DGX Spark (512 GB) |
| - | - | - | - |
| Total memory | 1,024 GB | 256 GB | 512 GB |
| Memory bandwidth | 2,400 GB/s (2x 1,200) | 546 GB/s (2x 273) | 1,092 GB/s (4x 273) |
| Interconnect | Thunderbolt 5 RDMA (16 GB/s) | ConnectX-7 RoCE (25 GB/s) | ConnectX-7 RoCE (25 GB/s) |
| V4.1 Flash at Q4 | \~20 tok/s | Does not fit | \~87-125 tok/s |
| V4.1 Flash at FP8 | \~40 tok/s | Does not fit | Does not fit |
| Price | \~\$22,600 | \~\$6,000 | \~\$12,000 |
| Power | 300W (2x 150W) | 400W (2x 200W) | 800W (4x 200W) |
| Form factor | Desktop | Small desktop | 4x small desktops |

**When to choose Mac Studio:**

* You need 512+ GB unified memory in a desktop form factor
* You want silent operation (Mac Studio is near-silent)
* You're already in the Apple ecosystem (macOS, Xcode, etc.)
* You need to run V4.1 Flash at FP8 (only 4x M5 Ultra fits this)

**When to choose DGX Spark:**

* You need maximum throughput (4x DGX Spark at 87-125 tok/s vs 2x M5 Ultra at 20 tok/s)
* You want lower cost per GB ($12,000 for 512 GB vs $22,600 for 1 TB)
* You prefer Linux-based serving (vLLM, SGLang)
* You need ConnectX-7 RoCE networking for low-latency clustering

## Software backend comparison

Mac Studio supports 4 primary inference backends. Each has different trade-offs for single-user interactive use vs multi-user API serving.

### MLX (mlx-lm)

**Best for:** Single-user interactive inference, latest model support

* **Pros:** Native Apple Silicon optimization, fastest single-user performance, supports latest models first
* **Cons:** Single-request at a time, no concurrent batching, macOS-only
* **Performance:** \~50-60 tok/s on 120B models, \~10-20 tok/s on frontier models
* **Setup:** `pip install mlx-lm`

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
mlx_lm.generate \
  --model Qwen/Qwen3.5-122B-A10B-MLX \
  --quantization q8 \
  --prompt "Write a poem about local LLMs"
```

### Ollama (with MLX backend)

**Best for:** Simple setup, one request at a time, desktop use

* **Pros:** Easiest setup, GUI available, supports MLX backend since v0.19
* **Cons:** One request at a time, no concurrent batching, slower than raw MLX
* **Performance:** \~40-50 tok/s on 120B models (10-20% slower than raw MLX)
* **Setup:** Download from [ollama.com](https://ollama.com), select MLX backend in settings

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
ollama run qwen3.5:122b-a10b-q8
```

### vLLM-MLX

**Best for:** Multi-user API serving, concurrent batching

* **Pros:** Concurrent batching, OpenAI-compatible API, production-grade
* **Cons:** Newer project, less stable than raw MLX, requires more setup
* **Performance:** \~35-45 tok/s per concurrent user (with batching overhead)
* **Setup:** `pip install vllm-mlx`

```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
vllm serve Qwen/Qwen3.5-122B-A10B-MLX \
  --quantization q8 \
  --max-num-seqs 10 \
  --api-server-host 0.0.0.0
```

### gmlx

**Best for:** Maximum throughput, prompt processing

* **Pros:** 1.7x faster prompt processing than llama.cpp, optimized for Apple Silicon
* **Cons:** Newer project, smaller community, limited model support
* **Performance:** \~60-70 tok/s on 120B models (fastest backend for prompt-heavy workloads)
* **Setup:** Build from source (see [gmlx GitHub](https://github.com/example/gmlx))

> [!TIP]
>
> **For most users, start with Ollama + MLX backend.** It's the simplest setup and good enough for single-user interactive use. Upgrade to vLLM-MLX when you need concurrent API serving.

## Model recommendations by use case

### Individual developer (256 GB config)

| Use case | Recommended model | Quantization | Memory | tok/s |
| - | - | - | - | - |
| Coding assistant | Qwen3-Coder-32B | Q8 | \~32 GB | \~70 |
| General chat | Qwen3.5-122B-A10B | Q8 | \~122 GB | \~55 |
| Reasoning | GLM-4.5-Air (106B MoE) | Q8 | \~106 GB | \~60 |
| Creative writing | Qwen3 70B | Q8 | \~70 GB | \~65 |
| Frontier (slow) | DeepSeek V4-Flash-0731 | Native | \~155 GB | \~25 |

### Research / frontier models (512 GB config)

| Use case | Recommended model | Quantization | Memory | tok/s |
| - | - | - | - | - |
| Frontier reasoning | DeepSeek V4.1 Flash | Q4 | \~402 GB | \~10-15 |
| Frontier reasoning (faster) | DeepSeek V4.1 Flash | Q2\_K | \~246 GiB | \~15-20 |
| Large MoE | GLM-5.2 (753B) | Q4 | \~418 GB | \~26 |
| Large dense | Qwen3.5-397B | Q8 | \~448 GB | \~25 |
| Multi-model serving | Multiple 120B models | Q4-Q8 | \~100-200 GB each | \~30-55 each |

### Team serving (4x M5 Ultra cluster)

| Use case | Recommended model | Quantization | Memory | tok/s |
| - | - | - | - | - |
| Frontier API | DeepSeek V4.1 Flash | FP8 | \~511 GB | \~40 |
| Large MoE API | GLM-5.2 | FP8 | \~753 GB | \~30 |
| Multi-tenant | Multiple 120B models | Q8 | \~120 GB each | \~55 each |

## Price-performance analysis

Is the Mac Studio M5 Ultra worth the premium over DGX Spark or RTX PRO?

### Cost per GB of unified memory

| Platform | Memory | Price | \$/GB | Notes |
| - | - | - | - | - |
| Mac Studio M5 Ultra 256 GB | 256 GB | \~\$8,500 | \$33/GB | Desktop form factor, silent |
| Mac Studio M5 Ultra 512 GB | 512 GB | \~\$11,300 | \$22/GB | Best value for large memory |
| 2x DGX Spark | 256 GB | \~\$6,000 | \$23/GB | Linux-based, higher throughput |
| 4x DGX Spark | 512 GB | \~\$12,000 | \$24/GB | Cluster, 87-125 tok/s on V4.1 Flash |
| RTX PRO 6000 | 96 GB | \~\$16,000 | \$167/GB | Professional GPU, NVLink |
| 4x RTX PRO 6000 Max-Q | 384 GB | \~\$34,000 | \$89/GB | Density-optimized |

**Mac Studio is 4-7x cheaper per GB than RTX PRO**, but RTX PRO has higher bandwidth (1,792 GB/s vs 1,200 GB/s) and supports NVLink for multi-GPU.

### Cost per tok/s (inference throughput)

| Platform | Model | tok/s | Price | \$/tok/s |
| - | - | - | - | - |
| Mac Studio M5 Ultra 512 GB | V4.1 Flash Q4 | \~15 | \~\$11,300 | \$753 |
| 2x DGX Spark | V4.1 Flash mixed | \~55 (avg) | \~\$6,000 | \$109 |
| 4x DGX Spark | V4.1 Flash NVFP4 | \~106 (avg) | \~\$12,000 | \$113 |
| 2x RTX PRO 6000 | V4-Flash-0731 FP8 | \~158 (avg) | \~\$32,000 | \$203 |
| 4x M5 Ultra cluster | V4.1 Flash FP8 | \~40 | \~\$45,200 | \$1,130 |

**DGX Spark is 5-7x cheaper per tok/s than Mac Studio** for frontier models. Mac Studio wins on silence, form factor, and ecosystem integration.

> [!TIP]
>
> **Choose Mac Studio if:** You value silence, desktop form factor, and Apple ecosystem integration over raw throughput.
>
> **Choose DGX Spark if:** You want maximum tok/s per dollar and prefer Linux-based serving.
>
> **Choose RTX PRO if:** You need professional features (ECC, ISV certs, warranty) and NVLink for multi-GPU.

## Setup guide

### Initial setup

1. **Unbox and connect**
   * Connect Thunderbolt 5 cable (if using distributed inference)
   * Connect power, display, keyboard, mouse
   * Boot into macOS

2. **Install prerequisites**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   # Install Homebrew (if not already installed)
   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

   # Install Python 3.11+
   brew install python@3.11

   # Install Xcode command line tools
   xcode-select --install
   ```

3. **Install MLX backend**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   pip3 install mlx-lm --upgrade
   ```

4. **Download a model**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   # Example: Qwen3.5-122B-A10B at Q8
   huggingface-cli download Qwen/Qwen3.5-122B-A10B-MLX \
     --local-dir ~/models/qwen3.5-122b-q8
   ```

5. **Run inference**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   mlx_lm.generate \
     --model ~/models/qwen3.5-122b-q8 \
     --prompt "Explain quantum computing in simple terms"
   ```

### Distributed inference setup (2+ Mac Studios)

1. **Connect Thunderbolt 5**
   * Use direct Thunderbolt 5 cable (up to 2m) or Thunderbolt 5 network switch
   * Verify connection: `system_profiler SPThunderboltDataType`

2. **Configure hosts**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   # On each Mac Studio, note the hostname
   hostname
   # Output: MacStudio-1.local, MacStudio-2.local, etc.
   ```

3. **Launch distributed inference**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   # On primary Mac Studio
   mlx.launch --hosts MacStudio-1.local,MacStudio-2.local \
     mlx_lm.generate \
     --model deepseek-ai/DeepSeek-V4.1-Flash-MLX \
     --quantization q4 \
     --tensor-parallel-size 2 \
     --prompt "Write a research paper on local LLM inference"
   ```

4. **Monitor performance**
   ```bash theme={"theme":{"light":"github-light","dark":"css-variables"}}
   # On each Mac Studio, monitor resource usage
   top -l 1 | grep "CPU usage"
   sudo powermetrics --samplers gpu_power -i 1000
   ```

## Next steps

> **[Hardware reference](/guides/hardware-reference.md)**
>
> Compare Mac Studio against DGX Spark, RTX PRO, and budget hacker tier.

> **[DGX Spark model compatibility](/guides/dgx-spark-model-compatibility.md)**
>
> See which models fit on DGX Spark's 128 GB unified memory (alternative to Mac Studio).

> **[DeepSeek V4.1 Flash on 2 DGX Sparks](/blog/deepseek-v41-flash-dgx-spark.md)**
>
> Community benchmarks for V4.1 Flash on DGX Spark cluster (higher throughput than Mac Studio).

> **[Model fit](/guides/model-fit.md)**
>
> Calculate which model sizes and quantizations fit your Mac Studio's unified memory.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.