メインコンテンツへスキップ
Find which open-weight models run on your hardware, or which hardware can run your target model. Toggle between “I have this hardware” and “I want to run this model” modes, adjust quantization and context length, and see the memory math for every combination.
Found a model that fits? Register it with Tokios to reach it as https://api.tokios.com from any OpenAI- or Anthropic-compatible client.

Interactive model-fit finder — select hardware or model, adjust quantization and context.

​
How it works

The widget estimates memory requirements using:
A model “fits” if the total estimate is within the hardware’s usable memory: MoE models (like Qwen3.8-2.4T-A95B or DeepSeek-V3) load all expert weights into memory, so total parameter count determines memory usage — even though only the active subset runs per token.

​
The sizing formula

Model weights take a fixed number of bytes per parameter, and that byte count depends entirely on precision: Raw weight size is not the whole story. You also need memory for the KV cache, activations, and framework overhead. The widget above adds a modest KV cache estimate (at the selected context length) plus a 5% overhead factor on weights and KV cache combined:
For quick mental math without the widget, a ×1.2 multiplier on raw weight size alone is a reasonable shortcut — it bakes in a rough KV cache estimate and overhead in one step, and is what the worked examples below use. Worked examples at ~Q4 (4-bit), including the ~1.2 overhead: For reference, higher precision scales the same models far past what a single consumer card can hold: 70B at FP16 is about 140 GB, and 120B at FP16 is about 230 GB. Hugging Face documents a similar pattern across model sizes: for example, about 29 GB at bf16, ~15 GB at 8-bit, and ~9.5 GB at 4-bit for a large model. See the Hugging Face guide to optimizing LLMs and the llama.cpp quantize README for the underlying numbers — GGUF Q4_K_M uses mixed precision that averages out to about 0.5–0.56 bytes per parameter. Ternary weight compression goes further than any standard quantization. Bonsai 2 27B stores every weight as -1, 0, or +1 — roughly 1.58 bits per weight — shrinking Qwen3.8-27B from 54 GB to 5.9 GB. That fits under 6 GB of VRAM on a consumer GPU. The calculator above does not model ternary format directly, since it is a training-time technique rather than a post-training quantization you can apply to any model.
These formulas estimate weight size. The actual number that matters is the GGUF file size listed on Hugging Face for the exact quantization you plan to run, plus the KV cache for your target context length. Always check that before committing to hardware.

​
GPU VRAM at a glance

Sources: NVIDIA GeForce RTX 50 series, NVIDIA professional GPUs, Wikipedia: GeForce RTX 40 series, Wikipedia: GeForce RTX 30 series, AMD Ryzen AI Max, Apple technical specifications. Dedicated GDDR memory (the NVIDIA cards above) is fast and fixed in size — you cannot add more later. Unified memory (Apple Silicon, Strix Halo) shares one pool between CPU and GPU, so it can be much larger, but it runs at lower bandwidth than a dedicated card.

​
What fits where

The table below estimates comfort level at ~Q4 quantization for a modest context length. Treat every entry as an estimate — verify against the actual GGUF file size plus KV cache before you commit. Ground truth is always the GGUF file size on Hugging Face for the quantization you choose, plus the KV cache your context length requires.
Most 30B-class agent models are MoE and need far more than their active-parameter count in memory. Muse Glimmer 30B is dense instead, so the whole model — not just an active slice — fits in 18 GB at 4-bit, comfortably inside the 24 GB tier above.
DeepSeek-V4-Flash-0731 has a roughly 162–167 GB weight floor in the published lossless GGUF and fused-weight formats. It needs a multi-device setup with measured KV-cache headroom; see DeepSeek V4 Flash local hardware for the supported, community-tested, and experimental paths.

Picked a model? Connect it

One authenticated endpoint at https://api.tokios.com — no inbound ports, about 5 minutes.

​
Why unified memory loads big models but runs them slower

Loading a model only requires enough memory capacity to hold the weights. Running it fast requires enough memory bandwidth to keep feeding the compute units, because single-stream decoding is memory-bandwidth-bound — each token generated has to read the model’s weights (or, for a mixture-of-experts model, the active experts) out of memory. Prefill (processing the prompt) is compute-bound instead, so it behaves differently. This is why the two memory types trade off differently:
  • Dedicated GDDR (RTX 4090, 5090, and similar): high bandwidth (roughly 1000–1800 GB/s), but capacity is fixed by the card.
  • Unified memory (Apple Silicon, Strix Halo): much larger capacity (up to 128 GB), but lower bandwidth (roughly 256–546 GB/s).
A 96 GB unified-memory machine can load a 120B model that would never fit on any single consumer GPU. It will still generate tokens more slowly than a dedicated GPU running a model that fits within the GPU’s own VRAM, because it has less bandwidth to move weights through per token. Do not rely on published tok/s numbers from other setups — hardware, driver versions, quantization, context length, and batch size all shift the result. Measure your own deployment with llama-bench and see Test your deployment for a repeatable way to compare deployments once your model is running.

​
Next steps

Get started with Tokios

Once a model fits your hardware, Tokios turns it into a stable API endpoint you can call from anywhere.

Pick the right model for your task

Sizing narrows down what fits. Task fit narrows down which family to load in the first place.