Skip to main content

​
Why model names matter

Browse Hugging Face or Ollama’s model library and you’ll see names like Qwen/Qwen3-235B-A22B-Instruct or bartowski/Llama-3.3-70B-Instruct-GGUF. These aren’t random strings — every segment encodes a decision you’ll need to make before downloading. The name tells you:
  • How big the model is — and therefore how much VRAM or system memory it needs.
  • What it was trained for — general chat, coding, reasoning, vision, or raw text completion.
  • How it was compressed — and what quality you’re trading off against size.
  • Who made it — and whether the weights are original or a third-party repackaging.
Learning to read these conventions takes 5 minutes and saves you from downloading a 70B model that doesn’t fit your GPU, or wasting VRAM on a base model that can’t follow instructions.

​
The anatomy of a model name

Most open-weight model names follow a pattern:
Not every name includes every segment, and publishers vary the separators — some use hyphens, others use slashes or nothing. But you can usually identify these parts: Real examples, parsed segment by segment: Qwen/Qwen3-235B-A22B-Instruct mistralai/Mistral-7B-Instruct-v0.3 TheBloke/Llama-2-7B-Chat-GGUF — TheBloke (third-party quantizer), Llama 2, 7B params, chat-tuned, GGUF format.
When the organization is a third-party quantizer like TheBloke, bartowski, or unsloth, the model itself was trained by someone else — the repo contains a repackaged or quantized version of the original weights.

​
Size labels decoded

The number before B is the parameter count in billions. It’s the single biggest driver of both capability and memory requirements.

​
MoE models: two numbers

MoE (mixture-of-experts) models list both total and active parameters, written as 235B-A22B or 30B-A3B:
  • The first number is the total parameter count — what you need to store in memory or on disk.
  • The second number (after A) is the active parameter count — how many parameters run per token.
A 235B-A22B model needs enough memory to hold 235B parameters, but only computes with 22B at a time, so it generates tokens much faster than a dense 235B model. The trade-off: MoE models are larger on disk and in memory than a dense model of the same active size, but faster at inference.
A 235B-A22B MoE model roughly matches a dense 22B model in per-token speed, but you still need VRAM or system RAM to store all 235B parameters. At Q4_K_M, that’s roughly 120 GB — far more than a dense 22B model’s 13 GB.

​
How parameter count maps to capability

More parameters generally means more capability, but the relationship isn’t linear. A 70B model isn’t 10x better than a 7B model — it’s noticeably better at reasoning, nuance, and complex tasks, but a well-tuned 7B model can still handle everyday chat and simple coding tasks well. The right size depends on your hardware and your task, not just a bigger-is-better rule.

​
Architecture suffixes

Publishers append suffixes to signal architecture variants beyond the base model. These affect what the model can do and what hardware it needs.
A dense model uses every parameter on every token. An MoE (mixture-of-experts) model routes each token through a subset of “expert” subnetworks. MoE models are larger in total parameters but faster per token because only the active parameters compute. If a model name includes two size numbers (like 235B-A22B), it’s MoE.
Models labeled -VL accept images or video in addition to text. Examples: Qwen3-VL-32B-Instruct, gemma-4-12b-it. VL models need extra VRAM for the vision encoder on top of the language model weights.
Models labeled -Coder are fine-tuned or trained primarily on code. They typically outperform general models of the same size on programming tasks but may be weaker at open-ended conversation. Example: Qwen3-Coder-480B-A35B-Instruct.
Models with -Math or reasoning-specific labels are trained on mathematical problem-solving and chain-of-thought data. They’re stronger at step-by-step logic than general models of the same size but narrower in scope.
Some publishers tag experimental releases as next, preview, or experimental. These may have incomplete training or untested chat templates — useful for evaluation, not production.

​
Training stage labels

The training stage tells you whether the model can hold a conversation or just complete text. This is the single most important segment for deciding whether a model is usable out of the box. Instruct and Chat are interchangeable for most purposes — both mean the model was fine-tuned to respond to user prompts. Chat is the older label (Llama 2 used it); Instruct is more common in newer families. Some publishers like Mistral use neither and instead rely on version tags (v0.3) alongside the base name.
For local use, you almost always want Instruct or Chat. A Base model can’t follow instructions — it only completes text — and will give confusing outputs if you prompt it like a chatbot.
If you see a model with no training stage label and no Base tag, check the model card before assuming it’s instruction-tuned. Some publishers release fine-tuned models without labeling them, but others release base models without labeling them either.

​
Quantization labels (the GGUF world)

GGUF quantization labels follow a naming scheme documented in the llama.cpp quantize README. The Q number is roughly how many bits per weight; higher means larger file and higher quality. K-quants add mixed precision — different tensors get different bit-widths depending on sensitivity — and the trailing S/M/L marks small/medium/large variants within that level. Here’s the full range you’ll encounter, with practical guidance:

​
Practical sweet spots

  • Q4_K_M — Start here. Good quality, fits consumer hardware.
  • Q5_K_M — Step up when your VRAM has headroom. Near-lossless for most tasks.
  • Q8_0 — Minimal loss. Use when quality is critical and memory is available.
  • Q3_K_M — Drop to this only to fit a larger model that won’t load at Q4_K_M. Expect a quality hit.

​
Size comparison: a 7B model at each quant

The file size of a GGUF quant is your ground-truth VRAM number for the weights. Add KV cache on top (see KV cache and context) for the total memory requirement.

​
I-quants (imatrix) vs K-quants

You may also see quant names starting with IQ — like IQ4_XS or IQ3_XXS. These are imatrix quants, built using an importance matrix: calibration data that tells the quantizer which weights matter most, so it can preserve them more carefully. I-quants can produce better quality than K-quants at the same bit-width, particularly at very low bit-widths, at the cost of a slower quantization process. If a repo offers both Q4_K_M and IQ4_M, the I-quant may be slightly better — but the difference is usually small enough that either works. For a deeper explanation of how quantization works and why K-quants outperform simple quants, see Quantization.

​
Other common suffixes

Beyond quantization labels, model names include format and precision tags that tell you which backend can load the file.
The format used by llama.cpp-based backends — Ollama, LM Studio, and any llama.cpp server. GGUF files bundle the weights, tokenizer, and chat template into 1 file. Most consumer hardware runs GGUF.
A GPU quantization format (typically 4-bit or 8-bit) that runs on NVIDIA GPUs via AutoGPTQ or vLLM. Faster than GGUF on GPU but NVIDIA-only and not compatible with llama.cpp backends.
Activation-aware weight quantization — another GPU format similar to GPTQ but calibrated using activation statistics. Supported by vLLM and some other inference engines. NVIDIA GPUs.
The ExLlamaV2 format — supports mixed-ratio quantization where different layers use different bit-widths. Optimized for NVIDIA GPUs and the ExLlamaV2 inference engine.
Full 16-bit precision. FP16 (float16) and BF16 (bfloat16) are both 2 bytes per parameter. BF16 has a wider exponent range and is more numerically stable during training; for inference, both behave similarly. Requires vLLM or similar engines.
8-bit floating point — half the size of FP16 with minimal quality loss. Used by inference engines like vLLM and SGLang on GPUs that support FP8 (NVIDIA Ada Lovelace and newer, AMD MI300).
NVIDIA’s 4-bit floating-point format, available on Blackwell GPUs including DGX Spark. Offers aggressive size reduction with hardware-native support — see DGX Spark model compatibility.

​
Quantization from the publisher vs third-party

Multiple parties often publish quantized versions of the same model:
  • Publisher quants — Qwen, Mistral, Google, and others increasingly ship their own GGUF files alongside full-precision weights. These are usually the safest choice because the publisher controls the original weights and chat template.
  • Unsloth — Ships GGUF quants with optimized chat templates and dynamic quantization. Also offers 1.58-bit (unsloth/...-1.58bit) variants that are extremely small. Generally reliable and widely used.
  • TheBloke — Prolific early quantizer whose repos helped define the GGUF distribution ecosystem. Well-organized but may lag behind the newest models or miss chat template updates for recent releases.
  • bartowski — Active community quantizer with consistent naming and good coverage of new releases. Often among the first to publish GGUF variants when a new model drops.

​
How to verify a third-party quant

  1. Check that the repo links back to the original model.
  2. Compare the file size to what you’d expect at that quant level (the table above is a reference).
  3. Look for the chat template metadata — a missing or wrong template degrades output silently.
  4. If perplexity benchmarks are published for the quant, compare them to the publisher’s own numbers.

​
Quick decision guide

Match your hardware to a model size and quantization level:

8 GB VRAM

7B Q4_K_M or 14B Q3_K_M. Stick with smaller models at comfortable quants — a 7B at Q4_K_M outperforms a 14B at Q2_K.

16 GB VRAM

14B Q5_K_M or 32B Q4_K_M. The 14B at Q5 is near-lossless; the 32B at Q4 trades some quality for significantly more capability.

24 GB VRAM

32B Q5_K_M or 70B Q4_K_M. The 32B at Q5 is the sweet spot for single-GPU quality. A 70B at Q4 pushes capability higher but fills most of the VRAM.

48 GB+

70B Q5_K_M or MoE models like 235B-A22B Q4_K_M with offloading. You have room for high-quality quants of the largest models.

128 GB (DGX Spark)

120B+ MoE models at FP8 or NVFP4. Full precision or near-full precision of frontier-scale models. See DGX Spark model compatibility.

CPU only

3B Q4_K_M or 7B Q3_K_M. CPU inference is slow — keep the model small. Expect 1–5 tokens per second.
These are starting points. Your actual VRAM usage also depends on context length (KV cache grows with context), batch size, and whether the backend offloads layers to system RAM. Use the model fit guide for precise sizing.

​
Naming red flags

Not every model name follows conventions, and some patterns warrant caution:
  • No size label — A model name without a parameter count could be anything. Check the model card before downloading.
  • “Uncensored” variants — Usually just safety training removed, not a capability improvement. Quality of the base model may be unchanged or worse if the removal broke alignment.
  • Merged models — Combine weights from 2 or more models (for example, merging a coding model with a chat model). Quality varies wildly and is hard to predict from the name alone. Look for the merge method (SLERP, DARE, TIES) and whether evaluation results are published.
  • Extremely low quants — Q2_K is almost always too lossy for anything beyond testing whether a model loads. Expect degraded reasoning, broken formatting, and hallucinations. If a model only ships Q2 quants, look for a different model at the same parameter count.
  • Version numbers without changelogs — v0.1 through v0.9 with no clear description of what changed between versions. Use the latest unless you have a specific reason not to.
  • “AWSD” or similar hype tags — Some repos add attention-grabbing suffixes that don’t correspond to any standard technique. If you can’t find the term elsewhere, treat it skeptically.

​
Putting it all together

Take unsloth/Qwen3-30B-A3B-Instruct-GGUF. Reading left to right:
  1. unsloth — quantized by Unsloth (third-party quantizer).
  2. Qwen3 — Qwen3 architecture, current generation from Alibaba.
  3. 30B-A3B — MoE model: 30B total parameters, 3B active per token.
  4. Instruct — instruction-tuned, ready for chat.
  5. GGUF — llama.cpp format, runs on Ollama, LM Studio, or any llama.cpp backend.
From the name alone you know: this is a small MoE model that runs fast, fits in 8 GB of VRAM at Q4_K_M, and can follow instructions. You don’t need to open the model card to decide whether to try it. Now compare meta-llama/Llama-3.3-70B-Instruct. That’s Meta’s Llama 3.3, 70B dense parameters, instruction-tuned — no GGUF tag means you’re looking at full-precision weights that need vLLM or a similar engine. The name tells you it’s a large, high-quality model that requires serious hardware.

​
Next: read the model card

Now that you can decode the name, the model card tells you everything the name leaves out — context length, license, chat template, and intended use.

Read a model card

Find the parameters, architecture, context length, and license that matter before you download.

Quantization

How quantization works and why K-quants outperform simple quants.

Model fit guide

Check whether a model-and-quant combination fits your hardware.