Skip to main content
A model card is the README.md on a Hugging Face model repo, and it’s where you should look before downloading anything. Most cards are long — this page tells you which fields matter for running the model locally and which you can skim past. Hugging Face’s own guidance on what a card should contain is in the Hub model cards docs. In practice, a handful of fields do most of the work when you’re deciding whether a model fits your hardware.

​
Parameter count

Usually in the model name itself (7B, 30B) or stated near the top of the card. This is the single biggest driver of memory requirements — see Memory and offloading for how parameter count turns into a VRAM number.

​
Architecture: dense vs. mixture-of-experts

Check whether the card describes the model as dense or mixture-of-experts (MoE). A dense model runs every parameter on every token. An MoE model — named like 30B-A3B, meaning 30B total parameters with 3B active — only runs a subset of “expert” parameters per token, so it decodes faster than a dense model of the same total size. The catch: you still need enough memory to hold all the parameters, not just the active ones, because any token might route to a different expert. Look for both numbers on the card: total parameters (what you need to store) and active parameters (what drives per-token speed).

​
Context length

The maximum number of tokens the model can hold in a single conversation, usually listed as something like “128K context” or max_position_embeddings in the config. This matters twice over: it caps how much you can send the model in one go, and it directly sets your ceiling on KV-cache memory. See KV cache and context for why longer context costs more VRAM.
A model’s maximum advertised context and its reliable context aren’t always the same thing. Some models degrade in recall well before they hit their stated limit. If long context matters for your use case, verify it rather than trusting the headline number.

​
License

Check the license field before you build anything on top of a model — it determines whether you can use outputs commercially, redistribute fine-tunes, or run it in a product at all. Licenses vary widely between families: some are fully permissive (Apache 2.0, MIT), others carry usage restrictions or require attribution. This is metadata Hugging Face surfaces directly on the card per its model cards documentation.

​
Files and repos: safetensors vs. GGUF

A single model is often published across multiple repos or file formats:
  • safetensors files are the original (or converted) weights, typically at full precision (FP16/BF16) or accompanied by other quantized formats. These are what backends like vLLM typically load.
  • GGUF files are the format llama.cpp-based backends (including Ollama and LM Studio) load. A GGUF repo usually offers several quantized variants of the same model as separate files.
A filename like …Q4_K_M.gguf tells you three things at a glance: it’s a Q4 (4-bit) quantization, from the K-quant family, at the M (medium) size/quality point within that family. See Quantization for how to read the rest of the naming scheme and pick a variant.
On a GGUF repo, the file size listed for each quant is your ground-truth memory number for the weights — no math required. Add your KV cache on top (see KV cache and context) to get the full picture.

​
Chat template

Most instruction-tuned models expect prompts wrapped in a specific chat template (special tokens marking system/user/assistant turns). Backends that serve the model — Ollama, llama.cpp, vLLM — typically apply the correct template automatically from metadata bundled with the model, but if you’re building raw prompts yourself, check the card for the expected format. Using the wrong template quietly degrades output quality without throwing an error.

​
Intended use

Cards typically state what the model was tuned for — general chat, coding, reasoning, instruction-following — and sometimes what it’s known to be weak at. This is worth a skim before you commit hardware to a model that turns out to be a poor fit for your task; see Which local model for which task? for matching models to tasks more systematically.

​
What matters once you serve it through Tokios

When you register a local model as a Tokios deployment, two model-card fields translate directly into setup decisions:
  • Upstream model id vs. deployment name. The id your backend serves (for example, an Ollama tag or a path a llama.cpp server was launched with) is different from the public deployment name you’ll give it in Tokios — the card tells you the former, you choose the latter.
  • Context length and license carry over as constraints on how you can use the deployment, regardless of which client calls it.

​
Go deeper

Read a model name

Decode parameter counts, quant labels, and architecture suffixes before you open the card.

Quantization

Read GGUF quant names and pick the right size/quality tradeoff.

Quantization and hardware

Which weight formats run on which hardware.

​
Once you’ve picked one and it’s running

Reading the card tells you what the model can do. Once you’ve downloaded it and it’s answering requests on localhost, the next step is making it reachable.

Give it a stable endpoint

Tokios doesn’t pick, host, or fine-tune the model — once it runs locally, Tokios turns it into a stable, authenticated endpoint you can call from anywhere.