Skip to main content
Pick by selection criteria, not by whichever name is trending this week: native tool-calling support if you’re running an agent, context window versus your VRAM budget, license, and how well the model follows instructions. Once you know the criteria, map them to a model family rather than a specific checkpoint — families like Qwen3, DeepSeek-Coder-V2, and Llama 3.1/3.3 keep shipping new versions, but the family’s strengths and quirks tend to hold steady across releases.
Specific model names and version numbers date fast — a model that’s state of the art this quarter may be superseded within months. Treat the tables below as a starting point for families and criteria, follow the source links, and re-check the current best release in each family before you commit hardware to it.

​
Match the task to a family

Tokios doesn’t serve embeddings on the gateway today — the API surface covers chat and messages endpoints only. The embeddings row above is for picking a model to run locally with your backend directly, not through a Tokios deployment.

​
Choosing a model for agents specifically

Coding agents and other tool-using clients are pickier than a chat UI. Use three criteria:
  1. Native tool/function calling. Agent loops depend on the model returning structured tool calls the client can parse. Structured-output reliability matters as much as raw coding skill — Qwen3 (via Qwen-Agent/MCP) and Llama 3.1 have well-documented native tool use; some smaller instruction-tuned models are noticeably weaker at this even when they chat well.
  2. Context window of at least 32–64K, ideally 128K or more. Agents accumulate file contents, diffs, and tool outputs turn over turn, so a short context fills up fast. Budget VRAM for the KV cache at your target context length on top of the model weights — see Fit a model to your GPU or unified memory.
  3. Instruction-following, structured output, and a permissive license. Apache-2.0 families like Qwen3 and gpt-oss are easier to adopt without licensing friction.
Clients like Roo Code hard-require native tool calling. If the underlying model doesn’t support it, the client can’t parse its responses into actions and will retry the same step instead of making progress — this is a model limitation, not a Tokios or connector issue.

​
Fit it to your hardware

Picking the right family is only half the decision — the model also has to fit in the VRAM or unified memory you actually have, at the context length you need. See Fit a model to your GPU or unified memory for sizing estimates by parameter count and quantization. For the workflow, backend, and naming side of the decision — which backend to run, and what to call the deployment once you’ve registered it — see Choose a local model and backend.

​
Once you’ve picked one

Tokios doesn’t choose the model for you; it gives whatever you’re running a stable deployment endpoint you can call from anywhere.

Register your deployment

Turn your chosen model into a Tokios deployment with a public name and API key.

Compare candidates

Benchmark two or more deployments side by side before you settle on one.