Skip to main content
Pick by selection criteria, not by whichever name is trending this week: native tool-calling support if you’re running an agent, context window versus your VRAM budget, license, and how well the model follows instructions. For DeepSeek, the current checkpoint to evaluate is deepseek-ai/DeepSeek-V4-Flash-0731; use R1 distills only when your hardware cannot support the V4 weight floor.
Specific model names and version numbers date fast — a model that’s state of the art this quarter may be superseded within months. Treat the tables below as a starting point for families and criteria, follow the source links, and re-check the current best release in each family before you commit hardware to it.

​
Match the task to a family

Tokios doesn’t serve embeddings on the gateway today — the API surface covers chat and messages endpoints only. The embeddings row above is for picking a model to run locally with your backend directly, not through a Tokios deployment.
For the DeepSeek-V4-Flash-0731 checkpoint, start with the local hardware guide. Its 1M-token specification does not mean every local configuration has enough KV-cache capacity for that context length.

​
Choosing a model for agents specifically

Coding agents and other tool-using clients are pickier than a chat UI. Use three criteria:
  1. Native tool/function calling. Agent loops depend on the model returning structured tool calls the client can parse. DeepSeek-V4-Flash-0731 is the current DeepSeek option for this workload. Validate it with the exact serving runtime, reasoning effort, and chat encoding you will use.
  2. Context window of at least 32–64K, ideally 128K or more. Agents accumulate file contents, diffs, and tool outputs turn over turn, so a short context fills up fast. Budget VRAM for the KV cache at your target context length on top of the model weights — see Fit a model to your GPU or unified memory.
  3. Instruction-following, structured output, and a permissive license. Apache-2.0 families like Qwen3 and gpt-oss are easier to adopt without licensing friction.
Clients like Roo Code hard-require native tool calling. If the underlying model doesn’t support it, the client can’t parse its responses into actions and will retry the same step instead of making progress — this is a model limitation, not a Tokios or connector issue.

​
Fit it to your hardware

Picking the right family is only half the decision — the model also has to fit in the VRAM or unified memory you actually have, at the context length you need. See Fit a model to your GPU or unified memory for sizing estimates by parameter count and quantization. For the workflow, backend, and naming side of the decision — which backend to run, and what to call the deployment once you’ve registered it — see Choose a local model and backend.

​
Once you’ve picked one

Tokios doesn’t choose the model for you; it gives whatever you’re running a stable deployment endpoint you can call from anywhere.

Register your deployment

Turn your chosen model into a Tokios deployment with a public name and API key.

Compare candidates

Test two or more deployments side by side before you settle on one.