Skip to main content
A local LLM (also called a self-hosted LLM) is an open-weight model you run on your own hardware — a laptop, a workstation GPU, a Mac, or a homelab server — instead of calling a hosted API you don’t control. This section covers the open-weight families that run well locally and how to pick between them. Tokios doesn’t pick, host, or fine-tune the model. You choose and run it locally; Tokios turns whatever you’re running into a stable, authenticated endpoint at https://api.tokios.com that any OpenAI- or Anthropic-compatible client can reach — no inbound ports, no VPN.

​
How to choose in three steps

1

Match the task

Coding agent, general chat, reasoning, vision, or edge/CPU work? The task narrows the family fast. See Which local model for which task? for a task-by-task breakdown with agent-specific criteria.
2

Fit it to your hardware

A model only helps if it fits in your VRAM or unified memory at the context length you need. See What size model fits your GPU? for the sizing math, and Quantization for the size/quality dial.
3

Check the license

Licenses vary widely — Apache-2.0 and MIT are permissive; others (Llama Community, Gemma, NVIDIA Open Model License) carry conditions. See How to read a model card for what to look for before you build on a model.

​
The families, and what’s current

Every family below has a dedicated page with tags, sizes, hardware-fit tables, and setup. Model versions move fast — this table is a starting point; follow each page for the current release.
“Current” means the newest open-weight release you can download and run yourself. Some families also ship larger frontier models that are API-only or multi-GPU-only (for example DeepSeek-V4-Pro or GLM-5.2, or the frontier-scale Kimi and MiniMax MoEs) — and Kimi K3 is hosted-only until its weights arrive, promised by July 27, 2026. Each family page calls out what is downloadable now, what needs a server, and what is still pending.

​
Pick by what you’re doing

General chat & agents

Qwen3.6, Llama, Gemma, and gpt-oss are strong all-rounders. Qwen3.6 is the current flagship.

Coding

Qwen3-Coder, Mistral’s Devstral, and GLM target agentic, repository-scale coding.

Reasoning

DeepSeek-R1, gpt-oss, Mistral’s Magistral, and Nemotron emit step-by-step reasoning.

Vision & multimodal

Qwen3-VL and Gemma 4 read text, images, and video — start with Qwen3-VL.

Small, edge & CPU

Gemma’s E2B, Phi-4-mini, Granite, and Mistral’s Ministral run on laptops and edge boxes.

Formats & hardware

Which weight format — GGUF, MLX, NVFP4, AWQ — runs on your GPU or Mac.

​
Once you’ve picked one

Tokios doesn’t choose the model for you; it gives whatever you’re running a stable, authenticated endpoint you can call from anywhere.

Quickstart

Pair a connector, register a deployment, and create your first key end to end.

Choose a model and backend

Match a model to your workflow, backend, and deployment naming.