Skip to main content
A local LLM (also called a self-hosted LLM) is an open-weight model you run on your own hardware — a laptop, a workstation GPU, a Mac, or a homelab server — instead of calling a hosted API you don’t control. This section covers the open-weight families that run well locally and how to pick between them. Tokios doesn’t pick, host, or fine-tune the model. You choose and run it locally; Tokios turns whatever you’re running into a stable, authenticated endpoint at https://api.tokios.com that any OpenAI- or Anthropic-compatible client can reach — no inbound ports, no VPN.

​
How to choose in three steps

1

Match the task

Coding agent, general chat, reasoning, vision, or edge/CPU work? The task narrows the family fast. See Which local model for which task? for a task-by-task breakdown with agent-specific criteria.
2

Fit it to your hardware

A model only helps if it fits in your VRAM or unified memory at the context length you need. See What size model fits your GPU? for the sizing math, and Quantization for the size/quality dial.
3

Check the license

Licenses vary widely — Apache-2.0 and MIT are permissive; others (Llama Community, Gemma, NVIDIA Open Model License) carry conditions. See How to read a model card for what to look for before you build on a model.

​
The families, and what’s current

Every family below has a dedicated page with tags, sizes, hardware-fit tables, and setup. Model versions move fast — this table is a starting point; follow each page for the current release.
Last verified: October 2026.
“Current” means the newest open-weight release you can download and run yourself. Some current releases still require multi-GPU hardware, including DeepSeek-V4.1-Flash, GLM-5.3, and the frontier-scale Kimi and MiniMax MoEs. Each family page calls out what is downloadable now, what needs a server, and what is still pending.
Need a current frontier-scale model you can still host yourself? DeepSeek-V4.1-Flash local hardware compares the 2× RTX PRO 6000, 2× DGX Spark, and 2× Strix Halo paths, including their support maturity and memory limits.

​
Pick by what you’re doing

General chat & agents

Qwen3.6, Llama, Gemma, and gpt-oss are strong all-rounders. Qwen3.8-27B is the newest Qwen release (vLLM/SGLang only for now); Qwen3.6 is the easiest verified path, and Muse Glimmer is the agent-focused pick that stays resident on a 24 GB card.

Coding

Qwen3-Coder, Mistral’s Devstral, and GLM target agentic, repository-scale coding.

Reasoning and agents

Start with DeepSeek-V4.1-Flash for the current DeepSeek reasoning, coding-agent, and tool-calling release. Use R1 distills for smaller hardware. For frontier-scale agentic coding on workstation hardware, Muse Spark Open leads the open-weight field.

Vision & multimodal

Qwen3-VL and Gemma 4 read text, images, and video — start with Qwen3-VL. MiMo V2.6 adds native audio input across all 3 variants.

Small, edge & CPU

Gemma’s E2B, Phi-4-mini, Granite, and Mistral’s Ministral run on laptops and edge boxes.

Formats & hardware

Which weight format — GGUF, MLX, NVFP4, AWQ — runs on your GPU or Mac.

​
Once you’ve picked one

Tokios doesn’t choose the model for you; it gives whatever you’re running a stable, authenticated endpoint you can call from anywhere.

Quickstart

Pair a connector, register a deployment, and create your first key end to end.

Choose a model and backend

Match a model to your workflow, backend, and deployment naming.