Coding agents that work with local models through Tokios

15 coding agents connect to local models through Tokios. All use standard OpenAI or Anthropic API format — you change 2 things: the base URL and your API key. Once your model is registered as a Tokios deployment, every agent below reaches it through the same endpoint.

Prerequisites: Before configuring any agent below, you need a running local model, a paired Tokios connector, a registered deployment, and a sk-tok-… API key. See the Quickstart if you haven’t set those up yet.

Agent compatibility matrix

Agent	Type	API format	Tool calling	Setup guide
Claude Code	Terminal	Anthropic	Yes	/guides/claude-code
OpenAI Codex	Terminal	OpenAI (Responses)	Yes	/guides/openai-codex
Aider	Terminal	OpenAI	Yes	/guides/aider
Cline	IDE	OpenAI	Yes	/guides/cline
Continue	IDE	OpenAI	Yes	/guides/continue
Roo Code	IDE	OpenAI	Yes	/guides/roo-code
OpenCode	Terminal	OpenAI	Yes	/guides/opencode
Goose	IDE	OpenAI	Yes	/guides/goose
Pi	Terminal	OpenAI · Anthropic	Yes	/guides/pi
Cursor	IDE	OpenAI	Yes	/guides/cursor
Zed	IDE	OpenAI · Anthropic	Unknown	/guides/zed
JetBrains AI Assistant	IDE	OpenAI	Unknown	/guides/jetbrains-ai-assistant
VS Code Copilot	IDE	OpenAI	Unknown	/guides/vscode-copilot
omp (oh-my-pi)	Terminal	OpenAI · Anthropic	Unknown	/guides/omp
Hermes Agent	Terminal	OpenAI	Yes	/guides/hermes-agent

“Tool calling” means the agent can send tools in the request and parse tool_calls from the response. Whether it works end-to-end depends on both the agent and the local model — Qwen3-Coder-30B, Llama-3-70B, and Mistral all handle tool calls well on vLLM.

IDE-based agents

These agents live inside your editor and send completions requests as you code. They all use the OpenAI Chat Completions API format, except where noted.

Cline and Roo Code

Cline and its fork Roo Code both use an “OpenAI Compatible” provider. Set 3 fields in the extension settings:

API Provider: OpenAI Compatible
Base URL: https://api.tokios.com/v1
API Key: your sk-tok-… key
Model ID: your deployment name (for example, qwen3-coder-tunnel)

Both are configured entirely through the settings panel — there is no config file to edit. See the Cline and Roo Code guides for full setup.

Continue

Continue reads ~/.continue/config.yaml. Add a model entry:

models:
  - name: Tokios
    provider: openai
    model: qwen3-coder-tunnel
    apiBase: https://api.tokios.com/v1
    apiKey: sk-tok-YOUR_KEY

See the Continue guide. Note that Continue supports both Chat Completions and Responses — use Chat Completions for local models.

Cursor

Cursor’s custom model override accepts a base URL and API key. Support varies by Cursor version — verify that custom base URL routing works in your build before relying on it:

Open Cursor Settings → Models.
Add a custom model with base URL https://api.tokios.com/v1 and your sk-tok-… key.

See the Cursor guide for version-specific notes.

Zed

Zed supports custom OpenAI-compatible providers in its settings.json:

{
  "language_models": {
    "openai_compatible": {
      "Tokios": {
        "api_url": "https://api.tokios.com/v1",
        "available_models": [
          { "name": "qwen3-coder-tunnel", "max_tokens": 128000 }
        ]
      }
    }
  }
}

The API key does not go in settings.json — enter it in Zed’s LLM provider UI, or export TOKIOS_API_KEY (Zed derives the variable name from the provider id). See the Zed guide.

JetBrains AI Assistant

JetBrains AI Assistant supports BYOK (Bring Your Own Key) through its plugin settings. Enter the base URL and API key in the custom provider fields:

Base URL: https://api.tokios.com/v1
API Key: your sk-tok-… key

See the JetBrains AI Assistant guide.

VS Code Copilot

GitHub Copilot’s BYOK support is emerging. When your VS Code version supports custom providers, point it at:

Base URL: https://api.tokios.com/v1
API Key: your sk-tok-… key

See the VS Code Copilot guide for the latest compatibility.

Terminal agents

These run in your shell and use environment variables or config files to connect.

Claude Code

Claude Code uses the Anthropic API format — not OpenAI. Set 2 environment variables:

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3-coder-tunnel

Note: the base URL has no /v1 suffix for Anthropic format. See the Claude Code guide for persisting these across sessions.

OpenAI Codex

OpenAI Codex uses the Responses API (wire_api=responses). Set the base URL in ~/.codex/config.toml:

[model_providers.tokios]
name = "Tokios"
base_url = "https://api.tokios.com/v1"
env_key = "OPENAI_API_KEY"
wire_api = "responses"

[profiles.tokios]
model = "qwen3-coder-tunnel"
model_provider = "tokios"
disable_response_storage = true

env_key names the variable holding your key rather than the key itself, and disable_response_storage is required because Tokios serves the Responses surface statelessly. Then run:

OPENAI_API_KEY=sk-tok-YOUR_KEY codex --profile tokios

See the OpenAI Codex guide.

Aider

Aider reads the base URL and key from the environment, and needs the openai/ model prefix:

export OPENAI_API_BASE=https://api.tokios.com/v1
export OPENAI_API_KEY=sk-tok-YOUR_KEY
aider --model openai/qwen3-coder-tunnel

See the Aider guide.

OpenCode

OpenCode uses the AI SDK (Vercel) under the hood. Add a custom provider in opencode.json:

{
  "providers": {
    "tokios": {
      "type": "openai",
      "baseURL": "https://api.tokios.com/v1",
      "apiKey": "sk-tok-YOUR_KEY"
    }
  },
  "models": {
    "qwen3-coder": {
      "provider": "tokios",
      "model": "qwen3-coder-tunnel"
    }
  }
}

See the OpenCode guide.

Goose

Goose’s built-in openai provider accepts a custom host. Goose persists these to ~/.config/goose/config.yaml:

export GOOSE_PROVIDER=openai
export GOOSE_MODEL=qwen3-coder-tunnel
export OPENAI_HOST=https://api.tokios.com
export OPENAI_API_KEY=sk-tok-YOUR_KEY

For a named provider instead, drop a JSON file in ~/.config/goose/custom_providers/. See the Goose guide.

Pi and omp

Pi uses a custom provider with api: openai-completions. omp (oh-my-pi) inherits the same pattern — providers is a mapping keyed by provider name, and the keys are camelCase. omp reads ~/.omp/agent/models.yml; Pi reads ~/.pi/agent/models.json:

providers:
  tokios:
    baseUrl: https://api.tokios.com/v1
    api: openai-completions
    apiKey: sk-tok-YOUR_KEY
    models:
      - id: qwen3-coder-tunnel
        input: [text]

See the Pi and omp guides.

Hermes Agent

Hermes Agent connects to any OpenAI-compatible /v1/chat/completions endpoint. It reads ~/.hermes/config.yaml, with the provider keys nested under model::

model:
  default: qwen3-coder-tunnel
  provider: custom
  base_url: https://api.tokios.com/v1
  api_key: sk-tok-YOUR_KEY

See the Hermes Agent guide.

Universal setup pattern

Every agent above follows the same 3-step pattern:

Set the base URL to https://api.tokios.com/v1 (OpenAI) or https://api.tokios.com (Anthropic).
Set the API key to your sk-tok-… key.
Set the model to your registered deployment name.

If your agent isn’t listed here but supports a custom OpenAI-compatible base URL, it almost certainly works. Try the universal env vars:

export OPENAI_BASE_URL=https://api.tokios.com/v1
export OPENAI_API_KEY=sk-tok-YOUR_KEY

Tool-calling compatibility by engine

Tool calling requires both agent support and backend support. Here’s which local engines handle it:

Engine	Tool calling	Notes
vLLM	Yes	Full OpenAI tool-call format; best compatibility
Ollama	Yes (v0.3+)	Via `/v1/chat/completions` with `tools` field
llama.cpp	Partial	`--chat-template` with tool format; varies by model
LM Studio	Yes	OpenAI-compatible tool-call format
SGLang	Yes	OpenAI-compatible; matches vLLM behavior

Tool calling depends on the model, not just the engine. A model must be fine-tuned or prompted to emit structured tool_calls in its response. Qwen3-Coder-30B, Llama-3-70B-Instruct, and Mistral-7B-Instruct all handle tool calls reliably. Muse Glimmer 30B is trained specifically for this: tool invocation against precise schemas, with failure recovery across multi-step workflows.

Best local models for coding agents

Model	Parameters	Best for	Engine recommendation
Muse Glimmer 30B	29.6B dense	Agentic coding, tool calling, single 24 GB GPU	Ollama, llama.cpp
DeepSeek-V4-Flash-0731	284B target / 13B active MoE	Frontier coding agents, tool use, long repositories	vLLM or SGLang (multi-device)
Qwen3-Coder-30B	30B dense	General coding, tool calling	llama.cpp (Vulkan), vLLM
Qwen3-32B	32B dense	Reasoning-heavy code tasks	vLLM, Ollama
DeepSeek-R1	671B MoE	Complex reasoning, architecture	vLLM (requires 2+ GPUs)
Mistral-7B-Instruct	7B dense	Fast completions, low-resource	Ollama, llama.cpp
Llama-3-70B-Instruct	70B dense	Tool calling, multi-file edits	vLLM (multi-GPU)

For DGX Spark (128GB unified memory), Qwen3-Coder-30B at Q4_K_M is the sweet spot — it fits easily, generates fast, and handles tool calls well. For Strix Halo, use llama.cpp with Vulkan for the best token generation speed.

DeepSeek-V4-Flash-0731 is the current DeepSeek choice for coding agents. Its official model card reports major gains over the preview on Terminal Bench 2.1, DeepSWE, and Toolathlon-Verified. It still needs a multi-device host. Review the local hardware guide, then validate tool calls with the exact runtime, reasoning_effort, context length, and sampling settings you plan to use.

Muse Glimmer 30B is the option here that runs fully resident on a single 24 GB GPU, and Meta trained it for agentic tool use — but its benchmarks are vendor-reported with no independent reproduction yet. Use Meta’s own sampling settings (temperature 1.0, top_p 0.95, top_k 64); reusing values tuned for another local model is a common cause of degraded output.

One endpoint serves them all

You don’t need to reconfigure your model when you switch agents. Register your deployment once in Tokios, create a sk-tok-… key, and point any agent at the same endpoint. Switch from Aider to Claude Code by changing 2 environment variables — your model, your connector, and your deployment stay the same.

Quickstart

Set up a connector and register your first deployment.

Compatible clients

The full matrix of 39 compatible clients, not just coding agents.

Choose a local model

Pick the right model for your hardware and use case.

API keys

Create, scope, and rotate keys for teammates and agents.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

Coding agents that work with local models through Tokios

Agent compatibility matrix

IDE-based agents

Cline and Roo Code

Continue

Cursor

Zed

JetBrains AI Assistant

VS Code Copilot

Terminal agents

Claude Code

OpenAI Codex

Aider

OpenCode

Goose

Pi and omp

Hermes Agent

Universal setup pattern

Tool-calling compatibility by engine

Best local models for coding agents

One endpoint serves them all

Quickstart

Compatible clients

Choose a local model

API keys

​Agent compatibility matrix

​IDE-based agents

​Cline and Roo Code

​Continue

​Cursor

​Zed

​JetBrains AI Assistant

​VS Code Copilot

​Terminal agents

​Claude Code

​OpenAI Codex

​Aider

​OpenCode

​Goose

​Pi and omp

​Hermes Agent

​Universal setup pattern

​Tool-calling compatibility by engine

​Best local models for coding agents

​One endpoint serves them all

Quickstart

Compatible clients

Choose a local model

API keys

Agent compatibility matrix

IDE-based agents

Cline and Roo Code

Continue

Cursor

Zed

JetBrains AI Assistant

VS Code Copilot

Terminal agents

Claude Code

OpenAI Codex

Aider

OpenCode

Goose

Pi and omp

Hermes Agent

Universal setup pattern

Tool-calling compatibility by engine

Best local models for coding agents

One endpoint serves them all