Use Coding Agents with Local Models via Tokios

Every major coding agent communicates with a model backend over an OpenAI- or Anthropic-compatible API. Point that API at Tokios, and your agent talks to a local model running on your own hardware — from any machine, not just the one running the inference server. Switch models by changing 1 field; the agent configuration stays the same.

How coding agents connect to backends

Coding agents send chat completions (and often tool calls) over HTTP. They read an endpoint URL and an API key from environment variables or a config file. Tokios exposes both an OpenAI-compatible and an Anthropic-compatible surface at the same URL, so most agents work with a single configuration change.

Agent	API format	Key env var	Base URL
Claude Code	Anthropic	`ANTHROPIC_API_KEY`	`ANTHROPIC_BASE_URL`
OpenCode	OpenAI	`options.apiKey`	`options.baseURL`
Aider	OpenAI	`OPENAI_API_KEY`	`OPENAI_API_BASE`
Cline	OpenAI	API key in UI	Custom endpoint
Continue	OpenAI	API key in config	`apiBase`
Roo Code	OpenAI	API key in UI	Custom endpoint

Prerequisites

A running Tokios connector paired with your local model — see Connector install
A registered deployment name — see Register a model
A Tokios API key — see API keys

Claude Code

Claude Code uses the Anthropic Messages API. Set 2 environment variables and pass your deployment name with --model:

macOS & Linux
Windows CMD
Windows PowerShell

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3-coder

set ANTHROPIC_BASE_URL=https://api.tokios.com
set ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model qwen3-coder

$env:ANTHROPIC_BASE_URL = "https://api.tokios.com"
$env:ANTHROPIC_API_KEY = "sk-tok-YOUR_KEY"
claude --model qwen3-coder

Claude Code’s built-in model picker only lists claude-* ids. Always pass --model explicitly with your Tokios deployment name. See the full Claude Code guide for persistence and troubleshooting.

OpenCode

OpenCode uses the AI SDK with an OpenAI-compatible provider. Add Tokios to opencode.json:

opencode.json

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "tokios": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Tokios",
      "options": {
        "baseURL": "https://api.tokios.com/v1",
        "apiKey": "sk-tok-YOUR_KEY"
      },
      "models": {
        "qwen3-coder": {
          "name": "qwen3-coder"
        }
      }
    }
  }
}

options.baseURL must include /v1. OpenCode resolves request paths relative to this URL, so omitting /v1 causes 404 errors.

See the full OpenCode guide for detailed setup and troubleshooting.

Aider

Aider reads OpenAI-compatible configuration from environment variables:

export OPENAI_API_BASE=https://api.tokios.com/v1
export OPENAI_API_KEY=sk-tok-YOUR_KEY
aider --model openai/qwen3-coder

The openai/ prefix tells Aider to use the OpenAI chat completions format. Aider supports tool calling with most models that implement the OpenAI function-calling schema.

Cline

Cline connects to custom OpenAI-compatible endpoints through its VS Code settings:

Open Cline settings in VS Code.
Under API Configuration, select OpenAI Compatible.
Set Base URL to https://api.tokios.com/v1.
Paste your sk-tok-YOUR_KEY in the API Key field.
Enter your deployment name (e.g. qwen3-coder) as the Model ID.

Cline uses tool calling extensively. Pick a model with strong tool-call support — see Model recommendations below.

Continue

Add Tokios as a custom provider in ~/.continue/config.json:

~/.continue/config.json

{
  "models": [
    {
      "title": "Tokios - Qwen3 Coder",
      "provider": "openai",
      "model": "qwen3-coder",
      "apiBase": "https://api.tokios.com/v1",
      "apiKey": "sk-tok-YOUR_KEY"
    }
  ]
}

Continue supports both chat completions and tool calling through this provider configuration.

Roo Code

Roo Code uses the same OpenAI-compatible endpoint pattern:

Open Roo Code settings.
Select OpenAI Compatible as the API provider.
Set Base URL to https://api.tokios.com/v1.
Enter your API key and deployment name.

Switch models without reconfiguring

Because every agent reads the model name from a single field — the model parameter in the API request — you can register multiple deployments on Tokios and swap between them without touching any agent config:

# Use Qwen3-Coder for code tasks
claude --model qwen3-coder

# Use GLM-4.5-Air for general reasoning
claude --model glm-4-chat

# Use Gemma for lightweight tasks
claude --model gemma-tunnel

The endpoint and API key stay the same. Tokios routes each request to the correct local backend.

Model recommendations for coding

Not all local models handle coding tasks equally well. Tool calling — the ability to invoke functions, read files, and run commands — is critical for agent-based workflows.

Model	Total params	Best for	Tool calling	Notes
Muse Glimmer 30B	29.6B	Agentic coding, tool use	Good	Trained for tool invocation against precise schemas with failure recovery; 4-bit build fits a single 24 GB GPU
Qwen3-Coder-30B	30B	Code generation, refactoring	Good	Confirmed working with Cline and Aider
Qwen3-32B	32B	General + coding	Good	Strong all-rounder; 43 tok/s on DGX Spark
GLM-4.5-Air	106B MoE	Complex reasoning	Good	12B active params; community-favorite for Spark
Qwen3.5-35B-A3B	35B MoE	Fast coding	Good	3B active; 120 tok/s with Atlas on Spark
Gemma-4-26B-A4B	26B MoE	Lightweight tasks	Moderate	4B active; fast but less capable on complex edits

vLLM on AMD GPUs has known issues with tool calling. If you run on Strix Halo or other AMD hardware, use llama.cpp with the Vulkan backend for cleaner tool-call support.

Tips for coding with local models

Use a model that supports tool calling. Agents like Claude Code, OpenCode, and Cline rely on structured function calls. If your model doesn’t support the OpenAI function-calling schema, the agent falls back to text-based tool invocation, which is slower and less reliable. Set context length appropriately. Coding agents send large prompts with file contents and conversation history. Set --max-model-len on your inference server to at least 8192 tokens. Models with 32K+ context handle multi-file tasks better. Prefer MoE models for throughput. Mixture-of-experts models like Qwen3.5-35B-A3B activate only a fraction of their parameters per token, delivering higher throughput than dense models of similar total size. For iterative coding tasks where latency matters, MoE is often the better choice. Test tool calls before relying on them. Send a quick test with a tool-calling payload to confirm your model and backend handle function calls correctly:

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-coder",
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "tools": [{"type": "function", "function": {"name": "calculator", "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}}}}]
  }'

If the response includes a tool_calls array, tool calling works.

One endpoint for all your agents

Tokios gives every coding agent the same URL and API key. Register your models once, generate 1 API key, and configure any number of agents to point at https://api.tokios.com. Switch models, add new backends, or move models to different hardware — your agent configurations don’t change.

Model routing

Learn how to register multiple models and route requests by deployment name.

Choose a local model

Compare backends and quantization options before committing a model to agentic workflows.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

Use Coding Agents with Local Models via Tokios

How coding agents connect to backends

Prerequisites

Claude Code

OpenCode

Aider

Cline

Continue

Roo Code

Switch models without reconfiguring

Model recommendations for coding

Tips for coding with local models

One endpoint for all your agents

Model routing

Choose a local model

​How coding agents connect to backends

​Prerequisites

​Claude Code

​OpenCode

​Aider

​Cline

​Continue

​Roo Code

​Switch models without reconfiguring

​Model recommendations for coding

​Tips for coding with local models

​One endpoint for all your agents

Model routing

Choose a local model

How coding agents connect to backends

Prerequisites

Claude Code

OpenCode

Aider

Cline

Continue

Roo Code

Switch models without reconfiguring

Model recommendations for coding

Tips for coding with local models

One endpoint for all your agents