Use Tokios with local AI workstations

Local AI workstations are powerful, but most clients still expect a cloud-style API endpoint. Tokios lets you keep the model on your own hardware while giving your tools a stable OpenAI- or Anthropic-compatible endpoint at https://api.tokios.com. Use this page if you run local models on hardware such as NVIDIA GB10 or DGX Spark systems, AMD Strix Halo or Ryzen AI Max systems, Apple silicon Macs with high unified memory, GPU workstations, mini PCs, or homelab servers.

What Tokios adds

Tokios does not replace your local model runtime. It sits between your clients and the backend you already run on the workstation.

You have	Tokios gives you
A model running on a local AI workstation	A public deployment name for API clients
Ollama, LM Studio, llama.cpp, vLLM, or another OpenAI-compatible backend	A stable endpoint at `https://api.tokios.com`
A machine behind NAT, a home router, or a private office network	An outbound-only connector tunnel
Several local models or backends	One API key and one endpoint for clients
Coding agents or SDK apps	OpenAI- and Anthropic-compatible request surfaces

Hardware this page is for

Tokios is useful when the machine has enough memory and compute to run local models, but the tools you use live somewhere else: a laptop, editor, coding agent, notebook, server process, or teammate’s client.

Hardware category	Common local AI reason	Tokios role
NVIDIA GB10 and DGX Spark systems	Run local AI development workloads on Grace Blackwell hardware	Give remote clients one stable API endpoint
AMD Strix Halo and Ryzen AI Max systems	Use high-memory desktop or mini-PC hardware for local inference	Expose the backend without opening inbound ports
Apple silicon Macs with high unified memory	Run local LLMs on a MacBook Pro, Mac Studio, or Mac mini	Connect Mac-hosted models to OpenAI- and Anthropic-compatible clients
GPU workstations and homelab servers	Serve models from a dedicated local machine	Route multiple deployments through one connector

Recommended setup

Start the local backend

Run your model server on the workstation. Tokios works with backends that expose an OpenAI-compatible Chat Completions API, such as Ollama, LM Studio, llama.cpp, or vLLM.

Install the connector on the workstation

Run tokios-connector on the same machine, or on a nearby machine that can reach the backend. The connector dials out to Tokios, so you do not need to open inbound ports.

Map the backend’s upstream model id to a public deployment name such as strix-coder, gb10-chat, or local-qwen. Clients send this deployment name in the model field.

Point your tools at Tokios

Use https://api.tokios.com/v1 for OpenAI-style clients and https://api.tokios.com for Anthropic-style clients.

Hardware-oriented workflows

Workflow	Why it fits local AI hardware	Tokios page
Coding agents on your own machine	Keep code context local while using familiar agent clients	Coding agents
Compare models on the same workstation	Change only the deployment name and keep hardware fixed	Benchmark deployment
Try several local runtimes	Route Ollama, LM Studio, llama.cpp, or vLLM behind one endpoint	Backends
Share one workstation endpoint with your own apps	Use API keys and deployment names instead of per-app backend URLs	Compatible clients
Move models between machines later	Keep the public deployment name stable while changing the backend route	Model routing

Naming deployments

Use names that describe the job, not only the hardware. That makes client configs easier to keep stable.

Avoid	Prefer	Why
`gb10`	`local-coder`	The client cares about the workflow
`strix-halo-qwen`	`fast-draft`	You can move the deployment later
`ollama-model-1`	`chat-local`	The backend is an implementation detail

If you upgrade from one workstation to another, keep the same deployment name and update the connector route. Your clients can keep using the same model value.

What to benchmark

Local AI workstation buyers often compare hardware by model size or raw tokens per second. For Tokios, measure the workflow you actually plan to run.

Signal	Why it matters
Time to first token	Shows whether the setup feels interactive
Streaming stability	Matters for coding agents and chat UIs
Total task time	Captures prompt processing, generation, and client behavior
Model quality	Determines whether the local model can do the job
Backend reliability	Catches restarts, context limits, and driver/runtime issues

Use Benchmark a Tokios deployment to compare deployments with the same prompt and hardware.

Next steps

Connect a backend

Configure Ollama, LM Studio, llama.cpp, or vLLM as your local upstream.

Install the connector

Run the connector beside your workstation model server.

Use coding agents

Connect Claude Code, OpenAI Codex, and compatible coding clients.

Benchmark a deployment

Compare local models and backends through the same Tokios endpoint.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

Use Tokios with local AI workstations

What Tokios adds

Hardware this page is for

Recommended setup

Hardware-oriented workflows

Naming deployments

What to benchmark

Next steps

Connect a backend

Install the connector

Use coding agents

Benchmark a deployment

​What Tokios adds

​Hardware this page is for

​Recommended setup

​Hardware-oriented workflows

​Naming deployments

​What to benchmark

​Next steps

Connect a backend

Install the connector

Use coding agents

Benchmark a deployment

What Tokios adds

Hardware this page is for

Recommended setup

Hardware-oriented workflows

Naming deployments

What to benchmark

Next steps