Use an SGLang server from anywhere

SGLang launches an OpenAI-compatible server, and Tokios forwards to any server that speaks the OpenAI Chat Completions API — so SGLang works as a generic OpenAI-compatible upstream. Keep SGLang on localhost and run the Tokios connector next to it: the connector dials out to https://api.tokios.com, and your tools, teammates, or agents call one authenticated endpoint from anywhere.

Advanced runtime. SGLang is a Python GPU server built for high-throughput serving, with more setup than a desktop tool. If you just want the easiest path, start with Ollama or LM Studio.

SGLang exposes an OpenAI-compatible Chat Completions API, so it works with Tokios like any OpenAI-compatible server: the connector forwards to whatever /v1 BaseUrl you set, regardless of which server implements that surface. Ollama, llama.cpp, vLLM, and LM Studio are the backends Tokios documents most closely (see Supported backends), but SGLang works the same way. If you hit a gap, it’s likely in SGLang’s OpenAI compatibility itself rather than in Tokios.

The outbound-only way

Run SGLang locally, as usual

Keep SGLang on 127.0.0.1 — don’t bind it wider than loopback. Launch its OpenAI-compatible server:

python -m sglang.launch_server --model-path <path-or-repo-id> --port 30000

The default --port value and whether SGLang binds to 127.0.0.1 or 0.0.0.0 out of the box can vary by version — bind it to loopback explicitly if it doesn’t already.

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing. Tokios shows a one-time claim code like TKS-XXXX-XXXX, valid for about 15 minutes.On the machine running SGLang, run the install one-liner for your OS:

Windows

irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'

macOS

curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX

Linux

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Already have tokios-connector installed? Pass the code directly instead:

tokios-connector TKS-XXXX-XXXX

Back in the console, approve the device on the Setup or Connectors tab. The connector’s token is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

Point the connector at SGLang's /v1 API

SGLang’s server speaks the same OpenAI Chat Completions surface Tokios expects. For the basic single-model path, the console’s Setup wizard lets you enter the upstream manually — use http://127.0.0.1:30000/v1. If you’re serving multiple models through a tokios-connector.json config, set the upstream BaseUrl explicitly, including /v1:

{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:30000/v1"
  }
}

See Connector config for the full multi-model file format.

In the Models tab, register the SGLang model and choose a public name — for example, sglang-tunnel. That public name, not SGLang’s local model id, is what clients send in the model field.

Create an API key

In the Keys tab, click Create key. Optionally scope it to specific deployments with model-name patterns (comma-separated globs, * for all). Copy the sk-tok-… key now — it’s shown only once.

SGLang has an --api-key launch option that requires a bearer token on requests to its own server. You can keep it as defense in depth on the loopback connection between the connector and SGLang, but it isn’t what authenticates callers reaching you through Tokios — that’s your sk-tok-… key. Check the exact flag syntax and whether it covers all SGLang endpoints or only /v1 paths against your SGLang version.

Use it from anywhere

Once you have a deployment name and an API key, any OpenAI- or Anthropic-compatible client can reach your SGLang model — from another machine, a teammate’s laptop, or a hosted agent.

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sglang-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="sglang-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model sglang-tunnel

The model field is always your registered deployment name (sglang-tunnel above), never SGLang’s local model id. See Use Claude Code for persisting the environment variables across sessions.

FAQ

Is SGLang officially supported by Tokios?

Ollama, llama.cpp, vLLM, and LM Studio are the backends Tokios documents most closely — see Supported backends. But the connector doesn’t inspect which server is behind the upstream — it forwards requests to whatever BaseUrl you configure, and SGLang’s launch server speaks the same OpenAI Chat Completions surface those backends do, so it works the same way. If SGLang’s OpenAI compatibility diverges from the others in some edge case, that’s a SGLang-side detail rather than a Tokios one.

Does streaming work?

Yes. Set "stream": true and Tokios returns standard server-sent events (SSE) in OpenAI or Anthropic format, depending on which API surface you called.

Can I serve more than one model through SGLang?

Register additional deployments in the Models tab, or use a tokios-connector.json config file to define multiple upstreams and routes for a single connector — useful if you run more than one SGLang process. See Connector config.

Your model never leaves your machine — your endpoint works everywhere

SGLang keeps doing exactly what it does today, on localhost, with no inbound ports and no endpoints exposed beyond /v1. The connector only dials out. Everything past that — auth, routing, streaming — is one endpoint you can call from any machine, teammate, or agent you choose to give a key to.

Supported backends

See the full list of named backends and default upstream ports.

API keys

Create, scope, and rotate sk-tok-… keys for teammates and agents.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

Use an SGLang server from anywhere — safely

The outbound-only way

Use it from anywhere

FAQ

Your model never leaves your machine — your endpoint works everywhere

Supported backends

API keys

​The outbound-only way

​Use it from anywhere

​FAQ

​Your model never leaves your machine — your endpoint works everywhere

Supported backends

API keys

The outbound-only way

Use it from anywhere

FAQ

Your model never leaves your machine — your endpoint works everywhere