Use a vLLM server from anywhere

By default, vLLM’s OpenAI-compatible server listens on http://127.0.0.1:8000/v1 — it isn’t reachable from another machine unless you bind it wider or forward the port. vLLM does ship an --api-key flag, but vLLM’s own docs are explicit that you shouldn’t rely on it exclusively. Keep vLLM on localhost and run the Tokios connector next to it instead: the connector dials out to https://api.tokios.com, and your tools, teammates, or agents call one authenticated endpoint from anywhere.

Advanced runtime. vLLM is a Python GPU server aimed at high throughput, with more setup and tuning than a desktop tool. If you just want the easiest way to serve a model, start with Ollama or LM Studio — Tokios connects to any of them the same way.

vLLM’s `--api-key` isn’t the whole story

vllm serve --api-key <key> requires a bearer token on requests, which is a reasonable first layer. The catch, per vLLM’s own security documentation, is scope: the key check only covers /v1 paths, and other endpoints on the same server are not protected by it. vLLM’s documented recommendation is to put a reverse proxy in front that allowlists only the endpoints you intend to expose — --api-key alone isn’t meant to be your perimeter.

This isn’t a knock on vLLM — --api-key does what it says for the paths it covers. It’s solving a different problem than “authenticate and route requests to this server from anywhere.” Tokios is purpose-built for that: it never opens an inbound port on your vLLM host at all, so there’s no listening surface to allowlist in the first place.

The outbound-only way

Run vLLM locally, as usual

Keep vLLM on 127.0.0.1 — don’t bind it wider than loopback. Start the OpenAI-compatible server:

vllm serve mistralai/Mistral-7B-Instruct-v0.2

Leave it listening on the default http://127.0.0.1:8000.

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing. Tokios shows a one-time claim code like TKS-XXXX-XXXX, valid for about 15 minutes.On the machine running vLLM, run the install one-liner for your OS:

Windows

irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'

macOS

curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX

Linux

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Already have tokios-connector installed? Pass the code directly instead:

tokios-connector TKS-XXXX-XXXX

Back in the console, approve the device on the Setup or Connectors tab. The connector’s token is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

Point the connector at vLLM's /v1 API

vLLM’s OpenAI-compatible server already speaks the surface Tokios expects. For the basic single-model path, the console’s Setup wizard uses http://127.0.0.1:8000/v1 as the upstream automatically. If you’re serving multiple models through a tokios-connector.json config, set the upstream BaseUrl explicitly:

{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:8000/v1"
  }
}

See Connector config for the full multi-model file format.

In the Models tab, register the vLLM model and choose a public name — for example, mistral-tunnel. That public name, not vLLM’s local model id, is what clients send in the model field.

Create an API key

In the Keys tab, click Create key. Optionally scope it to specific deployments with model-name patterns (comma-separated globs, * for all). Copy the sk-tok-… key now — it’s shown only once.

Serving LoRA adapters? vLLM can serve multiple adapters from one process when you start it with --enable-lora and --lora-modules, addressing each adapter by name in the model field of the request. Register each adapter as its own Tokios deployment with its own public name, the same way you’d register any other model — see Model routing for how the gateway maps a public deployment name to the id you send upstream.

Use it from anywhere

Once you have a deployment name and an API key, any OpenAI- or Anthropic-compatible client can reach your vLLM model — from another machine, a teammate’s laptop, or a hosted agent.

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="mistral-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model mistral-tunnel

The model field is always your registered deployment name (mistral-tunnel above), never vLLM’s local model id. See Use Claude Code for persisting the environment variables across sessions.

FAQ

Does streaming work?

Yes. Set "stream": true and Tokios returns standard server-sent events (SSE) in OpenAI or Anthropic format, depending on which API surface you called.

Can I serve more than one model or adapter through vLLM?

Yes. Register additional deployments in the Models tab, or use a tokios-connector.json config file to define multiple upstreams and routes for a single connector — useful if you run more than one vLLM process, or one process serving several LoRA adapters via --lora-modules. See Connector config.

Do I still need vLLM's --api-key flag?

You can keep it as defense in depth on the loopback connection between the connector and vLLM, but it isn’t what authenticates callers reaching you through Tokios. Client requests to https://api.tokios.com are authenticated with your sk-tok-… key; the connector is what’s allowed to reach 127.0.0.1:8000 at all.

Your model never leaves your machine — your endpoint works everywhere

vLLM keeps doing exactly what it does today, on localhost, with no inbound ports and no endpoints exposed beyond /v1. The connector only dials out. Everything past that — auth, routing, streaming — is one endpoint you can call from any machine, teammate, or agent you choose to give a key to.

Quickstart

Pair a connector and register your first model end to end.

API keys

Create, scope, and rotate sk-tok-… keys for teammates and agents.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

Use a vLLM server from anywhere — safely

vLLM’s `--api-key` isn’t the whole story

The outbound-only way

Use it from anywhere

FAQ

Your model never leaves your machine — your endpoint works everywhere

Quickstart

API keys

​vLLM’s --api-key isn’t the whole story

​The outbound-only way

​Use it from anywhere

​FAQ

​Your model never leaves your machine — your endpoint works everywhere

Quickstart

API keys

vLLM’s `--api-key` isn’t the whole story

The outbound-only way

Use it from anywhere

FAQ

Your model never leaves your machine — your endpoint works everywhere