Serve models from your homelab GPU server — to your whole team

Your homelab box already serves models fine on its own LAN. The hard part is letting your teammates — or an app that isn’t sitting on the same network — reach it, especially when your ISP’s CGNAT makes port forwarding impossible anyway. Run the tokios-connector next to your runtime and you get one authenticated endpoint at https://api.tokios.com that anyone you invite can call, with no inbound ports, no router changes, and no shared raw access to the box.

The homelab reachability problem

A homelab GPU server is usually headless, tucked in a closet or rack, and running Ollama, vLLM, or llama.cpp for whoever’s on the LAN. Sharing it beyond that runs into a few walls at once:

No public IP to forward to. If your ISP puts you behind carrier-grade NAT (CGNAT), there’s no unique inbound address for a port-forward rule to attach to in the first place — see Serve a local LLM from behind CGNAT for why that’s a router setting you can’t fix yourself.
You don’t want to expose it even if you could. Binding the runtime to 0.0.0.0 and forwarding a port turns your GPU box into an unauthenticated service on the open internet. Anyone who finds the port can send it requests.
Sharing access means sharing everything. Handing a teammate your Tailscale node or a raw bound port gives them (and anything on their machine) the same access you have — there’s no per-person key, and no way to cut off one person without disrupting everyone else.

Tokios is the connect layer that sits on top of the runtime you already run — it doesn’t replace Ollama, vLLM, or llama.cpp.

Set it up on your server

Run your runtime(s) on the server

Start your model server as you normally would, for example:

ollama serve

vllm serve meta-llama/Llama-3.1-8B-Instruct

Leave it listening on its default loopback address. Confirm it answers on its own OpenAI-compatible /v1 API before adding the connector.

Get a claim code and pair the connector

In the dashboard, open the Setup tab and select Start pairing to get a one-time claim code shaped TKS-XXXX-XXXX (valid for about 15 minutes). On the server, run:

Linux

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Already have tokios-connector installed? Pass the code directly instead:

tokios-connector TKS-XXXX-XXXX

Back in the dashboard, approve the device on the Setup or Connectors tab. The connector’s token (ct-tok-…) is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

Point the connector at your upstream(s)

The connector’s upstream is your runtime’s own OpenAI-compatible API on the server, not a port you expose externally. Default ports: Ollama 11434, llama.cpp 8080, vLLM 8000, LM Studio 1234 — all with /v1 appended. For Ollama, point at its /v1 OpenAI-compatible API, not the native /api. No inbound firewall rule is needed — the connector only dials out.

On the Models tab, map each upstream model id (for example llama3.1) to a public deployment name your team will use, such as homelab-llama. Clients send this deployment name in the model field of every request, never the upstream id.

Create per-teammate scoped keys

On the Keys tab, click Create key for each teammate or app. Scope each key to specific deployment name patterns (comma-separated globs, * for all) so a teammate only reaches the models you intend. Keys are sk-tok-… Bearer tokens, shown once at creation — copy it before leaving the page.

Serve several models at once

If your box runs more than one model, or splits traffic across Ollama, llama.cpp, and vLLM at the same time, use the connector config file instead of the single-upstream pairing flow. In tokios-connector.json, define Routes and Upstreams under the Connector section — one Upstream per backend, each with its own BaseUrl (including /v1) — and pass the file with --config. See Model routing for how to register and switch between the resulting deployments behind the same endpoint.

Use it from anywhere

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "homelab-llama",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello from the team laptop"}]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="homelab-llama",
    messages=[{"role": "user", "content": "Hello from the team laptop"}],
)

The model field is always the registered deployment name, never the upstream model id. See Use Claude Code for calling your homelab server from claude --model homelab-llama.

Why one endpoint for a team

No inbound ports on the box, ever. The connector only makes outbound connections, so teammates never need a VPN peer, an SSH tunnel, or a bound 0.0.0.0 port to reach the server — this holds whether or not you’re behind CGNAT.
Revoke one teammate’s key without disrupting anyone else. Because each person has their own sk-tok-… key, disabling or deleting one on the Keys tab stops that person immediately and leaves every other key working.
Swap models without touching client config. Point a deployment name at a different upstream model or backend, and every teammate and script already targeting homelab-llama keeps working.

Next steps

Quickstart

Go from zero to a working connector, deployment, and API key.

LLM behind CGNAT

Why port forwarding fails on residential ISPs, and how the connector sidesteps it.

Connector config reference

Serve multiple models from one server with tokios-connector.json.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

Serve models from your homelab GPU server — to your whole team

The homelab reachability problem

Set it up on your server

Serve several models at once

Use it from anywhere

Why one endpoint for a team

Next steps

Quickstart

LLM behind CGNAT

Connector config reference

Model routing

​The homelab reachability problem

​Set it up on your server

​Serve several models at once

​Use it from anywhere

​Why one endpoint for a team

​Next steps

Quickstart

LLM behind CGNAT

Connector config reference

Model routing

The homelab reachability problem

Set it up on your server

Serve several models at once

Use it from anywhere

Why one endpoint for a team

Next steps