Skip to main content
LibreChat reaches OpenAI-compatible APIs through custom endpoints defined in librechat.yaml. Add an entry under endpoints.custom with baseURL set to https://api.tokios.com/v1, apiKey set to your sk-tok-… key, and models listing your Tokios deployment name — and LibreChat can chat with your local model without that model’s port ever being exposed.

​
Prerequisites

Before you edit librechat.yaml, have these ready:

Connector paired

Install and pair tokios-connector next to your model.

Model registered

Register a deployment (public name) for the model you want LibreChat to see.

API key created

Create an sk-tok-… key, optionally scoped to that deployment.

Supported backends

Confirm your local backend (Ollama, llama.cpp, vLLM, LM Studio) is running.

​
Add Tokios as a custom endpoint

1

Open librechat.yaml

Open (or create) LibreChat’s librechat.yaml configuration file at the root of your LibreChat deployment. The exact default file path can vary by release — check yours.
2

Add a custom endpoint entry

Under endpoints.custom, add a block for Tokios with baseURL pointed at Tokios and models.default set to your public deployment name — for example, gemma-tunnel:
models.default should list the deployment name(s) you registered in the Models tab, not your Ollama, llama.cpp, vLLM, or LM Studio local model id. The exact YAML key names and nesting can shift between LibreChat releases, so confirm the keys against your install.
3

Provide the API key as an environment variable

Set TOKIOS_API_KEY (or whatever variable name you referenced above) to your sk-tok-… key in LibreChat’s .env file, rather than pasting the key directly into librechat.yaml:
.env
4

Restart LibreChat

Restart the LibreChat server so it reloads librechat.yaml. The Tokios endpoint should now appear in LibreChat’s endpoint picker.
Never commit a real sk-tok-… key into librechat.yaml or into version control. Reference it from an environment variable instead.

​
Your models come from your models list

LibreChat’s custom endpoints don’t automatically discover deployments the way some clients do — the models field in your librechat.yaml block is the source of truth for what shows up in the picker. List every Tokios deployment you want available under models.default. If your LibreChat version supports fetching models dynamically for custom endpoints, it does so by calling GET /v1/models against your baseURL. Tokios implements /v1/models and returns the deployments your key is scoped to, so a fetch-based configuration works too — check the current release’s docs for the exact toggle (for example models.fetch) and behavior.

​
Hosted UI, home model

The common shape this solves: you run LibreChat on a VPS or a container host, and you want it to reach a model that lives on your home machine or workstation, without exposing that machine’s port to the internet. Because the Tokios connector only dials outward from the machine running your model, you don’t need to:
  • Bind your backend to 0.0.0.0 (for example OLLAMA_HOST=0.0.0.0) just so a remote LibreChat instance can reach it.
  • Open an inbound port or punch a hole in your home router or firewall.
  • Set up a VPN or reverse proxy between the two machines.
LibreChat, wherever it’s hosted, talks to https://api.tokios.com/v1, authenticated with your sk-tok-… key. Tokios routes the request over the tunnel to the connector running next to your model, which stays on localhost the whole time.

​
Troubleshooting

Confirm librechat.yaml is in the path LibreChat expects (the exact config discovery path varies by version) and that you restarted the server after editing it.
The API key LibreChat sent is missing, malformed, or has been revoked. Confirm the environment variable referenced by apiKey holds the full sk-tok-… key with no extra whitespace, and check its status on the Keys tab.
The key exists but isn’t scoped to the deployment you listed in models, or your account is suspended. Check the key’s model patterns on the Keys tab.
The model value LibreChat sent doesn’t match a registered deployment. Confirm it exactly matches the public name on the Models tab — not your Ollama/llama.cpp/vLLM/LM Studio local model id.
The connector for that deployment is offline or at its concurrency cap. Check the Connectors tab — the connector needs to show as Online, and the machine running it needs to still be up and reachable.

​
Now put it to work everywhere else

Once LibreChat is talking to your model through Tokios, the same deployment and key work from any other OpenAI- or Anthropic-compatible client — no extra setup per client.

Compatible clients

See base URL and key settings for other clients.

API keys

Create separate scoped keys for teammates using the same LibreChat instance.