https://api.tokios.com/v1 with your sk-tok-… API key and your registered deployment name, and either app calls your local model — no host.docker.internal, no port forwarding, no VPN.
Prerequisites
Install and pair the connector
Run
tokios-connector next to your model and approve it in the console.Register a deployment
Give your model a public name, for example
gemma-tunnel — this is the model value clients send.Create an API key
Create an
sk-tok-… key on the Keys tab, optionally scoped to this deployment.Check your backend is supported
Ollama, llama.cpp, vLLM, and LM Studio all work behind the connector.
Tokios implements
GET /v1/models and normalizes the OpenAI API surface, so when AnythingLLM or Dify validates the provider (listing models, checking the endpoint), that check succeeds the same way it would against OpenAI itself.Add Tokios as a provider
- AnythingLLM
- Dify
1
Open LLM provider settings
In AnythingLLM, go to Settings → LLM Provider and choose Generic OpenAI from the provider list. The exact menu path and provider label can vary by release — check yours.
2
Set the base URL
In Base URL, enter:
3
Set the API key
In API Key, paste your Tokios key:
4
Set the model name
In Model name (also called Chat Model Name in some AnythingLLM versions), enter your Tokios deployment name, for example
gemma-tunnel — not the upstream model id your backend uses locally.5
Set token / context window
Set the Token context window field to match your model’s context length — the exact field label varies by version, and some versions require this value before they’ll save the provider.
6
Save and test
Save the provider, then send a message in a workspace to confirm AnythingLLM reaches your model through Tokios.
No more host.docker.internal
AnythingLLM and Dify both commonly run in Docker, and your model often runs on a different machine — or a different container — than the app. That split is where most self-hosted setups get stuck: localhost inside a container doesn’t reach the host, host.docker.internal doesn’t resolve the same way on every platform, and a model on a separate machine needs its own port forwarding or VPN entry.
Pointing AnythingLLM or Dify at https://api.tokios.com/v1 removes that problem entirely. The endpoint is public and stable regardless of which container, host, or network the app runs on — only the connector next to your model needs outbound access, and it never opens an inbound port.
Now that it’s connected
Your model is reachable from AnythingLLM or Dify wherever they run. The same deployment and API key work from any other OpenAI- or Anthropic-compatible client too.Compatible clients
See the full list of clients Tokios works with and which base URL each one needs.
API keys
Create a separate scoped key per app or teammate, and revoke access without touching your connector.