> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tokios.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Tokios exposes private local models through https://api.tokios.com using an outbound-only connector tunnel.
> Use 'connector' for the local tokios-connector process.
> Use 'deployment' for the public model name in the API model field.
> Use 'upstream model id' for the model served by Ollama, llama.cpp, vLLM, LM Studio, or another local backend.
> API keys use the prefix sk-tok-. Connector tokens use the prefix ct-tok-.

# Local LLM API (locallm API): expose your models with an OpenAI-compatible endpoint

> Give your local models (locallm) an authenticated, OpenAI-compatible API endpoint. Compare options (Tokios, LiteLLM, LocalAI) and set up team access in minutes.

Your local model runs on `localhost:11434` or `localhost:8000`. That's fine for you, but your team, your apps, and your remote clients can't reach it. You need a **local LLM API** (also written as **locallm API**) — an authenticated, OpenAI-compatible endpoint that lets anyone with an API key call your local model from anywhere.

This guide covers what a local LLM API is, the options for creating one, and how to set up team access with per-user authentication and model scoping.

Last verified: 2026-10-24

## The API gap

When you run a model with Ollama, vLLM, llama.cpp, or LM Studio, you get a local API on `127.0.0.1:PORT/v1`. That works for the machine running the model, but it doesn't solve:

* **Remote access** — your team can't reach `127.0.0.1` from their laptops
* **Authentication** — the runtime has no per-user auth (or a single shared key)
* **Model scoping** — everyone has access to every model, or no one does
* **Networking** — port forwarding, Tailscale, Cloudflare Tunnel, CGNAT

A **local LLM API** fills that gap. It sits between your runtime and your clients, providing:

* A public endpoint (e.g., `https://api.tokios.com/v1`) that routes to your local model
* Per-user API keys with model scoping
* Outbound-only tunneling (no inbound ports)
* OpenAI-compatible interface (any OpenAI SDK works)

## What is a local LLM API?

A local LLM API is an API layer that exposes your locally-running language models to remote clients through an authenticated, OpenAI-compatible endpoint. It typically includes:

* **Authentication** — API keys (`sk-tok-…` or similar) that identify each user or app
* **Authorization** — model scoping (who can access which models)
* **Routing** — mapping public deployment names to upstream model ids
* **Protocol translation** — converting between different API formats if needed
* **Tunneling** — connecting your local runtime to the public endpoint without inbound ports

The result: your team calls `https://api.tokios.com/v1` with their API key and the deployment name (e.g., `team-llama`), and the request routes through the tunnel to your local model.

## Options for exposing local models

### Raw runtime API

Ollama, vLLM, llama.cpp, and LM Studio all expose an OpenAI-compatible `/v1` API on localhost. This is the simplest option, but it has limitations:

* **No authentication** (Ollama) or **single shared key** (vLLM, llama.cpp, LM Studio)
* **Localhost only** — no remote access without additional networking
* **No model scoping** — everyone has access to every model
* **No per-user tracking** — can't tell who's making requests

**Best for:** personal use, single-machine setups, development and testing

**Not for:** team access, remote clients, production workloads

### LiteLLM

[LiteLLM](https://github.com/BerriAI/litellm) is an open-source proxy that provides an OpenAI-compatible API for 100+ LLMs (local and cloud). It's a popular choice in the r/LocalLLaMA community.

**Strengths:**

* Multi-provider routing (local + cloud in one endpoint)
* OpenAI-compatible interface
* Load balancing and fallback
* Active community

**Limitations:**

* Proxy model — your client calls LiteLLM, LiteLLM calls the runtime. Both need to be reachable.
* No built-in tunneling — you still need to solve networking (Tailscale, Cloudflare Tunnel, etc.)
* Single shared API key (no per-user auth out of the box)
* Self-hosted — you run the proxy server

**Best for:** multi-provider setups (local + cloud), teams comfortable managing their own proxy

**Not for:** teams who want a managed solution, users behind CGNAT, non-technical users

See [Tokios vs LiteLLM](https://docs.tokios.com/guides/tokios-vs-litellm.md) for the full comparison.

### LocalAI

[LocalAI](https://github.com/mudler/LocalAI) is a self-hosted, OpenAI-compatible API server that supports multiple model formats and modalities (text, images, audio, video).

**Strengths:**

* Broad modality support (text, images, audio, video)
* OpenAI-compatible interface
* Self-hosted, privacy-focused
* Active development

**Limitations:**

* Self-hosted — you run the server
* No built-in tunneling — you still need to solve networking
* Single shared API key (no per-user auth)
* More complex setup than Ollama or vLLM

**Best for:** multi-modal workloads (text + images + audio), teams who want a self-hosted OpenAI replacement

**Not for:** teams who want managed infrastructure, users behind CGNAT, simple text-only use cases

### Tokios

Tokios is a tunnel + API gateway designed specifically for team access to local models. It provides:

* **Outbound-only tunnel** — the connector dials out, so no inbound ports, no port forwarding, no Tailscale required. Works behind CGNAT, corporate firewalls, and NAT.
* **Per-user API keys** — each teammate gets their own `sk-tok-…` key, scoped to specific models
* **Model routing** — serve multiple models (Ollama, vLLM, llama.cpp) through one endpoint
* **OpenAI-compatible endpoint** — `https://api.tokios.com/v1` works with any OpenAI SDK
* **Managed infrastructure** — no proxy server to run, no tunnel to configure

**Strengths:**

* Purpose-built for team access to local models
* No networking headaches (outbound-only tunnel)
* Per-user authentication and model scoping
* Works behind CGNAT (Starlink, 5G, apartment ISPs)
* Managed service — no infrastructure to maintain

**Limitations:**

* Requires internet access (the tunnel needs outbound connectivity)
* Not suitable for air-gapped environments
* Managed service (not self-hosted)

**Best for:** teams who want to share local models with remote access, users behind CGNAT, non-technical users who want a simple setup

**Not for:** air-gapped environments, teams who require self-hosted infrastructure, multi-cloud routing

## Comparison matrix

| Feature | Raw runtime | LiteLLM | LocalAI | Tokios |
| - | - | - | - | - |
| **Authentication** | None or single key | Single key | Single key | Per-user API keys |
| **Model scoping** | No | No | No | Yes |
| **Networking** | Localhost only | Self-managed | Self-managed | Outbound-only tunnel |
| **CGNAT support** | No | Manual | Manual | Automatic |
| **Remote access** | No | Yes (with tunnel) | Yes (with tunnel) | Yes (built-in) |
| **Multi-model routing** | No | Yes | Yes | Yes |
| **Multi-provider** | No | Yes | No | No (local only) |
| **Setup complexity** | Low | Medium | Medium-High | Low |
| **Infrastructure** | None | Self-hosted proxy | Self-hosted server | Managed |
| **OpenAI-compatible** | Yes | Yes | Yes | Yes |

## Setting up a local LLM API with Tokios

**1. Install the connector on your model host**

On the machine running your model (Ollama, vLLM, llama.cpp, LM Studio), install the Tokios connector:

```bash Linux theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX
```

Get your one-time claim code `TKS-XXXX-XXXX` from the [dashboard](https://tokios.com/console) **Setup** tab.

**2. Claim the connector and point it at your runtime**

In the dashboard, **approve** the device. Then configure the connector to point at your runtime's local API:

* Ollama: `http://127.0.0.1:11434/v1`
* vLLM: `http://127.0.0.1:8000/v1`
* llama.cpp: `http://127.0.0.1:8080/v1`
* LM Studio: `http://127.0.0.1:1234/v1`

**3. Register your models as deployments**

On the **Models** tab, map each upstream model id (e.g., `llama3.1`) to a public deployment name (e.g., `team-llama`). Clients send the deployment name in the `model` field.

**4. Create API keys for your team**

On the **Keys** tab, create a key for each teammate. Scope each key to specific deployments. Share the key and the endpoint URL (`https://api.tokios.com/v1`) with your team.

Your team can now call your local model from anywhere:

```python theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="team-llama",
    messages=[{"role": "user", "content": "Hello from the team laptop"}],
)
```

See [Team access to local models](https://docs.tokios.com/use-cases/team-access.md) for the full walkthrough.

## Client integration examples

```bash curl theme={"theme":{"light":"github-light","dark":"css-variables"}}
curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "team-llama",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

```python OpenAI SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="team-llama",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
```

```javascript OpenAI SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.tokios.com/v1',
  apiKey: 'sk-tok-YOUR_KEY',
});

const response = await client.chat.completions.create({
  model: 'team-llama',
  messages: [{ role: 'user', content: 'Hello' }],
});
console.log(response.choices[0].message.content);
```

```typescript Anthropic SDK theme={"theme":{"light":"github-light","dark":"css-variables"}}
import Anthropic from '@anthropic-ai/sdk';

const client = Anthropic({
  baseURL: 'https://api.tokios.com/v1',
  apiKey: 'sk-tok-YOUR_KEY',
});

// Tokios translates Anthropic API calls to your local model
const response = await client.messages.create({
  model: 'team-llama',
  max_tokens: 100,
  messages: [{ role: 'user', content: 'Hello' }],
});
```

Any OpenAI-compatible client works — Python SDK, JavaScript SDK, curl, Continue, Cline, Open WebUI, and [dozens more](https://docs.tokios.com/guides/compatible-clients.md).

## FAQ

> **Can I use any OpenAI-compatible client?**
>
> Yes. Any client that speaks the OpenAI `/v1` API works — Python SDK, JavaScript SDK, curl, Continue, Cline, Open WebUI, Cherry Studio, and many more. See [Compatible clients](https://docs.tokios.com/guides/compatible-clients.md) for the full list.

> **What about streaming responses?**
>
> Tokios handles streaming correctly. Set `"stream": true` in your request and the response streams token-by-token. Some tunnel solutions (Cloudflare Tunnel) have SSE buffering issues that break streaming — see [Tunnels and streaming](https://docs.tokios.com/guides/tunnels-and-streaming.md) for details.

> **Can I serve multiple models?**
>
> Yes. Register multiple deployments (e.g., `team-llama`, `team-qwen`, `team-coder`) and route them to different upstream models. You can even serve models from different runtimes (Ollama + vLLM + llama.cpp) through the same endpoint. See [Model routing](https://docs.tokios.com/guides/model-routing.md).

> **How do I handle authentication?**
>
> Tokios provides per-user API keys. Create a key for each teammate, scope it to specific models, and share it securely. You can revoke any key without affecting others. See [API keys](https://docs.tokios.com/setup/api-keys.md) for full details.

> **What's the difference between a local LLM API and a cloud API?**
>
> A local LLM API exposes your locally-running models to remote clients. The model runs on your hardware, your data stays on your network, and you control the infrastructure. A cloud API (OpenAI, Anthropic) runs the model on their servers — you send data over the internet, they run inference, and you pay per token.

> **Can I use this with LiteLLM or LocalAI?**
>
> Tokios is an alternative to LiteLLM and LocalAI, not a complement. If you're already using LiteLLM or LocalAI, you can replace them with Tokios for simpler team access. If you need multi-provider routing (local + cloud), you can use Tokios alongside LiteLLM — see [Tokios vs LiteLLM](https://docs.tokios.com/guides/tokios-vs-litellm.md).

## Next steps

* [Team access to local models](https://docs.tokios.com/use-cases/team-access.md) — full walkthrough for giving your team API access
* [API keys reference](https://docs.tokios.com/setup/api-keys.md) — scoping, sharing, and revocation
* [Model routing](https://docs.tokios.com/guides/model-routing.md) — serve multiple models through one endpoint
* [Tokios vs LiteLLM](https://docs.tokios.com/guides/tokios-vs-litellm.md) — detailed comparison
* [Compatible clients](https://docs.tokios.com/guides/compatible-clients.md) — find your client


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.