Local LLM API (locallm API): expose your models with an OpenAI-compatible endpoint

​

The API gap

• Remote access — your team can’t reach 127.0.0.1 from their laptops
• Authentication — the runtime has no per-user auth (or a single shared key)
• Model scoping — everyone has access to every model, or no one does
• Networking — port forwarding, Tailscale, Cloudflare Tunnel, CGNAT

• A public endpoint (e.g., https://api.tokios.com/v1) that routes to your local model
• Per-user API keys with model scoping
• Outbound-only tunneling (no inbound ports)
• OpenAI-compatible interface (any OpenAI SDK works)

​

What is a local LLM API?

• Authentication — API keys (sk-tok-… or similar) that identify each user or app
• Authorization — model scoping (who can access which models)
• Routing — mapping public deployment names to upstream model ids
• Protocol translation — converting between different API formats if needed
• Tunneling — connecting your local runtime to the public endpoint without inbound ports

​

Options for exposing local models

​

Raw runtime API

• No authentication (Ollama) or single shared key (vLLM, llama.cpp, LM Studio)
• Localhost only — no remote access without additional networking
• No model scoping — everyone has access to every model
• No per-user tracking — can’t tell who’s making requests

​

LiteLLM

• Multi-provider routing (local + cloud in one endpoint)
• OpenAI-compatible interface
• Load balancing and fallback
• Active community

• Proxy model — your client calls LiteLLM, LiteLLM calls the runtime. Both need to be reachable.
• No built-in tunneling — you still need to solve networking (Tailscale, Cloudflare Tunnel, etc.)
• Single shared API key (no per-user auth out of the box)
• Self-hosted — you run the proxy server

​

LocalAI

• Broad modality support (text, images, audio, video)
• OpenAI-compatible interface
• Self-hosted, privacy-focused
• Active development

• Self-hosted — you run the server
• No built-in tunneling — you still need to solve networking
• Single shared API key (no per-user auth)
• More complex setup than Ollama or vLLM

​

Tokios

• Outbound-only tunnel — the connector dials out, so no inbound ports, no port forwarding, no Tailscale required. Works behind CGNAT, corporate firewalls, and NAT.
• Per-user API keys — each teammate gets their own sk-tok-… key, scoped to specific models
• Model routing — serve multiple models (Ollama, vLLM, llama.cpp) through one endpoint
• OpenAI-compatible endpoint — https://api.tokios.com/v1 works with any OpenAI SDK
• Managed infrastructure — no proxy server to run, no tunnel to configure

• Purpose-built for team access to local models
• No networking headaches (outbound-only tunnel)
• Per-user authentication and model scoping
• Works behind CGNAT (Starlink, 5G, apartment ISPs)
• Managed service — no infrastructure to maintain

• Requires internet access (the tunnel needs outbound connectivity)
• Not suitable for air-gapped environments
• Managed service (not self-hosted)

​

Comparison matrix

​

Setting up a local LLM API with Tokios

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="team-llama",
    messages=[{"role": "user", "content": "Hello from the team laptop"}],
)

​

Client integration examples

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "team-llama",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="team-llama",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.tokios.com/v1',
  apiKey: 'sk-tok-YOUR_KEY',
});

const response = await client.chat.completions.create({
  model: 'team-llama',
  messages: [{ role: 'user', content: 'Hello' }],
});
console.log(response.choices[0].message.content);

import Anthropic from '@anthropic-ai/sdk';

const client = Anthropic({
  baseURL: 'https://api.tokios.com/v1',
  apiKey: 'sk-tok-YOUR_KEY',
});

// Tokios translates Anthropic API calls to your local model
const response = await client.messages.create({
  model: 'team-llama',
  max_tokens: 100,
  messages: [{ role: 'user', content: 'Hello' }],
});

​

FAQ

​

Next steps

• Team access to local models — full walkthrough for giving your team API access
• API keys reference — scoping, sharing, and revocation
• Model routing — serve multiple models through one endpoint
• Tokios vs LiteLLM — detailed comparison
• Compatible clients — find your client