You fine-tuned a model. Now make it an API.

If you’ve already exported your fine-tune to GGUF and it’s running in Ollama or llama.cpp, you have a localhost endpoint but no way to call it from anywhere else. Keep the fine-tune exactly where it is, run the Tokios connector next to it, and register a deployment name for it — you get a real https://api.tokios.com endpoint, secured with an sk-tok-… key, that any OpenAI- or Anthropic-compatible client can call.

The last mile after fine-tuning

The documented fine-tuning path — for example with Unsloth — gets you as far as exporting a GGUF file and loading it with ollama create from a Modelfile, or serving it with llama.cpp’s server; check the exact export and ollama create command syntax against your Unsloth version. Both of those get your fine-tune answering requests on 127.0.0.1. That’s where the documentation usually stops. A localhost endpoint works from the same machine, in the same terminal, for as long as that process is running — it doesn’t work from a teammate’s laptop, a CI job, a hosted agent, or your own phone. Turning it into something callable from anywhere is the undocumented last mile, and it’s the specific problem Tokios solves: it doesn’t touch how the fine-tune was made or how it’s served, it just makes the endpoint reachable and authenticated.

Tokios doesn’t fine-tune, quantize, or export anything, and it doesn’t fix chat-template mismatches. It picks up wherever your runtime already exposes an OpenAI-compatible /v1 API and makes that endpoint callable outside localhost. Getting the fine-tune serving correctly, with the right chat template, is your runtime’s job — see Choose a local model if you’re still deciding how to serve it.

Steps

Confirm your fine-tune is serving locally

This guide assumes your GGUF is already loaded and answering requests through Ollama or llama.cpp’s server on localhost. For Ollama, that’s typically a model you loaded with ollama create <name> -f Modelfile and can already query with ollama run <name>. For llama.cpp, that’s llama-server listening on its default port — confirm the exact port against your install. Either way, don’t change how it’s bound — leave it on 127.0.0.1.

Pair the Tokios connector

Sign in at tokios.com/console, open the Setup tab, and click Start pairing. Tokios shows a one-time claim code like TKS-XXXX-XXXX, valid for about 15 minutes.On the machine running your fine-tune, run the install one-liner for your OS:

Windows

irm https://tokios.com/downloads/install/windows | iex; tokios-install -Code 'TKS-XXXX-XXXX'

macOS

curl -fsSL https://tokios.com/downloads/install/macos | sh -s -- --code TKS-XXXX-XXXX

Linux

curl -fsSL https://tokios.com/downloads/install/linux | sh -s -- --code TKS-XXXX-XXXX

Already have tokios-connector installed? Pass the code directly instead:

tokios-connector TKS-XXXX-XXXX

Back in the console, approve the device on the Setup or Connectors tab. The connector’s token is delivered over the tunnel once approved — you never see it on screen or paste it into a file.

Point the connector at your runtime's /v1 API

Tokios talks to whatever OpenAI-compatible /v1 surface your runtime exposes — it doesn’t matter that the model underneath is a fine-tune rather than a stock checkpoint. For the basic single-model path, the console’s Setup wizard fills in the upstream automatically based on the runtime you pick (http://127.0.0.1:11434/v1 for Ollama, http://127.0.0.1:8080/v1 for llama.cpp). If you’re serving multiple models or adapters through a tokios-connector.json config, set the upstream BaseUrl explicitly:

{
  "Upstream": {
    "BaseUrl": "http://127.0.0.1:11434/v1"
  }
}

See Connector config for the full multi-model file format.

In the Models tab, register your fine-tune and choose a public name — for example, my-finetune-tunnel. That public name, not the local model id you gave it in ollama create or llama.cpp, is what clients send in the model field. It’s also what tells your fine-tune apart from any base models or other fine-tunes you serve through the same connector.

Create an API key

In the Keys tab, click Create key. Optionally scope it to specific deployments with model-name patterns (comma-separated globs, * for all) — useful if you want a key that can only reach your fine-tune and nothing else. Copy the sk-tok-… key now — it’s shown only once.

Use your fine-tune from a coding agent

Once you have a deployment name and an API key, your fine-tune is reachable the same way any hosted model is — including from Claude Code or any OpenAI SDK client.

export ANTHROPIC_BASE_URL=https://api.tokios.com
export ANTHROPIC_API_KEY=sk-tok-YOUR_KEY
claude --model my-finetune-tunnel

from openai import OpenAI

client = OpenAI(
    base_url="https://api.tokios.com/v1",
    api_key="sk-tok-YOUR_KEY",
)

response = client.chat.completions.create(
    model="my-finetune-tunnel",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

curl https://api.tokios.com/v1/chat/completions \
  -H "Authorization: Bearer sk-tok-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-finetune-tunnel",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Claude Code’s model picker only lists claude* ids, so your fine-tune’s deployment name won’t show up in it. Pass --model my-finetune-tunnel explicitly, every time. See Use Claude Code for persisting the environment variables across sessions.

If responses look malformed, truncated, or off-tone, that’s almost always a chat-template mismatch between how the model was fine-tuned and how the runtime is prompting it — not something in the Tokios path. Tokios passes requests straight through to your runtime’s /v1 API and doesn’t rewrite prompts or templates. Fix the Modelfile or server prompt template at the runtime level, then re-test.

FAQ

Does Tokios care that this is a fine-tune and not a base model?

No. Tokios only sees an OpenAI-compatible /v1 endpoint and a deployment name you mapped to it. Whether the weights behind that endpoint are a stock checkpoint, a merged fine-tune, or a LoRA adapter makes no difference to the connector or the gateway.

Can I serve a base model and a fine-tune side by side?

Yes. Register each as its own deployment in the Models tab, or define multiple upstreams and routes in a tokios-connector.json config for one connector serving several models. See Connector config and Model routing.

Does streaming work with a fine-tune the same way?

Yes. Set "stream": true and Tokios returns standard server-sent events (SSE) in OpenAI or Anthropic format, depending on which API surface you called — the same as for any other deployment.

My fine-tune's chat template is wrong — can Tokios fix that?

No. Chat-template correctness — matching the special tokens and turn format the fine-tune was trained on — is entirely your runtime’s responsibility (Ollama’s Modelfile TEMPLATE, or llama.cpp’s server prompt template). Tokios forwards the request body as-is to your runtime’s /v1 API.

Your fine-tune never leaves your machine — your endpoint works everywhere

The weights stay on your hardware, in the same Ollama or llama.cpp process you already trust. The connector only dials out — no inbound port, no exposed API on your machine. Everything past that — auth, routing, streaming — is one endpoint you can call from any machine, teammate, or agent you choose to give a key to.

Quickstart

Pair a connector and register your first model end to end.

API keys

Create, scope, and rotate sk-tok-… keys for teammates and agents.

Hardware & fit

Serve & connect your runtime

Connect a client or app

Compare

You fine-tuned a model. Now make it an API.

The last mile after fine-tuning

Steps

Use your fine-tune from a coding agent

FAQ

Your fine-tune never leaves your machine — your endpoint works everywhere

Quickstart

API keys

​The last mile after fine-tuning

​Steps

​Use your fine-tune from a coding agent

​FAQ

​Your fine-tune never leaves your machine — your endpoint works everywhere

Quickstart

API keys

The last mile after fine-tuning

Steps

Use your fine-tune from a coding agent

FAQ

Your fine-tune never leaves your machine — your endpoint works everywhere