Skip to main content
Langfuse ships a drop-in replacement for the OpenAI SDK — from langfuse.openai import openai — so you get full call tracing by importing that instead of openai and pointing base_url at https://api.tokios.com/v1 with an sk-tok-… key. Because the wrapper mirrors the OpenAI client’s interface, anything you already send to Tokios (chat completions, streaming, tool calls) gets traced with no change beyond the import and the base URL.

​
Prerequisites

Langfuse account or self-host

Sign up for Langfuse Cloud or run a self-hosted instance, and grab your Langfuse public/secret key pair.

Install the connector

Pair tokios-connector with your local backend.

Register a deployment

Choose the public name your model field will use.

Create an API key

Generate an sk-tok-… key, scoped to the deployments you need.

​
Trace calls

1

Install both SDKs

2

Set your Langfuse credentials

Langfuse’s OpenAI wrapper reads its credentials from the environment:
Set LANGFUSE_HOST to your self-hosted instance’s URL if you’re not using Langfuse Cloud.
3

Import the wrapper and call your deployment

Import openai from langfuse.openai instead of the plain openai package, then use it exactly like the standard OpenAI client, with base_url pointed at Tokios and model set to your registered deployment name:
langfuse_tokios.py
Every call through this client is traced in Langfuse — no @observe decorator or manual span required for the call itself.
model is always the Tokios deployment name you registered in the Models tab, never your backend’s local model id, and the base URL must include /v1.

​
Cost tracking for local models

Langfuse calculates cost from a model’s per-token price, which it knows for hosted providers like OpenAI and Anthropic. A Tokios deployment fronts your own local model, so Langfuse has no built-in price for it — traces still capture prompts, completions, latency, and token counts, but cost may show as zero or missing until you tell Langfuse how to price that model.
To fix this, define a custom model in Langfuse’s model settings that matches on the deployment name you send as model, and set a price per token — including a $0 price if you just want usage tracked without a dollar cost. The exact steps can change between Langfuse releases, so check Langfuse’s own model-pricing documentation for the current flow before relying on cost figures.

​
Now trace it from anywhere

The tracing setup above works the same whether the client calling Tokios sits on your laptop or on a teammate’s machine — the connector still only dials out from your local backend. Once a teammate or a deployed agent has a scoped sk-tok-… key, their calls through the same langfuse.openai wrapper land in the same Langfuse project.

Compatible clients

See the full list of OpenAI- and Anthropic-compatible clients Tokios works with.

API keys

Create a separate scoped key per teammate or agent.

Request logging

Only need to read what was sent? Turn on request logging in the dashboard instead of running a tracing stack.