Skip to main content
Hermes Agent is an open-source autonomous agent runtime from Nous Research. Unlike a typical chat client, Hermes both exposes its own OpenAI-compatible endpoint for frontends to connect to, and consumes an OpenAI-compatible backend to do its own reasoning. This page covers the second part: pointing Hermes’s backend LLM provider at Tokios so its agent loop runs on your private local model.
Hermes Agent is a runtime, not a typical BYOK chat UI. Treat this integration as connecting one system to another (Hermes → Tokios), not as configuring a chat app.

​
Prerequisites

  • A running Tokios connector paired with your local model — see Connector install
  • A registered model deployment name — see Register a model
  • A Tokios API key (sk-tok-…) — see API keys
  • Hermes Agent installed

​
Configure Hermes Agent’s backend provider

Hermes reads its backend provider settings from ~/.hermes/config.yaml. The provider keys live under a top-level model: mapping — they are not top-level keys themselves:
~/.hermes/config.yaml
The nesting matters. If provider, base_url, and api_key sit at the top level, Hermes ignores them and silently falls back to its default provider — you get a working agent that never touches your local model.
Replace gemma-tunnel with the public deployment name you registered on the Models tab — not the local model id your backend (Ollama, llama.cpp, vLLM, or LM Studio) uses. Replace sk-tok-YOUR_KEY with your own key, and avoid committing it into a shared or version-controlled config file.
base_url must include /v1. Hermes’s custom-provider path sends requests to any server that implements /v1/chat/completions, so a base URL without /v1 fails to reach the Tokios API surface.
You can also set this interactively without editing YAML by hand:
Select Custom endpoint from the provider list, then supply the same base URL, API key, and deployment name. The exact wording and prompts of the hermes model interactive flow can vary by Hermes Agent release, so check yours.

​
Troubleshooting

The API key Hermes sent is missing, malformed, or has been revoked. Confirm api_key in config.yaml holds the full sk-tok-… key with no extra whitespace, and check its status on the Keys tab.
The model value doesn’t match a registered deployment. Confirm it exactly matches the public name on the Models tab — not your Ollama, llama.cpp, vLLM, or LM Studio local model id.
The connector for that deployment is offline or at its concurrency cap. Check the Connectors tab — the connector needs to show as Online, and the machine running it needs to still be up and reachable.

​
Next steps

Quickstart

Set up the connector and register your first model deployment.

Compatible clients

See base URL and key setup for other OpenAI- and Anthropic-compatible clients.