Hermes Agent: Switching Providers Mid-Run When a Provider Fails

Why this guide

Multi-provider setups fail in a thousand ways — Anthropic 529s, OpenAI 429s, local Ollama out-of-memory, OpenRouter budget caps, code-auth refresh failures, key rotations gone sideways. The fix isn’t to pray harder at the primary model; it’s to switch providers mid-task without losing context. Hermes v0.18.2 supports this three ways: a per-invocation --provider override, a persistent fallback_providers: chain in ~/.hermes/config.yaml, and the interactive hermes model picker for sticky swaps.

The per-provider failure shape

Every backend fails differently. The symptom in your terminal is what gives it away.

Provider	Failure	Code	What you see
Anthropic	API overloaded	HTTP 529	OverloadedError: 529 ... overloaded_error. Hermes retries up to api_max_retries (default 3), then falls back if a chain is set.
OpenAI / Codex	Rate limited	HTTP 429	RateLimitError plus a Retry-After header that Hermes honors before the next attempt.
OpenRouter	Budget cap / no credits	HTTP 402	payment_required; the auxiliary client marks OpenRouter unhealthy for 60s and skips it until the cooldown ends.
Mistral	Quota exceeded	HTTP 429	UsageLimitExceeded; same status code as OpenAI but no Retry-After and a different JSON shape.
Local Ollama	Out of memory	n/a (process)	Daemon returns model runner has unexpectedly terminated; Hermes treats it as a connection error.
Codex / OAuth	Auth refresh failed	HTTP 401	TokenExpiredError after a long idle; fix is hermes auth or wait for the next refresh tick.

The runtime fallback

Two ways to declare a fallback: a persistent chain in ~/.hermes/config.yaml, or a one-shot via the fallback subcommand.

Persistent chain (recommended for cron and unattended runs):

# ~/.hermes/config.yaml
fallback_providers:
  - provider: openrouter
    model: anthropic/claude-3-7-sonnet
  - provider: openai
    model: gpt-4o-mini

Managed interactively:

hermes fallback list                  # show current chain
hermes fallback add                   # picker — appends
hermes fallback remove                # picker — drops one
hermes fallback clear                 # empty the chain

Once a chain is set, if the primary fails with rate-limit, overload, or connection error, Hermes walks the chain in order. The chain is re-read on each chat invocation, so edits to ~/.hermes/config.yaml take effect on the next chat — no gateway restart needed.

The override flag

For a one-shot swap — testing a new model, debugging a 529, or running one query through OpenRouter — use the CLI flags:

• --provider <X> for the inference provider (minimax, openai, openrouter, anthropic, ollama-cloud, …)
• --model <X> for a specific model under that provider (anthropic/claude-sonnet-4, gpt-4o, minimax/MiniMax-M3)
• --toolsets <Y> for the toolset override (comma-separated)
• -m and -t are the short forms

All three live on hermes chat and on the top-level hermes invocation. There is no --fallback flag — fallback is configuration, not a flag.

To swap providers permanently (sticky across all chats), use the picker:

hermes model              # launches the TUI model picker; saves to config.yaml

hermes model does not take a --provider argument — it’s a picker, not a setter. To change provider non-interactively: edit ~/.hermes/config.yaml (model.provider: line) or run hermes config set model.provider <name>.

When to use which override

Situation	Command	Why
Anthropic 529 today, want OpenAI for one chat	hermes chat --provider openai -q "..."	Per-invocation swap; your default stays put
Local Ollama OOM, want Anthropic via OpenRouter	hermes chat --provider openrouter --model anthropic/claude-3-7-sonnet -q "..."	OpenRouter is the only path that gives you Anthropic without its own API key
Make the swap permanent	hermes model then pick	Updates ~/.hermes/config.yaml; next hermes chat uses the new default
Cron job — primary might be down	Set fallback_providers: in config; don’t hard-code --provider	The fallback chain is what survives a primary outage
Need a different toolset for one session	hermes chat --toolsets research,web -q "..."	Override --toolsets without touching the default list
Quick smoke test of a provider	hermes chat --provider <X> -q "Reply with OK"	One-liner; exits with the response or the error

How to read the failure log

Two places to look:

1. Live log — ~/.hermes/logs/agent.log (not chat.log; that name doesn’t exist in v0.18.2). Filter for the failing provider:

   tail -F ~/.hermes/logs/agent.log | grep -E "\[provider\]|overloaded|429|payment_required|unhealthy"

   Typical lines:

   WARNING agent.auxiliary_client: Auxiliary: marking openrouter unhealthy for 60s
       (payment / credit error). Subsequent auxiliary calls will skip it until HH:MM:SS.
   INFO agent.auxiliary_client: Auxiliary call: skipping openrouter (recently returned
       payment error, retry in 59s)

2. Session DB — ~/.hermes/state.db (SQLite, FTS5-enabled). Open it with the sqlite3 CLI or via hermes sessions. The messages table carries the assistant-side reasoning and any provider_error payloads for failed turns:

   sqlite3 ~/.hermes/state.db "SELECT role, substr(content,1,200) FROM messages
                               WHERE session_id='<sid>' AND content LIKE '%error%';"

   Look for the assistant turn where content contains ProviderError, RateLimitError, or OverloadedError; the user turn immediately before it is the one that triggered the failed request.

3. Status snapshot — hermes status prints the model + provider, every API key (truncated), and component health in one screen. There is no --provider-status flag — that command is fictional. Run hermes status plain, or hermes status --all for a redacted-for-sharing variant.

What not to do

• Don’t switch providers inside an in-flight session. A --provider override applies at chat-start only. If the session is mid-turn and the primary dies, let the turn settle, /exit, then restart with the override. Hot-swapping leaves the open SSE/HTTP connection holding stale auth headers.
• Don’t hard-code --provider X in a cron job if X is a single point of failure. If X is down, the cron will retry until api_max_retries (3 by default) and then fail. Set a fallback_providers: chain instead — that’s the whole point of the chain.
• Don’t change API keys while a chat is mid-turn. The open connection may already hold an OAuth bearer or cached API key; the new key won’t apply until the next chat. Use hermes logout <provider> then hermes auth between sessions, not during.
• Don’t confuse hermes model --provider with a flag — hermes model is an interactive picker only; it has no --provider argument. To set a provider non-interactively: hermes config set model.provider <name>.
• Don’t treat a 429 from Mistral like a 429 from OpenAI. Mistral doesn’t send Retry-After; a blind exponential backoff will work, but the retry budget may exhaust before the cap lifts. Re-check the cap, then either reduce request rate or use the fallback chain.

Verification checklist

#	Command	What it proves
1	hermes chat -q "Reply with OK"	Default provider (minimax on this box) is reachable end-to-end
2	hermes chat --provider openrouter -q "Reply with OK"	Per-invocation override is honored — same Hermes, different backend
3	tail -20 ~/.hermes/logs/agent.log	Last chat attempt is logged with provider name and timing
4	hermes fallback list	Persistent fallback chain (or “No fallback providers configured”)
5	hermes status	All configured providers + their API-key status (use this, not the fictional --provider-status)
6	sqlite3 ~/.hermes/state.db ".tables"	Session DB is reachable and the schema is intact

Sources

• Hermes Agent providers
• Hermes CLI commands reference
• Hermes fallback providers guide
• ABS companion: Anthropic provider setup on Hermes
• ABS companion: OpenAI and Codex auth
• ABS companion: OpenRouter for one-key multi-model routing

Last verified: 2026-07-14 against Hermes v0.18.2 (2026.7.7.2) with minimax primary, OpenAI for image gen, OpenRouter Anthropic.