Credential Pools — Rotate API Keys

Imagine showing up at an amusement park with a single ticket, only to be told mid-ride that your daily quota is exhausted. Frustrating, right? Now imagine holding a stack of tickets — when one runs out, you simply pull out the next. That’s exactly what a credential pool does for your AI agent.

Why You Need a Credential Pool

AI providers like OpenRouter and Anthropic enforce rate limits and billing caps on every API key. When you hammer a single key with requests, you’ll inevitably hit 429 (too many requests) or 402 (payment required) errors.

Hermes Agent’s credential pool lets you register multiple keys for the same provider. When one key taps out, the system automatically rotates to a healthy one — your conversation never skips a beat.

Note: A credential pool rotates keys within the same provider. A “fallback provider” switches to a completely different service. The system tries the pool first, and only falls back when all keys are exhausted.

How It Works

Your request
  → Pick a key from the pool (round-robin / least-used / fill-first / random)
  → Send to provider
  → Hit 429 rate limit?
      → If "usage limit reached" (e.g., ChatGPT's message) → switch key immediately (retrying is pointless)
      → If temporary throttle → retry once; if it fails again, switch keys
      → All keys exhausted → activate fallback provider
  → Hit 402 billing error?
      → Switch keys immediately (the failed key cools down for 1 hour)
  → Hit 401 auth expired?
      → Try refreshing the token (OAuth)
      → Refresh fails → switch keys
  → Success → continue the conversation

Quick Start

If you already have one API key (in your .env file), Hermes treats it as a single-key pool automatically. To enjoy multi-key rotation, just add more:

hermes auth add openrouter --api-key sk-or-v1-your-second-key

Run the command again with additional keys to grow your pool.

⚠️ Important: Key Rotation Resets Prompt Caching

Providers (Anthropic, OpenAI, OpenRouter) bind prompt caches to specific accounts/API keys. When the system switches to a fresh key, that key has no cache of your conversation — the next request must read the full history at full price. If you later rotate back to a previous key, its cache may still be valid (typically minutes to hours), but every rotation triggers at least one full-price read.

What this means: In long conversations, each key rotation adds a small cost — but the benefit is uninterrupted dialogue. It’s a trade-off worth understanding.

💡 Pro Tip

Credential pools shine for providers using API keys. If you’re on Nous Portal (one OAuth token grants access to 300+ models), you usually don’t need a pool — a single Portal account suffices.

Summary

A credential pool gives your AI assistant several “spare keys” — when one hits a limit or billing error, it automatically switches to the next, keeping your conversation flowing. For heavy API key users, this is a genuinely practical feature. Just keep the cache-reset cost in mind and plan your key count wisely for long sessions.

Practical advice: If you frequently hit rate limits, start by adding 2–3 keys. If that’s still not enough, consider configuring a fallback provider as a safety net.