Api Key Rotation For Llm Providers

The $12,000 Git Push

In March 2024, an engineer at a mid-sized SaaS company accidentally committed an OpenAI API key to a public GitHub repository. Within 47 seconds, an automated scraper found it and started making requests. By the time the team noticed, the bill had hit $12,000. Now imagine that key wasn't just for text generation — it had access to fine-tuned models, stored embeddings, and a production deployment serving 50,000 users. Static API keys are ticking bombs. If you're building on LLM providers and you're not rotating keys, you're one `git push` away from a very bad day.

The Problem with Static Keys

LLM provider API keys are different from traditional API credentials. They carry direct financial liability — every request costs money — and they often gate access to proprietary data: fine-tuned models, uploaded documents, conversation history. A compromised database password can be changed in minutes with zero customer impact. A compromised LLM key can drain your budget, exfiltrate your training data, and generate harmful content under your account, all before you finish reading the alert email.

Most teams handle key rotation the same way they handle database passwords: manually, infrequently, and usually after something goes wrong. This approach doesn't work for LLM integrations because the blast radius is larger and the attack surface is wider. Keys live in environment variables, CI/CD secrets, container orchestrators, lambda functions, and developer laptops. Each location is a potential leak point.

Think of Keys Like Milk, Not Like Wine

Keys don't get better with age — they get more dangerous. The longer a key exists, the more places it gets copied, the more likely it ends up somewhere it shouldn't. Rotation is the practice of expiring and replacing keys on a schedule, with enough overlap to avoid service disruption.

Think of it like a hotel key card system. When you check out, your card stops working — but the hotel doesn't disable it the instant you hand it back. There's a grace period. New cards are issued before old ones are deactivated. The front desk always has a working card ready. API key rotation works the same way:

1. Issue a new key while the old one is still active

2. Deploy the new key to all services

3. Verify the new key works everywhere

4. Revoke the old key after a grace period

The grace period matters because deployment isn't atomic. You might update your Kubernetes secrets, but a pod is still running with the old key cached in memory. If you revoke too early, you get failed requests. If you never revoke, you've just accumulated keys.

Building a Rotation Manager

Here's a practical implementation using only Python's standard library. This manager tracks multiple keys per provider, handles grace periods, and determines which key is currently active:

import json
import secrets
from pathlib import Path
from datetime import datetime, timedelta, timezone


class KeyRotationManager:
    """Manages API key rotation for LLM providers with grace periods."""

    def __init__(self, state_file="keys.json", rotation_days=30, grace_days=7):
        self.state_file = Path(state_file)
        self.rotation_days = rotation_days
        self.grace_days = grace_days
        self.state = self._load_state()

    def _load_state(self):
        if self.state_file.exists():
            return json.loads(self.state_file.read_text())
        return {"providers": {}}

    def _save_state(self):
        self.state_file.write_text(json.dumps(self.state, indent=2))

    def _now(self):
        return datetime.now(timezone.utc)

    def add_key(self, provider, key_value=None):
        """Add a new key for a provider."""
        if provider not in self.state["providers"]:
            self.state["providers"][provider] = []

        entry = {
            "key": key_value or f"sk-{secrets.token_urlsafe(32)}",
            "created_at": self._now().isoformat(),
            "status": "active",
            "last_rotated": self._now().isoformat(),
        }
        self.state["providers"][provider].append(entry)
        self._save_state()
        return entry

    def get_active_key(self, provider):
        """Returns the newest active key, falling back to grace-period keys."""
        keys = self.state["providers"].get(provider, [])
        now = self._now()

        for entry in reversed(keys):
            if entry["status"] == "active":
                return entry["key"]

        # Fall back to keys still within grace period
        for entry in reversed(keys):
            if entry["status"] == "rotating":
                rotated_at = datetime.fromisoformat(entry["last_rotated"])
                if now - rotated_at < timedelta(days=self.grace_days):
                    return entry["key"]

        raise RuntimeError(f"No usable key for provider: {provider}")

    def check_rotation(self, provider):
        """Returns the key entry if rotation is due, None otherwise."""
        keys = self.state["providers"].get(provider, [])
        now = self._now()

        for entry in keys:
            if entry["status"] != "active":
                continue
            created = datetime.fromisoformat(entry["created_at"])
            if now - created > timedelta(days=self.rotation_days):
                return entry
        return None

    def rotate(self, provider, new_key_value=None):
        """Marks current key as rotating, adds a new active key."""
        keys = self.state["providers"].get(provider, [])
        now = self._now()

        for entry in keys:
            if entry["status"] == "active":
                entry["status"] = "rotating"
                entry["last_rotated"] = now.isoformat()

        new_entry = self.add_key(provider, new_key_value)

        # Clean up keys past grace period
        self.state["providers"][provider] = [
            e for e in self.state["providers"][provider]
            if e["status"] == "active" or
            (e["status"] == "rotating" and
             now - datetime.fromisoformat(e["last_rotated"])
             < timedelta(days=self.grace_days))
        ]

        self._save_state()
        return new_entry

    def revoke_expired(self, provider, revoke_callback=None):
        """Revokes keys past the grace period. Returns list of revoked keys."""
        keys = self.state["providers"].get(provider, [])
        now = self._now()
        revoked = []

        for entry in keys:
            if entry["status"] == "rotating":
                rotated_at = datetime.fromisoformat(entry["last_rotated"])
                if now - rotated_at >= timedelta(days=self.grace_days):
                    if revoke_callback:
                        revoke_callback(entry["key"])
                    entry["status"] = "revoked"
                    revoked.append(entry["key"])

        self._save_state()
        return revoked

Using the Manager

Wire this into a scheduled job that runs daily:

# Daily rotation check — run via cron, systemd timer, or cloud scheduler
manager = KeyRotationManager(rotation_days=30, grace_days=7)

for provider in ["openai", "anthropic", "google"]:
    needs_rotation = manager.check_rotation(provider)
    if needs_rotation:
        print(f"Rotating key for {provider}")
        new_key = fetch_new_key_from_provider(provider)
        manager.rotate(provider, new_key)

    revoked = manager.revoke_expired(
        provider, revoke_callback=call_provider_revoke_api
    )
    if revoked:
        print(f"Revoked {len(revoked)} expired keys for {provider}")

The `get_active_key` method is what your application calls at runtime. It always returns the newest active key, with automatic fallback to a grace-period key if rotation is mid-flight. This means zero downtime — even if a pod restarts during rotation, it picks up a working key.

Key Takeaways

• **Rotate on a schedule, not on a panic.** 30-day rotation cycles are a reasonable baseline. High-stakes deployments should rotate weekly.
• **Always use a grace period.** Revoking a key the moment you deploy a new one guarantees failures. Seven days gives you room to catch missed deployments.
• **Track key age, not just key existence.** A key that's been active for six months is a liability, even if it hasn't been compromised.
• **Automate the full lifecycle.** Creation, deployment, verification, revocation — if any step is manual, it won't happen consistently.
• **Use your provider's dashboard API.** OpenAI, Anthropic, and Google all expose APIs for key management. Automate key creation and revocation programmatically.
• **Audit key usage.** Most providers expose usage logs per key. If a key suddenly spikes in usage, that's a rotation trigger, not just a billing alert.
• **Store keys in a secrets manager.** The JSON file in this example is for illustration. In production, use Vault, AWS Secrets Manager, or GCP Secret Manager.

Next Steps

If you're running LLM workloads in production, key rotation is table stakes — but it's just one piece of a broader security posture. Check out our companion code repository for a complete working example including provider-specific revoke callbacks. For a deeper dive into securing your entire LLM pipeline, read our earlier post on secrets management for AI workloads and our guide to rate limiting as a cost-control mechanism.

Companion code

Written with AI assistance — reviewed by Toc Am