LLM Provider Failover

Why it matters

Failover shapes reliability. Understanding shapes design.

The architecture

Detect failure (5xx, timeout, rate limit).

Route to fallback provider.

Return normal response.

How it works end to end

Circuit breaker per provider.

Model-equivalent fallbacks.

Prompt-cache invalidation.

Cost tracking on fallback.