Six nines out of ninety-fours.
Frontier providers hover around 94% single-region uptime. Stack three, route around outages, and combined availability crosses 99.9999% — the router does it per request, mid-stream.
How failover works
- 1. Router selects primary provider per the auto:* policy.
- 2. On 5xx / timeout / rate-limit, retry a healthy peer.
- 3. Mid-stream disconnect? Resume against a peer with the partial context replayed.
- 4. Provider health signals decay over 60s — recovering providers get traffic back.
- 5. Client sees a single response with
x-llmcloud-providerattributing the winner.
{
"model": "auto:quality",
"route": {
"prefer": ["anthropic/claude-4.5-sonnet"],
"fallback": [
"openai/gpt-5",
"google/gemini-3-pro",
"fireworks/deepseek-v3.1"
],
"on_error": ["timeout", "rate_limit", "5xx"],
"max_retries": 3
}
}Built into every request
Idempotent auto-retry
Timeouts, 429s and 5xx retry against a healthy peer with exponential backoff and a max-latency budget.
Mid-stream continuation
If a stream drops halfway, the router resumes on a peer and re-emits only the missing tokens — clients never see a broken stream.
Live provider scoring
Real-time p50 latency, error rate and rate-limit headroom feed the router. Sick providers get de-prioritized within seconds.
Multi-region posture
us-east, eu-west and apac clusters serve traffic locally with cross-region failover as a last resort.
99.99% managed SLA
Enterprise plans include a 99.99% router SLA with service credits — independent of any single upstream.
Public status page
Live incident feed and per-provider health at status.llmcloud.ai. Subscribe by webhook, email or RSS.
How does mid-stream failover not duplicate tokens?+
The router keeps a rolling window of what the client has already received. On resume it re-prompts the peer with the original messages plus an assistant continuation prefix, then dedupes the overlap before emitting deltas.
Does failover work with tool calls?+
Yes. Function-call schemas are provider-neutral. If a call is emitted before the failure, the router replays the tool_result on the peer and continues.
Can I disable failover for a request?+
Pass route.max_retries: 0 or set X-LLMCloud-Retry: none. Useful for evals where you want exact-provider attribution.