Six nines out of ninety-fours.
Frontier providers hover near 94% single-region uptime. Stack three and route around outages — combined availability crosses 99.9999%, per request, mid-stream.
How failover works
- 1. Router selects primary provider per the auto:* policy.
- 2. On 5xx / timeout / rate-limit, retry a healthy peer.
- 3. Mid-stream disconnect? Resume against a peer with the partial context replayed.
- 4. Provider health signals decay over 60s — recovering providers get traffic back.
- 5. Client sees a single response with
x-llmcloud-providerattributing the winner.
{
"model": "auto:quality",
"route": {
"prefer": ["anthropic/claude-4.5-sonnet"],
"fallback": [
"openai/gpt-5",
"google/gemini-3-pro",
"fireworks/deepseek-v3.1"
],
"on_error": ["timeout", "rate_limit", "5xx"],
"max_retries": 3
}
}Built into every request
Idempotent auto-retry
Timeouts, 429s and 5xx retry on a healthy peer with exponential backoff and a latency budget.
Mid-stream continuation
If a stream drops, the router resumes on a peer and re-emits only the missing tokens.
Live provider scoring
Real-time p50 latency, error rate and rate-limit headroom feed the router. Sick providers de-prioritize in seconds.
Multi-region posture
us-east, eu-west and apac clusters serve local traffic, with cross-region failover as a last resort.
99.99% managed SLA
Enterprise plans include a 99.99% router SLA with service credits — independent of any upstream.
Public status page
Live incidents and per-provider health at status.llmcloud.ai. Subscribe by webhook, email or RSS.
How does mid-stream failover not duplicate tokens?+
The router tracks what the client received. On resume it re-prompts the peer with the original messages plus an assistant prefix, then dedupes overlap before emitting deltas.
Does failover work with tool calls?+
Yes. Function-call schemas are provider-neutral. If a call already emitted, the router replays the tool_result on the peer and continues.
Can I disable failover for a request?+
Pass route.max_retries: 0 or X-LLMCloud-Retry: none. Useful for evals where you want exact-provider attribution.