Migrate · LiteLLM
Retire the proxy pager.
LiteLLM is great for hacking. When you're paged at 3am for a proxy OOM, llmcloud.ai gives you the same OpenAI-compatible interface without operating anything.
LiteLLM proxy config.yaml
model_list:
- model_name: fast
litellm_params:
model: groq/llama-3.3-70b
api_key: os.environ/GROQ_API_KEY
- model_name: smart
litellm_params:
model: anthropic/claude-3-5-sonnet
api_key: os.environ/ANTHROPIC_API_KEY
router_settings:
routing_strategy: usage-based-routing
fallbacks:
- fast: [smart]llmcloud.ai — one policy, no yaml
# nothing to deploy. one HTTP call:
POST https://api.llmcloud.ai/v1/chat/completions
Authorization: Bearer $LLMCLOUD_API_KEY
{
"model": "auto:speed",
"route": {
"prefer": ["groq/llama-3.3-70b"],
"fallback": ["anthropic/claude-4.5-sonnet"]
},
"messages": [{ "role": "user", "content": "Hi" }]
}Feature map
| LiteLLM | llmcloud.ai |
|---|---|
| router_settings.fallbacks | route.fallback in the request body |
| routing_strategy: usage-based-routing | auto:quality / auto:cost / auto:speed |
| cache: type=redis | Semantic caching, on by default, with cache-hit routing |
| success_callback: langfuse | Built-in analytics + OTel export + log webhooks |
| Virtual keys via /key/generate | Scoped virtual keys with budgets & RBAC |
| Guardrails plugins | PII, prompt-injection and content policies at the router |
| Deploy: helm, k8s, RDS, Redis, KMS | Zero deploy · or self-host the same stack |
Can I keep LiteLLM as the client SDK?+
Yes. Point litellm.completion at api.llmcloud.ai and it works — LiteLLM speaks the OpenAI protocol we serve.
What about self-hosting?+
We ship an open-source router that runs the same routing, caching and guardrails stack in your VPC. See Self-host.