Apps · Infra & proxies

LiteLLM

Proxy and SDK that normalizes 100+ providers.

Tokens / 7d

118B

Routing directive

auto

On our metal

21%

Licence

Open source

Teams already running LiteLLM point one upstream at llmcloud and delete a pile of per-provider keys. The gateway becomes the single billing and failover surface behind their existing proxy.

→ litellm.ai → All apps → Integration docs

How it connects

Single upstream entry in the LiteLLM model_list.
Provider keys consolidated into one gateway key, or BYOK passthrough.
Spend tracking reconciled against gateway analytics.

Traffic pattern

Aggregated server-side traffic — no per-user shape, just steady throughput with sharp burst peaks.

claude-sonnet-4.5gpt-5.5deepseek-v4

Point it at the gateway

model_list:
  - model_name: auto
    litellm_params:
      model: openai/auto
      api_base: https://api.llmcloud.ai/v1
      api_key: os.environ/LLMCLOUD_KEY