Your VPC. Same router.
Deploy the open-source router on Docker, Kubernetes, ECS or your favorite runtime. Routing, caching, guardrails, keys and analytics — all local to your account.
Open-source core
The router, cache layer, key store and dashboard are open source under a permissive license. Vendor a fork if you need to.
Docker & Helm
One-line Docker Compose for local dev; production Helm chart for EKS/GKE/AKS with HPA, PDB and network policies.
Bring your own storage
Postgres for state, Redis for cache, S3 or R2 for logs. Or use the built-in embedded mode for evaluations.
OTel-native
Emit OpenTelemetry traces and metrics. Ship to Datadog, Honeycomb, Grafana Cloud, or your own collector.
Local key store
Provider credentials and virtual keys never leave your VPC — sealed with your KMS.
Managed control plane (optional)
Point at the managed dashboard for orchestration while keeping the data plane local. Ideal for hybrid teams.
# docker-compose.yml
services:
router:
image: ghcr.io/llmcloud/router:latest
ports: ["8080:8080"]
env_file: .env
redis:
image: redis:7-alpine
postgres:
image: postgres:16-alpine
environment: { POSTGRES_PASSWORD: dev }helm repo add llmcloud https://charts.llmcloud.ai helm install router llmcloud/router \ --set postgres.dsn=$PG_DSN \ --set redis.url=$REDIS_URL \ --set kms.provider=aws \ --set observability.otel.endpoint=$OTEL
What's the license?+
Router, cache, dashboard and CLI are Apache-2.0. Some enterprise modules (SSO, SCIM, private connectors) require a commercial license.
Do I need to open outbound traffic to llmcloud.ai?+
No. Fully self-hosted deployments talk directly to upstream providers. Optional control-plane telemetry can be disabled with one flag.
Can I run air-gapped?+
Yes. Pull images to your registry, provide model & provider metadata via a signed bundle, and run without any egress to us.