Your own replicas, your own limits
Shared serverless is the default. When you need predictable latency, a fixed precision or rate limits nobody else can eat into, take dedicated capacity on the same fleet.
No noisy neighbours
Replicas serve only your traffic. Queue depth and tail latency stop depending on someone else's batch job.
Pinned region & precision
Choose the data center and the served precision. Neither changes without a change request from you.
Limits you set
Per-key RPM and TPM ceilings are yours to allocate across teams, not a shared pool.
Autoscale with a floor
Set a minimum replica count for latency and a ceiling for cost. Burst overflow can spill to shared capacity or fail closed.
Contracted SLA
99.9% availability standard, 99.95% with multi-region replicas, with service credits written into the order form.
GPU-hour billing
You pay for the reservation, not per token. Cheaper than serverless above roughly 40% sustained utilisation.
Sizing
| Tier | Accelerators | Fits | Throughput | Price |
|---|---|---|---|---|
| S | 1× H200 | 8–30B dense, or a LoRA fleet | ~90 tok/s/stream | $3.20 / GPU-hr |
| M | 4× H200 | 70–120B dense, MoE up to 235B | ~140 tok/s/stream | $12.40 / GPU-hr |
| L | 8× H200 (NVLink) | 400B–1T MoE at BF16/FP8 | ~210 tok/s/stream | $24.60 / GPU-hr |
| XL | Multi-node, reserved | Custom weights, long-context serving | Sized with you | Annual commit |
Illustrative rates for planning. Final pricing is on the order form.
{
"model": "llmcloud/llama-4-maverick",
"provider": {
"endpoint": "dedicated:acme-prod",
"region": "eu-west",
"fallback": "shared"
},
"messages": [{"role":"user","content":"hi"}]
}When is dedicated cheaper than serverless?+
Roughly above 40% sustained utilisation of a replica. Below that, shared serverless per-token pricing usually wins — we will say so if your usage says so.
Can I run a model that isn't in the fleet?+
Any open-weight model we can legally serve, plus your own fine-tunes. Custom weights start at the M tier.
Does the gateway still fail over?+
Yes, if you allow it. Set provider.fallback to shared capacity or to third-party providers, or leave it unset to fail closed.
Which plan includes it?+
Dedicated endpoints are an Enterprise entitlement. Every tier can call the same hosted models per token as an ordinary provider; reservations, pinned precision and the SLA come with Enterprise.
How do I start?+
Join the beta from signup and tell us the model, region and expected sustained tokens per second.