The default routing rule most teams write is 'use Claude for code, GPT for chat, Gemini for long context'. It works — until it doesn't. Real traffic is messier than three buckets, and every model has topic-level strengths that static rules can't capture.
Our router model
We trained a 120M-parameter classifier on 8M prompts labeled with (winning model, task category, difficulty). Inference takes ~2 ms on CPU. Its output is a distribution over the eligible model pool, which we then combine with live latency and price signals in the bandit stage.
| Routing method | Avg quality | Cost | p50 latency |
|---|---|---|---|
| Static rules | 76.2 | 1.00× | 410 ms |
| Prompt-classifier only | 82.8 | 1.03× | 412 ms |
| Classifier + bandit (ours) | 90.1 | 0.91× | 395 ms |
You can BYO
If you already have a routing preference table, upload it and we'll blend it with the classifier's output. Great for enterprises with regulated topic → model mappings.