What Is AI Model Routing? How Production Systems Use Tiered Models to Cut API Costs

AI model routing is an architecture pattern that directs different tasks within an AI system to different model tiers based on what each task actually requires. A data extraction task goes to the cheapest model (Claude Haiku at $0.25/million tokens). A summarisation task goes to a mid-tier model (Claude Sonnet at $3/million tokens). Complex multi-step reasoning — if it exists in the workflow at all — goes to the most capable model (Claude Opus at $15/million tokens).

In production AI systems we've re-architected, model routing reduces monthly API costs by 80–95%. It is the single highest-impact cost optimisation available — and most development teams skip it entirely.

How does AI model routing work?

A routing layer sits between the application logic and the AI provider's API. When a task enters the pipeline, the router evaluates what kind of task it is and sends it to the appropriate model tier. The router doesn't change the task, the input, or the expected output — it only changes which model processes it.

The prerequisite is task decomposition. Most AI "features" are actually 3–7 distinct tasks bundled into one API call. Routing requires separating those tasks so each can be evaluated, prompted, and routed independently.

Which tasks go to which model tier?

Cheap tier (Haiku-class, $0.25/million tokens): structured data extraction, text classification, entity recognition, format conversion (JSON/CSV/XML), input validation, and routing decisions within agent workflows. These tasks are pattern-matching operations where the cheapest model matches the most expensive one on accuracy.

Mid tier (Sonnet-class, $3/million tokens): nuanced summarisation with tone control, content generation with brand voice, multi-step data analysis, document comparison, and quality assurance checks on cheaper model outputs.

Expensive tier (Opus-class, $15/million tokens): complex reasoning across large contexts, tasks with ambiguous inputs requiring domain judgment, architectural code generation, and edge cases where cheaper models demonstrably fail.

In most production systems, 70–85% of calls are cheap-tier tasks, 10–20% are mid-tier, and under 5% genuinely need the expensive tier.

What are the three routing strategies?

Static routing assigns a fixed model to each task type at build time. The mapping is determined during architecture by testing each task across tiers and selecting the cheapest one that meets the quality bar. This handles 90% of use cases.

Complexity-based routing analyses the input before choosing a model. Simple, structured inputs go to the cheap tier. Long, unstructured, edge-case-heavy inputs go to a more capable tier. This works for tasks where input complexity varies significantly.

Fallback routing starts cheap and escalates. Send to Haiku first. If the output fails validation (schema check, confidence score, format verification), retry on Sonnet. If that fails, escalate to Opus. Most calls resolve on the first attempt.

Why don't most development teams implement model routing?

Because it requires 3–5x more upfront engineering work for a benefit the client doesn't see until production volume arrives. A team building without routing picks one model, writes one prompt set, and ships. A team building with routing must decompose workflows, evaluate each task across tiers, write tier-specific prompts, build the routing logic, and instrument monitoring. The demo looks identical either way.

Vendors on fixed-price contracts have no incentive to do this work. They're paid to deliver a working product. The client pays the API bill. The cost problem only surfaces at scale — 10,000+ daily calls — when the monthly bill becomes impossible to ignore.

Can you add model routing to an existing system?

If tasks are already separated into distinct function calls, adding routing is a configuration exercise — 2–4 weeks. If the system uses monolithic prompts (multiple tasks in one API call), the pipeline must be decomposed first — 6–10 weeks. In both cases, the rebuild pays for itself within 2–3 months through reduced API costs.

The product doesn't change. Same inputs, same outputs, same user experience. What changes is the engineering underneath — and the monthly bill. We build AI systems with routing from day one — and re-architect existing ones that weren't built this way.