Qolca Team · 2026-07-15 · 10 min read
Not every step in an AI product needs a frontier model. Classification, extraction, routing, and short replies are handled beautifully by a small, fast model at a fraction of the cost and latency. Here is how to decide which step needs which model, how to measure it, and how to fall back safely.
The single biggest lever on the cost and speed of an AI product is also the one teams reach for last: which model runs which step. The default is to pick the smartest model available and route everything through it, because it obviously works. It does work — and it is almost always the wrong default. Most of the work an AI product does is not hard. Deciding whether a message is a complaint or a question, pulling a date out of an email, choosing which of five branches to take: a small, fast model does all of these as well as a frontier model, at a fraction of the cost and a fraction of the latency. Model routing is the discipline of sending each step to the cheapest model that can do it well, and reserving the expensive model for the few steps that genuinely need it.
If you profile a real AI product, the calls sort into two piles. The large pile is mechanical: classify, extract, validate, route, summarize, produce a short structured answer. The small pile is generative and open-ended: write the persuasive reply, reason through the ambiguous case, produce the long-form draft. The mechanical pile is usually the large majority of your call volume, and it is exactly the work small models are good at. A small fast model in the Haiku class costs roughly a third to a fifth of a frontier model per token and returns in a fraction of the time. Routing the mechanical 80% to it, and keeping the frontier model for the generative 20%, is the highest-leverage optimization most AI products never make.
Small models shine when the task is well-scoped, the output is short or structured, and the space of correct answers is narrow. In those conditions there is simply not much room for the extra capability of a frontier model to matter. These are the workloads we route to the cheap model without a second thought.
The expensive model earns its price when the task is open-ended, the quality of the wording matters, or the reasoning has real depth and consequence. This is where the gap between a good model and a great one is visible to the user, and where trying to save money produces output that is obviously worse.
Notice that "the final answer" appears here. A common and effective pattern is to do all the cheap preparatory work — classify, retrieve, extract, route — on a small model, and then spend the frontier model exactly once, on the single generative step the whole flow was building toward. That is a different design from "use the big model for everything," and it is usually cheaper by a large multiple for the same user-facing quality.
Our production AI sales assistant is a concrete example of this split. When a visitor sends a message, the language-detection and intent-classification steps both run on a small, fast Haiku-class model, in parallel. A conditional context-summary step, which only fires on longer conversations, also runs on the cheap model. All of that is the mechanical 80%: labels and summaries, short in and short out. Only the step that writes the actual reply — the part a visitor reads and judges — runs on the more capable frontier-class model, and it receives a lean, pre-processed context rather than a giant prompt. The classify-cheap, generate-expensive split is deliberate, and it is the reason the assistant stays fast and inexpensive while still producing replies that read like a thoughtful human wrote them.
This is the same architecture we reach for whenever we build a customer-facing assistant or an AI agent for a business. It pairs naturally with the token discipline we describe in our piece on why you should stop sending giant system prompts — routing decides which model runs a step, and lean prompting decides how much context that step carries. Do both and the savings compound.
Routing decisions should be measured, not guessed. The good news is that the mechanical steps are exactly the ones that are easy to evaluate, because they have right answers. Here is the process we use to decide, for a given step, whether the cheap model is good enough.
Routing to a cheap model does not mean abandoning it to fail silently. The mechanical steps should be built with guardrails, because they feed everything downstream. Classifiers should return a value from a known, fixed set — and when the model returns something outside that set, the code should fall back to a safe default rather than propagate garbage. Extraction steps should validate their output against the shape you expect before trusting it. And the whole pipeline should have a graceful failure path: if any step errors, the system should degrade to a sensible response — a friendly message with a way to reach a human — rather than crash. Cheap models make these guardrails cheaper to run, because you can afford to validate, retry, and double-check without the cost of a frontier call each time.
Frontier models are extraordinary, and most of your product does not need one. The skill is not picking the smartest model — it is knowing, step by step, exactly how much intelligence each job requires, and refusing to overpay for the rest.
If your AI product runs everything through a single expensive model, there is almost certainly a cheaper, faster architecture hiding inside it. Mapping that out — which steps drop to a small model, which stay on the frontier one, and how to measure the swap — is the kind of work we do. Book a free initial consultation at https://calendly.com/qolca-info/consultoria-inicial-gratuita, or message us on WhatsApp at https://wa.me/51991376769, and we will help you find the 80% you are overpaying for.