Moving away from NLU-first architectures was the right call. Retiring NLU entirely is the overcorrection. The thing teams are leaving on the table isn't AI capability — it's a fast, cheap intent signal the orchestrator can act on before committing to a full model call. That's a different argument than "NLU is better than the LLM." It isn't, for most tasks. It's cheaper and faster for the specific tasks where that's what matters.
What changed, and what didn’t
Frontier models closed the capability gap. A well-prompted LLM now handles intent classification, entity extraction, and ambiguity resolution well enough for most production tasks — without a separate training pipeline, a versioned intent taxonomy, or a slot-filling schema that someone has to update every time the product changes. NLU-first made sense in 2020. It doesn’t make sense now. That part of the correction is right.
What didn’t change: a lightweight classifier still returns a structured result in under 100 milliseconds at a fraction of the cost of a frontier model call. For a support system handling 40,000 conversations a month where 85% fall into eight known categories, that signal has real value — not because the classifier is smarter than the LLM, but because the LLM doesn’t need to run to handle the clear cases. The capability argument and the cost-and-latency argument are separate. Winning the first doesn’t settle the second.
The right use of NLU
The LLM orchestrates. That’s the correct frame — it owns the decision about what to do with input, not the NLU pipeline. But “orchestrates” doesn’t mean “does everything itself.” It means the LLM decides when to call a tool, and NLU is a tool.
Two patterns are worth distinguishing. The first is pre-routing: a lightweight classifier runs before the LLM call and returns a signal — intent, category, confidence — that the orchestrator uses to skip work. High-confidence, well-understood traffic takes a fast path. Ambiguous or low-confidence inputs go to the model. The LLM never processes the clear cases at full cost. The second is mid-reasoning tool use: the LLM is already handling the interaction and calls the classifier to get a structured result it needs — precision extraction, a calibrated confidence score, entity tagging on a document. The pattern is the same in both cases; what differs is when in the flow the NLU step runs.
Confidence calibration matters here more than it sounds. Trained classifiers give you a real number — 0.91 means something specific and testable. “I’m fairly confident this is a billing inquiry” from a prompted LLM doesn’t translate to a threshold you can put in an escalation path. For workflows with human review or handoff logic — which is most serious production systems — you need a score, not a sentiment. Discarding the tool that gives you one is a step backward for operational reliability, independent of the AI capability comparison.
Where teams go wrong
Keeping NLU in the orchestrator role out of inertia. The existing pipeline works, the migration is expensive, the team that built it has context. The cost is an architecture where the LLM is fighting the intent system — stuffing complex reasoning into response generation because the routing layer won’t let it get upstream of the problem. If the LLM is downstream of your NLU, it can’t fix what the NLU got wrong.
Retiring NLU because “the model can do it.” True for the median case. Not the point for pre-routing, where the question isn’t capability but cost and latency at volume. Teams that migrate everything to the LLM and then hit 5–10x the inference cost they budgeted, or find their human escalation rate climbed because they lost a real confidence threshold, made a different version of the same mistake: choosing an architecture for the wrong reason.
Takeaway
The LLM should orchestrate. That’s settled. The open question is what it orchestrates with — and a fast, cheap, well-calibrated intent signal is a useful instrument, not a legacy artifact. The case for keeping NLU isn’t that it’s smarter than the model. It’s that the model doesn’t always need to do the work itself to get what it needs.