Most teams are still asking the wrong question. "Should we build or buy?" was the right frame in 2021. In 2026, the AI capability gap between layers has largely closed. The infrastructure gap has not. The decision has gotten more precise, not easier. The right question is: *where does the hard part of your problem actually live?*
What changed
Two things happened simultaneously that most teams haven’t fully processed together.
Frontier model releases made the upper layers good enough for most tasks. Intent classification, document triage, structured extraction, decision support — these used to require specialized models or managed NLU pipelines to handle reliably. A well-crafted prompt against a current frontier model now handles most of them without the overhead of a managed platform’s opinionated flow structure.
At the same time, proficient coding assistants made building at lower layers cheap. The historical case for managed platforms was always speed and reduced maintenance burden — not just better AI. That advantage has compressed significantly. A developer working with a coding assistant today can build and maintain custom orchestration that previously required a team and a meaningful engineering investment. The “build” option is more accessible than it has ever been.
Together, these shifts mean the managed platform no longer wins on AI capability and developer productivity. It used to win on both.
What hasn’t changed
The infrastructure layer is still real, and underestimating it has consequences.
One pattern I’ve seen repeatedly: skilled engineering teams build something that works well in the POC — the AI experience is sound, the logic is correct — but it can’t meet production latency requirements. A recent example involved a team migrating customer service workflows to a custom ADK implementation, connecting from a cloud environment to on-prem infrastructure. The experience handling was solid. But they couldn’t return responses in under 8 seconds.
What they skipped: measuring baseline network latency across the cloud-to-on-prem path before building anything; implementing streaming so time-to-first-token wasn’t held hostage to full generation; connection pooling across a VPN; load testing under realistic concurrency; and explicitly testing the security boundary where on-prem firewall inspection adds latency that never shows up in a dev environment. Each of those is a known, solvable problem — but each requires investment that doesn’t advance the AI story, so it gets deferred until it becomes a crisis.
This isn’t isolated. Across teams I’ve worked with and spoken to, the pattern is consistent: custom solutions are excellent for POCs and often impressive in demos. Production is where the infrastructure bill comes due. The managed platform had already paid that bill.
The decision
Build at layers 2–3 (raw API or orchestration framework) when the hard part is the AI task itself, and the infrastructure requirements are low or already solved. A content pipeline processing raw congressional bill data into user-facing summaries is the clear case: the work is genuinely complex — legislative language is difficult to transform into something readable and useful — there’s no latency requirement, no telephony, no concurrent session management. A managed platform would add overhead and constraint with no infrastructure benefit. Full control over the prompt, chunking strategy, and output format is what the task actually needs.
Lightweight conversational handling fits here too — classification, intent detection, simple extraction early in an interaction where speed matters and persistence doesn’t. The productive pattern isn’t always choosing one layer for the whole system. A well-designed architecture may handle early conversational traffic at layer 2 and escalate to a managed platform once it becomes clear the interaction requires contextual persistence, deep integration, or long session state. Each layer does what it’s built for.
Use a managed platform when the hard part is the infrastructure — scale, latency, telephony, compliance, or multi-channel deployment. When the team that will own the system long-term isn’t primarily engineering. When the platform’s integrations represent work that isn’t your differentiator and would take significant time to rebuild correctly.
Where teams go wrong
Reaching for managed platforms for the wrong reason. The most common mistake: choosing a managed NLU platform because the AI felt safer in 2022 and never revisiting that decision. If you’re now fighting the platform’s intent system to do work the model should be doing directly, you’re paying infrastructure costs for AI capability that isn’t there anymore.
Treating POC performance as a production baseline. A custom orchestration layer that responds in 2 seconds against a local dev environment is not a system that responds in 2 seconds under 30 concurrent customer service conversations routed through a VPN to on-prem infrastructure. The infrastructure work — streaming, connection management, load testing, latency budgeting across every network hop — is unglamorous and doesn’t advance the AI story, so it gets deferred until it becomes a crisis.
Takeaway
Choose the layer where your problem is hard. Use managed infrastructure when the infrastructure is the hard part — it earns its cost, and it has earned it over years of production engineering. Build when the AI task is the hard part and your team can own the result. And be honest about which situation you’re actually in, because confusing the two is expensive in either direction.