Building Effective AI Agents

When Anthropic's engineering team published their guide to building production agents in December 2024, synthesizing lessons from dozens of real deployments, the headline finding wasn't a new framework or a novel architecture — it was a warning. The teams doing best weren't the ones with the most sophisticated orchestration stacks. They were the ones who had resisted the urge to build sophisticated orchestration stacks in the first place.

What's Converging

The agent tooling market has grown faster than the engineering judgment needed to use it. Since late 2024, the landscape of open-source orchestration libraries — LangGraph, CrewAI, AutoGen, Pydantic AI, and others — has expanded substantially, each offering its own abstractions for tool-calling, memory, and multi-model coordination. The implicit promise is consistent: complexity is the obstacle between an idea and a working agent, and the library absorbs it for you. This framing has proven irresistible to developers under deadline pressure.

The failure reports from teams running agents in production tell a different story. When a multi-step agent misbehaves inside three layers of framework abstraction, identifying the failure point is genuinely hard — the library hides both the symptom and the fix. Several engineering teams have publicly noted that moving back to direct API calls, with explicit step-by-step code, dramatically reduced debugging time even when it meant writing more boilerplate. The frameworks that survived scrutiny were the ones adding observability and retry infrastructure, not the ones adding orchestration opinions.

Cost pressure has reinforced the same lesson. Inference prices for frontier models have fallen considerably since 2024, but agent loops multiply per-query expense by the number of steps in the chain. A five-step agent loop handling a task that a single well-optimized prompt could manage adequately costs five to ten times as much for no measurable output improvement. Teams running agentic workloads at scale have grown acutely aware of this ratio, and it is pushing architecture decisions back toward simplicity.

The Specific Development

The core conceptual contribution of Anthropic's engineering post is a clean distinction between workflows and agents — two categories that practitioners frequently conflate. A workflow follows a code-defined path: the programmer specifies the sequence of steps, and the model executes within that structure. An agent, by contrast, lets the model decide its own next steps dynamically, choosing tools and constructing plans at runtime without a fixed script. These profiles are fundamentally different in their risk and cost characteristics. Workflows are predictable, testable, and auditable. Agents are flexible but error-prone, with mistakes compounding across each self-directed step.

The post describes five patterns that together cover most real production use cases. Prompt chaining links LLM calls sequentially — each output becomes the next input, with programmatic gates that can halt or branch the chain at any point. Routing classifies an input first, then dispatches to a specialized downstream prompt tailored to that category, avoiding the performance degradation that comes from asking a single generic prompt to handle wildly different task types. Parallelization either splits a task into independent subtasks that run simultaneously or runs the same task multiple times and takes a majority vote when output confidence matters. The orchestrator-workers pattern uses one LLM to decompose a complex goal and assign components to worker models that execute independently. The evaluator-optimizer closes a feedback loop: one model generates an output, a second evaluates it against defined criteria, and the cycle repeats until the result passes.

Of those five, the evaluator-optimizer pattern is the one we think is most underused. The common instinct is to route a task through a single generation pass and accept whatever comes back. Adding a second LLM pass that reasons against an explicit acceptance checklist — not simply verifying an exit code, but asking whether the output actually satisfies the stated goal — catches a class of errors that automated testing misses entirely. The cost is one additional API call; at typical usage levels, that's negligible against the value of catching a reasoning failure before it reaches a human reviewer. Teams building internal developer tooling, where the standard for "done" is nuanced and task-dependent, stand to gain the most from this pattern.

The post also makes a pointed observation about tool documentation that deserves more attention than it typically gets. Every tool an agent can invoke needs the same engineering care as the system prompt itself — a precise statement of purpose, an explicit input schema, a clear description of what success and failure output look like, and at least one concrete example. Anthropic's finding from production deployments is that vague tool definitions are a primary driver of agent errors, not model capability limitations. Unlike model capability, this is a problem the developer controls entirely, which makes it unusually tractable.

What's Likely Next

The open question the post frames without resolving is what role orchestration frameworks should play in a world where direct API calls outperform them on debugging clarity and cost efficiency. The framework vendors that survive the next twelve months will likely be the ones that reposition as infrastructure rather than orchestration — providing retry logic, parallel execution plumbing, and observability tooling while leaving the decision-making structure to the developer. The ones that continue competing on abstraction depth will face increasing pressure from teams that have tried both and chosen the simpler path.

Evaluator quality is likely to become a first-class research focus in the second half of 2026. The evaluator-optimizer loop is only as reliable as the evaluator's ability to reason against acceptance criteria rather than surface heuristics like output length or formatting. Teams that develop strong LLM-based evaluators — models or prompts that correctly judge whether a complex task was actually completed, not just whether it looks complete — unlock the ability to run fully autonomous loops with meaningful quality gates. The benchmark for that isn't passing unit tests. It's producing output a domain expert would sign off on without re-review.

Source