An AI agent is a program that makes a language model act in a loop. It reads files, runs commands and writes edits, until a task is done. The language model is the program that reads and writes text behind an AI assistant. A team of agents brings several together, sharing the work.
A course reports an experiment: the same task, writing a program, given to four teams of agents organized differently. All had the same model. One of them, called a pipeline, lined up eleven agents in a chain: each took over the previous one's work. It had $50 to pay for the model's usage.
The pipeline didn't crash. It worked: plans, breakdowns, architectures. It went through five planning stages, meaning preparing the work without writing it yet. Then it ran out of money before writing its first function. Final output: zero lines of code.
A single agent, with the same model, got the top score: 28 out of 28.
The experiment comes from Beyond Code, a 2026 video course by Jeremy McEntire. I read it as a transcript for my book review, my critical summary of the course.
It bothered me, because I lived the opposite last spring. I had 26 AI agents work together on my own code. They found a real security vulnerability, a flaw an attacker can exploit.
So, single agent or multi-agent (a team)? Both results are true, because they don't measure the same kind of work.
Four agent setups, one task, 28 points to 0
The course's task: build the server side (the backend) of a vacation rental booking platform. It is split into seven microservices, seven small independent programs that talk to each other over the network. Every setup got the same language model and the same task. The pipeline had a $50 budget. The only thing that changed was how the work was organized:
- Single agent: one agent does everything, start to finish. Score: 28 out of 28.
- Hierarchy: a coordinator hands work to specialists. Score: 18. The coordinator ended up coding everything itself. Two limits held it back. First, a cap on steps per agent: past that number, it has to stop. Second, a smaller context: what the agent can keep in view at once. A comparison: a work table of limited size.
- Swarm: eight agents work at the same time on shared files, with no leader. They share tasks through a board where each one writes them down. Score: 9. The swarm delivered five services out of seven, 152 files, and service boundaries that were all incompatible. A boundary is the way two services exchange data. In the run, half the agents named fields in snake_case, like
customer_name. The other half wrote camelCase, likecustomerName. The services no longer understood each other. - Pipeline: eleven specialized agents in sequence, one per stage. The stages: diagnosis, decomposition, architecture, execution, testing, verification, review, and so on. A review is an agent that rereads another agent's work and can reject it. Score: 0.
The course puts it bluntly: "The pipeline spent its entire budget on five planning stages. It produced zero lines of implementation code." In other words, the $50 went to planning before a single line of code existed. The single agent cost roughly as much as the pipeline's planning alone, and it scored 28 out of 28.
Where do the 28 points come from? From a rubric of seven criteria, which checks whether the program follows a specification written in advance. The course names only four: endpoints, data models, API contract consistency and error handling. Endpoints are the addresses a service offers. Data models describe the shape of the information, such as a customer or a booking. An API contract is the written description of what a service promises the others: endpoints, formats, errors. The course doesn't detail how the 28 points are split across these criteria.
This rubric doesn't measure quality. The course admits it: it is a proxy. The protocol isn't published, the text doesn't name the model, and it doesn't say how many times each setup was run. Nobody is named behind "the researchers." The author also warns that these systems don't represent the state of the art. So 28 to 0 is an experiment story, not a measurement. I read it as an illustration of a mechanism, not as proof.
Part of the mechanism can be computed. When every agent has to agree with all the others, each pair opens a communication channel. With 5 agents, that makes 10 channels. With the swarm's 8 agents, 28. With 10 agents, 45. The formula is n(n-1)/2, for n agents. Coordination cost grows much faster than the agent count.
The pipeline, though, is a chain: each agent waits for the previous one's work. The course also explains its failure through another mechanism. At each handoff, an agent passes on only a summary of its work. What is lost never comes back. In another run, with a similar architecture on a more complex version of the task, the review stage rejected seven pieces of work out of eight. Four of those rejections cited no actual problem, only subjective objections, 15 to 23 each.
The course also reports a fifth setup, which wrote the contracts before the code. Its contracts were typed, meaning written in a strict format, and tests replaced reviews. It avoided the subjective rejections. In return, it burned its budget on ever more elaborate contracts: 607 lines of JSON for a four-function module. A contract, too, can become the goal instead of the means.
My 26 audit agents found a real vulnerability
Last spring, I had 26 agents audit ShareBox, one of my side projects: a PHP site that streams videos. An audit is a methodical read of the code to look for defects. The full story is in my 26-agent AI code audit.
Eleven reader agents each read a part of the code, in parallel. Two agents linked their reports. Twelve agents each scored a single axis: security, performance, architecture, tests. A last agent, the judge, was told to be strict. It reread all the scores to bring down the too-kind ones. That makes 11 + 2 + 12 + 1 = 26 agents.
The agents found a code injection: a value that ends up executed as program code. It hid in the application's startup script. That application runs in Docker, a tool that packs a program so it runs anywhere. The injection sat right under a comment, a note left in the code, claiming to prevent it. A real vulnerability, which I checked in the source before believing it.
My audit and the course's pipeline don't contradict each other. My audit also chained stages: read, link, score, judge. The difference lies in what each agent had to produce. My reader agents read. Each had its own slice, and none waited for another's work to start. Their output, a list of findings, didn't have to fit with the neighbor's. The final judge scored harshly, and I checked every flaw in the source.
The course's pipeline, on the other hand, was building. Each agent waited for the previous one's output, and everything had to come together into one working system.
Here's a rule I draw from it, and it's mine: the course doesn't state it this way. Parallel agents work on independent slices to read, when every finding can then be checked in the code. A chain of agents that must coordinate to build something burns itself out preparing.
The course's swarm confirms the rule from the other side. Its eight agents did work in parallel, but they were building, with no contract between them. The course sums it up: "Parallel decomposition without contracts produced volume without coherence." Plainly put, splitting the work in parallel without contracts produced lots of files, not a system whose parts fit together.
Feeling faster with AI: the METR study and the perception gap
The course cites a study you may have come across. It serves my topic: with AI, you can feel faster without being faster. METR is a nonprofit research organization that evaluates AI models. From February to June 2025, it ran a randomized controlled trial: chance decides, task by task, whether AI is allowed.
Sixteen experienced developers of open source projects, whose code is public, worked through 246 real tasks. Each task was randomly assigned: AI allowed or not. The main tool was Cursor Pro (a code editor with built-in AI), with Claude 3.5 and 3.7 Sonnet.
Before the study, the developers expected AI to make them 24% faster. Afterwards, they believed it had made them 20% faster.
On the clock, they took 19% longer. So they were slower, while feeling faster. That's a 39-point gap between perception and measurement (20 + 19). Sources: METR's write-up and the paper by Becker, Rush, Barnes and Rein on arXiv.
The limits are real. Sixteen people is a small sample. The tasks were on mature projects these developers knew inside out, with five years of experience on average. METR itself framed the result as a snapshot of early-2025 AI.
In February 2026, METR published a follow-up and now flags the 2025 study as out of date. It covers 57 developers. The new margins of uncertainty (confidence intervals) include zero: they cover both a gain and a loss. So no effect is demonstrated, either way.
METR calls these numbers unreliable: some developers now refused to work without AI. Its unquantified view is that early-2026 tools help more than the 2025 ones did.
So the 2025 number no longer holds, and no new one has replaced it. Still, nothing suggests the gap between perception and measurement has gone away. The eleven-agent pipeline is its extreme form: plenty of visible activity, plans, documents, and nothing that runs.
Mechanical verification over an opinion
The course's practical lesson fits in two sentences: "Advisory review asks whether someone did the work. Mechanical verification asks whether the work succeeded." An opinion asks whether someone did the job. A mechanical check asks whether the job works.
An opinion, even an agent's, says the work looks done. A test, a compiler or a script answers yes or no, and gives the same answer every time. The course calls it a mechanical gate: an automated check that blocks the next step until it passes.
In my setup, with Claude Code (Anthropic's coding agent, which you drive from a terminal), the idea took four forms. Each one answers a real problem.
- A hook that proofreads prose. A hook is a script Claude Code runs on its own before or after an action. After every edit to a book review, mine runs a prose check. It counts sentences over 25 words and spots semicolons and a list of banned words. It gives no opinion: it answers yes or no. The agent gets the feedback and fixes it before moving on, without me repeating the rule. More examples in my article on Claude Code hooks.
- A written instruction that loses to a default setting. My instruction file said: never put an AI signature in a commit (a saved set of changes in Git, the tool that keeps a project's history). By default, the tool adds a
Co-Authored-Byline naming Claude. The default won. The line landed in the shared history, the list of changes saved by everyone, and I had to rewrite that history. Since then, a hook rejects any commit or pull request command whose message carries that signature. A pull request is a request to merge your work into the main project. An instruction hopes to be read. A hook runs every time. - A colon that switches off a skill. A skill is an instruction file Claude Code loads when its description matches the request. That description is written at the top of the file (the header), in YAML: a configuration format where the colon separates a name from its value. A colon followed by a space, in unquoted text, makes the header invalid. Claude Code then loads an empty header without any message, and the skill stops triggering. I saw it on three skills in a single day. Only a script that reads every header one by one catches it.
- An AI judge that copies the score it's shown. Each of my book reviews carries a score out of 10. I had agent judges re-score four of them, with the published score as reference. First test: the judge first saw a proposed score, deliberately off by 1.5 points. Its score landed 0.95 points from the published one on average. Second test: the judge saw no score, but got a rubric and book-to-book comparisons. It landed 0.4 points away. Four books is a small sample. The published scores used as reference were themselves set by an AI, and I only adjusted a few of them. The test mostly measures resistance to anchoring: the first number you see pulls the next ones toward it. When you still need an opinion, remove whatever biases it (more in the article on why the reviewer is never the author).
Single agent or agent team: the rule I follow
The course offers a split I find sound:
- A single agent as long as the task fits in its context (what it can keep in view at once).
- Coordination through shared traces for loosely coupled work, meaning work whose parts depend little on each other. This requires mechanical checks that reject what doesn't fit. The agents read these traces and extend them. The traces are the files, the tests and continuous integration, a tool that reruns the tests on every push of code.
- A hierarchy only for ambiguous trade-offs, when several choices are defensible.
My current settings point the same way. These days, never more than two or three agents in parallel. One agent for a batch of small, similar tasks, rather than one agent per task. These rules came from a subscription that caps usage (a quota) and that I didn't want to burn through. They help quality too: each agent gets its slice, and nothing else.
That last point ties into another idea from the course: "The well-written tutorial is a trap." Well-written but off-topic text, placed in the agent's context, hurts the model more than random noise. Noise is easy to spot and ignore. A clean tutorial on the wrong subject, the model follows.
The course's example comes from AWS, Amazon's cloud host. A tutorial on the pricing of one of its services (EC2) is pasted in for a question about the pricing of another (Lambda). The two topics look alike, so the model believes it has the right page. Handing every agent the whole project code "just in case" sets exactly that trap.
To go further, two other articles on the blog summarize books on the subject. The first: what two books say about parallel agents. The second: four books on Claude Code.
Conclusion
The eleven-agent pipeline wasn't short on intelligence. Each agent produced a document for the next one, and none could fail a test, because there was nothing to run. My audit worked for the opposite reason: every finding could be checked in the code, and I checked it before believing it.
Adding an agent mostly adds coordination. When an agent team stalls, it rarely needs one more agent. It almost always needs something that can say no without arguing: a test, a hook, a script.