Vals AI: agent teams cost 1.8-5.1x more, and only 1 of 4 gains was significant

Vals AI, an evaluation company, published a case study on 9 October 2026 testing whether agent teams are worth their cost. It ran GPT 6 Sol and Claude Opus 5.5 on its Vibe Code Bench, building 50 complete web apps from a product spec, each as a single agent and as a team, at medium and max reasoning effort. Vals reports that teams cost 1.8 to 5.1 times as much, and only one of the four score differences was statistically significant.

How the test was run

In the team setup a lead agent could hand work to up to five subagents at once; subagents could not spawn their own. Both leads got the same short instruction to split the work, delegate, then integrate and verify. Prompt, sandbox and grader were otherwise identical. Score is the share of UI tests passed, graded by browser agents. Each of the eight setups ran once.

What it found

• GPT 6 Sol, medium effort: single agent 77.6% at $1.22 per app; team 84.9% at $3.07. The +7.3 points is significant (p = 0.005).
• GPT 6 Sol, max effort: single 89.0% at $4.82; team 90.4% at $8.54. Not significant.
• Claude Opus 5.5, medium effort: single 91.5% at $4.08; team 91.2% at $9.10. Not significant.
• Claude Opus 5.5, max effort: single 89.8% at $23.77; team 93.2% at $122. The +3.4 points is not significant (95% interval -1.4 to +8.3).

Vals attributes most of the extra spend to cached input: each subagent holds its own copy of the spec and working state, re-sent on every call. At max effort, Opus subagents read a median 224 million cached tokens per app, against 55 million for the single agent.

Effort versus team

For Sol, raising effort helped more than adding a team: max effort lifted the single agent by 11.4 points, and its score sat 4.1 points above the medium-effort team (not significant, p = 0.08), at 1.6 times the cost and three times the runtime. For Opus, max effort alone did not raise the score at all (p = 0.48). Vals concludes that neither option was a clear default.

Time and orchestration

Sol's team finished faster at max effort, 27.6 minutes against 38.4. Opus's teams were slower, up to 2.3 times the single agent's runtime, because the lead wrote a shared contract file first and then delegated in sequential waves. Four Opus max-effort team runs took more than four hours and one hit the 5.25-hour limit.

Limits

This is one benchmark of full-stack web apps, one run per setup, and a prompt Vals wrote. Vals says the results may not carry over to other kinds of work. The Decoder, which reported the study on 11 October, also cites Noam Brown on a podcast saying multi-agent systems mainly buy speed rather than quality; that is a spoken remark, not a measurement, and I have not checked it against a transcript.