Agent cost numbers are wrong until you count the whole tree

205,800 tokens. That’s what one arm of a two-arm agent benchmark spent inside sub-agents it spawned on its own initiative, none of it visible in top-line telemetry. The experiment, posted to r/LLMDevs in the last day, asked a plain question: does a persistent code graph save a coding agent tokens compared to grep, on a 2,125-file NestJS repo? The surface numbers said the graph-less arm won a subsystem-understanding task, 56k tokens to 76k. The transcripts said that arm had unfolded five sub-agents and actually spent about 262k, 3.5 times the arm it supposedly beat. The conclusion generalizes well past code graphs: “agent with tool X versus without” comparisons systematically undercount whichever arm delegates, because modern agents fan out sub-agents unprompted. A harness that reports only the top-level session is, in the author’s phrase, measuring politeness, not cost, and any published “our tool saves N%” figure now deserves the whole-tree question.

The per-class results under whole-tree accounting are worth keeping. Impact analysis over a literal symbol name: parity, 71.4k versus 71.9k, because grep on an exact string is cheap even at that repo size. Subsystem understanding: a measured 71% saving, 75.9k against roughly 262k, because the graph scoped the subsystem instantly and one agent answered what forced the other arm to orchestrate five. Full-repo audit: parity again, around 445k versus 500k. Where the saving exists it comes from orchestration you no longer need, not from cheaper individual calls. It’s n=1 per cell, one repo, one model tier, and the author flags all of it; the accounting critique stands regardless.

Cost runs through everything else in the last day too. Simon Willison shipped sqlite-utils 4.0rc2 with Claude Fable doing most of the work, and put an exact figure on it: $149.25 of estimated unsubsidized API cost, computed by running AgentsView inside the same session. The work is the more interesting half. A single review prompt against 4.0rc1 surfaced five release blockers, the worst a delete_where() that never committed and left the connection inside an open transaction, so the delete and every subsequent write were silently rolled back on close. Data loss, caught before the stable release. From there: 37 prompts, 34 commits, +1,321/−190 across 30 files. A cross-model pass followed, GPT-5.5 reviewing the RC2 diff, and it turned up two more P1 transaction bugs (db.query("update ...") committing before it raises, and INSERT ... RETURNING only committing once its generator is exhausted), which a fresh Fable session confirmed by experiment and fixed. Two habits worth stealing: review the documentation edits first to build a mental model of the change, and have one vendor’s best model routinely review the other’s. Willison used to consider that superstition and now does it habitually because it keeps finding real bugs.

At the heavier end of the spend curve, Lovable’s Alexander Lebedev has put $85,000 into tokens since January: $600 a month before Lovable, $25,000 a month by May. Output moved from 20-30 merged PRs a week to 150+, peaking at 293. The structure behind it is one human over six or seven agents, each with its own subagents; a workflow that classifies PRs by risk so human review concentrates on high-risk changes; roughly 75% of tokens on implementation and a growing 25% on automation, including automated skill authoring (“the last time I wrote a skill manually was in January”). The open problem he names is second-order: code review used to diffuse knowledge through a team, and nothing in the agent workflow has replaced that yet.

Two papers put sharper edges on the same themes. When Agents Do Not Stop names the failure class behind runaway spend: infinite agentic loops, feedback paths through model calls, tools, workflow transitions, and agent handoffs that nothing effectively bounds. The authors’ static analyzer, IAL-Scan, abstracts framework-specific agent code into a common IR, recovers explicit and framework-induced feedback paths, and confirmed 68 real infinite-loop defects across 47 of the 6,549 agent repositories it scanned, at 91.9% precision. An unbounded handoff cycle amplifies a single request into cost exhaustion and repeated external side effects; the same money that hides in sub-agent trees can also burn in control flow nobody bounded.

TestEvo-Bench is the evals item: a live, executable benchmark for test-and-code co-evolution, 746 test-generation and 509 test-update tasks mined from 152 open-source Java projects out of 59,950 candidate records, each anchored to a real commit and packaged with a runnable environment, so pass rate, coverage, and mutation score are execution-grounded rather than metadata guesses. Frontier combinations (Claude Code, Gemini CLI, and SWE-Agent over Opus 4.7 and Gemini 3.1 Pro) reach 77.5% on generation and 74.6% on update. Both numbers drop on tasks mined after training cutoffs, and drop significantly under per-task cost limits. The constrained score is the number that matters, for the same reason the A/B post matters.

Yohei Nakajima’s field notes from AI Engineer close the loop from the hallway track: the local-model room was packed, mostly for cost reasons; model routing is seeing real innovation, deterministic classifiers included; the conference had a dedicated area for “token billionaires,” people spending a billion tokens a week; and getting coding agents reliable over longer horizons on bigger codebases was the constant discussion. Also circulating: “company brain,” documenting how a company makes decisions rather than what data it holds, which is agent memory phrased as an organizational problem.

Watch two things from here: whether agent harnesses start exposing nested sub-agent cost as a first-class number instead of a transcript-archaeology project, and whether live benchmarks promote the cost-bounded score to the headline. A billion tokens a week is a budget line now; the tooling that accounts for it is still being invented.