The harness, not the model, is where the reviews got cheaper

GitHub swapped Copilot’s bespoke code-review tools for the better-maintained grep, glob, and view that already power the Copilot CLI, and reviews got worse: cost went up, fewer real issues got caught. The postmortem GitHub published yesterday is worth reading because the tools were never the problem. The instructions were. The shared tools carried CLI-shaped guidance that told the agent to map an area before touching it, so faced with a pull request it browsed the repository, guessed likely paths, read broadly, and carried all that context forward into later reasoning, where every extra file it opened stayed in the window as tokens it kept paying for. Rewriting the tool instructions for how a reviewer actually reads a diff, start from the change, grep for callers, glob for likely handlers, view only the narrow ranges that answer the question, turned that regression into roughly 20% lower average review cost at the same quality. The counterexample is the part to keep: the same focused instructions applied back inside the CLI did nothing, because an interactive coding session has no single diff to anchor on. Shared tools scale across products; the workflow around them does not transfer for free.

That lesson, that the harness often decides more than the model does, showed up in three research drops over the same 36 hours, each attacking reliability from the program around the weights rather than the weights themselves. TrajAudit takes on failure diagnosis for repository-level coding agents, whose trajectories run long and are packed with low-signal noise, redundant program structure and verbose context that drowns the actual causal chain of a failure. Its investigator agent folds failure-irrelevant context by semantic saliency and seeds the search with priors pulled from the test failure report, then unfolds detail on demand. On RootSE, a new benchmark of 102 real repository tasks each annotated with the earliest decisive error step, it beats the strongest baseline by 10.8% and 21.6% on exact failure localization with and without a reference (paper). The annotation choice is the interesting bit: not “did the run fail” but “which step first doomed it,” which is the question a human triaging an agent’s transcript actually asks.

TTHE pushes the idea further and asks whether the harness can be rewritten during evaluation itself, from nothing but the unlabeled traces the agent produces on the test inputs. It keeps a population of candidate harnesses around one frozen model, has a proposer agent reason over their execution traces to suggest edits, and a judge commit an improved harness on execution-derived proxy signals, no gold labels, no weight updates, no separate adaptation model. Solver, proposer, and judge are just different programs wrapped around the same LLM, so every gain is a change to the surrounding code. Across text-to-SQL, competitive programming, software engineering, data science, and tool-use tasks it improves fixed ReAct-style baselines and leaves behind an inspectable program rather than a pile of per-query retries. The open problem the authors name is the honest one: how far you can trust an execution-derived proxy for correctness when no label is available.

The most concrete version of harness-over-model this cycle is a production deployment. Tool-Making and Self-Evolving LLM Agents compiles repeated standard-operating-procedure steps into validated, versioned tools before deployment instead of regenerating the same code on every request. The tool-maker watches live execution traces, reads the backend schemas, synthesizes candidate tools, and repairs them against labeled cases; at runtime the agent calls the tool directly and only falls back to code generation when it has to. In an Amazon fulfillment-center alarm-triage system running a 44-node SOP over heterogeneous metric backends, the tool calls cut p50 latency 42% and, across 1,500 historical alarms, dropped end-to-end error rate by up to 53% by killing the run-to-run variance in repeated steps. Because the tools return compact structured verdicts, a simpler direct-call architecture shaved another 62% off p50 in a controlled ablation. Versioning the tools also surfaced specification gaps and upstream data drift, which is the kind of operational payoff that rarely shows up in a benchmark table.

The frontier that all of this runs on moved yesterday too. OpenAI shipped GPT-5.6 as Sol, Terra, and Luna, and put a number on it: on the Artificial Analysis Coding Agent Index, Sol posts 80.0, 2.8 points above Claude Fable 5, while spending under half the output tokens, taking under half the time, and costing about a third less, per the AINews recap. Altman’s framing was 54% more token-efficient on agentic coding than the prior generation, with pricing from $1 to $5 per million tokens and a 90% cache-read discount. Read the leaderboard delta with the usual caution about index composition, but the efficiency figure is the one that carries: at agent scale the metric teams feel is cost per resolved task, not raw pass rate, and cache-read pricing is aimed squarely at the long, repetitive context that agentic loops generate.

Under all the tooling sits a human question nobody has settled. 3100 Opinions on Code Review in an AI World mines practitioner discourse to build a causal account of what happens when coding agents author entire pull requests: whether review becomes the bottleneck, whether human review is still load-bearing, and whether it erodes the codebase understanding that reviewing used to build. Repository-mining studies catch the surface trend and miss the mechanism, and the trends themselves keep flipping, which is exactly why the argument stays unresolved.

Worth watching next: whether anyone reports the GPT-5.6 token-efficiency claim on a codebase they don’t control, the way Databricks did its own agent benchmark on a multi-million-line repo. Vendor numbers on vendor indexes are a starting gun, not a finish line.