The harness, not the model, moved this week's numbers

Six foundation models, twenty-two locked tasks, one thing changed: the orchestration layer around the model. Everything else stayed frozen. Cost per task fell 41 percent, from $0.21 to $0.12. Median wall-clock fell 44 percent, 48 seconds down to 27. Tokens per task fell 38 percent, and task-completion held flat. That result, from “The Harness Effect”, is the week in miniature. The model was the constant. The scaffolding around it (context assembly, tool exposure, turn sequencing, delegation) was the variable, and the scaffolding is where the numbers moved. Nearly every strong paper this week points the same direction.

Benchmarks stopped scoring a single bit

The SWE-bench recipe, mine merged fixes from public GitHub and grade them with the tests that shipped alongside, has two problems a high score hides: the fix was probably in pretraining, and the shipped tests were written to confirm one specific patch rather than grade an arbitrary correct solution. “DeepSWE” closes both. Its 113 tasks are written from scratch across 91 active repositories and five languages, never contributed upstream, and each is graded by a hand-written verifier that accepts any working implementation. The payoff is measurable: when an independent LLM judge re-reviews graded runs, it disagrees with DeepSWE’s verifier 1.4 percent of the time, against 32.4 percent for SWE-Bench Pro’s inherited tests. The prompts run about half as long, the reference solutions touch 5.5 times more code, and frontier agents spread across a wider score band instead of clustering at the top of a saturated leaderboard.

“AgentLens” attacks the same single-bit problem from the usage side. A pass/fail score discards everything the person running the agent actually watches: how it follows instructions, uses tools, verifies its own work, recovers from mistakes. AgentLens grades the whole trajectory, pairing formal verification where an objective check exists with LLM-written trajectory reviews and side-by-side comparisons, so each run yields a readable reason for its score. The authors run it as a nightly pipeline to catch regressions in their own agent, which is the tell: this is an eval built to be lived with, not to top a table once.

If the benchmarks are getting sharper, “Reasoning effort, not tool access, buys first-try reliability” is the sharpest practitioner finding under them. Ninety independent runs built the same app from one spec, scored on a fixed 14-criterion rubric. Capability tier dominated: frontier models clustered near the 42-point ceiling while a low-cost local model fell to 24 to 37. The extras teams reflexively add did little. A browser-based testing tool raised cost 42 to 68 percent without improving functional score or reliability, even on interface-visible criteria. Reasoning moved the needle instead. Pushing effort from High to xHigh lifted first-try-perfect runs from 28 percent to 89 percent and cut corrective prompts roughly fivefold, for 9 to 29 percent more cost. Match the fix to the failure: most first-run failures came from weak reasoning, not from visible flaws a checker would catch.

The harness is the lever, and it has a tax

The harness-as-variable result is not an isolated benchmark. “SwarmResearch” makes it structural for open-ended coding work. A single long-running agent tends to lock onto one high-level approach early, then grind low-level edits while better approaches go unexplored. SwarmResearch puts a Shepherd Agent with global context in charge of a population of Search Agents, each working local context in its own git branch, and it beats or matches state-of-the-art LLM-guided evolution on 13 of 15 optimization tasks. The gain comes from higher-level exploration and from adapting how much parallelism to spend at different search depths, not from more raw compute thrown in a straight line.

The tax on all this iterative machinery is that it may never stop. “When Agents Do Not Stop” names the failure Infinite Agentic Loops: an agent repeatedly calling models, tools, or handoffs when the feedback path has no effective bound, burning budget and racking up repeated external side effects. These are not ordinary code loops; they emerge from the interaction of agent logic, framework semantics, and runtime observations, which is why they slip past normal review. The authors’ static analyzer, IAL-Scan, abstracts heterogeneous agent code into a framework-independent IR, builds a dependence graph over feedback paths, and flags paths that can reach costly operations without a bound. Across 6,549 agent repositories it confirmed 68 real loop failures in 47 projects at 91.9 percent precision. The orchestration layer that buys the gains is also the layer that can bankrupt a run.

Full history made agents worse than no memory at all

Three memory papers converged on one uncomfortable point: persisting more is often persisting worse. The cleanest number comes from Apple’s “Shared Selective Persistent Memory”. Across three enterprise scenarios, an agent given nothing scored 79 percent task completion. An agent given its full prior-session history scored 71 percent, below the blank slate, because stale reasoning traces anchored it to old solution paths. An agent given only four structured categories (task specs, data schemas, tool configs, output constraints) with the reasoning traces discarded hit 96 percent. Their complementary trick, decoupling generated programs from runtime data so a refresh needs zero model calls, cut recurring task time 14-fold. The lesson they draw is the one the week keeps repeating: context management, not model capability, is the lever.

“When Not to Write Memory” asks the governance version of the question. If agents write reusable memories from their own traces, the safety-critical decision is when to refuse. Repeated observations across agents are not independent evidence; the same claim can be copied from a shared source, induced by a shared prompt, or valid only in a narrow scope. Their GovMem reference policy estimates dependency-aware support, retrieves counterevidence, and returns promote, reject, or needs-review, cutting false promotion from 0.597 to 0.040 on synthetic stress tests. The real-trace result is the honest part: on 133 high-impact external coding-agent candidates, none were safe for automatic promotion, and all 11 that passed a naive verification gate were rejected as boilerplate, shared-tool artifacts, or non-reusable debugging traces. Memory write paths, the authors argue, should be treated as risk-controlled evidence systems, not append logs.

“Remember When It Matters” takes the read side. In long-horizon tasks, decision-relevant state (a requirement, an environment fact, a prior failed attempt) scatters across an expanding trajectory and gets buried or pushed out of the window right when a decision needs it. Its proactive memory agent surfaces the relevant fragment before the acting agent asks, rather than leaving retrieval to a once-per-turn store. Read together, the three papers redraw memory as an editorial function: choosing what to keep, what to refuse, and what to resurface, with forgetting as a feature.

Context is a budget, and a third of it is slack

If context is the lever, its cost is the observation stream. “CoACT” compresses the tool outputs an agent accumulates, but gates compression on a concrete signal: next-action preservation. A compressed observation is acceptable only if it induces the same next action as the raw one. A teacher model proposes several compressions, an action-preservation reward discards any that would change the agent’s move, and a length reward picks the most compact survivor to train a lightweight compressor. On SWE-bench Verified across three models, that cut total token consumption 33 percent while holding task-solving close to the uncompressed agent. “ContextSniper”, AntTrail’s code-memory layer for repository-level repair, targets the same waste from the input side, where whole-file reads and broad searches spend budget on code and logs that never decide anything. Both treat retrieval and context as an efficiency problem with real money attached, which is where the field has landed.

Review is the bottleneck; adoption is already real

The most grounded number on agentic review this week is a rejection rate. Mining “developer feedback to CodeRabbit” across 31,073 review-comment pairs on 10,191 pull requests in 239 repositories, the study found 56.3 percent of agent review comments rejected, 36.4 percent accepted, and 7.3 percent sparking discussion. Rejections clustered on false positives, redundant notes, and out-of-scope suggestions, and the agent skewed toward functional comments over maintainability ones while being more likely to be wrong. A lightweight classifier predicted rejection at up to 76 percent F1, which means the noise has learnable structure worth filtering before it reaches a human. The qualitative companion, “3,100 Opinions on Code Review in an AI World”, builds causal theory from practitioner discourse and finds no consensus: engineers disagree on whether review is now the bottleneck, whether human review stays necessary, and whether agent-authored PRs erode the understanding that review used to build.

Whether or not review is settled, use is not hypothetical. “Microsoft’s early-2026 rollout” of Claude Code and Copilot CLI, measured across tens of thousands of engineers, found first use spread mainly through social networks, retention tracked coding activity rather than demographics, and adopters merged roughly 24 percent more pull requests than they otherwise would have, a lift that held across a four-month window. Merged PRs are an imperfect proxy for value, and the authors say so, but the effect is neither uniform adoption nor a novelty spike.

Watch what happens when these threads meet. Evals are moving to grade trajectories and harnesses instead of models; memory is learning to refuse writes and forget on purpose; context is being priced by the token. The open question is whether harness-swap methodology and “when not to persist” governance show up as defaults in the tools people actually ship, or stay in the papers that measured them.