Agents finish their own jobs, and their merges still need fixing more often

Merged pull requests from AI coding agents attract verified follow-up fixes at 1.62 times the odds of merged human pull requests from the same repositories over the same period. That number comes from a study of 6,774 merged agent PRs across Codex, Copilot, Devin, Cursor, and Claude Code, set against 5,044 contemporaneous human PRs in repositories with at least 500 stars. The second half of the finding is the part worth carrying around: 69.6% of the verified fixes on agent merges came from the same agent, and 76.4% of the fix PRs were agent-authored in every commit. Agents are finishing their own jobs. They are also leaving more to finish.

This week’s issue runs through three threads that keep circling that pattern: agents working in parallel and stepping on each other, the cost of letting a single agent run long, and evaluations that measure something other than what they claim. Then the semantic-governance desk has its strongest week in a while, with three papers on what it takes to let a model query a governed database without producing a plausible answer to the wrong question.

Parallel agents pass alone and fail together

stale, a benchmark from Illinois and Columbia, isolates one failure mode of parallel coding agents: two patches that each pass their tests, merge without a textual conflict, and fail when combined because one changed an interface the other still relies on. The interference metric counts only tests that pass with each patch alone and fail with both, run on the same combined suite, and that detail turns out to matter. The team’s first grading procedure ran each solo patch against its own task’s tests and the merged patches against both, so a patch that broke the other task’s test on its own got counted as interference. After the correction, 834 runs on 417 mined Django PR pairs produced exactly one case. Reviewed history has already had its conflicts resolved.

The mechanism is real, though. On 36 constructed tasks that apply renames, required-argument additions, and return-type changes to 12 real Django helpers, GPT-5.5 produced interference in 105 of 108 blind runs, with a clean textual merge every time. A message of roughly 130 tokens describing the completed concurrent change recovered 82% of those runs, and 93% of the ones that still merged cleanly. The paper is careful about what it does not show: the constructed rates say nothing about how often this happens before review in practice, and the recovery result assumes a complete, accurate description of a finished change, which a running system rarely has.

Microsoft’s Agensh takes the opposite bet and removes the orchestrator entirely. Workers run a cooperation loop against a shared workspace, a message interface, and a shared context store, claiming subtasks and merging progress asynchronously. On the five hardest ProgramBench tasks with GPT-5.6-sol, scaling from 1 to 128 agents raised the mean final test-pass rate from 19.31% to 28.78%. On pandoc, 1,024 agents reached 55.06% against 33.89% for one. The headline is a new scaling dimension. The caveat is that these are pass rates under hard latency constraints, and the paper does not report what 1,024 agents cost per point gained. Read it beside stale and the open question writes itself: the coordination messages that recover 82% in the controlled setting are exactly what an orchestrator-free swarm has to generate on its own, mid-flight, from unfinished work.

Long-horizon agents waste money in three specific ways

Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents examined 1,200 trajectories from Claude Code and Mini-SWE-Agent on SWE-bench Verified and named three recurring wastes: subsumed retrieval, similar script generation, and test re-execution. Between them they touch 79% to 98% of tasks and account for up to 22.75% of task cost. The mitigation comparison, run over 10,000 further trajectories on held-out Verified and Pro tasks, is the useful part. Structure-aware retrieval added overhead and changed how the agent delegated, raising cost by up to 28.14% in some configurations. Skills the agent synthesized for itself came out low-level and trace-specific. Skills written by developers, high-level and trace-agnostic, cut cost by up to 41.73%, about double the best agent-synthesized result. The hand-written project instructions file is still earning its keep.

CliffCompaction, from a team including Tim Dettmers, works the other cost lever: context. The technique reduces cost by up to 50% under a bounded context while holding or improving Terminal-Bench, and it does so with two rules worth stealing. Compaction only truncates or drops content, never rephrases it. And it never compacts a compaction; each pass works on original content and discards the prior compacted output, so drift cannot accumulate. Under parallel test-time scaling, the per-rollout savings added over 10 points on Terminal-Bench for less than the cost of two full-context runs, and let Kimi K2.6 match Opus 4.7 at lower cost. On KernelBench the same technique sustained a session past a million tokens and reached CUDA kernel speedups of 2.23x after 200 steps and 3.58x after 400, ahead of specialized search algorithms. There is an open-source, scaffold-agnostic API-proxy implementation that sits in front of Claude Code, Codex, and other harnesses.

The memory story is less flattering. VibeMemBench evaluates memory systems on 111 coding targets from 90 SWE-rebench V2 repositories with 3,634 history trajectories, and keeps a target only when injecting verified prior experience improves its executable outcome in a reference setting. Transferred to five held-out solvers, direct injection of that verified experience raised resolution by 1.1 to 4.5 points on four of them and lowered step counts on all five. When four existing memory systems had to construct and retrieve the experience themselves from the same history, eleven of twelve solver-and-system pairings failed to beat the memory-off baseline. The useful experience is in the history. The systems are not delivering it.

Evaluations are cheaper, leakier, and more contract-dependent than they look

Three eval papers this week each undercut a different assumption. Trajectory-aware benchmark subset selection targets the cost of regression testing an agent, which can run to hundreds of millions of tokens per full benchmark pass. Across 76 configurations and three regression scenarios, selecting the instances closest to the centroid of each pass/fail outcome group in trajectory-embedding space beat random and stratified sampling on both average and worst-case error. A 10% subset keeps median estimation error under 5% while cutting token cost by roughly 90%.

SchrodingerRepo treats the test repository as a latent variable instantiated only when the agent enters the environment, eroding familiar cues through four transformation levels: problem-statement reconstruction, namespace remapping, intra-file layout reordering, and behavior-preserving rewriting. On SWE-bench Verified and SWE-QA, removing those cues consistently degraded performance and raised interaction cost across models, with the extra cost concentrated in exploration and localization. Some fraction of what leaderboards measure is familiarity with the repository rather than the ability to reason about one.

The sharpest of the three is a controlled Eurostat benchmark from Sabina-Cristiana Necula. Thirty tasks over seven Eurostat datasets, four conditions, three replicates, 360 task-runs of Claude Sonnet 5 generating Python against the live API with gold answers frozen in advance. Adding a frozen metadata card raised exact correctness from 27.8% to 51.1% and cut wrong-filter outcomes from 39 to four. Adding a three-attempt repair loop driven by sanitized execution diagnostics raised it to 76.7%. So did a control that got the same three attempts and no diagnostics at all: 69 of 90 in both, twelve discordant pairs in each direction, a paired difference of exactly zero. The retry budget did the work; the diagnostics rode along. The companion result is the one for anyone who builds evaluators. The first run used an output contract that never told the model which JSON key a ranking should carry or whether units were codes or labels, and the evaluator required both. Correct answers scored as wrong, the repair condition was understated by 23.4 points, and a significance test flipped. Even in the best condition, one executable answer in five was silently wrong. Total API cost for the primary experiment: about $6.48.

Failure is the main path in a 20-step loop

Petar Ivanov’s 5 Resilience Patterns for AI Agents opens with arithmetic that belongs on a wall. At 98% success per call, only about 67% of 20-call runs complete clean, so one run in three hits a failure. At 99% it is one in five. The request-level patterns protect a single call; agents are long, stateful, and act on the world, so each pattern needs an adjustment. Retry the step, not the run, and sort failures into what time fixes, what the model fixes when the error comes back as a tool result, and what nothing fixes. Let one layer own retries, because three step attempts on top of SDK defaults can mean nine model calls. Give every writing tool an idempotency key built from stable IDs. Checkpoint after every step so a crash costs one step, and version the saved state, because runs outlive deploys. Fall back across failure domains, which usually means the same model on another cloud rather than a sibling model at the same provider, and only to a model your evals passed. Cap steps, tokens, and wall-clock time, and note that the SDK default of a 10-minute timeout with two retries can stall a run for half an hour. The examples use Inngest, but the patterns transfer to Temporal or anything else with step-level records.

Governed data access is where the silent errors live

The semantic-governance desk had three papers that agree on one point: successful execution is not correctness, and the failures that matter return plausible numbers.

Bayer’s Ascent exposes medical coding, cohort building, and epidemiological question answering as a 31-tool MCP surface over both OMOP and native vendor schemas, with the caller’s identity propagated so queries run under the user’s own Snowflake grants. The evaluation set, EpiTrap, is built around recognized pharmacoepidemiological errors, such as counting prevalent cases as incident or applying one glucose threshold to mixed units, and it grades whether the reasoning avoided the trap regardless of the path taken. Against a fixed pipeline at 50% on the native schema and 49% on OMOP, agentic configurations with Sonnet 5 and DeepSeek-V4-Pro gained 26.7 and 20.4 points on average, at more tool calls and longer runtimes. Two negative results are worth as much as the positive one: a reflection prompt added 27% more tool calls and 13% more runtime without consistent gains, and withholding the specialized text-to-SQL tools from Sonnet 5 left accuracy unchanged while tool calls rose 1.5 to 2.1 times. From 12 July to 21 September, 36 users made 13,002 calls. The lessons section names the failure modes seen in production: plausible answers to the wrong question, silent scope drift in code lists, sparse fields reported without caveat. Epidemiological plausibility, not query success, became the validation gate.

Generating Query Context for Relational Databases, from Alekh Jindal’s team at Tursio, addresses the cold-start side. Natural-language interfaces need sample questions paired with data-model fragments, and producing that context by hand requires SQL and schema expertise and rots as schemas evolve. The paper defines query flows that capture common retrieval and analysis patterns, traverses them to generate data models, and instantiates those with values sampled from the database under weighted strategies for diversity and coverage. The experience report covers more than 50 real-world databases.

Benchmarking Text-to-SQL under Role-Based Access Control, revised this week and accepted at SIGMOD 2027, augments existing benchmarks with synthesized user roles and access policies. Role synthesis runs as structured reasoning over the schema, inferring the application context first and then deriving responsibilities and access scopes, with expert screening on top. The metrics separate SQL utility from access-control compliance, and the finding is that many systems with high unrestricted scores, open-weight models especially, degrade sharply once constraints are in place, because they violate the policy rather than answering with the permitted data.

Retrieval infrastructure wants chunks, and enterprise answers want receipts

Semantics Delivery Network argues that web retrieval infrastructure is still built for human clients: search returns URLs and snippets ranked for generic relevance, and CDNs cache URL-addressed objects without knowing which passage an agent needs. The proposal is an origin-authorized, hierarchical edge substrate that indexes, searches, and caches at chunk granularity with tenant-specific retrieval policies. The hard problem the authors name is that semantic retrieval has no explicit miss signal, so the network has to estimate when its corpus is incomplete or stale and trigger scoped discovery. Preliminary probes show a large gap between page content processed and chunks consumed, and higher answer quality per context token from chunk delivery.

EvidenT is the enterprise-assistant counterpart: a pipeline that verifies extracted evidence against retrieved documents before generation, using structured passage extraction plus deterministic lexical alignment, with no retraining. On roughly 500 real enterprise queries it improved gold-source hit rate by an average of 29% over prompting baselines and produced zero citations to URLs that were never retrieved.

What to watch

The stale authors want execution records of parallel agent work captured before review, which is the dataset that would turn a 97% constructed rate into a real prevalence estimate. Watch for whether Agensh-style swarms publish per-point cost. On evals, the Eurostat result that an under-specified output contract can reverse a significance test should push anyone running a leaderboard to publish the full contract shown to the model alongside the scoring code. And on governance, the pattern across Ascent, the Eurostat benchmark, and the RBAC work is that the validator, not the repair loop, is the component that buys correctness. Expect the next round of text-to-SQL systems to ship one.

Source note: the code-intel-copilot feed reported mirror mode with a last sync of 2026-09-01 at generation time, though the items above carry publication dates through 2026-09-25. Dates were taken from the items themselves.