Inherited work cuts coding-agent success by up to 53 points

In 58.9 to 88.2 percent of failed runs on DeepSWE, a frontier coding agent handed a half-finished repository submitted with exactly the same tests still failing, tests it passes most of the time from a clean checkout. That is the core result of SWE-Residual, posted yesterday, and the authors call the behavior acceptance bias: the agent reads the inherited work as the definition of what remains instead of checking it against the original request. The construction is tight. They take 198 residual states from 111 tasks in DeepSWE and Terminal-Bench, each one the state a predecessor agent left after declaring the task complete while hidden tests still failed, and give the evaluated agent only the original request and that state, with no trajectory and no hint that the earlier attempt failed. Across five frontier agents, success drops 20.6 to 53.5 points on DeepSWE and 20.6 to 34.9 on Terminal-Bench, and the DeepSWE ranking reverses. Anyone running agents in relay, whether a second session resuming a branch or a reviewer agent inheriting a worker’s diff, is working in exactly this regime, and clean-start leaderboards say nothing about it.

The same day, the SWE-Bench Pro maintainers published SWE-Bench Pro V2, and the size of the repair job shows how far a widely cited benchmark had drifted. Agentic audits, expert adjudication and blind re-implementation repaired or retired 480 of the original 731 tasks; 23 contracted engineers spent 404 hours on it, each in a language they write professionally, and a second engineer verified every revised instruction by solving the task from the text alone. The protocol now restricts network access to the model endpoint and re-grades every patch in a clean container, closing two routes the original harness left open: fetching the upstream fix, and collecting credit for changes outside the submitted patch. Frontier agents reach 99.4 percent on the 642 V2 tasks, so the full split is near saturation on release, while the 51-task Hard split still spreads systems from 52.9 to 98.0 percent. Set beside SWE-Residual, the eval problem looks less like missing headroom and more like unmeasured starting conditions, since a model that clears 99 percent from a clean container can still lose 50 points when the container is dirty.

Meta’s engineering blog made a related argument from production. The Thin Orchestrator describes a 1,300-line router that sends 20+ workflows to 14 specialist subagents for data-column onboarding across Meta’s recommendation pipelines, a job spanning three repositories, three languages and up to six pipeline stages. The first build was one agent loaded with every schema, codegen rule and population pattern for 10+ data sources, and past a certain context size it began applying one subsystem’s field-naming conventions to another. A separate single-agent design reached 70 to 80 percent success only when an engineer wrote a custom system prompt per task, and 10 to 20 percent without one. In the replacement, only one of the orchestrator’s seven fixed steps calls a model, delegation, and that call runs inside a subagent that loads only its own domain files. New use cases arrive as workflow templates that resequence existing agents, so pipeline migration shipped with zero new agent code. Column config prep went from three to four days by hand to under an hour, and about two-thirds of changes land without significant human edits. The figures are self-reported with no ablation, but the design rule is easy to copy: keep the component whose mistakes reach everything downstream out of domain reasoning.

Silent failure is also Laminar’s pitch. Their Launch HN post introduces flow-1, a small open model fine-tuned and then RL-trained on thousands of synthetic investigations of agent workflows, which they report matches GPT-6-sol at finding errors in agent traces at 23x lower cost. The harness is the part worth borrowing: flow-1 treats a trace like a repository with each span as a file, greps across a compressed representation of the whole trace, and reads the relevant spans in full. Findings come back as JSON in a user-defined schema and get clustered, and a cluster becomes an eval set for a coding agent to hill-climb against. The benchmark is Laminar’s own, so the 23x figure needs outside replication, but the target is correct; an agent that cites the wrong document while every call completes successfully never moves an error-rate dashboard.

Two papers on structured data carry the theme into analytics, where the silent failure is a plausible answer to the wrong question. Adversarial Intent Resolution drops the assumption that a natural-language question maps to one correct SQL query. AIR builds competing interpretations, executes them to confirm they actually return different results, and then asks a clarification question grounded in the observed data difference, all without task-specific training. On AMBROSIA with schema-only context, which the authors treat as the realistic production case because exposing the full database is often infeasible, AIR beats the RL-trained IntentRL on F1 (0.679 vs 0.618) and degrades about 7 percent when the database is withheld, against roughly 20 percent for IntentRL. On AmbiQT, where ambiguity comes from schema design rather than phrasing, it leads under both context settings. Execution-verified divergence makes a better clarification trigger than a model’s self-reported uncertainty: it is checkable, and it gives a governed analytics agent two concrete result sets to put in front of the user.

Data.Readme, posted October 18 by Jiaming Liang, Yeye He, Haoyu Dong, Zachary Ives and Surajit Chaudhuri, works on the step before SQL: finding the right tables in a lake full of near-duplicates. An offline pass infers and indexes table-to-table relationships (confusable, joinable, union-able), and a small set of agent tools queries that index at task time. On lakes of up to 12K tables it improves accuracy by as much as 40 points across discovery, analysis and data-science tasks, beating coding agents given unlimited time and budget, and adding it to an existing coding agent cuts latency up to 6x. The confusable relation is the one a semantic layer should adopt, because it records the wrong-table error as machine-readable metadata before any agent makes it.

On retrieval, Yongjie Zhou, Shuai Wang, Bevan Koopman and Guido Zuccon ask whether conversational agents need explicit memory at all. They compare six state-of-the-art explicit memory systems with raw-history RAG on LongMemEval-S and BEAM-100K, under each system’s original retrieval settings and under a controlled one. Gains appear only for particular question types and tight context budgets, with no consistent advantage over a strong raw-history baseline, even though memory construction can take hours per conversation. Oracle retrieval separates two failure modes: some systems discard the answer-supporting information while building memory, and others keep it but cannot retrieve it. For a team weighing a memory layer on a long-running agent, the paper argues for spending on retrieval over what is already stored before building anything richer.

Two things to watch. Residual states are cheap to generate from any benchmark that logs failed submissions, and if Terminal-Bench or the Pro V2 maintainers adopt them, a dirty-start score could sit beside the clean-start one, closer to how agents are actually used. And the Pro V2 Hard split is the only slice with real spread left; whether it survives the next frontier release, or the next revision has to be organized around resumption rather than harder issues, is the open question for the next round of coding evals.

---

Sourcing note: the feed’s mirror_status call reported a last sync of 2026-09-01, but every item above was ingested on 2026-10-20 with publication dates of October 18 to 20, so the status timestamp appears out of date rather than the data.