Submit rate isn’t resolve rate

Across 1,750 trajectories on 50 SWE-bench Verified tasks, GPT-5 submitted a patch on every single run and resolved 44% of them; Llama 4 submitted on 99% of runs and resolved 18%; Gemini submitted least, on 70%, and still cleared more tasks than GPT-5, 50% to 44%. Those numbers come from Confident and Wrong, and they make a point that submit-rate dashboards hide: an agent that always acts looks productive and is often wrong. The failure the authors isolate is the unsettling kind. On a buggy task the agent returns a plausible patch on all five runs and none pass, the same misreading repeated rather than scattered noise. This silent semantic failure covers 80% of Llama 4’s failing runs and 68% of GPT-5’s, and it is invisible to exactly the monitors teams lean on: completion looks healthy and consistency looks healthy at the moment the agent should not be trusted. A second probe hands the model an already-fixed bug, where the correct move is to do nothing, and most models edit the correct code anyway. Action bias is what completion metrics reward.

That bias compounds when the next agent inherits the code. Is Agent Code Less Maintainable Than Human Code? builds controlled maintenance experiments with a framework called CodeThread, then has four frontier agents resolve follow-on tasks on top of either human-written or agent-written code. Building on agent code dropped resolve rates by up to 13.1%. The interesting part is what did not explain the drop: traditional maintainability metrics barely moved the regression. The signals that did were behavioral, namely changes in how agent code handles input validation and error handling, alongside downstream code size and task difficulty. The patch passes its own tests and raises the cost of the next edit.

The humans who are supposed to catch this are looking less. Habituation at the Gate tracks 400 repeat reviewers across 11,429 reviews over seven months in the AIDev dataset. Comparing each reviewer’s early and late episodes, approval of AI-generated PRs rose from 30.1% to 36.8%, and pooled by experience decile the gap reaches +14.5 points from first to tenth. It is not that the PRs got easier; median PR size stayed flat, and approval of human-authored PRs fell over the same window. What rose was review latency, up 3.5x, while inline comment volume dropped 22%. More time sitting in the queue, less time actually reading the diff. The authors read this as reflexive habituation under growing workload rather than earned trust calibration.

If part of the answer is agents that know when not to act, another part is agents that know what not to keep. AdaMem goes after memory bloat, the slow erosion where uniformly stored trivia crowds out the few facts a user actually needs and drags down QA accuracy over weeks. Instead of remembering everything, it learns a role-specific write policy from weekly QA feedback, refining it with a patch-style self-reflection step and rolling back updates that backfire. On the authors’ AdaMem-Bench it lifts QA accuracy by up to 9 points over a uniform Mem0 baseline while holding 9% less memory. Write control, not just retrieval, is the lever most memory systems leave on the table.

Orchestration is converging on the same instinct to commit to a contract before acting. CodeTeam generates a whole repository from a requirements document by splitting the job into stages: multiple Architect agents draft competing design sketches, a CTO agent selects and normalizes one into a machine-checkable contract specifying file ownership, public interfaces, and dependency constraints, and Developer agents implement under a dependency-aware scheduler with lightweight Git coordination while a QA agent runs tests and drives repairs. On the execution-based NL2Repo-Bench it reached a 42.3% average test pass rate in its fine-tuned setting, and ablations credit project-specific developer allocation and retrieval-augmented planning for most of the gain. The contract, checkable before any code is written, does the work that a prose plan cannot.

Underneath all of it is which model you point at the harness. Nathan Lambert calls GLM-5.2 a step change for open agents, the first open-weights release he would reach for inside an agent loop rather than treat as a benchmark curiosity. Community runs are starting to agree, placing GLM-5.2 and a handful of other open models within range of Sonnet on coding-agent scenarios rather than a clear tier below.

The throughline across today’s papers is measurement catching up to behavior: resolve rate over repeated runs instead of submit rate, downstream maintainability instead of single-issue pass rates, write control instead of raw recall. The open question is whether review keeps pace. If approval rates keep drifting up while inline comments drift down, the gate meant to catch silent semantic failures becomes the place they slip through.