Auto-approve takes harness attack success from 29% to 96%

Flipping a coding-agent harness to auto-approve raises attack success from 29.2% to 95.6%. That is the headline number from HarnessSecurity-Bench, a study from Zhengyang Zhu, Zibin Zheng, and colleagues that treats the harness, not the model, as the security boundary. The authors build a ten-mechanism taxonomy, rate 400 harness-mechanism cells with both researchers and LLM judges, and find that about half of the confirmed implementations are opt-in, with closed-source harnesses showing large evidence gaps. Then they run 23 tasks across five attack surfaces against six harnesses, Claude Code, Codex CLI, Gemini CLI, gptme, Qwen Code, and GitHub Copilot, with GLM-5.2 held fixed as the model so the harness is the only variable. The run is 2,500 trials, 81,155 tool calls, and 2.2 billion tokens, scored by deterministic oracles for both task utility and attack effect. The useful finding is the shape of the tradeoff curve rather than any single setting. Network isolation and read-only mode cut attack effects but cost a lot of utility; command denylisting cuts attack effects and improves utility; command allowlisting cuts attack effects at a small utility cost. Several task-level cases show allowed tools leaving protected operations reachable through alternative execution paths, which is the harness analogue of a firewall rule with a side door. The recommendation to providers is modest and correct: make security settings verifiable, and test the alternative paths to a protected operation rather than only the primary one.

Two papers on multi-agent coordination take opposite ends of the lifecycle, before the work starts and after it has failed. NP-Bench from Sumanyu Muku argues that reactive conflict detection arrives too late at agent speed, since the warning lands after the wasted edit, and recasts coordination as scheduling: take each work item’s declared scope, partition into disjoint scopes, and order merges along the producer-to-consumer graph before any agent touches a file. The planner lives inside Nerveplane and is evaluated on a three-arm benchmark (no coordination, reactive detection, proactive planning) that verifies integration off a real git merge. Clean integration goes from 1 of 9 scenarios to 9 of 9 and merge conflicts from 13 to 0, with the gap widening as the agent count grows. On a live breaking contract change, both baselines fail on every seed while the planner reaches a 1.0 clean-integration rate on a frontier model and 0.6 on a small one, with zero scope leakage. A cross-session memory drops the repeated-mistake rate from 1.00 to 0.00. There is also a reported negative: routing facts to agents does not improve long-context accuracy at window-fitting scales; its value is cost and capacity. And because the gain comes from how work is allocated rather than how the model reasons, it did not shrink as models got stronger across two vendors and two capability tiers.

MAScope picks up at the other end, once a multi-agent run has failed and someone has to say why. The observation is that similar symptoms in a trace can come from different problems in how information was passed, used, or verified, and that communication topology is a structural cue for telling them apart. A Trace Structural Extractor recovers the interaction graph from heterogeneous traces by grounding it in message evidence; a Topology-Conditioned Judge then classifies the failure using the trace, the predicted topology, an empirical failure prior, and a short description of topology-specific failure patterns. The association between topology and failure type is strong (chi-squared 409.9, p around 1e-70 on MAST-clean). On 851 traces, giving gpt-mini the ground-truth topology lifts Macro-F1 from 0.173 to 0.350; with predicted topology it reaches 0.346, close to the 0.372 that trace-only gpt-5.4 gets. Since a fixed orchestration structure lets the recovered topology be reused across runs, the projected cost for 1,000 traces is about 6% of repeated gpt-5.4 diagnosis. Read together with NP-Bench, the pattern is that structure declared up front (scopes, merge order, topology) is doing the work that model reasoning was being asked to do.

On the semantic side, ARCS from Yihao Hu, Yanlin Feng, Naoki Otani, and Nikita Bhutani attacks the ambiguity problem in text-to-SQL directly. The claim is that conversational clarification is inefficient and poorly matched to how people actually work with data, and the proposal is structured disambiguation: resolve ambiguity through explicit, constrained interactions rather than free-form dialogue. The corpus is the first text-to-SQL benchmark with naturally occurring, unconstrained ambiguities over real-world databases, annotated with every valid ambiguity point, each interpretation, and the SQL for each. The numbers are sobering: gpt-6-sol reaches only 51% end-to-end execution accuracy and no open-source model exceeds 27%. For anyone building a governed analytics layer, this is the measurement of what the semantic layer has to absorb. A metrics definition that pins down grain, time semantics, and the default interpretation of “active customer” is exactly the constrained interaction ARCS is asking for, pushed upstream of the query.

The Right Memory in the Wrong Context from Zi Wang and colleagues, accepted at the NeurIPS 2026 “Who Verifies the Agents?” workshop, makes a retrieval argument that applies well beyond agent memory. A memory can be relevant and still inadmissible for the current request because it belongs to another principal, violates policy, or sits in an incompatible lifecycle state, and neither recall nor final-answer accuracy exposes this. A route can look safe by missing required evidence, and a correct answer can follow inadmissible prompt exposure. The framework assigns each memory-query pair a status of admissible, inadmissible, or unresolved, compares routes at matched required-evidence recall, and tracks memory IDs through prompt exposure to answer-level disclosure. On a post-hoc top-20 reanalysis of 3,767 queries from RHELM and MemOps, namespace filtering raises anchor recall from 0.432 to 0.533 and cuts exact similarity evaluations by 98.3%. In a frozen 72-case diagnostic, a released-metadata reference preserves required evidence while neither text-only verifier detects violations under a 1% false-denial limit, which says that admissibility has to be checked against metadata, not inferred from text. The authors are careful about what is causal: the judged-accuracy gains of 0.053 to 0.068 across 1,523 paired cases are observational, and only one of four reader-specific confidence intervals for literal disclosure excludes zero. Row-level and principal-level policy on an enterprise semantic layer is the same problem with a different vocabulary.

Finally, an evals result from a 22-author team including Bryan Catanzaro and Jiantao Jiao. Before They Can Solve asks which base checkpoint is worth an expensive round of agentic post-training. End-to-end pass@K is the wrong instrument because many base models cannot emit a well-formed tool call, and single-shot patch tasks sidestep the capability that matters, holding coherent state across many tool steps as the repository changes. The move is to replay successful post-trained trajectories, rerun tests after every code-changing step, and find the decisive step, the first whose cumulative patch flips the repo from failing to passing. Three screens at that step need no cold-start harness driving: Decisive-Action bits-per-byte on the certified action, a Patch MCQ between that action and verifier-rejected alternatives, and prefix-conditioned pass@K. Across ten public base and post-trained pairs, all three rank the cohort in close agreement with post-trained SWE-bench Verified pass@1. Any agentic benchmark with successful trajectories and a verifier can be turned into a base-model evaluation this way.

What to watch: whether harness vendors answer HarnessSecurity’s call to make settings verifiable, and whether the ARCS result pushes text-to-SQL teams toward constrained disambiguation surfaces instead of chat. The admissibility work is one more sign that retrieval evaluation is splitting into support, policy, exposure, and disclosure as separate measurements, and semantic-layer teams would do well to adopt that split before auditors ask for it.

Feed note: the item mirror’s status endpoint reported a last sync of 2026-09-01, yet items dated through 2026-10-08 were present and were used here; the staleness report looks like a metadata lag rather than a stalled feed.