Coding-agent reliability gets attacked from the harness and the crowd

Three coding agents voting together fail roughly a third as often as one working alone. In a study posted today, N-Version Programming with Coding Agents puts hard numbers on it: across the classic Knight-Leveson Launch Interceptor spec, the mean failure count for a single agent-written implementation is 387.44; route the same task through three independently generated versions under majority voting and that drops to 130.99, with 11,844 of the three-version units showing zero failures across a million randomized test inputs. The catch is the same one Knight and Leveson found in 1986: diversity does not eliminate common-mode failure. The implementations still fail together, and the co-occurring failures cluster exactly where the specification is ambiguous or hard. Redundancy buys you a lot, but it cannot vote its way out of an underspecified problem.

That ambiguity finding rhymes with the day’s other reliability paper. AgentArmor sorts destructive coding-agent failures into three mechanisms: underspecification, where the model’s default behavior is unsafe; capability errors, where the safe action exists but the model doesn’t take it; and harness errors, where the model picks the safe action but the harness fails to execute it. The authors built 8 evaluations from real deployment failures, spanning 20 coding environments and 59 synthetic transcript templates, then proposed a set of harness modifications: an extended system prompt, a separate command classifier, a “3 strikes” policy, deterministic guardrails, and tools that let the agent edit its own context. The gains are statistically significant, and the framing is the useful part. Reliability work is migrating into the harness, the layer around the model, rather than waiting on better weights.

Memory is moving the same way. AtomMem argues that most memory systems store too coarsely and update too loosely, so it extracts high-value atomic facts from long interactions, organizes them into hierarchical event structures and temporal profiles that track how a user’s attributes change over time, and connects fragments through an associative graph at retrieval. It reports state-of-the-art results on LoCoMo, the standard long-conversation memory benchmark. The design bet is that value-dense, stable storage beats large reactive caches, and the temporal profile is the interesting wrinkle: memory that models change over sessions rather than treating every fact as timeless.

The sharper memory idea today reframes it as retrieval across a population of agents. Multi-Agent Transactive Memory, from a group including Fernando Diaz, points out that agent trajectories encode reusable procedural knowledge and then get thrown away after a single use, forcing every newly spawned agent to rediscover solutions others already found. MATM treats trajectories the way a search engine treats documents: producer agents write them to a shared repository, consumer agents retrieve them. On ALFWorld and WebArena, retrieving past trajectories improved task performance and cut interaction steps with no coordination and no joint training. It is RAG pointed at agent-generated artifacts instead of human-written ones, and it gives multi-agent systems a way to accumulate experience without retraining anything.

Scale is where orchestration stops behaving. Autonomous Event-Driven Multi-Agent Orchestration for Enterprise AI at Scale compares DAG Plan-and-Execute against ReAct across 208 production-derived scenarios at three sizes: under 10 agents, 20 to 80, and 200. The headline finding is that scale, not task complexity, dominates orchestration quality. Both architectures handle small deployments well and degrade at 200 agents as discovery noise, the difficulty of finding the right agent, becomes the bottleneck. Counterintuitively, simple tasks degrade more sharply than complex ones, and DAG planning’s structured parallelism wins on precision at small scale but its overhead turns against it at enterprise size, where ReAct’s incremental failure handling holds up better. Their Task Manager, which does priority inference, related-event merging, and preemption, cut high-priority queue latency by 14 to 75 percent and improved related-event correctness by more than 20 points at the largest scale.

One practitioner note worth tracking, from Jo Kristian Bergum: a case for “code mode” in retrieval. The argument is that agents are already good at writing code, so retrieval should be expressed as code the agent composes against a search API, rather than a fixed tool call with a query string. It lines up with MATM and the broader move to treat retrieval as a first-class agent capability instead of a bolted-on lookup.

The through-line across all six is that the reliability and capability gains this week live outside the model weights, in harness guards, redundant voting, structured memory, trajectory reuse, and orchestration schedulers. Worth watching: whether anyone combines the N-version voting result with AgentArmor’s harness guards, since the failures that survive both are the ones rooted in ambiguous specs, and no amount of infrastructure fixes a spec.