Agent risk lives in the repository, not the agent

Across more than 930,000 agent-authored pull requests, about half the variation in integration friction stays with the repository after you account for the contribution, its author, its size, and the agent that wrote it. That number, from Daniel Russo’s Govern the Repository, Not the Agent, reframes a question the field keeps asking the wrong way. We benchmark coding agents one at a time, on isolated tasks, the way we’ve always evaluated a component. But agents that each pass their own tests still leave codebases that accumulate problems no single PR accounts for. Russo measures integration friction, the cost of merging a contribution into a codebase that other contributors are concurrently changing, and finds agent-authored changes concentrate repository-level friction roughly twice as much as human ones: an intraclass correlation of 0.30 versus 0.16, holding after controls for codebase size, age, task shape, process maturity, and merge path. The risk is a property of the ecosystem, not the agent. That has a governance consequence: a SWE-bench score tells you nothing about what a fleet of agents does to a shared repo over a quarter.

If the agent isn’t the right unit to govern, the verifier might be. Glite ARF makes that literal. It runs up to twelve LLM coding agents (Claude Code, Codex CLI) in parallel on a research repository, with a three-role split: a human picks hypotheses, agents implement tasks under a fixed structure, and deterministic Python verifier scripts enforce task isolation, immutability of completed work, and a materialized project overview. The authors call it verifier-driven research, and the phrasing is the point: the rules of the process live in code that fails loudly when violated, not in prose the agents are merely asked to follow. The payoff shows up in a real campaign. Using the framework, they took first place in the closed track of the BEA 2026 vocabulary-difficulty task and second in the open track across Spanish, German, and Mandarin, cutting baseline RMSE by 29.9% and 35.9%. The campaign ran 273 tracked tasks across 129 feature sets for about $450 in API spend, orchestrated from a single laptop. The structural machinery added roughly 1% of wall-clock time, and per-fold provenance caught four target-leaking feature sets, correcting an implausible 0.609 RMSE up to a believable 0.802. Cheap structure beats trusting the model to behave.

NOVA carries the same instinct into production. It’s a verification-aware harness for evolving recommender-model architectures inside an industrial advertising system, where runnable code is the easy bar and a valid architecture is the hard one. Generic coding agents optimize for code that executes; candidates pass local tests and then cause silent failures that quietly degrade ranking. NOVA runs a verification cascade, structure semantics, then local executability, then offline effectiveness, then online impact, and blocks invalid candidates early, recording failure patterns as forbidden directions for the next modification. Task-level L1–L4 control routes high-risk changes to a human via Copilot. Deployed, it hit effective pass rates of 54.5% and 60.0% on its harder task tiers, shortened one literature-to-production cycle by more than 13x in human-attended time, and the selected candidate moved GMV on three objectives by +1.25%, +1.70%, and +2.02% in online A/B tests. The signal that drives the next edit isn’t a chat history; it’s aggregated verification diagnostics and metric feedback.

Routing is shifting on the same axis, from static classification to accumulated experience. Agent-as-a-Router shows the bottleneck for model routers is information deficit: bolting task-dimension-level performance statistics onto a vanilla LLM router buys a 15.3% relative gain on its own. Their ACRouter formalizes routing as a Context→Action→Feedback→Context loop with an orchestrator, a verifier, and a memory module, accumulating execution-grounded experience during deployment instead of guessing from the prompt. They release CodeRouterBench, around 10K task instances with verified scores from eight frontier models, scored by cumulative regret on streaming tasks rather than one-shot accuracy. A router that remembers what each model actually did is a different object than a classifier.

Retrieval is the other place the work is moving off the prompt. REQL is a local repository context engine that indexes files, symbols, imports, calls, tests, and docs into a graph and exposes a query language plus CLI, Python API, and an optional MCP server. Tree-sitter parsing covers 30-plus languages with deeper extraction for Python, JavaScript, and TypeScript, and there are no mandatory LLM calls in the core indexing and retrieval path. The pitch is a compact, connected, source-grounded view of a codebase instead of scanning everything or hoping the right files fall into the context window. It’s alpha, but it sits in the same lane as the benchmarks (CORE-Bench, ContextBench) that keep finding context retrieval, not generation, is where agents lose repository tasks.

The framing worth keeping came from Jon Udell, quoted by Simon Willison: drop “human in the loop,” because it cedes authority to the machine. It’s our loop; we work the way we always have and recruit agents onto the team. The papers from the last day all point the same direction. The thing you measure, verify, route, and retrieve against is the codebase and the process, with the agent a participant inside it. Watch whether ecosystem-level metrics like Russo’s integration friction start showing up next to SWE-bench when teams report on agent fleets, because that’s the number that predicts what a quarter of merged agent PRs leaves behind.