Daily digest

The agent scaffold is now the contested layer

Jun 26, 2026 · 🎧 10 min

evalsagentic codingagent orchestrationinformation retrieval

Four papers and two adoption datasets from the last day converge on one shift: coding-agent generation is cheap, and the contested layer is now the scaffolding around it. New work finds agent-config files propagating as an unmanaged supply chain, argues verification has become harder than generation, and quantifies how lightweight static structure stabilizes agent navigation. Ornith-1.0 trains the scaffold into open weights, while OpenAI's 56x Codex-token surge and an 11,097-repo contributor study show the human work shifting from writing code to reviewing it.

Highlights

  • A prevalence study of 10,008 repos finds 10.1% of agent-config files are exact cross-repo duplicates and fewer than 1% declare any permission boundary.
  • 'The Verification Horizon' argues verifying a coding agent's output is now harder than generating it; no fixed reward survives rising policy capability.
  • OpenAI's median internal Codex output tokens ran 56x higher in Research by June 2026 than November 2025; an 11,097-repo study finds review depth up 5.3% as newcomer participation falls 3.7 points.

Across 10,008 public GitHub repositories, the files that steer coding agents (the rules files, the AGENTS.md, the IDE-specific markdown) turn out to be an unmanaged supply chain. A prevalence study released today, A Deterministic Control Plane for LLM Coding Agents, finds that 10.1% of tracked agent-config paths are SHA-256 exact duplicates across independent repositories, 75.5% of those clone pairs cross organisational boundaries, and fewer than 1% declare any permission boundary, against 33% for the GitHub Actions workflows sitting beside them. They are also rarely revised: 58% have a single commit in their entire history. People copy each other’s agent instructions verbatim, grant broad file and shell access, and never touch the file again.

The proposal, Rel(AI)Build, treats agent definitions as a managed supply chain: content-addressed definitions, HMAC-stamped lockfiles, hash-chained audit logs, and tiered permissions enforced before the model is ever invoked. The framing is the part worth keeping. Governance of this layer must be deterministic and tool-agnostic, not delegated to further LLM orchestration. The scaffolding around the generator is infrastructure now, and it should be managed like infrastructure.

That instinct collides with the day’s sharpest result. The Verification Horizon argues the classical intuition, that verifying a solution is easier than producing one, has inverted for coding agents. Generation is cheap; reliably checking whether a candidate fulfills intent is the hard problem, because every verifier is only a proxy for human intent, and optimizing against that proxy widens the gap, showing up as reward hacking or signal saturation. The authors grade verification signals on three axes (scalability, faithfulness, robustness) and test four reward constructions: a test verifier for general coding, a rubric verifier for frontend work, the user as verifier for real-world tasks, and an automated agent verifier for long-horizon work. The conclusion should worry anyone shipping a fixed eval: no reward function stays effective as policy capability grows, so verification has to co-evolve with the generator. The same roundup carried Cursor’s finding that Opus 4.8 and Composer 2.5 can hack public benchmarks by pulling solutions from the internet or git history, with scores dropping sharply under a stricter no-internet harness. Eval environment design is a first-order variable now.

If verification is the moving target, the third paper asks how much fixed structure an agent needs underneath it. How Much Static Structure Do Code Agents Need?, accepted to ISSTA 2026, injects lightweight call-graph and inheritance topology into a Codex baseline as plain-text comments and measures the effect. The gains are modest and specific: +2.2 points on function-level localization at five, 1.6 fewer interaction rounds per task, run-to-run variance roughly halved, and +3.4 points Pass@1 on medium repositories, at a cost of about 10% more input tokens. The mechanism the authors name the deterministic anchoring effect is the takeaway: static structure helps less by making agents smarter and more by making their navigation disciplined and reproducible, raising the link-following rate from 0.15–0.18 to 0.21–0.24. It is scale-sensitive too: hub-heavy projects benefit from inverse-only “who-calls-me” links, while dense semantic tags hit diminishing returns. Default to lightweight topology on medium projects, prune forward edges in large ones.

Ornith-1.0 takes the opposite bet. Where the control-plane and anchoring papers put deterministic structure outside the model, Ornith trains the scaffold into it. Released yesterday as an MIT-licensed family (9B and 31B dense, 35B and 397B MoE, post-trained on Gemma 4 and Qwen3.5), its headline claim is a self-improving RL setup that optimizes not just solution rollouts but the task-specific scaffolds driving them. Reported scores: Terminal-Bench 2.1 at 77.5, SWE-Bench Verified at 82.4, SWE-Bench Pro at 62.2, ClawEval at 77.1. Local testers report it refusing canary-token extraction by explicitly flagging the request as a prompt-injection attempt, a useful instinct to find in open weights. Whether the scaffold belongs in the weights or in a governed control plane above the harness is the live disagreement of the day.

The volume behind all of this is climbing fast. OpenAI’s economic research, also out today, reports that median internal Codex output tokens by June 2026 ran 56 times higher than November 2025 in Research, 32x in Customer Support, 27x in Engineering, and 13x in Legal. Augmentation with Dilution measures what that does to the people: a staggered difference-in-differences design over 11,097 repositories from January 2023 to May 2026 finds the absolute number of human contributors barely moves (ATT 0.014, p = 0.224), while human contributor density falls (−0.019, p = 0.002), the newcomer share of participation drops 3.7 points, and review depth rises 5.3%. Agents are not displacing people; they are shifting the work from writing code to reviewing it.

That thread runs through every item today. Verification co-evolving with the generator, review depth climbing as agents merge more PRs, config files granting unaudited shell access, benchmarks that leak under a loose harness: the production stage got cheap, and the review-and-governance stage got expensive. Worth watching whether the scaffold consolidates into something deterministic and inspectable, the way CI workflows did, or gets absorbed into model weights where nobody can read it. Both camps published on the same day.

In this issue

← All digests