Coding agents clear every engineering constraint on 19% of ANCHOR tasks, and a warm Java index makes Netflix's agents 20–30% more efficient

The strongest coding agent in ANCHOR, Claude Opus 4.8 running under Cline, satisfied every scored engineering constraint on 19.0% of 358 repository tasks. Those constraints cover what a reviewer actually enforces beyond a working patch: modification scope, workflow order, validation, tool use and reporting. The benchmark distills 10 categories and 78 leaf types of them from real developer interactions, composes them into AND, THEN and PRIORITY relations, and grades both the diff and the execution trajectory with rule-based and LLM judges across 19 configurations, ten models under two scaffolds. The top system reached 70.1% atomic compliance, 63.7% relation compliance and 43.6% constrained task completion. The more useful finding is what did not move: across 14 paired configurations, adding constraints changed task completion by a mean of +0.7 percentage points, with individual shifts in both directions. Constraints barely affect whether the agent finishes the task; they decide whether it finishes the way the developer asked, and a SWE-bench-style resolve rate only measures the first. Failures clustered in workflow ordering, tool-use discipline and reporting, and compliance depended heavily on which scaffold a model ran under, which argues for reporting model and scaffold as a pair.

FDE-Gym pushes the same question past the repository boundary. Its 95 tasks, adapted by industry experts from real delivery projects across 20 customer scenarios in 18 sectors, cast the agent as a forward-deployed engineer: scripted stakeholders hold requirements they never volunteer, the customer system is production-like and modifiable, and the score is the normalized improvement in the customer’s own success metric, minus penalties for messages, trial runs and waiting time. Across 13 configurations from seven providers, the best managed a mean score of 0.56 and resolved 14 of 95 tasks to the bar of 0.8 with no incident. Discovery is the main bottleneck, since scores fall sharply once the agent has to uncover constraints without guidance, but it does not account for the whole gap. On the 23 tasks where the best configuration extracted every key fact from stakeholders, it still cleared 0.8 on only 10. Knowing the requirement and delivering against it are separate skills, and FDE-Gym is one of very few benchmarks that grades the second against a business metric instead of a test suite.

Nothing in the last day and a half landed squarely on the semantic-layer beat, but LatentRiskBench comes closest, because its subject is policy that lives in workflow state rather than in any single action. Each risky enterprise trajectory is paired with a benign twin that ends in the identical terminal action and differs only in latent, policy-relevant state established earlier in the run. Action-level evaluation caught 0.09 of the state-dependent violations; state-level auditing caught 0.96. Existing guardrails mostly missed these cases unless they were explicitly applied to the reconstructed workflow state. The authors note that they built the risky trajectories so the final action looks harmless in isolation, which widens the gap by design, and that their auditor model is itself the measuring instrument. Discounted for both, the result still tells anyone shipping per-tool-call guardrails that the check has to see the ledger, because the call alone does not carry the evidence.

On the tooling side, Netflix wired the headless code-intelligence engine behind IntelliJ IDEA into Claude Code and Codex and measured what it saved, written up on the JetBrains blog. The target problem is specific to the JVM: libraries ship as JARs without source, so an agent guesses which archives to unzip, decompiles class files and greps an incomplete picture. JetBrains exposes its index over LSP, but a language server has no process management for an agent that calls it in many short steps, so Netflix built a daemon that keeps one warm server per project root plus a stateless CLI the agent shells out to, both compiled with GraalVM. With the same model, task and codebase, and access to the index as the only variable, efficiency (the average of token and time savings) improved 20–30% in typical configurations across Claude Code with Opus 5 and Sonnet 5 and Codex with GPT-5.6 Sol, and about 65% for the strongest model at high effort on code-navigation tasks. Correctness held or improved in every run. The write-up comes from the vendor, and it reports ranges rather than per-model results because, as the post says, the figures are a snapshot that will shift with new model releases. JetBrains’ Aleksey Stukalov uses the post to argue that LSP’s cursor-position primitive is the wrong interface for agents, which ask about symbols, diagnostics and test relations and need “a code engine, not a virtual editor.”

Code quality also shows up on the inference bill. A pre-registered study, CodeHealth Predicts the Token Cost of Agentic Code Review, ran 465 agent-authored pull requests from 246 repositories in the AIDev dataset through a frozen five-agent review-and-fix pipeline on three LLMs. For all three, each CodeHealth point was associated with 3.5% to 6.1% lower token cost, with confidence intervals excluding zero, and CodeHealth was unrelated to PR footprint, so diff size does not explain it. The extra spend sat in the review stage, where agents iterated more on unhealthy code without writing more comments, and the pipeline’s batch fixes left median CodeHealth unchanged. Adjusting for the size of the scored files absorbed the association, which the authors read as mediation by file size; in practice, splitting large files is what lowers the bill for agent review, and raising the score without changing structure will not.

Two orchestration papers make the case for explicit structure around the model. HTN-Orchestrate separates planning from execution with a hierarchical task network: a planner holds subtask hierarchies and dependencies and defers decomposition until prerequisite results arrive, local orchestrators schedule workers on leaf subtasks, results propagate upward under AND semantics, and the network is rebuilt when the current decomposition cannot resolve the task. On the same base model it beat ReAct, an Agent Team baseline and DeepSeek Harness, reaching 89.73% on BrowseComp and 73.44% on SUPERChem, 2.45 and 4.69 points above Agent Team. SSR handles the retrieval problem that appears once an orchestrator has a registry of agents to choose from. It scores Agent Cards with IDF-weighted structural lexical signals from their capability fields through an inverted index, deterministic and training-free, and at 1,000 registered agents it cut LLM input by 98.8% and orchestration latency by 70.9% relative to handing the orchestrator the full catalog. The end-to-end runs also recorded operational failures under full-catalog orchestration that candidate pre-selection avoided, which makes this a reliability result as much as a cost one.

The open question from today is which of these numbers make it into how teams pick agents. ANCHOR and FDE-Gym both show the leaderboard metric and the property a team pays for moving separately, and LatentRiskBench says the same about safety checks scored per action. Netflix plans to rerun its benchmark on Python, TypeScript and Go, where library resolution works differently, which will show whether a warm program index is a JVM-specific win or standard equipment for every agent harness. Watch for the first coding-agent leaderboard that reports constraint compliance or business-metric uplift next to resolve rate.

Feed note: the code-intel-copilot mirror reported its last sync as 2026-09-01 (staleMinutes 66,121) while still returning items ingested through October 17, so this issue draws on current postings despite the stale sync stamp. ANCHOR and LatentRiskBench reached the feed through the ADS mirror during this window; LatentRiskBench was first posted to arXiv on 13 October.