Frontier agents ace one task and stall across a stream of them

Twelve frontier models, four agent frameworks, and the same collapse: scores above 80% on isolated coding tasks fall to at most 38% once the tasks arrive as a continuous stream. That result comes from SWE-Milestone, posted today, which rebuilds “milestone DAGs” out of real commit logs so an agent has to carry a codebase through a sequence of functionally cohesive goals instead of solving one ticket in a vacuum. The failure mode is error accumulation: each milestone inherits the debt and mistakes of the last, and the models that look strong on a single SWE-bench-style task struggle to keep a system coherent over time.

A second benchmark out the same day, Long-Horizon-Terminal-Bench, lands in the same place from a different direction. Its 46 tasks each run hundreds of episodes, close to 9.9M tokens and roughly 85 minutes on average, and grade partial progress rather than only the final state. The strongest of 15 models tested clears 15.2% of tasks at a 0.95 reward threshold; the mean across all of them is 4.3%. Both papers measure the thing production teams actually feel, that a coding agent which nails a well-scoped change drifts badly once it has to plan, hold context, and debug across a long session. If you run agents as long-lived systems, read the two together.

The counterpoint is what the numbers look like when the work is scoped tightly. Ploy migrated a production agent to GPT-5.6 and reported 2.2x faster runs at 27% lower cost, the first concrete production migration data on OpenAI’s new frontier family since last week’s launch. The distance between that and the benchmark scores is the whole game right now: constrained, well-instrumented agent workloads keep getting cheaper and faster, while open-ended autonomy stays hard.

Nathan Lambert spent the day pointing at open-weight models and the policy risk over them. He cites unofficial reports of White House discussions on a possible executive order governing open models, likely aimed at Chinese-origin weights and government use, and argues it would be the first regulation with real enforcement teeth applied specifically to open release. There is no official text yet, so treat the specifics as early. The stakes he lays out are not: an ecosystem that spent three years assuming open weights stay freely available is now planning around the chance that they don’t.

Underneath the frontier, the vibe-coding crowd spent the last day auditing its own output. The r/vibecoding thread “Why your vibe-coded app scores 34 on PageSpeed and what actually fixes it” walks through the performance debt that ships by default when a model generates a whole frontend, and it landed the same day as “Vibecoded apps are a security nightmare” and a guide to un-vibe-coding an existing site. The tools that get people to a working app in an afternoon are the reason those apps are slow and leaky, and the community is starting to treat that as an engineering problem rather than a vibe.

Two Show HN launches went up in the last day aimed at exactly the weak spots those benchmarks expose. Capn-hook gives a coding agent persistent memory so it stops re-deriving the same facts about a codebase on every run, tagline “don’t grep the same mystery twice.” Peek-cli lets an agent see the browser it’s driving, closing the loop between a change and its rendered result. Both are small and early, and both target the two things the long-horizon benchmarks say agents lose: memory and feedback across an extended session.

Joe Reis argues the constraint isn’t only the model. In “Your Agents Are Stuck In Your Org Chart” he makes the case that agent deployments inherit the silos and handoffs of the teams that build them, so an agent scoped to one team’s slice of a workflow hits the same walls a human in that seat would. It’s a useful lens on the week’s benchmark numbers: some of the long-horizon failure is the model, and some of it is that we keep handing agents the fragmented context our org charts produce, then acting surprised when they can’t maintain a coherent picture.

What to watch: whether any lab publishes long-horizon numbers on its own models now that two public benchmarks exist to score against, and whether the open-models executive order moves from newsletter speculation to a draft anyone can read.