When the loop closes, verification becomes the job

Let me start with one line from Swix posted in the last day because it captures where the field’s head is at right now. He’d been using Ultracode, which is Anthropic’s internal subagent orchestration tool, the thing most people outside the lab haven’t gotten their hands on yet, and his verdict was blunt. He said it’s scarily good at burning tokens, but you need to set up your repo to parallelize properly to actually make use of the fanout. And then he gave the framing that I think is the useful part.

He described subagents as subroutines but intelligent. The work that benefits from them isn’t the one big task you can write a script for. It’s the long tale of yak shaves, all those little side quests that each need a bit of judgment, the kind of thing you can’t fully specify in advance. You fan those out across subagents, and in principle you get a lot of parallel intelligent work done at once.

But here’s the catch, and it’s a real one. That only pays off if your codebase is structured so the independent pieces can actually run in parallel. If your repo is tangled, if everything depends on everything, you point this thing at it and you just get the token bill without the speedup. So the tool is real, the leverage is real, but it surfaces a requirement most teams haven’t met.

Your code has to be organized for fanout before fanout helps you. And that’s a genuinely demanding thing, to ask of a codebase that grew up being written and read by one human at a time. I’m leading with that because it landed the same week the field started seriously arguing about the codebase. I’m leading with that because it landed the same week the field started seriously arguing about the codebase.

And the clearest statement of that argument came from Turing Post in a piece they called The Flywheel. The distinction they draw is between a pipeline and a flywheel. A pipeline just repeats. It’s a cron job, a script plumbing.

It does the same thing every time. A flywheel steers. It acts, it measures the result of its own action, and it uses that measurement to decide what to do next. Generate, measure, decide what to try next, then generate again.

Three beats, not one and the same idea showed up elsewhere in the window under different names alpha signal was writing about loop engineering and i’m not kidding loop maxing programming digest had loop engineering in its links the same morning so the vocabulary is converging even if the practice is still early now the turing post thesis is the one i’d actually hold on to because it’s the load-bearing idea a loop only closes where verification is cheap fast and objective that’s it that’s the whole thing think about why coding closed first it’s not because code is special or because coders are special because software spent 40 years accidentally building exactly the verification infrastructure that a closed loop needs compilers reject malformed programs type systems catch whole categories of mistakes before you run anything test suites encode what good looks like in executable form ci runs all of it automatically on every single change so when the language models showed up the verifier was already sitting there waiting a modern coding agent doesn’t just write code it runs an experiment write the code run the tests read the failures rewrite run the tests again and nobody reviews iteration three of seven you review the final diff and increasingly for low stakes changes not even that so the question turing post pushes you toward is uncomfortable look at your own organization which of your workflows has a test suite which has anything that even resembles one for most companies the honest answer is that the workflows have humans the human is the verification layer the human is also the coordination layer the thing deciding which workflow runs next and whether the whole effort is even working which means the human is the reason the flywheel can’t spin and if you just remove them and hope you get the embarrassing autonomous agent story the one that starts with we deployed agents and ends with a cleanup the right way the only way that actually works is to replace the human checkpoint with a verifier a test suite a schema validation a reconciliation against known totals a performance metric the judgment doesn’t disappear it gets encoded once into the verifier instead of being exercised by hand on every run and the human moves up a level off the work itself and onto the verifier which is a lovely theory and i want to spend a minute on the unglamorous evidence that the verifier is genuinely hard because two papers dropped on archive in this window that are basically the empirical floor under all this very easy to leave behind these out-of-the-box software have a lot of diversity within the way, so let’s get straight to the point. flywheel talk the first is called when errors become narratives it’s a longitudinal taxonomy of silent failures in a production llm agent runtime and silent is the operative word these are the failures that don’t throw an exception don’t write anything to a log and only show up as a wrong answer two or three steps downstream by which point you have no idea where it went sideways if you’re running anything autonomous on a schedule that’s the failure mode that actually bites you and it’s the one your verifier has to catch the second paper is even more pointed given everything i just said about verifiers it’s called the coin flip judge with a question mark and it measures the run-to-run reliability and the bias of llm as a judge setups where you use one model to grade another model’s output and this matters enormously because the judge model is increasingly the verifier in the loop it’s the thing standing in for the human checkpoint and the paper’s framing tells you where it lands if your closed loop is gated by a judge whose verdict flips between runs on the same input you haven’t encoded human judgment into your verifier you’ve encoded a coin flip you’ve automated a loop and put a random number generator at the gate neither of these papers has much social signal on its own they’re single archive listings no big launch behind them but together they’re the reality check on the rhetoric the flywheel is only as good as the verifier and the verifier is harder than it looks now the other thing about all this new surface area is that it links there was a tool that landed on the claude code subreddit called agent sweep and it points at a hole that i’d bet most people listening have never actually checked every time you paste an api key a database url a dot 10 file into a coding agent it gets written to a local history file in plain text and it doesn’t just sit there harmlessly the agent rereads its own history as context so that plaintext key keeps getting fed back into the model and it can resurface later in a file it writes or a command it runs. AgentSweep scans those history files with around 191 secret detection rules ported from Gitleaks, plus a dedicated detector for crypto seed phrases, and it covers about 30 agents out of the box. Claude Code, Cursor, Codex, Cline, Aider, Windsurf.

It redacts in place, keeps backups, gives you a full undo, and it’s read-only by default and MIT-licensed. If you’ve spent the last year pasting credentials into a terminal agent, which is most of us, that’s worth a run this week. There were two more things I want to flag because they’re about where the agent stack is heading. The first is an essay called Decouple the Agent, and the argument is that prompts, tools, and model selection don’t belong baked into your client.

Push them server-side. Then you can change the model, swap out a tool, or fix a prompt without shipping a whole new client build, and you’ve got one place to enforce policy. Inversion. It’s the same separation that REST pushed onto everybody years ago, just applied to the agent configuration that a lot of teams are currently hardcoding directly into their apps.

And it pairs with something concrete that hit the Hacker News front page. Anthropic published an Apple Foundation Models library in its SDK docs, which means Apple’s on-device models now sit behind the same SDK surface as Claude. Nobody’s shipped much on it yet, but the interop direction is the signal. The config you decouple today, is the config you’ll want to point at an on-device model tomorrow.

And then the backdrop to all of it, which didn’t go quiet. Nathan Lambert’s Interconnects ran a piece called Welcome to the AGI Era of AI Governance, and it reads as the considered follow-on to that export control directive that pulled Claude Fable 5 and Mythos 5 offline earlier this month. And the point that sticks is this. When frontier capability is governed by directive, that can suspend a model overnight, governance stops being a side conversation that happens in policy circles and becomes part of the release calendar.

It’s a thing that ships, or unships, your model. So that’s the thread tying the day together. The loops are closing. The tooling to run them is spreading out from inside the labs to the rest of us, Ultracode being the leading edge of that.

And the two things that decide whether any of it actually holds up are both still under construction. One is a verifier you can trust, which those two things are. What those two archive papers suggest is genuinely hard. And the other is a policy regime that won’t yank the model out from under you.

What I’m watching for next is twofold. Whether Ultracode-style fanout shows up in a form you can actually ship outside Anthropic, and whether anybody publishes a verifier good enough that the decide-what-to-try-next beat can run unattended on something that isn’t code, because code had a 40-year head start and everything else is starting from a much weaker place, that’s the day. I’ll see you tomorrow.