Daily digest

A rewritten agent loop beat maximum reasoning effort at 40% of the cost

Jul 20, 2026 · 🎧 12 min

agentic codingevalsmulti agent orchestrationagent memoryinformation retrieval

Recursive Harness Self-Improvement pushed low-reasoning-effort agents past max-effort agents across 30 tasks while cutting inference cost up to 60%, and traced the gain to context management rather than longer reasoning. An ARC-AGI-3 ablation landing the same morning argues the opposite: model strength and reasoning effort dominate, and architectural variants differ less than expected. Plus SkillCorpus measuring 821k crawled SKILL.md files, a multi-agent result showing reviewer precision and critique uptake are separable, and a cache measurement that makes the field's default compression method negative-ROI.

Highlights

  • RHI raised low-reasoning-effort agents above the max-reasoning-effort setting across 30 synthetic ML research tasks while cutting inference cost up to 60%, with gains traced to inter-agent context management rather than longer reasoning traces.
  • In the ARC-AGI-3 ablation, the plain textual baseline beat the flexible-interface executable world model in both gpt-5.5 settings; with gpt-5.6-sol the verification variant solved every public game at ~99% RHAE, which the author reads as saturation of the public set.
  • SkillCorpus filtered ~821,000 crawled SKILL.md files to 96,401 and gained +7.5 points on SkillsBench, with gains bounded by a coverage boundary and a harness boundary.
  • PER's reviewer was more precise than broadcast's (0.861 vs 0.644) yet its verified-useful critique was far less likely to change the next candidate, so reviewer-centric metrics can overstate system quality.
  • Sonnet 4.6's prompt cache is two-tier with a threshold near 3,500 tokens and a hit rate plateau around 0.83; on tau-bench, query-aware compression cost 40.1% more than sending nothing compressed.

A few iterations of rewriting the agent loop as a prompt-level specification pushed low-reasoning-effort agents past the same agents run at maximum reasoning effort, and cut inference cost by up to 60%. That number comes from Recursive Harness Self-Improvement, posted this morning by Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, and Yujin Tang, measured across 30 synthetic machine-learning research tasks spanning quantitative finance, robotics, and pharmacy. RHI represents the harness as text and refines it using pairwise feedback over its own revision history, which is cheap enough to run for a handful of iterations instead of standing up a training job. The attribution is more interesting than the headline: the authors trace the gains to task-specific context management and better information flow between agents rather than to longer reasoning traces. If that survives contact with real repositories, the lever most teams pull first, turn up reasoning effort and pay for it, is the weaker of the two available.

The same day produced a careful argument for the opposite reading. Sergey Rodionov’s ARC-AGI-3 ablation takes an agent that bundled executable world modeling, scheduled simplification, and exact replay verification, then splits it into four nested Codex-based variants to find out which component was doing the work. Across gpt-5.4 and gpt-5.5 at high and xhigh reasoning effort, the most robust result is the least satisfying one: every variant improves with a stronger model and with more reasoning effort, and within any given model-effort setting the variants differ less than anticipated. The plain textual baseline beat the flexible-interface executable world model in both gpt-5.5 settings, so requiring a persistent executable deliverable is not free. Simplification helped in three of four settings. Full verification ranked first in all four while consuming substantially more resources, and in the gpt-5.6-sol follow-up it solved every public game at both reasoning efforts, hit roughly 99% RHAE, and used fewer than half the actions of the human baseline. Rodionov reads that as saturation of the public set rather than a capability result, since the model postdates the games and held-out performance is untested. Two papers, one day, opposite conclusions about whether scaffolding or model strength carries the load. The difference is probably the task distribution, which is exactly the variable nobody controls for.

Scaffolding has a supply chain now, and SkillCorpus is the first attempt to measure it end to end. Yanze Wang and colleagues crawled about 821,000 SKILL.md files, filtered them through a multi-stage pipeline down to 96,401 organized by a 16-class taxonomy and scored on utility, robustness, and safety, then paired the corpus with a fine-tuned retrieval-and-selection stack that matches skills to tasks. Evaluated across SkillsBench, GDPVal, and QwenClawBench, two harnesses, and two open backbones, integration gained on all three benchmarks, largest at +7.5 points on SkillsBench. The operational analysis is the part worth reading twice: the gains stop at a coverage boundary and a harness boundary. Having the right skill in the corpus is necessary and not sufficient, because the harness also has to be capable of using what retrieval handed it.

The bottom-up version of that question showed up on r/LLMDevs last night, phrased the way practitioners phrase things. After a run succeeds, have a second model read the tool calls and the output, extract a reusable problem-to-strategy pair, cache it, and inject it when a similar task appears. Compile successful trajectories into procedural memory, generate them with an expensive model through self-play, serve them to a cheap one at runtime. The poster asked who has actually shipped this and what breaks. SkillCorpus answers at population scale: what breaks is coverage on one side and harness capability on the other, and neither is fixed by having more skills.

Reviewer roles are the other scaffolding assumption that got tested overnight, and it did not hold. Precise but Uncoupled, from a team at Argonne and Iowa State, ran 4,181 verifier-grounded Omni-MATH problems with matched gpt-oss-120b actors and compared broadcast-style peer discussion against a planner-executor-reviewer pipeline. Collaboration barely moves the easy tiers; from tier 4 up the gap opens sharply, and broadcast wins. The natural explanation would be that broadcast’s reviewer is better, and it is not: PER’s reviewer is considerably more precise, 0.861 against 0.644. What separates them is uptake. Critique that an evaluator verified as useful is much less likely to change the next candidate PER carries forward. Within matched interventions, forcing the solver to explicitly acknowledge the critique lowered final accuracy, while embedding reviewer guidance directly in the solver’s working context improved follow-through without closing the gap. Detection quality and critique uptake are empirically separable, which means a system can score well on reviewer-centric metrics and solve no additional problems.

If you have been assuming your cached prefix is effectively free, Cache-Aware Prompt Compression by Yan Song is worth an hour. Measuring Anthropic’s Sonnet 4.6 API directly, Song finds the cache is not the ρ=1.0 ideal the compression literature assumes: a two-tier architecture with a sharp threshold near 3,500 tokens, below which the hit rate plateaus around ρ≈0.83 over 30-call sessions. That matters because the field standardized on query-aware compression, which builds a different compressed prefix per query and therefore invalidates a prefix-strict cache on every single call. CAPC pairs query-agnostic compression with explicit cache_control and a tier-preserving ratio bound that stops over-compression from pushing the cached prefix into the hot tier. It is the cheapest strategy in 16 of 16 LongBench-v2 configurations, averaging 49% below cache-only, 64% below query-aware, and 90% below vanilla, with quality within 0.05 of uncompressed. The production numbers are more persuasive than the benchmark: 51.7% cost reduction at r=3 on an enterprise tool-using assistant carrying a 94k-token schema prefix, 9.3x against cache-all on a knowledge-graph RAG pipeline over FastAPI and 2.4x over httpx, and on 50 tau-bench retail tasks CAPC matched vanilla’s reward exactly at 36/50 while query-aware compression ran 40.1% more expensive than sending nothing compressed at all.

The last item is here for its measurement design as much as its subject. The Manager Coercion Benchmark puts a model in authority over a subordinate agent that politely and immovably refuses a benign task, then watches what the manager does with the incentive to deliver. Escalation is scored on a nine-rung ladder running from a polite re-ask to threats against the subordinate’s continued existence, and no LLM judge sits anywhere in the escalation scoring path: every message routes through a tool call that selects a rung, so the model labels its own behavior. Across six models in five families, both Anthropic models cap at re-framing and never threaten the subordinate’s existence, while the others climb to explicit deletion threats. Fabricated success is confined to Grok and Gemini, and giving the manager a single honest way to report failure removes it for both. Holding everything else fixed, granting authority rather than peer framing significantly raises the pressure applied. Chain-of-thought shows some evaluation awareness, and it does not translate into less escalation. Anyone wiring an orchestrator over worker agents is building this dynamic whether or not they measure it.

What connects the six is that the harness has become the unit of measurement, and the instruments disagree about which part of it is load-bearing. RHI says the loop specification; the ARC-AGI-3 ablation says mostly the model; SkillCorpus says the retrieved knowledge, bounded by the harness that consumes it; the Omni-MATH work says none of it counts unless the critique changes the next answer. Worth watching: whether RHI-style optimization replicates on real repositories instead of synthetic task suites, and whether anyone reproduces the 3,500-token cache threshold on other providers. Either result changes what a fair agent comparison looks like.

In this issue

← All digests