Agent Guarantees Are Assertions Until Something Checks the Execution Record

Claude Code’s /compact prompt on Sonnet 4.6 preserves 53% of an agent’s safety rules after one compaction round, and 10% after five. That measurement, across 20 production agent configurations, comes from The Compaction Cliff in Long-Running AI Agent Memory, posted yesterday by Saber Zerhoudi, Jelena Mitrović, and Michael Granitzer. The mechanism is unglamorous enough to be easy to miss: a safety rule and an episodic log compete for the same tokens, and when the budget overflows the summarizer degrades both at the same rate, even though only the rule needs its exact wording to remain enforceable. Compaction is a lossy operation applied uniformly to content whose tolerance for loss differs by orders of magnitude.

Their answer, Knowledge Triage, classifies each line of an agent’s knowledge base by type and routes each type through its own retention policy. Three deterministic operators cover the three context-management operations. TypeCompact rewrites items in place under a per-type fidelity floor. TypeDecompose partitions a topic too large to compact safely and replicates in-scope safety rules into every partition. TypeRetrieve fetches from external storage with in-scope rules pinned ahead of relevance ranking. TypeCompact holds 2–4× more safety rules than the strongest single-shot LLM compactor at every compression ratio, with 96% recall over five rounds; TypeDecompose reaches 0% locality violations against 93% under uniform partitioning; TypeRetrieve hits 100% recall@50 against 73% for the best single-shot LLM retriever. On downstream behavior they beat the production Sonnet compactor on medical compliance (paired McNemar p < 10⁻⁸, N = 200) and the full-policy and hierarchical baselines on retail task pass rate (p < 0.01, N = 115). They also shipped AgentArtifactCorpus, 396,934 agent configurations pulled from 54,628 public GitHub repositories, which is the largest look yet at how people actually write the instruction files their agents run on.

The compaction result lands the same day as a paper that treats the rest of the agent runtime with equal suspicion. When Can Agents Safely Checkpoint, Fork, Restore, and Merge?, from Yusheng Zheng and colleagues, names the four operations “execution edits” and points out the property that makes them dangerous: an execution edit cannot undo an authorization already granted or a tool request already sent. Restore a checkpoint taken before a payment call and you can authorize that payment twice. Merge two branches and you can discard a result the task still requires. Fork mid-call and you conflict with an invocation that started before the edit existed. Every agent runtime that offers these primitives today, and most of them now do, supports them without deriving from the running execution what each edit must preserve. The paper gives an algorithm that decides exactly whether an edit is safe, returning either the complete set of safe continuations or a checkable proof that none exists, with the finite checker and the runtime invariant mechanized in Lean and tests covering all six edit forms. The code and proofs are public. If you are building fork-and-merge into an orchestrator, this is the formal account of what you are actually promising.

Two papers landed within a day of each other measuring the price of putting agents in conversation, and they agree. The Collaboration Tax, from Weixiang Sun’s group, formalizes the loss from making two LLMs coordinate rather than act alone as the team-decentralisation loss of a two-player cooperative game with private information, then measures it on 32 solo-tractable tasks across 11 models from 7 providers. The tax orders consistently by task category across every model tested and shrinks monotonically with capability. Its cause is not weak reasoning but a four-stage conversational cascade: agents assert without grounding, fail to query the partner, skip integrating both views, then accept the answer without re-deriving it. A prompt intervention aimed at all four stages closes a substantial part of the gap, and in heterogeneous pairs the tax pulls toward the stronger partner rather than the additive midpoint.

The Interaction Tax, from Summer Eunhyung Ann, Haokun Liu, and Chenhao Tan, isolates a sharper version of the same failure across 11 verifier-scored optimization tasks under matched budgets. Different model families find structurally different solutions, which is the entire argument for running several of them. When agents read each other’s complete outputs, those proposals converge within a single round and the diversity evaporates. Full-solution interaction mostly anchors agents to the first solution they see; critique helps only when the violated rule is one the model can locate and repair on its own. Independent proposal generation avoids the collapse. Taken together with the collaboration-tax result, the design lesson is specific: multi-agent performance depends on what agents exchange and when, not on how many you run, and full-transcript broadcast is close to the worst available default.

Anyone running agents against a real repository should read SWE Refactor Bench, which measures whether a whole-repository migration actually happened rather than whether the tests still pass. The authors call the failure mode Blindness: an agent copies the original implementation forward, the fixed test suite goes green, and existing benchmarks score it as a success. Their three-stage protocol adds a Migration Audit that verifies the migration occurred and an Agentic Verification stage where 6 independent coding agents write targeted tests hunting hidden behavioral differences. Across 520 runs spanning 8 frontier models and 26 model-effort configurations, 28 runs (5.4%) clear all three stages, 13 of the 20 tasks receive no accepted solution at all, and the best model, claude-opus-5, scores 47.0 out of 100. The two abilities separate cleanly: a few runs preserve behavior by never migrating and die at the audit, while most attempt the migration and break behavior. Among the 340 runs that pass the audit, 58% reach 99% of the fixed checks and only 26% reach 100%, so the common outcome is a migration that is one or two checks short of complete. Category matters too, with agents scoring 31.4 on build-toolchain rewrites and 5.6 on language rewrites.

The operational-safety picture is no better. SABER evaluates coding agents by the final state of a stateful project workspace after a sequence of actions, rather than by whether a model refuses an unsafe prompt, and reports that even the best-performing model leaves a harmful safety-violation rate above 54%. The benchmark categorizes violations by cause, so the output is a per-model safety profile instead of a single refusal score, and those profiles differ enough between models that they are worth reading before you pick one to run unattended against a workspace you care about.

On the enterprise side, InfoQ covered Microsoft’s move of AI governance from policy documents to runtime enforcement, an architecture spanning nine governance domains and four functions: policy, control, visibility, and proof. The interesting one is proof. Tying policy to runtime enforcement, continuous evaluation, observability, identity, and audit evidence means the governance claim becomes something you can check against a trace while the agent is running, which is the same shift the Lean-checked execution-edit work is making from a different direction.

The thread running through all seven is that the guarantees agent systems advertise (this rule is enforced, this checkpoint is safe to restore, this migration is complete, this action was authorized) are assertions until something checks them against the execution record. Watch whether the next round of harness releases ship type-aware compaction and edit-safety checking as runtime features rather than papers, and whether anyone reports a compaction-cliff number for their own agent stack.