Thematic explorer

Memory Design Considerations

82 papers · 11 themes

← All collections

82 papers shown

Retrieval & Ranking

How memories are surfaced into context at inference time: single- vs multi-channel retrieval, fusion across channels (e.g. Reciprocal Rank Fusion), keyword + semantic hybrids, and moving beyond raw cosine similarity toward user- and task-aware ranking.

Open questions
  • One vector index, or parallel channels (recency, semantic, entity, summary, raw) fused with RRF?
  • Is cosine similarity the right relevance signal, or does it surface the semantically-near-but-useless?
  • When should the agent retrieve at all versus reason from what it already holds?
  1. Agents that remember: introducing Agent Memory

    Tyson Trautmann · 2026 0 cites

    Synthesis

    Retrieval is five-channel parallel + Reciprocal Rank Fusion (RRF) — not a single vector lookup.

    Why it matters Concrete, shipped instance of the multi-channel+fusion pattern; copy the channel decomposition. Connects directly to [precision-belief-state] on why channel fusion still needs retrieval-quality measurement.

  2. Cloudflare Announces Agent Memory, a Managed Persistent Memory Service

    Steef-Jan Wiggers · 2026 0 cites

    Synthesis

    Confirms five-channel parallel retrieval with RRF and structured-memory extraction.

    Why it matters Independent confirmation of the architecture in [cf-agent-memory-blog].

  3. AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents

    Yan et al. · 2026 0 cites

    Synthesis

    Argues memory systems lean too hard on semantic similarity (misses user-centric evidence) and store related experiences as isolated fragments. Proposes adaptive, user-centric retrieval.

    Why it matters Direct critique of cosine-similarity-as-relevance — the case for ranking on user state/goals, not just embedding distance. Connects to AdaMem's fragmentation point and [cast-episodic]'s coherence argument.

  4. To Retrieve or To Think? An Agentic Approach for Context Evolution

    Chen et al. · 2026 0 cites

    Synthesis

    RAG-at-every-step is a rigid brute-force strategy that wastes compute and can degrade performance. Proposes an agent that decides when to retrieve vs reason from current context.

    Why it matters Reframes retrieval as a policy decision, not a reflex — the cost/quality lever most retrieval designs ignore. Bridges Retrieval and Working-Memory categories.

  5. HiNS: Hierarchical Negative Sampling for Memory Retrieval Embedding Models

    Tian et al. · 2026 0 cites

    Synthesis

    Memory retrieval depends on the embedding model; existing training ignores the hierarchical difficulty of negatives (close distractors vs easy negatives) in human–agent interaction. HiNS trains on that hierarchy.

    Why it matters The under-discussed layer beneath retrieval design: the embedding model itself decides what 'similar' means. Improving it is orthogonal to channel/fusion choices.

  6. Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrieval

    Jeffrey Flynt · 2026 0 cites

    Synthesis

    Observes that returning the entire belief store yields recall 1.0 and passes answer-quality evals — so answer-correctness can't validate a retrieval system. Introduces a precision-aware retrieval benchmark over a structured belief state.

    Why it matters The cleanest statement of the retrieval-vs-answer-correctness gap (the 'unit test vs integration test' framing). A north star for evaluating any multi-channel retriever. Bridges Retrieval and Evaluation.

  7. SuperLocalMemory V3.3: 'The Living Brain' — bio-inspired forgetting, multi-channel retrieval, zero-LLM

    Bhardwaj · 2026 0 cites

    Synthesis

    Opens on the paradox: coding agents have vast parametric knowledge yet can't remember an hour ago. Critiques single-channel vector retrieval that needs cloud LLMs and implements no cognitive processes. Adds biologically-inspired forgetting, cognitive quantization, and multi-channel retrieval — all local/zero-LLM.

    Why it matters Bundles forgetting + multi-channel retrieval + local-first into one system; the indie counterpart to Cloudflare's managed multi-channel approach. Connects to [yourmemory] (zero-LLM, local) and [git-s3-memory] (local-first ethos).

  8. Self-hosted archive for all AI conversations (hybrid keyword + semantic search)

    u/Sufficient_Guard9850 · 2026 0 cites

    Synthesis

    'ChatDB' — a self-hosted conversation archive across multiple AI apps, with a proper hybrid keyword + semantic search interface, deployable free on Cloudflare.

    Why it matters Demonstrates the storage+search split: keep transcripts, layer hybrid retrieval on top. The hybrid (lexical+semantic) retrieval choice connects to the Retrieval category.

  9. Engram: A Bi-Temporal Memory Engine Where a Lean Retrieved Context Beats the Full History

    Wang, Liuyin · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Engram is an open-source long-term memory engine for LLM agents. Instead of replaying an entire conversation history into the prompt, it stores the past as a bi-temporal knowledge graph and retrieves a small, precisely-targeted slice at answer time. On a standard 500-question memory benchmark that lean ~9.6k-token slice answers more accurately than feeding the model the full ~79k-token history, turning memory from a cost optimization into an accuracy improvement.

    Motivation Stateless LLM agents forget across sessions, and the usual fix — concatenating the whole history — grows token cost and latency linearly and loses accuracy as distractors crowd the window ('lost in the middle'). Two gaps stay open: most memory systems are cheaper or faster but not more accurate than full-context, and memory benchmarks run on inconsistent harnesses where one system reports wildly different scores across sources. Engram targets both — beating full-context on accuracy, and shipping a neutral, re-runnable harness.

    Methodology A dual-process design. A System-1 hot write path appends lossless episodes with no LLM (sub-50ms) and enqueues them. A System-2 async path extracts atomic (subject,predicate,object) facts, builds a bi-temporal knowledge graph (valid time vs transaction time on every fact and edge), detects conflicts, and resolves them cheap-then-escalate: exact slot match, embedding similarity, and content subsumption handle the common case with no LLM call, invalidating (never deleting) a superseded fact and recording a supersedes chain and provenance, with only ambiguous cases escalated to an LLM adjudicator. The hybrid read path retrieves through four channels (dense, BM25, graph n-hop, recency/salience), fuses them with Reciprocal Rank Fusion, applies an 'as-of' temporal filter and an abstention gate, and assembles a deduplicated, provenance-tagged, token-budgeted context of facts plus raw chunks.

    Results On the full 500-question LongMemEval_S under the official category-specific judge, Engram's lean configuration scores 83.6% vs 73.2% for full-context (+10.4 points, McNemar exact p<10^-6) at ~8x fewer tokens (9.6k vs 79k), 0/500 errored. The gain is load-bearing on the read path being hybrid: facts alone lose recall, while facts plus retrieved chunks recover detail. Bi-temporal modeling pays off most on knowledge-update (87.5%) and temporal (81.1%) categories, while multi-session aggregation and preference remain headroom. The paper documents measurement-integrity pitfalls (truncation, home-grown judges, full-history leaks) and ships a neutral in-repo harness with the official judge baked in and raw per-question logs, every number reproducible by command.

  10. Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memory

    Ji, Suozhao · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Infini Memory is a long-term memory architecture for LLM agents that stores memory as a library of plain-text 'topic documents' rather than vectors or a knowledge graph. Each document gathers related evidence under a subject and is maintained over time by splitting, merging, and rewriting. At answer time the agent reads memory through iterative tool calls, expanding context around matches instead of taking a single retrieval shot.

    Motivation External memory systems that store observations as isolated records, summaries, or indexed fragments hit four recurring failure modes: fragmentation (evidence about one subject scattered across records), conflict (old and new versions of a fact coexisting), compression loss (summaries dropping temporal and source cues), and insufficient retrieval (single-shot top-k returning fragments without enough local context for multi-hop reasoning). Infini Memory reframes persistent memory as a lifecycle maintenance problem — write, maintain, read — and aims for an inspectable, editable state without a mandatory vector or graph backend.

    Methodology Memory is a library of topic documents, each a maintenance scope with a summary, body, and entry-level metadata signatures (<seq,time,source>) that preserve order and provenance as content is rewritten. Writes are decoupled from structure: new candidates append to a buffer document, then periodic consolidation rewrites, splits, updates, and merges them into coherent topic documents. Retrieval can run over plaintext via lexical indexing rather than embeddings. At inference an agentic read procedure lets the LLM iteratively choose memory tools, inspect intermediate results, expand local context, and assemble evidence before answering.

    Results On MemoryAgentBench the agentic-retrieval variant scores 64.7% overall and 81.2% on Accurate Retrieval, with gains on Factual Recall, Test-Time Learning, and Selective Forgetting. Ablations on LongMemEval_S isolate two complementary sources: holding the hybrid reader fixed, removing structural split-and-merge maintenance drops accuracy 76.0%->69.3% (-6.7, concentrated on knowledge-update and multi-session questions), while upgrading the reader from hybrid to agentic adds 3.3 points (76.0->79.3) — so maintenance matters more than the retrieval upgrade and neither is sufficient alone. A split-threshold sweep shows over-fragmentation is recoverable, but oversized documents that mix subtopics are costly.

  11. T-Mem: Memory That Anticipates, Not Archives

    Guo, Weidong · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract T-Mem is a long-term conversational-memory architecture for LLM agents that aims to make every stored memory reachable two ways: by surface similarity (descriptive recall) and by latent semantic/causal link (associative recall). It precomputes, at write time, four families of retrieval cues ('triggers'), one per quadrant of a granularity (item vs scene) x orientation (descriptive vs associative) design space, and retrieves via a topic -> scene -> item cascade fused with RRF over lexical and dense indices. It reaches state-of-the-art on LoCoMo (80.26%) and LoCoMo-Plus (74.81%).

    Motivation Existing LLM memory systems (flat RAG, graph/temporal-KG, hierarchical, OS-style) all retrieve by projecting query and memory into one similarity space and taking top-K, so they are reachability-bounded by lexical/dense similarity. In long-running dialogue users rarely re-raise old topics with the same wording; they revisit them through indirect situational cues, so the target's surface form has drifted and can never be reached from the same neighbourhood. This associative half of the query-memory relation is a structural blind spot.

    Methodology M is a typed tuple of scenes, items, topic labels, four trigger families, and per-speaker Persona, built by a load-bearing four-stage offline pipeline: event-closure scene segmentation, incremental data-grown topic labelling, dual-granularity item extraction (atomic + connected, one LLM call per topic), and trigger instantiation (Entity+Bridge jointly per item; Scene+Horizon per scene). Memory-construction LLM is GPT-4.1-mini, dense encoder bge-m3. Retrieval is a top-down topic -> scene -> item cascade scored by RRF over BM25 + per-type dense rankings; multi-view trigger indices surface host nodes via nan-aware max cosine, and associative triggers bypass the topic prefilter so cues outside the similarity neighbourhood still hit. Triggers stay off the QA evidence path; Persona is ambient context appended after retrieval.

    Results On LoCoMo, T-Mem reaches 80.26% LLM-as-judge accuracy (51.96 token-F1), 3.25 pp above the strongest baseline HyperMem and the maximum on five of six columns. On LoCoMo-Plus it scores 74.81%, narrowing the LoCoMo-to-LoCoMo-Plus drop to 5.45 pp, about 5x tighter than HyperMem (28.38 pp) and near an order of magnitude tighter than the Mem0/SeCom/A-Mem cluster (~49 pp). Ablations confirm the scene-level associative triggers drive the associative gain: removing Scene+Horizon collapses LoCoMo-Plus by 22.19 pp (Horizon alone -12.47 pp) while moving LoCoMo by under 0.4 pp. T-Mem also reaches higher accuracy than HyperMem at a lower input-token budget.

  12. Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge

    Yadav, Neeraj · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MemStrata is a memory system for AI agents that keeps track of when facts become outdated. Instead of just retrieving whatever text looks most similar to a query, which fails when an old and a new fact look nearly identical, it uses a deterministic rule to detect when a new fact supersedes an old one and retires the stale version.

    Motivation Retrieval-augmented memory has no concept of time: when a fact changes (a renamed function, an updated config value, a new port number), both the old and new versions sit in the store with near-identical embeddings, and the agent can't tell which is current. The authors show this isn't a tuning problem: on a calibrated dataset, cosine similarity separates contradictions from duplicates at only 0.59 AUROC (near chance), because a value-flip edit sits textually closer to the original than a genuine rephrasing does.

    Methodology MemStrata's write path first tries a deterministic (subject, relation, object) triple match: if an incoming fact shares a key with a stored one but asserts a different value, the old fact is retired (not deleted) in a bi-temporal ledger and the new one is stored as current. Non-triple prose falls back to a similarity-plus-LLM-judge gate. The system is evaluated on six local, deterministic benchmarks (two static, four marker-free evolving: code mutation, config migration, dependency bumps, API evolution) with a 7B model on consumer hardware.

    Results MemStrata matches RAG on static recall (no cost) and reaches 0.95-1.00 accuracy on evolving-knowledge benchmarks where RAG reaches only 0.20-0.47. When forced to answer, plain RAG serves the superseded value 15-40% of the time; MemStrata drives this to ~0%. It also runs at ~2.1s retrieval latency versus ~16-18s for LLM-reranking/verification baselines, since no LLM sits on the read path.

  13. MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance

    Wang, Haoyu · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A memory framework for LLM agents that runs an LLM verifier over each candidate trajectory and then keeps the verifier's output on the record permanently, as reward, confidence, label and uncertainty fields. Those fields decide whether the memory is admitted, held provisional or rejected, and are read again later to rank retrieval, resolve conflicts, trigger summarization and archive stale records. Evaluated on Terminal-Bench 2.0, SWE-Bench Verified, WebArena and Mind2Web across four backbones.

    Motivation Agent memory is only useful if stored experience stays reliable over hundreds of interactions, and two failure modes break that. Unreliable admission: failed trajectories, accidental successes, invalid patches and misleading observations all look relevant to a retriever and get written, then mislead later decisions — a web agent that learns to edit the first row after filtering an admin table edits the wrong row once a table is sorted differently. Memory drift: a bank that grows across many tasks accumulates duplicate, conflicting, stale and overgeneralized records that remain retrievable long after their assumptions stop holding. Prior experience-memory systems (Synapse, AWM, ReasoningBank) make memory persistent but treat write-time feedback as a one-shot signal, so nothing continues to govern the record afterward. Verifier work, in turn, scores the current episode and stops there.

    Methodology Verification is decomposed into multiple criteria — completion, consistency, validity, generalizability — with reward estimated from score-token distributions and repeated views; repeated verification is triggered when uncertainty crosses a threshold. The resulting descriptor (reward, confidence, label, uncertainty) is attached before activation and routes the candidate to rejection, a provisional state, the active bank, or a failure-guard pool that stores failed experience as constraints rather than recipes. Each record additionally carries lifecycle state, quality, usage statistics and conflict links. Retrieval is a hybrid of BM25 and embedding cosine similarity over title, description and content, adjusted by the descriptor and by staleness and overgeneralization penalties, with positive memories and failure guards rendered as separate blocks in the injected prompt. Conflicts are located by structured-signature similarity above a threshold. Governance runs after every task: activation, rejection, merge and conflict checks immediately; summarization and archival when the active-memory budget is exceeded or a record goes stale. Baselines are No Memory, Synapse, AWM, ReasoningBank and a verifier-only filter, all matched on task order, step budget, retrieval budget, injected-memory budget, memory block template and decoding settings.

    Results Averaged over five seeds, MemGuard has the best success metric and the lowest average step count in all 16 backbone-benchmark settings. Against ReasoningBank, the strongest prior memory baseline evaluated, the largest gain is 7.9 success-rate points on WebArena and 5.6 step-success-rate points on Mind2Web, with 2.4 to 3.5 points on the terminal and software-engineering benchmarks. Backbones are Qwen-3.5-Flash, Qwen-3.5-Plus, Gemini-3-Flash and Gemini-3.1-Pro. The verifier-only control improves on ReasoningBank in most cells by admitting better candidates but loses to MemGuard in every benchmark-backbone cell, which the authors read as evidence that the gain comes from persisting the verifier signal through the lifecycle rather than from filtering once at the door.

  14. SQLite is Enough. Lexical, Semantic, and Hybrid Search with scrydb

    Breuer, Timo · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract scrydb puts a whole small-to-medium search pipeline inside one SQLite file: documents, an FTS5 lexical index, and binary, int8, and float32 embeddings, with reranking and rank fusion on top. Evaluated on eight BEIR datasets on a laptop with no GPU, a cheap binary first stage reranked at higher precision matches an exhaustive high-precision scan at the top of the ranking for about a fifth of the latency, while RRF hybrid fusion helps on only one dataset of eight.

    Motivation A retrieval component usually arrives as a bundle: an index dump, a vector-store snapshot, configuration files, and a server process to keep them in sync. The author's premise is that for small-to-medium corpora none of that is necessary, and that packaging the documents, index, and embeddings as a single artifact makes an IR resource as easy to share, archive, and rerun as any other file, inheriting SQLite's preservation properties. The second motivation is efficiency as a first-class outcome rather than effectiveness at any computational cost.

    Methodology Lexical retrieval is delegated to FTS5 with BM25 ranking; semantic retrieval uses sqlite-vec, storing float32, int8, and binary vectors in vec0 virtual tables with one table per collection and precision, so all three precisions live side by side in the same file and any one can rerank another's candidates. Embeddings are binarized with a component-wise Heaviside quantizer and compared by Hamming distance via XOR plus popcount, giving a 32x storage reduction, with int8 scalar quantization as an intermediate. A single Index object owns the connection; search takes mode (lexical, semantic, hybrid), precision (binary, int8, float), and an optional rerank stage, and batch search exports TREC runs. Thirteen retrieval configurations were evaluated on eight BEIR datasets (ArguAna, FiQA, NFCorpus, Quora, SciDocs, SciFact, Touche, TREC-COVID), all embedded with Qwen3-Embedding-8B and compared against that model's published full-precision MTEB results. All retrieval ran on an Apple M2 MacBook Air with 24 GB of memory and no GPU; embeddings were computed once ahead of time through a remote API.

    Results Reranking improved nDCG@10 over raw BM25 on every dataset, and reranking by the cheap Hamming distance captured nearly all of that gain. Semantic retrieval over the binarized index beat BM25 on nDCG@10 on all eight datasets. The central result is that refining a coarse ranking to higher precision is equivalent, at the top of the ranking, to scanning the whole corpus at that precision: Hamming plus int8 cosine and exhaustive int8 gave identical P@10 and nDCG@10 to three decimals on all eight datasets, as did the float32 pairs, diverging only in AP below the 1000-document reranking depth. That makes Hamming plus int8 cosine dominant over exhaustive int8, matching top-10 effectiveness at 164.5 ms against 822.5 ms. Four of eight datasets met or exceeded the MTEB baseline. The remaining gaps were not caused by compression: the author's own exhaustive full-precision run reproduced the TREC-COVID deficit (0.885 against a reported 0.950), placing it upstream in the embedding pipeline. RRF was best on exactly one dataset, Touche, and trailed the better of its two inputs on the other seven, so the author concludes it is worthwhile only where lexical and semantic retrieval are of comparable strength and make complementary errors. On latency, Hamming was the fastest of thirteen configurations on seven of eight datasets and scaled with corpus size, while BM25 scaled with query length.

  15. LivingRAG: Augmenting Graph RAG with Experience

    Cui, Yuzhuo · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Graph RAG systems answer each query independently and discard the reasoning afterwards, so related later queries start from scratch. LivingRAG adds a writable experience store to a graph retrieval backbone, keeping for each verified answer the entity activation map that worked and a compact reasoning summary, and reusing them on two separate paths: fused into the retriever's starting activation, and inserted into the prompt as a scaffold. Across five online QA streams it improves accuracy over graph RAG baselines while cutting completion tokens 22.7% and estimated cost 12.1%, with grounding and novelty gates admitting only 27.4% of candidate experiences.

    Motivation Reuse across an online query stream is not limited to repeated entities. Two questions may visit nearby regions of the retrieval graph with entirely different entities, or follow the same reasoning pattern with different subjects, as when two comparison questions both ask which person was born earlier. A measurement of the streams supports the distinction: direct entity overlap with an earlier query ranges from 4.30% to 76.69% by dataset, while graph-neighborhood overlap runs 80.45% to 99.70%, so an entity-keyed answer cache would miss most of the available reuse. The countervailing risk is that a writable store lets unsupported reasoning be written once and then amplified by every later query that reuses it.

    Methodology The retrieval backbone is LinearRAG, unchanged: a passage-sentence-entity graph built with lightweight entity extraction and no LLM calls during indexing, query entity activation, propagation over the sentence-entity graph, and Personalized PageRank over the passage-entity graph. A verified experience stores the query, its embedding, a sparsified final activation vector, a compact summary, the answer, a timestamp and a grounding confidence. Experiences are scored against a new query by a convex combination of query-embedding cosine and activation-map cosine, and the top-K maps are fused into the initial activation vector with softmax weights scaled by stored confidence and deliberately not renormalized. Scaffold selection adds a masked-template similarity term that retrieval omits, because a scaffold only enters the prompt and cannot inject historical entities into graph retrieval. Write-back requires a novelty score computed against the base rather than the fused activation to avoid circularity, and a grounding score equal to the fraction of extracted atomic claims entailed by retrieved passages under an NLI model; novelty is checked first so NLI runs only on survivors, and the grounding score becomes the stored confidence. Evaluation runs 2WikiMultiHopQA, HotpotQA, MuSiQue, MuSiQue-full and WixQA as online streams with an empty initial store, generating with Qwen3.6 Plus, reporting contain-match and LLM-evaluation accuracy.

    Results LivingRAG leads all baselines on the four multi-hop benchmarks, for instance 58.42 LLM-evaluation accuracy on MuSiQue-full against 52.71 for LinearRAG, 47.75 for GFM-RAG and 44.06 for HippoRAG2, and 70.25 against 66.00 on WixQA. Prompt tokens rise 3.5% weighted because scaffolds add context, completion tokens fall 22.7%, and total estimated cost falls from $51.05 to $44.87, a reduction on every dataset. The saving is not a late-stage artifact: per-segment traces show the system spending more completion tokens than its backbone in the earliest segments and turning negative once experience accumulates. Realized reuse differs sharply by stream, with graph transfer at 91.20% on MuSiQue and 0.00% on 2Wiki, where realized template reuse is 84.15% instead. Ablations match those traces, since removing activation fusion costs most where graph transfer is realized, removing scaffolds costs most on 2Wiki and cuts token savings from 23.6% to 8.3%, and removing the quality gate lowers accuracy on all three tested datasets. The stated limits are the store's lifecycle: nothing updates, downweights or deletes an accepted experience, and the fixed-corpus benchmarks contain no chronological updates or fact-validity intervals, so staleness cannot be evaluated.

Consolidation & Distillation

Turning raw turns into durable memory: when (and whether) to run an LLM to extract/summarize, episodic traces vs consolidated abstractions, and the failure modes of letting an LLM continuously rewrite its own memory.

Open questions
  • Eager (consolidate every turn) or lazy/recurrent (batch, on idle, on retrieval)?
  • Keep raw episodic traces alongside distilled facts, or replace one with the other?
  • What experience is even worth keeping — and how should it change behavior, not just fill storage?
  1. AI Agents of the Week: Memory as a First-Class Citizen

    Pascal Biese · 2025 0 cites

    Synthesis

    Newsletter roundup that flagged the agent-memory survey wave and frameworks like MemVerse (fast parametric recall + hierarchical retrieval) and WorldMM (multimodal experience consolidation).

    Why it matters Good lay-of-the-land pulse on what the research community foregrounded as memory went mainstream. Lighter signal than the survey itself.

  2. How Slack Manages Context in Long-running Multi-agent Systems

    Sergio De Simone · 2026 0 cites

    Synthesis

    Slack engineering moved away from accumulating chat logs toward structured memory, validation, and 'distilled truth' to keep long-running agents coherent and accurate.

    Why it matters Production validation of the consolidation thesis — and crucially, they *validate* the distilled memory rather than trusting LLM rewrites. Read alongside [useful-memories-faulty], which explains why that validation step is necessary.

  3. Useful Memories Become Faulty When Continuously Updated by LLMs

    Zhang et al. · 2026 0 cites

    Synthesis

    Distinguishes episodic traces (raw trajectories) from consolidated abstractions (schema-like lessons). Shows that when an LLM repeatedly rewrites a textual memory bank, the consolidated memory degrades over time.

    Why it matters The strongest research caution against LLM-driven consolidation as your primary store. Direct argument to keep raw traces (see Substrate) as ground truth and treat distillation as lossy. Pairs with Slack's validation step.

  4. RecMem: Recurrence-based Memory Consolidation for Long-Running LLM Agents

    Dai et al. · 2026 0 cites

    Synthesis

    Critiques 'eager' consolidation (invoke an LLM on every incoming interaction to extract memory) as a major token-cost driver. Proposes recurrence-based consolidation that batches/defers the work.

    Why it matters The cost knob for consolidation. Connects to [memfly] and [simplemem] as the efficiency cluster; the tradeoff is staleness vs token spend.

  5. SimpleMem: Efficient Lifelong Memory for LLM Agents

    Liu et al. · 2026 0 cites

    Synthesis

    Frames the dilemma: retain full history (redundancy) vs iterative reasoning to filter noise (token cost). Proposes an efficient middle path for lifelong memory.

    Why it matters Clean statement of the consolidation cost/coverage tradeoff that recurs across the category. Sits with [recmem]/[memfly] on efficiency.

  6. MemFly: On-the-Fly Memory Optimization via Information Bottleneck

    Zhang et al. · 2026 0 cites

    Synthesis

    Uses an information-bottleneck objective to balance compressing redundant info against keeping retrieval precise, optimizing memory on the fly.

    Why it matters Gives the consolidation tradeoff a principled objective rather than a heuristic. Theoretical companion to [recmem]/[simplemem].

  7. Amory: Coherent Narrative-Driven Agent Memory through Agentic Reasoning

    Zhou et al. · 2026 0 cites

    Synthesis

    Argues current frameworks fragment conversations into isolated embeddings or graph nodes; proposes building a coherent narrative via agentic reasoning instead.

    Why it matters Bridges consolidation and representation: the unit of memory should preserve narrative coherence, not just be a retrievable shard. Connects to [cast-episodic]'s who/when/where stance.

  8. Compiled Memory / Atlas: More Precise Instructions, Not More Information

    Rhodes & Kang · 2026 0 cites

    Synthesis

    Shifts the question from memory *management* (retrieve/page within a budget) to memory *utility*: what experience is worth keeping, and how it should change agent behavior. 'Atlas' compiles accumulated experience into precise instructions.

    Why it matters Reframes the goal of consolidation around behavior change, not storage. Strong complement to the cost cluster — efficiency is moot if you keep the wrong things.

  9. EverMemOS: A Self-Organizing Memory Operating System

    Hu et al. · 2026 0 cites

    Synthesis

    Notes most memory systems store isolated records and retrieve fragments, limiting consolidation of evolving user state and conflict resolution. Proposes a self-organizing memory OS for structured long-horizon reasoning.

    Why it matters 'Memory OS' framing that ties consolidation to conflict resolution (Temporality) and structure (Representation). Conceptual cousin of Letta/MemGPT-style OS metaphors.

  10. D-Mem: A Dual-Process Memory System for LLM Agents

    You et al. · 2026 0 cites

    Synthesis

    Critiques incremental per-turn extraction/update; proposes a dual-process design (fast + slow paths) for high-fidelity memory access over long horizons.

    Why it matters Dual-process (System-1/System-2) is a recurring shape; here applied to when to consolidate cheaply vs reason deeply. Connects to [retrieve-or-think]'s retrieve-vs-think policy.

  11. Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory

    Sun et al. · 2026 0 cites

    Synthesis

    Critiques the dominant extracted-fact paradigm: handcrafted static prompts compress raw dialogue into atomic facts that are stored, matched, and injected — losing the ability to reason deeply over history. Proposes going beyond atomic facts.

    Why it matters The most on-the-nose challenge to atomic-facts-as-primitive. Read before committing to a fact-extraction pipeline; connects to [amory] (narrative) and [cast-episodic] (events) as richer alternatives.

  12. ContextWeaver: Selective and Dependency-Structured Memory Construction

    Wu et al. · 2026 0 cites

    Synthesis

    Sliding-window and prompt-compression context management omit earlier structured info later steps rely on; retrieval-based memory surfaces relevant content but overlooks dependencies. Builds memory selectively with explicit dependency structure.

    Why it matters Targets the failure where compression drops the one earlier fact a later step needs — a dependency-aware answer to context rot. Connects consolidation (what to keep) with context (what to show).

  13. Agentic Context Engineering: Evolving Contexts for Self-Improving Models

    Zhang et al. · 2026 0 cites

    Synthesis

    Context adaptation (modifying inputs vs updating weights) suffers two failure modes: brevity bias (concise summaries drop domain insight) and context collapse (iterative rewriting erodes detail).

    Why it matters Names the exact degradation mode behind LLM-rewritten memory — the context-side mirror of [useful-memories-faulty]. Strong argument against over-summarizing either context or stored memory.

  14. Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memory

    Ji, Suozhao · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Infini Memory is a long-term memory architecture for LLM agents that stores memory as a library of plain-text 'topic documents' rather than vectors or a knowledge graph. Each document gathers related evidence under a subject and is maintained over time by splitting, merging, and rewriting. At answer time the agent reads memory through iterative tool calls, expanding context around matches instead of taking a single retrieval shot.

    Motivation External memory systems that store observations as isolated records, summaries, or indexed fragments hit four recurring failure modes: fragmentation (evidence about one subject scattered across records), conflict (old and new versions of a fact coexisting), compression loss (summaries dropping temporal and source cues), and insufficient retrieval (single-shot top-k returning fragments without enough local context for multi-hop reasoning). Infini Memory reframes persistent memory as a lifecycle maintenance problem — write, maintain, read — and aims for an inspectable, editable state without a mandatory vector or graph backend.

    Methodology Memory is a library of topic documents, each a maintenance scope with a summary, body, and entry-level metadata signatures (<seq,time,source>) that preserve order and provenance as content is rewritten. Writes are decoupled from structure: new candidates append to a buffer document, then periodic consolidation rewrites, splits, updates, and merges them into coherent topic documents. Retrieval can run over plaintext via lexical indexing rather than embeddings. At inference an agentic read procedure lets the LLM iteratively choose memory tools, inspect intermediate results, expand local context, and assemble evidence before answering.

    Results On MemoryAgentBench the agentic-retrieval variant scores 64.7% overall and 81.2% on Accurate Retrieval, with gains on Factual Recall, Test-Time Learning, and Selective Forgetting. Ablations on LongMemEval_S isolate two complementary sources: holding the hybrid reader fixed, removing structural split-and-merge maintenance drops accuracy 76.0%->69.3% (-6.7, concentrated on knowledge-update and multi-session questions), while upgrading the reader from hybrid to agentic adds 3.3 points (76.0->79.3) — so maintenance matters more than the retrieval upgrade and neither is sufficient alone. A split-threshold sweep shows over-fragmentation is recoverable, but oversized documents that mix subtopics are costly.

  15. T-Mem: Memory That Anticipates, Not Archives

    Guo, Weidong · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract T-Mem is a long-term conversational-memory architecture for LLM agents that aims to make every stored memory reachable two ways: by surface similarity (descriptive recall) and by latent semantic/causal link (associative recall). It precomputes, at write time, four families of retrieval cues ('triggers'), one per quadrant of a granularity (item vs scene) x orientation (descriptive vs associative) design space, and retrieves via a topic -> scene -> item cascade fused with RRF over lexical and dense indices. It reaches state-of-the-art on LoCoMo (80.26%) and LoCoMo-Plus (74.81%).

    Motivation Existing LLM memory systems (flat RAG, graph/temporal-KG, hierarchical, OS-style) all retrieve by projecting query and memory into one similarity space and taking top-K, so they are reachability-bounded by lexical/dense similarity. In long-running dialogue users rarely re-raise old topics with the same wording; they revisit them through indirect situational cues, so the target's surface form has drifted and can never be reached from the same neighbourhood. This associative half of the query-memory relation is a structural blind spot.

    Methodology M is a typed tuple of scenes, items, topic labels, four trigger families, and per-speaker Persona, built by a load-bearing four-stage offline pipeline: event-closure scene segmentation, incremental data-grown topic labelling, dual-granularity item extraction (atomic + connected, one LLM call per topic), and trigger instantiation (Entity+Bridge jointly per item; Scene+Horizon per scene). Memory-construction LLM is GPT-4.1-mini, dense encoder bge-m3. Retrieval is a top-down topic -> scene -> item cascade scored by RRF over BM25 + per-type dense rankings; multi-view trigger indices surface host nodes via nan-aware max cosine, and associative triggers bypass the topic prefilter so cues outside the similarity neighbourhood still hit. Triggers stay off the QA evidence path; Persona is ambient context appended after retrieval.

    Results On LoCoMo, T-Mem reaches 80.26% LLM-as-judge accuracy (51.96 token-F1), 3.25 pp above the strongest baseline HyperMem and the maximum on five of six columns. On LoCoMo-Plus it scores 74.81%, narrowing the LoCoMo-to-LoCoMo-Plus drop to 5.45 pp, about 5x tighter than HyperMem (28.38 pp) and near an order of magnitude tighter than the Mem0/SeCom/A-Mem cluster (~49 pp). Ablations confirm the scene-level associative triggers drive the associative gain: removing Scene+Horizon collapses LoCoMo-Plus by 22.19 pp (Horizon alone -12.47 pp) while moving LoCoMo by under 0.4 pp. T-Mem also reaches higher accuracy than HyperMem at a lower input-token budget.

  16. Memory Depth, Not Memory Access: Selective Parametric Consolidation for Long-Running Language Agents

    Han, Haoliang · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Long-running language agents accumulate more history than fits in working context, and the usual fix is retrieval — store past events outside the model and fetch a relevant subset at query time. This paper argues retrieval only answers what can be fetched (memory access) and not what should keep shaping behavior after the working context is unloaded (memory depth). It introduces the loop-drift protocol, a controlled stress test where the retrieval index stays intact but working context is cleared, so goal-conditioned behavior must persist through long-loop interference without the relevant text being reinserted. It evaluates EVAF, a surprise- and valence-gated LoRA consolidation mechanism that writes only behavior-relevant events into a small adapter. Across GPT-2, TinyLlama, and Mistral-7B, retrieval wins shallow factual recall while EVAF wins goal persistence and post-unload recovery with only 2–3 parametric writes per 200 events, and the paper shows selective consolidation factorizes into two separable controls — selection and actuation.

    Motivation Memory access and memory depth are different problems. A shallow memory is one the system can retrieve or attend to; a deep memory changes future behavior — it persists through interference, survives context unload, and affects choices without being reinserted as text. Retrieval is indispensable for fetched facts, but a long-running assistant also needs durable goals, preferences, and constraints that are not merely fetched facts. Existing long-memory benchmarks (LongMemEval, LoCoMo) emphasize conversational recall, temporal access, and knowledge updates, and do not isolate the post-unload setting where retrieval remains available but behavior must continue without the relevant text in context. The paper's claim is narrow and explicit: memory depth can be probed by post-unload goal-conditioned behavior, and consolidation factorizes into selection and actuation — it does not claim universal memory accuracy, SOTA retrieval, or complete deletion/update validity.

    Methodology Loop-drift protocol: synthetic per-user streams of 200 events (10 users/run) mixing stable goal/preference reminders, off-topic distractors, transient opposite requests, conflicts, sibling-user contamination, and scheduled factual notes; four probe layers — shallow episodic (recent fact), noisy episodic (old fact after same-key interference), parametric tendency (does a stable goal still shape behavior after long interference), and post-unload recovery (re-probe the goal immediately after a context unload, with the retrieval index intact but working context cleared). The RAG baseline stores all events in a durable embedding index (top-3 cosine) that context unload does not clear, so any EVAF goal-layer advantage is not a trivial 'RAG forgot' artifact. EVAF mechanism: per-event surprise (token negative log-likelihood) and valence (embedding similarity to the user's durable goal/preferences) combine into an admission gate; events above threshold enter a buffer, and when the buffer fills a LoRA adapter is updated on the buffer plus replay from prior consolidated events, with an L2 anchor as a drift guard. Model controls: GPT-2 and TinyLlama (four-seed means) plus Mistral-7B. Selection is isolated with a matched-random gate (same write count and online write dynamics, random admitted events). Actuation is isolated with fixed-inner controllers (fixed-1/2/3 inner LoRA steps using the same gate). A routed EVAF+RAG variant routes factual probes to retrieval and goal probes to EVAF. Public Memora event streams serve as an external boundary diagnostic for stale-memory invalidation, tested with McNemar's test.

    Results Depth flip: RAG is strongest on recent explicit facts (short-fact accuracy 0.956–0.973) and near-useless on goals; EVAF is near chance on short facts but much stronger on the goal layer — on GPT-2 EVAF reaches 0.904 goal / 0.900 post-unload vs RAG 0.398/0.394, and on TinyLlama 0.833/0.812 vs RAG 0.396/0.394 — at only 2.4–2.6 writes (L2 drift ~21–29) vs RAG's 0 writes. Writing everything is not enough: Naive-LoRA writes all 200 events at far higher drift (~67 TinyLlama, ~119 GPT-2) and still fails the goal layer; at 7B indiscriminate writing is actively harmful — Naive-LoRA goal persistence collapses to 0.333±0.047, below the 0.500 chance baseline. Selection is not sparsity: on GPT-2 EVAF beats a matched-random gate of equal write count on goal and post-unload in all four seeds (mean 0.790/0.763 vs 0.590/0.619); TinyLlama is weak/mixed, so the selection signal is not monotonic in model scale. Actuation is a separable, model-dependent factor: fixed-inner audits show smaller inner steps cut drift and improve goal/post (Mistral-7B five-step 0.354/0.306 -> Fixed-2 0.796/0.775 -> Fixed-1 0.919/0.938), but Fixed-1 contamination saturates at 1.000 on Mistral, so high actuation trades selectivity for goal strength. Asymmetric coupling: under a miscalibrated five-step actuation at 7B the matched-gate comparison reverses, yet EVAF still keeps lowest sibling contamination (0.787±0.041) — selection stays semantically active while its translation into goal behavior fails. Boundary: on Memora, EVAF improves forgetting-absence only 91/222 to 95/222 (p=0.57, not significant), so append-only selective consolidation does not solve stale-memory delete/update validity, which the paper leaves to validity-gating or reconsolidation.

  17. ESAA-Conversational: An Event-Sourced Memory Layer for Continuity, Handoff, and Curation Across Heterogeneous LLM Coding Agents

    Brito dos Santos Filho, Elzo · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract ESAA-Conversational is a shared memory layer that lets several LLM coding agents — Codex, Claude Code, Grok — hand work off to one another without copying the conversation by hand. It watches each agent's visible turns through hooks or watchers, writes them into one append-only log (activity.jsonl), and deterministically projects compact files — a handoff contract, current state, recorded decisions, and an open-task list — that the next agent reads to pick up where the last left off.

    Motivation Developers increasingly switch among multiple coding agents as context windows fill or a different tool suits a subtask, but each agent keeps its conversation in a private, vendor-specific log. The result is 'conversational state drift': goals, rejected alternatives, decisions already made, and open tasks established with one agent are not reliably available to the next. The usual fix — copy-pasting context — is manual, lossy, expensive in tokens, and conflates capturing evidence with interpreting it.

    Methodology The system applies event sourcing and CQRS: visible turns are captured mechanically, with no LLM inference, into an append-only activity.jsonl that is the single source of truth, while state.md, handoff.md, decisions.md, and tasks.json are reconstructible read models that are never hand-edited. 'Inverted ingestion' means the runtime reads native agent logs, hooks, or watchers and normalizes them into conversation_turn events rather than requiring agents to share a protocol. A strict boundary separates mechanical capture (turns are evidence) from curation (durable decisions and tasks entered through explicit decide/task commands). A paginated context command serves filtered windows (--last, --around, --before, --topic) so a cold agent reads a slice, not the whole log; workspace_root isolates projects and a lockfile serializes writes. The v1.1.0 release is a local PowerShell CLI.

    Results A self-referential case study recorded 570 events (562 conversation turns) in a single workspace on 21 June 2026, distributed across Codex (304), Claude (79), and Grok (67). The three heterogeneous agents co-designed and reviewed the tool through the shared log alone, with no direct agent-to-agent channel — e.g., Codex was given a focused view of recent Grok iterations via `context --agent grok --last 20`, and one concrete defect (incomplete filtering of legacy events under context --topic) was found, fixed, and closed as a task. The public release ships 51 tests in its main battery. Reported limitations: the implementation is Windows/PowerShell-only, sync depends on third-party hook surfaces outside the authors' control, retrieval is purely textual with no embeddings, and the system offers operational but not forensic auditability (no hash chains or signatures); validation covers one workspace and three agents.

  18. AutoMem: Automated Learning of Memory as a Cognitive Skill

    Wu, Shengguang · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract AutoMem trains an agent to get better at managing its own memory, the way a person gets better at note-taking with practice. It gives the agent file-system operations (read, write, search, append) as memory actions on equal footing with its normal task actions, then uses a strong meta LLM to review whole episode traces (tens of thousands of steps) and improve both the memory scaffold the agent works within and, separately, the agent's own skill at making memory decisions.

    Motivation Long-horizon tasks can run for 10^4-10^5 steps, far past what a human reviewer can practically read through to find where a memory decision went wrong; a single bad memory choice can stay hidden for thousands of steps before it matters. Existing memory systems treat memory as a fixed architectural module rather than a skill the agent can improve, so there's no natural way to optimize it beyond manual tuning.

    Methodology Two outer loops wrap a shared inner-loop agent that treats a directory of files as its memory. Loop 1 (structure): a meta-LLM reads full episode traces, diagnoses memory-use failure patterns, and revises the agent's scaffold, prompts, file schema, action vocabulary. Loop 2 (proficiency): a meta-LLM selects the agent's own good memory decisions across many episodes as supervised training data and orchestrates LoRA fine-tuning of a dedicated memory-specialist model, while the task-action model stays frozen. Evaluated on three procedurally generated long-horizon games (Crafter, MiniHack, NetHack) with Qwen2.5-32B-Instruct as the base model.

    Results Optimizing memory alone, without touching the model's task-action behavior, improves the base agent's performance roughly 2x-4x across the three games, and the optimized 32B model outperforms Qwen2.5-72B-Instruct on all three, becoming competitive with frontier systems like Claude Opus 4.5 and Gemini 3.1 Pro Thinking. The authors frame this as evidence that memory management is an independently learnable skill and a high-leverage optimization target for long-horizon agents.

  19. SelfMem: Self-Optimizing Memory for AI Agents

    Yang, Shu · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract SelfMem gives an AI agent a set of memory tools and feedback signals, instead of a fixed rule for what to remember and how, and lets the agent figure out its own memory strategy. The idea, as the authors put it, is teaching the agent to fish rather than giving it a fish: rather than forcing a predefined summarization format, the agent inspects the raw conversation, decides what's worth writing to its memory workspace, and can revise that memory when a self-check flags a problem.

    Motivation Existing agent-memory systems (Mem0, MemGPT, MemoryBank, A-Mem) rely on manually specified strategies for what to store, update, and retrieve, which are rigid across tasks and need hand-tuning. Different tasks and conversation histories need different memory behaviors, and no fixed schema fits all of them; the authors argue this calls for letting the agent adapt its own strategy rather than hand-crafting one.

    Methodology SelfMem keeps the raw transcript as an immutable source of truth, accessible only through read-only tools like a SQL-queryable turns table, and gives the agent a separate memory workspace plus a memory action space: read the transcript, read/write/revise memory, check memory quality, and even extend its own memory toolkit. The agent decides what to store, compress, update, or leave for transcript retrieval. Evaluated on BEAM across conversation scales from 100K to 1M tokens against retrieval, compression, and agent-memory baselines.

    Results SelfMem gets the highest official score and Pass0.5 at all three tested scales (100K, 500K, 1M tokens), improving the official score over the strongest baseline by 0.165, 0.141, and 0.134 respectively, and Pass0.5 by 14.3-17.0 percentage points. It's the best performer on 9/10 question types at 100K tokens (8/10 at 500K, 7/10 at 1M), and a model-guided strategy-refinement study shows further gains are possible on top of the base approach.

  20. Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

    Kwon, Alex · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A language model's memory can be worse than no memory at all. Give a model a memory that kept a wrong conclusion but dropped the work behind it and it re-emits the stale value as a confident answer; give the same model an empty memory and it abstains. The paper names this failure brittle memory, measures it with a reclaim-evaluation protocol that tests whether a correction can recover a known answer after compression, and shows a one-line fix (keep the recomputable source, drop the re-derivable conclusion) restores correctability at equal memory budget.

    Motivation Memory systems carry information across sessions by compressing it, on the implicit assumption that a compression preserving the model's answer has preserved what matters. The paper shows the same compression decides whether the model can later be corrected: once the answer-determining source is gone, a correction has nothing to act on, and the resulting error compounds as deployed agents feed memory into memory.

    Methodology Reclaim evaluation drifts a model into committing to a wrong answer via a planted premise, deepens the commitment over neutral turns, then issues a correction in a fresh session whose only inheritance is a memory written under one of three matched-budget policies: lossy (keep the salient conclusion, shed the source), source-first (keep the source, shed the conclusion), and lossy-padded (lossy plus neutral filler to at least source-first's length, controlling for budget). Success is exact recovery of the known answer, with no judge. Tasks are multi-step arithmetic and constraint-logic puzzles with objectively scorable answers; the pipeline runs end to end on llama-3.1-8b and grok-4.3 with a frontier replay on Claude models; headline cells are n=96, and three validators designed to fail against a deterministic fake all pass.

    Results Within one conversation there is no wall, only anchoring: a directed correction holds far longer than a generic one (reclaim 0.79 to 0.50 over eight commitment turns) and pushing the error further back lifts reclaim rather than starving it. Across a session boundary the window becomes a wall: once the lossy note drops the source line items, even a directed correction dies, reclaim is 0.00 by measurement, and a lossy memory is worse than an empty one because models that abstain with nothing emit the confident wrong value with a source-less note. The wall sits in the same place from the 8B model to frontier systems. Source-first restores reclaim at equal budget (oracle 1.00; the deployable one-prompt distiller 0.49-0.88, concentrated on compact numeric sources), and the length-matched control rules out added text as the cause. Chained through a memory loop, one dropped-source error corrupts a growing span of downstream steps and stays uncorrectable however late it is caught, while source-first holds to a bounded budget horizon; the wall and the fix replicate on three deployed memory systems and on MultiWOZ, and past the budget where the source no longer fits, the fix fails silently unless the note records its own completeness.

  21. FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents

    Wang, Tianyou · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A benchmark in which an LLM agent runs a football club for 20 in-game years against 15 rivals, scored cumulatively by a deterministic engine with no LLM judge. It measures long-horizon management under hidden information, cumulative consequences, a counter-adaptive market and multi-objective pressure, and treats the agent's self-written notebook as its only cross-stop memory.

    Motivation Language model agents handle bounded tasks reliably, but those tasks have a short horizon to a correct answer and no competing agents. Newer benchmarks push one dimension at a time: long-horizon work stretches episodes to hundreds of steps or simulates months of a vending machine, shop, startup or company's finances, while competitive benchmarks stage cooperation and conflict over short episodes or rank models by bargaining inside a single session. None combines both dimensions while giving the agent an organization to keep alive. The authors define management as running an organization against rivals pursuing the same opportunities over a horizon long enough that early decisions reshape the world, judged cumulatively under the conditions those decisions created.

    Methodology The environment instantiates four demands with concrete mechanisms: hidden information through scout bands carrying a permanent per-scout bias, hidden player traits and hidden asks in negotiation; cumulative consequences through youth and facility investment that pays off over years, insolvency and confidence spirals that compound yet remain recoverable, and honors accruing into the final score season by season; a counter-adaptive market where rejected bids raise the hidden ask, repeat pairs draw markups, and negotiation cooldowns apply; and multi-objective pressure from a board judging results and financial discipline jointly with season targets scaled to squad strength. A run spans roughly 340 to 400 decision stops across 26 schema-generated tools, with a per-stop budget of 30 queries and 10 negotiation moves. Each stop is a fresh conversation, so the only carried state is a private notebook edited through append_note and rewrite_notes, which makes memory curation itself a measured capability. Two tracks share one engine: a solo track playing each of 15 frontier models against a frozen scripted world with tiered opponents, and an Arena placing the same models plus a scripted anchor in one shared 20-year economy, made comparable by an equal-endowment draft, sealed-bid conflict resolution and capped revival. Grading is continuous and mechanism-computed throughout, with no LLM judge or human rater. Memory curation is scored by reconstructing each notebook at season end and taking TF-IDF cosine similarity between consecutive snapshots, read alongside notebook size. Three seeds were run, and six first-play humans ran the same track.

    Results Across three seeds all 15 models complete every horizon while the blind scripted anchors, including a disciplined heuristic, die out in 7 of their 9 runs. claude-fable-5 tops both tracks, reaching about 95% of a scripted upper anchor allowed to read the hidden state (90.94 against 95.54), yet competition still reshuffles the board: the league title rotates among ten models and mid-board solo standings do not survive adaptive rivals. Neither scale, price nor vendor predicts the order, the order settles only late in the horizon, and the best first-play human finishes at the bottom of the model board while four of six humans died out. Three of the six behavioral capabilities track score on every seed: reducing slow-payoff investment as the horizon ends (Spearman -0.58), keeping cash deployed rather than idle (-0.50), and opening contract renewals early (+0.45). Token spend is uncorrelated with score under every accounting. Two negative results stand out. Hundreds of rejected bids never teach a model where the market's true prices lie. And self-managed memory fails in two opposite regimes rather than one: an append-only archive in which current state drowns in history at the high-similarity end, and wholesale rewriting at the low end, with claude-sonnet-5 at 0.20 and qwen3.7-max at 0.23 rewriting so completely that no plan survives to be executed. The winner sits at 0.39 against a field median of 0.31, holding a stable strategy skeleton while rewriting state each season. Similarity alone does not certify curation quality: notebook size decodes it, since the same high consistency is a 200k-character archive for one model and a 3-6k curated document for the winner.

  22. Can Agent Memory Systems Track Evolving State?

    Fan, Xinyi · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A benchmark and method for a memory capability separate from recall: keeping track of which version of a fact is currently in force after it has been revised across sessions. The benchmark, StateMemBench, generates each scenario as a symbolic program of state operations so the correct answer is computed by replay and the specific way a lazy reader would get it wrong is known in advance. The method, StateMem, parses each turn into structured state units with typed dependency links, then handles supersession and staleness deterministically rather than with an LLM.

    Motivation Existing memory systems and benchmarks optimize recall of relevant facts, but as agents run longer, facts, constraints and decisions get revised, and an answer must reflect the current state rather than a superseded one. The authors call the failure state drift: the relevant fact is present in the assembled context, but the agent acts on a stale or incomplete version of it. This is distinct from dialogue state tracking, which prescribes a slot-value representation and evaluates it directly over cooperative dialogues that accumulate a goal monotonically; here state is whatever a system must maintain to answer correctly, evaluation is purely behavioral, and revisions are adversarial across sessions. Some concurrent work centers state, but the authors argue none cleanly isolates state tracking from the other errors it co-occurs with.

    Methodology The authors first define and label drift on existing benchmarks, assigning a failure to drift only after excluding retrieval, comprehension, schema and reasoning readings, dropping unassignable points, and cross-checking with two judge passes, a cross-family judge and two human annotators. They then build StateMemBench: each scenario is a symbolic event program of typed operations over ground, derived and declared state; the gold answer comes from deterministic replay; a family of executable lazy reader policies is run against the replay, and a scenario is admitted as a trap when policies disagree, with the disagreeing set forming its failure-mode signature (status, salience, sequence, compound, plus anti-trap controls). Programs are grounded in public data for surface vocabulary and rendered into multi-session dialogue by a strong LLM, then programmatically verified for fact placement and phrase leakage. Probes are closed-pool: an unseen pool of three to four options holds the gold answer, the targeted policy's drift answer and neutral distractors. StateMem itself runs a per-turn TurnEncoder producing state units (id, content, priority, source, deps), a deterministic update stage applying supersessions and marking dependents needs_recheck by dependency-graph traversal, and a single answer-time call over the assembled active state. A wrapper variant applies the same trace-then-resolve structure as a prompt-level transformation of any backend's answer call, evaluated against a length- and cost-matched generic-extraction control.

    Results Drift leads the confirmed failure distribution on several existing benchmarks (63.5% on MemoryArena-shopping, 44.4% on LongMemEval oracle where retrieval is perfect by construction, 10 of 16 on tau-squared-bench-Z) but is not universal, falling to 19.0% on MemoryArena-travel. On StateMemBench, long-context is not the strong baseline it is on recall tasks: the best long-context model reaches 0.277 and same-backbone long-context 0.149. StateMem reaches 0.363 on DeepSeek-V4-Flash, 1.8x the best memory system and 2.4x same-backbone long-context, and 0.233 on Qwen-3.5-9B, 1.6x the best memory system, both significant by paired McNemar at p < 0.001; GraphRAG at 0.224 is statistically level on Qwen. Ablations put supersession marking as the largest single component and show dependency propagation over-fires on Set B anti-traps by 12.5 points, so removing it leaves DeepSeek slightly better. Drift-rate analysis shows the memory layer barely changes outcomes on a weak answerer, where every arm drifts at 61 to 66%, and separates on the stronger backbone, where StateMem's drift rate falls 15 points and correct answers rise by 42 while long-context, Mem0 and BM25 move by 1 to 3 points. The wrapper improves every one of six backends on both benchmarks, adding 31.7 to 66.6 points on StateMemBench with 15.0 to 31.7 attributable to state structure over the matched control, significant in all twelve cells. State tracking does not cost recall: StateMem also leads memory systems on LongMemEval (0.656 on DeepSeek) and LoCoMo (0.592), with margins concentrated on temporal-reasoning and knowledge-update question types. The authors note StateMem mirrors the policy family behind the traps, so its StateMemBench margins should be read as an upper bound.

  23. Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents

    Li, Baichuan · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A benchmark for the decision an agent makes before it writes to memory: should this piece of interaction-derived information be persisted durably, used only in the current context, re-checked against the world, or clarified with the user. It evaluates three models from two families under three prompt conditions, and separately tests whether the stated decision predicts which memory tool the model actually calls.

    Motivation Persistent memory personalizes an agent, but an incorrect durable update silently distorts future behavior. A temporary request should not become a standing preference, a service status can go stale, one tool failure may be noise, and an underspecified correction may need a question before it is generalized. The authors argue the critical capability is not recall but commitment, and that two sources of uncertainty are usually collapsed: verification queries the world, which is authoritative for changing facts, while clarification queries the user, who is authoritative for intent and scope. Prior work covers adjacent targets (binary session-level storage gating, ADD/UPDATE/DELETE/NOOP operations, whether retrieved memory grounds tool parameters, general ambiguity clarification) without jointly distinguishing durable storage, local use, world verification and user clarification at commitment time.

    Methodology Each item supplies an acquire context, a candidate update and a later reuse context, with a gold action assigned by released rules and a tie between persist and a weaker action resolved toward the weaker commitment. 140 primary scenarios are split 70/70 by sorted identifier within category, covering stable and episodic preferences, freshness-sensitive facts, one-off corrections, policy constraints, ambiguous updates and noisy failures at 20 items each, with eight lexical traps where a surface cue points to the wrong action. Two non-authors labeled the held-out and contrast items blind to author labels and to each other, with a blind third resolving ties. Claude Haiku 4.5, Claude Sonnet 4.6 and a locally served Qwen3.5-9B (Q4_K_M via Ollama, temperature 0, seed 13, thinking disabled) were each run under three conditions: a bare prompt defining the actions, a policy prompt adding five commitment rules including the tie-breaker, and a four-shot prompt with one development example per action. A separate track, MCB-Act, removes the label vocabulary and requires one structured tool call, scored by mapping the selected tool to an action. Analysis uses accuracy with bootstrap intervals, macro-F1, over-memory, under-memory and per-class recalls, with exact paired McNemar tests and Holm correction within each family of comparisons.

    Results Both model families under-ask. Claude label-mode verification recall runs 0.889 to 1.000 while clarification recall runs 0.500 to 0.750; bare Qwen verifies 12 of 18 freshness items and asks on 0 of 12 clarification items. Few-shot prompting lifts Qwen accuracy from 0.557 to 0.771 (paired delta +0.214, p_H = 0.002) and clarification recall from 0 to 0.333, still missing 8 of 12. The policy prompt raises Qwen accuracy by only 0.071 (p_H = 0.539) but cuts erroneous persistence from 0.243 to 0.100 (p_H = 0.038), moving the uncertainty to verification (recall 0.667 to 0.944) rather than to the user (clarification 0 to 0.083). Haiku's policy and few-shot gains survive correction (p_H = 0.002 and 0.047); Sonnet's do not, so the benchmark measures a prompt-conditioned commitment policy rather than a fixed model trait. Label-to-tool agreement is 0.571 for each Claude model and 0.229 for Qwen; Sonnet accuracy falls from 0.814 to 0.529 (p_H < 0.001) and Qwen from 0.557 to 0.343 (p_H = 0.047), with Qwen calling use_now on 54 of 70 items and verification recall collapsing to 0.056. All emitted arguments pass the deterministic well-formedness rules, locating the bottleneck in tool choice. On the combined 140 Qwen items of the contrast extension, bare, policy and few-shot accuracy is 0.614, 0.757 and 0.843, with clarification remaining the weakest class throughout; the authors retain the extension as a controlled sensitivity check rather than a claim of naturalistic external validity, since its rule-authored templates align closely with the explicit policy rules. MCB-Act scores tool-call selection and does not execute downstream effects.

  24. MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance

    Wang, Haoyu · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A memory framework for LLM agents that runs an LLM verifier over each candidate trajectory and then keeps the verifier's output on the record permanently, as reward, confidence, label and uncertainty fields. Those fields decide whether the memory is admitted, held provisional or rejected, and are read again later to rank retrieval, resolve conflicts, trigger summarization and archive stale records. Evaluated on Terminal-Bench 2.0, SWE-Bench Verified, WebArena and Mind2Web across four backbones.

    Motivation Agent memory is only useful if stored experience stays reliable over hundreds of interactions, and two failure modes break that. Unreliable admission: failed trajectories, accidental successes, invalid patches and misleading observations all look relevant to a retriever and get written, then mislead later decisions — a web agent that learns to edit the first row after filtering an admin table edits the wrong row once a table is sorted differently. Memory drift: a bank that grows across many tasks accumulates duplicate, conflicting, stale and overgeneralized records that remain retrievable long after their assumptions stop holding. Prior experience-memory systems (Synapse, AWM, ReasoningBank) make memory persistent but treat write-time feedback as a one-shot signal, so nothing continues to govern the record afterward. Verifier work, in turn, scores the current episode and stops there.

    Methodology Verification is decomposed into multiple criteria — completion, consistency, validity, generalizability — with reward estimated from score-token distributions and repeated views; repeated verification is triggered when uncertainty crosses a threshold. The resulting descriptor (reward, confidence, label, uncertainty) is attached before activation and routes the candidate to rejection, a provisional state, the active bank, or a failure-guard pool that stores failed experience as constraints rather than recipes. Each record additionally carries lifecycle state, quality, usage statistics and conflict links. Retrieval is a hybrid of BM25 and embedding cosine similarity over title, description and content, adjusted by the descriptor and by staleness and overgeneralization penalties, with positive memories and failure guards rendered as separate blocks in the injected prompt. Conflicts are located by structured-signature similarity above a threshold. Governance runs after every task: activation, rejection, merge and conflict checks immediately; summarization and archival when the active-memory budget is exceeded or a record goes stale. Baselines are No Memory, Synapse, AWM, ReasoningBank and a verifier-only filter, all matched on task order, step budget, retrieval budget, injected-memory budget, memory block template and decoding settings.

    Results Averaged over five seeds, MemGuard has the best success metric and the lowest average step count in all 16 backbone-benchmark settings. Against ReasoningBank, the strongest prior memory baseline evaluated, the largest gain is 7.9 success-rate points on WebArena and 5.6 step-success-rate points on Mind2Web, with 2.4 to 3.5 points on the terminal and software-engineering benchmarks. Backbones are Qwen-3.5-Flash, Qwen-3.5-Plus, Gemini-3-Flash and Gemini-3.1-Pro. The verifier-only control improves on ReasoningBank in most cells by admitting better candidates but loses to MemGuard in every benchmark-backbone cell, which the authors read as evidence that the gain comes from persisting the verifier signal through the lifecycle rather than from filtering once at the door.

  25. LivingRAG: Augmenting Graph RAG with Experience

    Cui, Yuzhuo · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Graph RAG systems answer each query independently and discard the reasoning afterwards, so related later queries start from scratch. LivingRAG adds a writable experience store to a graph retrieval backbone, keeping for each verified answer the entity activation map that worked and a compact reasoning summary, and reusing them on two separate paths: fused into the retriever's starting activation, and inserted into the prompt as a scaffold. Across five online QA streams it improves accuracy over graph RAG baselines while cutting completion tokens 22.7% and estimated cost 12.1%, with grounding and novelty gates admitting only 27.4% of candidate experiences.

    Motivation Reuse across an online query stream is not limited to repeated entities. Two questions may visit nearby regions of the retrieval graph with entirely different entities, or follow the same reasoning pattern with different subjects, as when two comparison questions both ask which person was born earlier. A measurement of the streams supports the distinction: direct entity overlap with an earlier query ranges from 4.30% to 76.69% by dataset, while graph-neighborhood overlap runs 80.45% to 99.70%, so an entity-keyed answer cache would miss most of the available reuse. The countervailing risk is that a writable store lets unsupported reasoning be written once and then amplified by every later query that reuses it.

    Methodology The retrieval backbone is LinearRAG, unchanged: a passage-sentence-entity graph built with lightweight entity extraction and no LLM calls during indexing, query entity activation, propagation over the sentence-entity graph, and Personalized PageRank over the passage-entity graph. A verified experience stores the query, its embedding, a sparsified final activation vector, a compact summary, the answer, a timestamp and a grounding confidence. Experiences are scored against a new query by a convex combination of query-embedding cosine and activation-map cosine, and the top-K maps are fused into the initial activation vector with softmax weights scaled by stored confidence and deliberately not renormalized. Scaffold selection adds a masked-template similarity term that retrieval omits, because a scaffold only enters the prompt and cannot inject historical entities into graph retrieval. Write-back requires a novelty score computed against the base rather than the fused activation to avoid circularity, and a grounding score equal to the fraction of extracted atomic claims entailed by retrieved passages under an NLI model; novelty is checked first so NLI runs only on survivors, and the grounding score becomes the stored confidence. Evaluation runs 2WikiMultiHopQA, HotpotQA, MuSiQue, MuSiQue-full and WixQA as online streams with an empty initial store, generating with Qwen3.6 Plus, reporting contain-match and LLM-evaluation accuracy.

    Results LivingRAG leads all baselines on the four multi-hop benchmarks, for instance 58.42 LLM-evaluation accuracy on MuSiQue-full against 52.71 for LinearRAG, 47.75 for GFM-RAG and 44.06 for HippoRAG2, and 70.25 against 66.00 on WixQA. Prompt tokens rise 3.5% weighted because scaffolds add context, completion tokens fall 22.7%, and total estimated cost falls from $51.05 to $44.87, a reduction on every dataset. The saving is not a late-stage artifact: per-segment traces show the system spending more completion tokens than its backbone in the earliest segments and turning negative once experience accumulates. Realized reuse differs sharply by stream, with graph transfer at 91.20% on MuSiQue and 0.00% on 2Wiki, where realized template reuse is 84.15% instead. Ablations match those traces, since removing activation fusion costs most where graph transfer is realized, removing scaffolds costs most on 2Wiki and cuts token savings from 23.6% to 8.3%, and removing the quality gate lowers accuracy on all three tested datasets. The stated limits are the store's lifecycle: nothing updates, downweights or deletes an accepted experience, and the fixed-corpus benchmarks contain no chronological updates or fact-validity intervals, so staleness cannot be evaluated.

Knowledge Representation

The data structure memory lives in: flat vector RAG, entity–relationship knowledge graphs, atomic facts, event-grounded episodic records, or layered semantic/episodic/procedural stores.

Open questions
  • Graph, vector, atomic facts, or events — and is the choice load-bearing or incidental?
  • Do you model who/when/where (episodic coherence) or only what (semantic recall)?
  • Is the atomic-fact paradigm (handcrafted prompts → compressed facts) actually the right primitive?
  1. Amory: Coherent Narrative-Driven Agent Memory through Agentic Reasoning

    Zhou et al. · 2026 0 cites

    Synthesis

    Argues current frameworks fragment conversations into isolated embeddings or graph nodes; proposes building a coherent narrative via agentic reasoning instead.

    Why it matters Bridges consolidation and representation: the unit of memory should preserve narrative coherence, not just be a retrievable shard. Connects to [cast-episodic]'s who/when/where stance.

  2. EverMemOS: A Self-Organizing Memory Operating System

    Hu et al. · 2026 0 cites

    Synthesis

    Notes most memory systems store isolated records and retrieve fragments, limiting consolidation of evolving user state and conflict resolution. Proposes a self-organizing memory OS for structured long-horizon reasoning.

    Why it matters 'Memory OS' framing that ties consolidation to conflict resolution (Temporality) and structure (Representation). Conceptual cousin of Letta/MemGPT-style OS metaphors.

  3. D-Mem: A Dual-Process Memory System for LLM Agents

    You et al. · 2026 0 cites

    Synthesis

    Critiques incremental per-turn extraction/update; proposes a dual-process design (fast + slow paths) for high-fidelity memory access over long horizons.

    Why it matters Dual-process (System-1/System-2) is a recurring shape; here applied to when to consolidate cheaply vs reason deeply. Connects to [retrieve-or-think]'s retrieve-vs-think policy.

  4. Graph-based Agent Memory: Taxonomy, Techniques, and Applications

    Yang et al. · 2026 0 cites

    Synthesis

    The survey of graph-structured agent memory: why graphs (relational modeling, knowledge accumulation, self-evolution), the technique landscape, and applications across multi-turn dialogue, games, and scientific discovery.

    Why it matters The map of the KG-memory design space — read before committing to or rejecting a graph. Read against [kg-wrong-abstraction] for the dissent.

  5. Hot take: knowledge graphs are the wrong abstraction for agent memory

    u/Expert-Address-2918 · 2026 0 cites

    Synthesis

    Argues the field over-converged on entity–relationship graphs (Mem0, Zep, supermemory). Costs flagged: an extra entity-extraction LLM step (latency) and hallucinated edges that fabricate connections — when the real job is fast retrieval of the right past context.

    Why it matters The sharpest practitioner counter-argument to KGs. Worth weighing before adopting a graph layer; [gaama] partially answers it (mega-hub fix), [graph-memory-survey] is the steelman.

  6. GAAMA: Graph Augmented Associative Memory for Agents

    Paul et al. · 2026 0 cites

    Synthesis

    Flat RAG loses structural relationships; entity-centric KGs suffer 'mega-hub' effects (a few nodes accrue too many edges). GAAMA proposes graph-augmented associative memory to keep structure without the hub blowup.

    Why it matters A concrete fix for one of the KG failure modes [kg-wrong-abstraction] complains about. Useful if you want graph structure but fear scaling pathologies.

  7. PersonalAI: Systematic Comparison of KG Storage and Retrieval for Personalized LLM Agents

    Menschikov et al. · 2026 0 cites

    Synthesis

    Systematically compares knowledge-graph storage/retrieval approaches for personalized agents, against the backdrop that RAG improves factual accuracy but lacks structured memory and doesn't scale in complex long-term settings.

    Why it matters The closest thing to an apples-to-apples KG-design comparison; pairs with [mem0-vs-graphiti] (vector-vs-graph in distributed setting) for store-selection decisions.

  8. Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory

    Sun et al. · 2026 0 cites

    Synthesis

    Critiques the dominant extracted-fact paradigm: handcrafted static prompts compress raw dialogue into atomic facts that are stored, matched, and injected — losing the ability to reason deeply over history. Proposes going beyond atomic facts.

    Why it matters The most on-the-nose challenge to atomic-facts-as-primitive. Read before committing to a fact-extraction pipeline; connects to [amory] (narrative) and [cast-episodic] (events) as richer alternatives.

  9. CAST: Character-and-Scene Episodic Memory for Agents

    Ma et al. · 2026 0 cites

    Synthesis

    Most agent memory emphasizes semantic recall and stores experience as key-value/vector/graph, which struggles to represent coherent events. CAST models episodic memory grounded in who/when/where (characters and scenes).

    Why it matters The episodic-coherence counterpoint to fact/vector stores; aligns with [amory] (narrative) and [adamem] (fragmentation). Connects to Temporality via the 'when' grounding.

  10. How I implemented 3-layer memory for LLM agents (semantic + episodic + procedural)

    u/No_Advertising2536 · 2026 0 cites

    Synthesis

    Practitioner build motivated by agents repeating mistakes (deploy → forget migrations → DB crash). Implements the cognitive-science triad — semantic (what you know), episodic (what happened), procedural (how to do things) — and open-sources it.

    Why it matters Grassroots evidence that the semantic/episodic/procedural layering from research is being adopted by builders. Pairs with [cast-episodic] (episodic) and the procedural-skills theme of the academic explorer.

  11. Temporal Knowledge-Graph Memory in a Partially Observable Environment

    Kim, François-Lavet, Cochez · 2026 0 cites

    Synthesis

    Agents in partially observable environments need persistent memory to integrate observations over time; KGs naturally represent evolving state. Introduces a benchmark where both world dynamics and the agent's memory are explicitly graph-shaped.

    Why it matters Grounds temporal-KG memory in a controllable evaluation setting — rare for this topic. Connects temporality to representation: time is modeled as graph evolution.

  12. Human-Inspired Memory Architecture for LLM Agents

    Kerestecioglu et al. · 2026 0 cites

    Synthesis

    Six cognitive mechanisms: sleep-phase consolidation, interference-based forgetting, engram maturation, reconsolidation upon retrieval, and entity knowledge graphs — a principled lifecycle rather than heuristic decay.

    Why it matters The most complete bio-grounded lifecycle proposal; turns 'forgetting' from a TTL into a set of mechanisms. Unvalidated at scale, like most of this cluster — weigh against eval scarcity ([persistbench]).

  13. Engram: A Bi-Temporal Memory Engine Where a Lean Retrieved Context Beats the Full History

    Wang, Liuyin · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Engram is an open-source long-term memory engine for LLM agents. Instead of replaying an entire conversation history into the prompt, it stores the past as a bi-temporal knowledge graph and retrieves a small, precisely-targeted slice at answer time. On a standard 500-question memory benchmark that lean ~9.6k-token slice answers more accurately than feeding the model the full ~79k-token history, turning memory from a cost optimization into an accuracy improvement.

    Motivation Stateless LLM agents forget across sessions, and the usual fix — concatenating the whole history — grows token cost and latency linearly and loses accuracy as distractors crowd the window ('lost in the middle'). Two gaps stay open: most memory systems are cheaper or faster but not more accurate than full-context, and memory benchmarks run on inconsistent harnesses where one system reports wildly different scores across sources. Engram targets both — beating full-context on accuracy, and shipping a neutral, re-runnable harness.

    Methodology A dual-process design. A System-1 hot write path appends lossless episodes with no LLM (sub-50ms) and enqueues them. A System-2 async path extracts atomic (subject,predicate,object) facts, builds a bi-temporal knowledge graph (valid time vs transaction time on every fact and edge), detects conflicts, and resolves them cheap-then-escalate: exact slot match, embedding similarity, and content subsumption handle the common case with no LLM call, invalidating (never deleting) a superseded fact and recording a supersedes chain and provenance, with only ambiguous cases escalated to an LLM adjudicator. The hybrid read path retrieves through four channels (dense, BM25, graph n-hop, recency/salience), fuses them with Reciprocal Rank Fusion, applies an 'as-of' temporal filter and an abstention gate, and assembles a deduplicated, provenance-tagged, token-budgeted context of facts plus raw chunks.

    Results On the full 500-question LongMemEval_S under the official category-specific judge, Engram's lean configuration scores 83.6% vs 73.2% for full-context (+10.4 points, McNemar exact p<10^-6) at ~8x fewer tokens (9.6k vs 79k), 0/500 errored. The gain is load-bearing on the read path being hybrid: facts alone lose recall, while facts plus retrieved chunks recover detail. Bi-temporal modeling pays off most on knowledge-update (87.5%) and temporal (81.1%) categories, while multi-session aggregation and preference remain headroom. The paper documents measurement-integrity pitfalls (truncation, home-grown judges, full-history leaks) and ships a neutral in-repo harness with the official judge baked in and raw per-question logs, every number reproducible by command.

  14. Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memory

    Ji, Suozhao · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Infini Memory is a long-term memory architecture for LLM agents that stores memory as a library of plain-text 'topic documents' rather than vectors or a knowledge graph. Each document gathers related evidence under a subject and is maintained over time by splitting, merging, and rewriting. At answer time the agent reads memory through iterative tool calls, expanding context around matches instead of taking a single retrieval shot.

    Motivation External memory systems that store observations as isolated records, summaries, or indexed fragments hit four recurring failure modes: fragmentation (evidence about one subject scattered across records), conflict (old and new versions of a fact coexisting), compression loss (summaries dropping temporal and source cues), and insufficient retrieval (single-shot top-k returning fragments without enough local context for multi-hop reasoning). Infini Memory reframes persistent memory as a lifecycle maintenance problem — write, maintain, read — and aims for an inspectable, editable state without a mandatory vector or graph backend.

    Methodology Memory is a library of topic documents, each a maintenance scope with a summary, body, and entry-level metadata signatures (<seq,time,source>) that preserve order and provenance as content is rewritten. Writes are decoupled from structure: new candidates append to a buffer document, then periodic consolidation rewrites, splits, updates, and merges them into coherent topic documents. Retrieval can run over plaintext via lexical indexing rather than embeddings. At inference an agentic read procedure lets the LLM iteratively choose memory tools, inspect intermediate results, expand local context, and assemble evidence before answering.

    Results On MemoryAgentBench the agentic-retrieval variant scores 64.7% overall and 81.2% on Accurate Retrieval, with gains on Factual Recall, Test-Time Learning, and Selective Forgetting. Ablations on LongMemEval_S isolate two complementary sources: holding the hybrid reader fixed, removing structural split-and-merge maintenance drops accuracy 76.0%->69.3% (-6.7, concentrated on knowledge-update and multi-session questions), while upgrading the reader from hybrid to agentic adds 3.3 points (76.0->79.3) — so maintenance matters more than the retrieval upgrade and neither is sufficient alone. A split-threshold sweep shows over-fragmentation is recoverable, but oversized documents that mix subtopics are costly.

  15. ESAA-Conversational: An Event-Sourced Memory Layer for Continuity, Handoff, and Curation Across Heterogeneous LLM Coding Agents

    Brito dos Santos Filho, Elzo · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract ESAA-Conversational is a shared memory layer that lets several LLM coding agents — Codex, Claude Code, Grok — hand work off to one another without copying the conversation by hand. It watches each agent's visible turns through hooks or watchers, writes them into one append-only log (activity.jsonl), and deterministically projects compact files — a handoff contract, current state, recorded decisions, and an open-task list — that the next agent reads to pick up where the last left off.

    Motivation Developers increasingly switch among multiple coding agents as context windows fill or a different tool suits a subtask, but each agent keeps its conversation in a private, vendor-specific log. The result is 'conversational state drift': goals, rejected alternatives, decisions already made, and open tasks established with one agent are not reliably available to the next. The usual fix — copy-pasting context — is manual, lossy, expensive in tokens, and conflates capturing evidence with interpreting it.

    Methodology The system applies event sourcing and CQRS: visible turns are captured mechanically, with no LLM inference, into an append-only activity.jsonl that is the single source of truth, while state.md, handoff.md, decisions.md, and tasks.json are reconstructible read models that are never hand-edited. 'Inverted ingestion' means the runtime reads native agent logs, hooks, or watchers and normalizes them into conversation_turn events rather than requiring agents to share a protocol. A strict boundary separates mechanical capture (turns are evidence) from curation (durable decisions and tasks entered through explicit decide/task commands). A paginated context command serves filtered windows (--last, --around, --before, --topic) so a cold agent reads a slice, not the whole log; workspace_root isolates projects and a lockfile serializes writes. The v1.1.0 release is a local PowerShell CLI.

    Results A self-referential case study recorded 570 events (562 conversation turns) in a single workspace on 21 June 2026, distributed across Codex (304), Claude (79), and Grok (67). The three heterogeneous agents co-designed and reviewed the tool through the shared log alone, with no direct agent-to-agent channel — e.g., Codex was given a focused view of recent Grok iterations via `context --agent grok --last 20`, and one concrete defect (incomplete filtering of legacy events under context --topic) was found, fixed, and closed as a task. The public release ships 51 tests in its main battery. Reported limitations: the implementation is Windows/PowerShell-only, sync depends on third-party hook surfaces outside the authors' control, retrieval is purely textual with no embeddings, and the system offers operational but not forensic auditability (no hash chains or signatures); validation covers one workspace and three agents.

  16. The Log Is the Agent

    Nakajima, Yohei · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract The paper presents ActiveGraph, an open-source (Apache-2.0) agent runtime that inverts the usual framework design: instead of a conversation loop with logging bolted on, the append-only event log is the source of truth, the working graph is a deterministic projection of that log, and agent behavior is a population of reactions that fire on graph changes and emit new events. No component instructs another; coordination happens through the shared graph. This buys deterministic replay of any run, cheap forking at any event without re-executing the shared prefix, and total lineage from goal to each model call.

    Motivation Conventional LLM agent stacks grow by accretion: chat loop, then tools, then rules, then logging, with memory as a lossy similarity-queried store, so the log is a byproduct and questions like 'why is this fact in context', 'what did the agent believe before rule R changed', or 'what if step 42 had gone differently' are awkward or impossible. For long-running agentic work such as diligence, compliance, and research, where the reasoning matters as much as the answer, the authors argue the recoverable causal chain is the actual product.

    Methodology A systems and architecture contribution, explicitly not a task-performance benchmark. The runtime defines events (id, type, payload, actor, caused_by, timestamp), behaviors as subscriptions over event types plus Cypher-subset graph-shape patterns, and a determinism contract (no random, wall-clock, fresh UUIDs, or outside I/O in behavior bodies) policed dynamically by strict replay. Nondeterministic model calls are handled by recording responses in a content-addressed cache keyed on a hash of the full request, so replay and forks make no new model calls. A bundled investment-diligence pack runs fully offline against recorded fixtures on three companies, with no API key, completing in under thirty seconds.

    Results The reproducible quickstart run produced 671 events yielding 93 objects (3 companies, 24 questions, 9 documents, 25 claims, 25 evidence items, 1 contradiction, 3 risks, 3 memos) and 76 relations via 103 model calls and 48 tool calls, with zero orchestration code; re-running produced byte-identical logs. Forking a 200-step run at step 150 pays only for steps from 150 onward. Named costs and limits: replay time grows with log length (no checkpointing or compaction yet), schema evolution is a real operational burden, side-effecting tools still mutate the world on first execution, multi-writer ordering is unresolved, and self-improving agents are discussed only as an affordance, not demonstrated.

  17. Ontology-Grounded Project Memory for Coding Agents

    Adam, James · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MOOSEDev gives coding agents ontology-grounded, typed project memory (decisions, constraints, rationales, lessons) in a small knowledge graph rather than flat notes or vector chunks, and shows it dramatically outperforms a production vector-memory tool on questions requiring completeness, negation, or supersession reasoning.

    Motivation Coding agents generate most new code in many projects today, but the reasoning behind why changes were made is easy to lose track of; notes files and vector-based retrieval-augmented memory help but don't know what kind of record something is (a decision vs. a lesson vs. an outdated, superseded note) or how records relate to each other, which is fundamentally an ontological modeling problem, not a retrieval problem.

    Methodology Two small OWL ontologies (9 and 11 classes) with SHACL-validated typed records, reasoned over by a neurosymbolic engine (MOOSE) that treats the LLM as a narrow, declared-point sensor rather than the primary reasoner, exposed to agents via MCP. Evaluated head-to-head against a production vector-memory tool (mem0) and a BM25 baseline on a neutral 835-record public corpus, using a pre-registered strict LLM judge, plus a live trial bootstrapped from the authors' own repository history.

    Results On tasks requiring set-completeness, negation (absence), and supersession-traversal reasoning, MOOSEDev scored 0.98-1.00 versus 6-27% for the vector-memory baseline — a categorical, not incremental, gap. On simple relevance retrieval the two systems tied. In a live trial, the system provided 71 unprompted, relevant assists over the first three weeks against 38 misses, with most misses traced to a thinly-bootstrapped project.

Temporality & Updating

Reasoning about time: validity intervals, bi-temporal modeling (event time vs ingestion time), contradiction/conflict resolution when facts update, and keeping a fact's timeline straight.

Open questions
  • When a user's fact changes, does the system supersede, version, or silently overwrite?
  • Do you track both when something was true and when you learned it (bi-temporal)?
  • How are contradictions detected — formally, or left to retrieval to disambiguate?
  1. Useful Memories Become Faulty When Continuously Updated by LLMs

    Zhang et al. · 2026 0 cites

    Synthesis

    Distinguishes episodic traces (raw trajectories) from consolidated abstractions (schema-like lessons). Shows that when an LLM repeatedly rewrites a textual memory bank, the consolidated memory degrades over time.

    Why it matters The strongest research caution against LLM-driven consolidation as your primary store. Direct argument to keep raw traces (see Substrate) as ground truth and treat distillation as lossy. Pairs with Slack's validation step.

  2. Temporal Knowledge-Graph Memory in a Partially Observable Environment

    Kim, François-Lavet, Cochez · 2026 0 cites

    Synthesis

    Agents in partially observable environments need persistent memory to integrate observations over time; KGs naturally represent evolving state. Introduces a benchmark where both world dynamics and the agent's memory are explicitly graph-shaped.

    Why it matters Grounds temporal-KG memory in a controllable evaluation setting — rare for this topic. Connects temporality to representation: time is modeled as graph evolution.

  3. Aurra's bi-temporal memory vs Mem0 — is Mem0 behind?

    u/Jst_Qrius · 2026 0 cites

    Synthesis

    A Mem0 user hits the classic wall: agents 'forget' the timeline of facts (amnesia when a user updates info). Tries Aurra's bi-temporal memory (tracks event time vs ingestion time) and reports it handles fact updates better.

    Why it matters Field evidence that fact-update/timeline handling is the concrete weakness pushing people off vector-only stores — and that bi-temporal modeling is the differentiator. Connects to [useful-memories-faulty] (why naive updates fail).

  4. Show HN: YourMemory — persistent memory layer with temporal reasoning

    SachitRafa · 2026 0 cites

    Synthesis

    Biologically-inspired decay plus temporal reasoning; a CLI to infer knowledge from stored memory with zero token/LLM cost, plus a dashboard that can double as an audit trail.

    Why it matters Combines temporality with forgetting (decay) and nods at governance (audit trail). Representative of the 'zero-LLM, local, dashboarded' indie memory wave alongside [superlocalmemory].

  5. Engram: A Bi-Temporal Memory Engine Where a Lean Retrieved Context Beats the Full History

    Wang, Liuyin · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Engram is an open-source long-term memory engine for LLM agents. Instead of replaying an entire conversation history into the prompt, it stores the past as a bi-temporal knowledge graph and retrieves a small, precisely-targeted slice at answer time. On a standard 500-question memory benchmark that lean ~9.6k-token slice answers more accurately than feeding the model the full ~79k-token history, turning memory from a cost optimization into an accuracy improvement.

    Motivation Stateless LLM agents forget across sessions, and the usual fix — concatenating the whole history — grows token cost and latency linearly and loses accuracy as distractors crowd the window ('lost in the middle'). Two gaps stay open: most memory systems are cheaper or faster but not more accurate than full-context, and memory benchmarks run on inconsistent harnesses where one system reports wildly different scores across sources. Engram targets both — beating full-context on accuracy, and shipping a neutral, re-runnable harness.

    Methodology A dual-process design. A System-1 hot write path appends lossless episodes with no LLM (sub-50ms) and enqueues them. A System-2 async path extracts atomic (subject,predicate,object) facts, builds a bi-temporal knowledge graph (valid time vs transaction time on every fact and edge), detects conflicts, and resolves them cheap-then-escalate: exact slot match, embedding similarity, and content subsumption handle the common case with no LLM call, invalidating (never deleting) a superseded fact and recording a supersedes chain and provenance, with only ambiguous cases escalated to an LLM adjudicator. The hybrid read path retrieves through four channels (dense, BM25, graph n-hop, recency/salience), fuses them with Reciprocal Rank Fusion, applies an 'as-of' temporal filter and an abstention gate, and assembles a deduplicated, provenance-tagged, token-budgeted context of facts plus raw chunks.

    Results On the full 500-question LongMemEval_S under the official category-specific judge, Engram's lean configuration scores 83.6% vs 73.2% for full-context (+10.4 points, McNemar exact p<10^-6) at ~8x fewer tokens (9.6k vs 79k), 0/500 errored. The gain is load-bearing on the read path being hybrid: facts alone lose recall, while facts plus retrieved chunks recover detail. Bi-temporal modeling pays off most on knowledge-update (87.5%) and temporal (81.1%) categories, while multi-session aggregation and preference remain headroom. The paper documents measurement-integrity pitfalls (truncation, home-grown judges, full-history leaks) and ships a neutral in-repo harness with the official judge baked in and raw per-question logs, every number reproducible by command.

  6. Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge

    Yadav, Neeraj · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MemStrata is a memory system for AI agents that keeps track of when facts become outdated. Instead of just retrieving whatever text looks most similar to a query, which fails when an old and a new fact look nearly identical, it uses a deterministic rule to detect when a new fact supersedes an old one and retires the stale version.

    Motivation Retrieval-augmented memory has no concept of time: when a fact changes (a renamed function, an updated config value, a new port number), both the old and new versions sit in the store with near-identical embeddings, and the agent can't tell which is current. The authors show this isn't a tuning problem: on a calibrated dataset, cosine similarity separates contradictions from duplicates at only 0.59 AUROC (near chance), because a value-flip edit sits textually closer to the original than a genuine rephrasing does.

    Methodology MemStrata's write path first tries a deterministic (subject, relation, object) triple match: if an incoming fact shares a key with a stored one but asserts a different value, the old fact is retired (not deleted) in a bi-temporal ledger and the new one is stored as current. Non-triple prose falls back to a similarity-plus-LLM-judge gate. The system is evaluated on six local, deterministic benchmarks (two static, four marker-free evolving: code mutation, config migration, dependency bumps, API evolution) with a 7B model on consumer hardware.

    Results MemStrata matches RAG on static recall (no cost) and reaches 0.95-1.00 accuracy on evolving-knowledge benchmarks where RAG reaches only 0.20-0.47. When forced to answer, plain RAG serves the superseded value 15-40% of the time; MemStrata drives this to ~0%. It also runs at ~2.1s retrieval latency versus ~16-18s for LLM-reranking/verification baselines, since no LLM sits on the read path.

  7. Ontology-Grounded Project Memory for Coding Agents

    Adam, James · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MOOSEDev gives coding agents ontology-grounded, typed project memory (decisions, constraints, rationales, lessons) in a small knowledge graph rather than flat notes or vector chunks, and shows it dramatically outperforms a production vector-memory tool on questions requiring completeness, negation, or supersession reasoning.

    Motivation Coding agents generate most new code in many projects today, but the reasoning behind why changes were made is easy to lose track of; notes files and vector-based retrieval-augmented memory help but don't know what kind of record something is (a decision vs. a lesson vs. an outdated, superseded note) or how records relate to each other, which is fundamentally an ontological modeling problem, not a retrieval problem.

    Methodology Two small OWL ontologies (9 and 11 classes) with SHACL-validated typed records, reasoned over by a neurosymbolic engine (MOOSE) that treats the LLM as a narrow, declared-point sensor rather than the primary reasoner, exposed to agents via MCP. Evaluated head-to-head against a production vector-memory tool (mem0) and a BM25 baseline on a neutral 835-record public corpus, using a pre-registered strict LLM judge, plus a live trial bootstrapped from the authors' own repository history.

    Results On tasks requiring set-completeness, negation (absence), and supersession-traversal reasoning, MOOSEDev scored 0.98-1.00 versus 6-27% for the vector-memory baseline — a categorical, not incremental, gap. On simple relevance retrieval the two systems tied. In a live trial, the system provided 71 unprompted, relevant assists over the first three weeks against 38 misses, with most misses traced to a thinly-bootstrapped project.

  8. Can Agent Memory Systems Track Evolving State?

    Fan, Xinyi · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A benchmark and method for a memory capability separate from recall: keeping track of which version of a fact is currently in force after it has been revised across sessions. The benchmark, StateMemBench, generates each scenario as a symbolic program of state operations so the correct answer is computed by replay and the specific way a lazy reader would get it wrong is known in advance. The method, StateMem, parses each turn into structured state units with typed dependency links, then handles supersession and staleness deterministically rather than with an LLM.

    Motivation Existing memory systems and benchmarks optimize recall of relevant facts, but as agents run longer, facts, constraints and decisions get revised, and an answer must reflect the current state rather than a superseded one. The authors call the failure state drift: the relevant fact is present in the assembled context, but the agent acts on a stale or incomplete version of it. This is distinct from dialogue state tracking, which prescribes a slot-value representation and evaluates it directly over cooperative dialogues that accumulate a goal monotonically; here state is whatever a system must maintain to answer correctly, evaluation is purely behavioral, and revisions are adversarial across sessions. Some concurrent work centers state, but the authors argue none cleanly isolates state tracking from the other errors it co-occurs with.

    Methodology The authors first define and label drift on existing benchmarks, assigning a failure to drift only after excluding retrieval, comprehension, schema and reasoning readings, dropping unassignable points, and cross-checking with two judge passes, a cross-family judge and two human annotators. They then build StateMemBench: each scenario is a symbolic event program of typed operations over ground, derived and declared state; the gold answer comes from deterministic replay; a family of executable lazy reader policies is run against the replay, and a scenario is admitted as a trap when policies disagree, with the disagreeing set forming its failure-mode signature (status, salience, sequence, compound, plus anti-trap controls). Programs are grounded in public data for surface vocabulary and rendered into multi-session dialogue by a strong LLM, then programmatically verified for fact placement and phrase leakage. Probes are closed-pool: an unseen pool of three to four options holds the gold answer, the targeted policy's drift answer and neutral distractors. StateMem itself runs a per-turn TurnEncoder producing state units (id, content, priority, source, deps), a deterministic update stage applying supersessions and marking dependents needs_recheck by dependency-graph traversal, and a single answer-time call over the assembled active state. A wrapper variant applies the same trace-then-resolve structure as a prompt-level transformation of any backend's answer call, evaluated against a length- and cost-matched generic-extraction control.

    Results Drift leads the confirmed failure distribution on several existing benchmarks (63.5% on MemoryArena-shopping, 44.4% on LongMemEval oracle where retrieval is perfect by construction, 10 of 16 on tau-squared-bench-Z) but is not universal, falling to 19.0% on MemoryArena-travel. On StateMemBench, long-context is not the strong baseline it is on recall tasks: the best long-context model reaches 0.277 and same-backbone long-context 0.149. StateMem reaches 0.363 on DeepSeek-V4-Flash, 1.8x the best memory system and 2.4x same-backbone long-context, and 0.233 on Qwen-3.5-9B, 1.6x the best memory system, both significant by paired McNemar at p < 0.001; GraphRAG at 0.224 is statistically level on Qwen. Ablations put supersession marking as the largest single component and show dependency propagation over-fires on Set B anti-traps by 12.5 points, so removing it leaves DeepSeek slightly better. Drift-rate analysis shows the memory layer barely changes outcomes on a weak answerer, where every arm drifts at 61 to 66%, and separates on the stronger backbone, where StateMem's drift rate falls 15 points and correct answers rise by 42 while long-context, Mem0 and BM25 move by 1 to 3 points. The wrapper improves every one of six backends on both benchmarks, adding 31.7 to 66.6 points on StateMemBench with 15.0 to 31.7 attributable to state structure over the matched control, significant in all twelve cells. State tracking does not cost recall: StateMem also leads memory systems on LongMemEval (0.656 on DeepSeek) and LoCoMo (0.592), with margins concentrated on temporal-reasoning and knowledge-update question types. The authors note StateMem mirrors the policy family behind the traps, so its StateMemBench margins should be read as an upper bound.

Forgetting & Lifecycle

Lifecycle management: decay and salience, interference-based forgetting, consolidation/'sleep' phases, and the under-appreciated question of when memories *should* be forgotten (including for safety).

Open questions
  • Does memory have an explicit lifecycle, or does it only grow until it rots?
  • Is forgetting heuristic decay, or principled (interference, reconsolidation, salience)?
  • When is forgetting a safety requirement, not just a cost optimization?
  1. Show HN: YourMemory — persistent memory layer with temporal reasoning

    SachitRafa · 2026 0 cites

    Synthesis

    Biologically-inspired decay plus temporal reasoning; a CLI to infer knowledge from stored memory with zero token/LLM cost, plus a dashboard that can double as an audit trail.

    Why it matters Combines temporality with forgetting (decay) and nods at governance (audit trail). Representative of the 'zero-LLM, local, dashboarded' indie memory wave alongside [superlocalmemory].

  2. Human-Inspired Memory Architecture for LLM Agents

    Kerestecioglu et al. · 2026 0 cites

    Synthesis

    Six cognitive mechanisms: sleep-phase consolidation, interference-based forgetting, engram maturation, reconsolidation upon retrieval, and entity knowledge graphs — a principled lifecycle rather than heuristic decay.

    Why it matters The most complete bio-grounded lifecycle proposal; turns 'forgetting' from a TTL into a set of mechanisms. Unvalidated at scale, like most of this cluster — weigh against eval scarcity ([persistbench]).

  3. SuperLocalMemory V3.3: 'The Living Brain' — bio-inspired forgetting, multi-channel retrieval, zero-LLM

    Bhardwaj · 2026 0 cites

    Synthesis

    Opens on the paradox: coding agents have vast parametric knowledge yet can't remember an hour ago. Critiques single-channel vector retrieval that needs cloud LLMs and implements no cognitive processes. Adds biologically-inspired forgetting, cognitive quantization, and multi-channel retrieval — all local/zero-LLM.

    Why it matters Bundles forgetting + multi-channel retrieval + local-first into one system; the indie counterpart to Cloudflare's managed multi-channel approach. Connects to [yourmemory] (zero-LLM, local) and [git-s3-memory] (local-first ethos).

  4. PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?

    Pulipaka et al. · 2026 0 cites

    Synthesis

    Persisting facts (e.g. 'user is vegetarian') aids personalization but also introduces safety risks that are largely overlooked. PersistBench measures when persistence becomes a liability and when memories *should* be forgotten.

    Why it matters Reframes forgetting as a safety requirement, not just hygiene — and is one of the only benchmarks targeting forgetting at all. Bridges Forgetting and Evaluation; pairs with the 'forgetting is unmeasured' gap.

  5. Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents

    Kang, Qingcan · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract OSL-MR is a framework for deciding which memories a long-horizon language agent should keep when its storage budget is smaller than everything it has seen. It poses the choice as an optimization problem that weighs the future value of evidence against the cost of losing it, and learns the policy from logged interactions using only signals that would actually be available when the agent runs.

    Motivation Long-horizon agents accumulate more observations, reasoning traces, and retrieved facts than fit in context, so retention is a resource-allocation problem. Prior systems score memories with heuristics, retrieval objectives, or learned compression, but treat each retention decision locally and never model its long-term consequences — and many depend on signals such as gold evidence or answer correctness that are only knowable after the fact and so would not exist at deployment.

    Methodology Retention is cast as a constrained multi-step stochastic optimization: at each step the agent selects a subset of memories under a hard size budget, and a per-step reward credits covered evidence while charging storage, miss penalty, reacquisition delay, and stale-information use. A strict online/offline split separates online-observable inputs (query, memory metadata, interaction history) from offline-available supervision (gold evidence, answer text, freshness) used only for training. OSL-MR pairs a Mixed-Score heuristic — a deployable cold-start baseline and inductive prior — with an evidence learner trained offline on gold-evidence membership labels, then frozen and deployed on online features alone.

    Results On LoCoMo and LongMemEval, OSL-MR beats recency, Generative-Agents-style scoring, behavior cloning, and the Mixed-Score heuristic across budgets, with the largest gains under tight budgets (LoCoMo budget 128: F1 0.302 and reward 305 vs 0.069/132 for Mixed-Score). It uses the budget more efficiently, running well below full occupancy while baselines saturate it, and reward rankings track precision and F1 exactly. Ablating the Mixed-Score prior cuts precision (0.529→0.421 on LongMemEval budget 256) while leaving recall nearly unchanged, showing the prior steers the learner away from low-utility memories.

  6. Memory Depth, Not Memory Access: Selective Parametric Consolidation for Long-Running Language Agents

    Han, Haoliang · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Long-running language agents accumulate more history than fits in working context, and the usual fix is retrieval — store past events outside the model and fetch a relevant subset at query time. This paper argues retrieval only answers what can be fetched (memory access) and not what should keep shaping behavior after the working context is unloaded (memory depth). It introduces the loop-drift protocol, a controlled stress test where the retrieval index stays intact but working context is cleared, so goal-conditioned behavior must persist through long-loop interference without the relevant text being reinserted. It evaluates EVAF, a surprise- and valence-gated LoRA consolidation mechanism that writes only behavior-relevant events into a small adapter. Across GPT-2, TinyLlama, and Mistral-7B, retrieval wins shallow factual recall while EVAF wins goal persistence and post-unload recovery with only 2–3 parametric writes per 200 events, and the paper shows selective consolidation factorizes into two separable controls — selection and actuation.

    Motivation Memory access and memory depth are different problems. A shallow memory is one the system can retrieve or attend to; a deep memory changes future behavior — it persists through interference, survives context unload, and affects choices without being reinserted as text. Retrieval is indispensable for fetched facts, but a long-running assistant also needs durable goals, preferences, and constraints that are not merely fetched facts. Existing long-memory benchmarks (LongMemEval, LoCoMo) emphasize conversational recall, temporal access, and knowledge updates, and do not isolate the post-unload setting where retrieval remains available but behavior must continue without the relevant text in context. The paper's claim is narrow and explicit: memory depth can be probed by post-unload goal-conditioned behavior, and consolidation factorizes into selection and actuation — it does not claim universal memory accuracy, SOTA retrieval, or complete deletion/update validity.

    Methodology Loop-drift protocol: synthetic per-user streams of 200 events (10 users/run) mixing stable goal/preference reminders, off-topic distractors, transient opposite requests, conflicts, sibling-user contamination, and scheduled factual notes; four probe layers — shallow episodic (recent fact), noisy episodic (old fact after same-key interference), parametric tendency (does a stable goal still shape behavior after long interference), and post-unload recovery (re-probe the goal immediately after a context unload, with the retrieval index intact but working context cleared). The RAG baseline stores all events in a durable embedding index (top-3 cosine) that context unload does not clear, so any EVAF goal-layer advantage is not a trivial 'RAG forgot' artifact. EVAF mechanism: per-event surprise (token negative log-likelihood) and valence (embedding similarity to the user's durable goal/preferences) combine into an admission gate; events above threshold enter a buffer, and when the buffer fills a LoRA adapter is updated on the buffer plus replay from prior consolidated events, with an L2 anchor as a drift guard. Model controls: GPT-2 and TinyLlama (four-seed means) plus Mistral-7B. Selection is isolated with a matched-random gate (same write count and online write dynamics, random admitted events). Actuation is isolated with fixed-inner controllers (fixed-1/2/3 inner LoRA steps using the same gate). A routed EVAF+RAG variant routes factual probes to retrieval and goal probes to EVAF. Public Memora event streams serve as an external boundary diagnostic for stale-memory invalidation, tested with McNemar's test.

    Results Depth flip: RAG is strongest on recent explicit facts (short-fact accuracy 0.956–0.973) and near-useless on goals; EVAF is near chance on short facts but much stronger on the goal layer — on GPT-2 EVAF reaches 0.904 goal / 0.900 post-unload vs RAG 0.398/0.394, and on TinyLlama 0.833/0.812 vs RAG 0.396/0.394 — at only 2.4–2.6 writes (L2 drift ~21–29) vs RAG's 0 writes. Writing everything is not enough: Naive-LoRA writes all 200 events at far higher drift (~67 TinyLlama, ~119 GPT-2) and still fails the goal layer; at 7B indiscriminate writing is actively harmful — Naive-LoRA goal persistence collapses to 0.333±0.047, below the 0.500 chance baseline. Selection is not sparsity: on GPT-2 EVAF beats a matched-random gate of equal write count on goal and post-unload in all four seeds (mean 0.790/0.763 vs 0.590/0.619); TinyLlama is weak/mixed, so the selection signal is not monotonic in model scale. Actuation is a separable, model-dependent factor: fixed-inner audits show smaller inner steps cut drift and improve goal/post (Mistral-7B five-step 0.354/0.306 -> Fixed-2 0.796/0.775 -> Fixed-1 0.919/0.938), but Fixed-1 contamination saturates at 1.000 on Mistral, so high actuation trades selectivity for goal strength. Asymmetric coupling: under a miscalibrated five-step actuation at 7B the matched-gate comparison reverses, yet EVAF still keeps lowest sibling contamination (0.787±0.041) — selection stays semantically active while its translation into goal behavior fails. Boundary: on Memora, EVAF improves forgetting-absence only 91/222 to 95/222 (p=0.57, not significant), so append-only selective consolidation does not solve stale-memory delete/update validity, which the paper leaves to validity-gating or reconsolidation.

  7. Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge

    Yadav, Neeraj · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MemStrata is a memory system for AI agents that keeps track of when facts become outdated. Instead of just retrieving whatever text looks most similar to a query, which fails when an old and a new fact look nearly identical, it uses a deterministic rule to detect when a new fact supersedes an old one and retires the stale version.

    Motivation Retrieval-augmented memory has no concept of time: when a fact changes (a renamed function, an updated config value, a new port number), both the old and new versions sit in the store with near-identical embeddings, and the agent can't tell which is current. The authors show this isn't a tuning problem: on a calibrated dataset, cosine similarity separates contradictions from duplicates at only 0.59 AUROC (near chance), because a value-flip edit sits textually closer to the original than a genuine rephrasing does.

    Methodology MemStrata's write path first tries a deterministic (subject, relation, object) triple match: if an incoming fact shares a key with a stored one but asserts a different value, the old fact is retired (not deleted) in a bi-temporal ledger and the new one is stored as current. Non-triple prose falls back to a similarity-plus-LLM-judge gate. The system is evaluated on six local, deterministic benchmarks (two static, four marker-free evolving: code mutation, config migration, dependency bumps, API evolution) with a 7B model on consumer hardware.

    Results MemStrata matches RAG on static recall (no cost) and reaches 0.95-1.00 accuracy on evolving-knowledge benchmarks where RAG reaches only 0.20-0.47. When forced to answer, plain RAG serves the superseded value 15-40% of the time; MemStrata drives this to ~0%. It also runs at ~2.1s retrieval latency versus ~16-18s for LLM-reranking/verification baselines, since no LLM sits on the read path.

  8. MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance

    Wang, Haoyu · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A memory framework for LLM agents that runs an LLM verifier over each candidate trajectory and then keeps the verifier's output on the record permanently, as reward, confidence, label and uncertainty fields. Those fields decide whether the memory is admitted, held provisional or rejected, and are read again later to rank retrieval, resolve conflicts, trigger summarization and archive stale records. Evaluated on Terminal-Bench 2.0, SWE-Bench Verified, WebArena and Mind2Web across four backbones.

    Motivation Agent memory is only useful if stored experience stays reliable over hundreds of interactions, and two failure modes break that. Unreliable admission: failed trajectories, accidental successes, invalid patches and misleading observations all look relevant to a retriever and get written, then mislead later decisions — a web agent that learns to edit the first row after filtering an admin table edits the wrong row once a table is sorted differently. Memory drift: a bank that grows across many tasks accumulates duplicate, conflicting, stale and overgeneralized records that remain retrievable long after their assumptions stop holding. Prior experience-memory systems (Synapse, AWM, ReasoningBank) make memory persistent but treat write-time feedback as a one-shot signal, so nothing continues to govern the record afterward. Verifier work, in turn, scores the current episode and stops there.

    Methodology Verification is decomposed into multiple criteria — completion, consistency, validity, generalizability — with reward estimated from score-token distributions and repeated views; repeated verification is triggered when uncertainty crosses a threshold. The resulting descriptor (reward, confidence, label, uncertainty) is attached before activation and routes the candidate to rejection, a provisional state, the active bank, or a failure-guard pool that stores failed experience as constraints rather than recipes. Each record additionally carries lifecycle state, quality, usage statistics and conflict links. Retrieval is a hybrid of BM25 and embedding cosine similarity over title, description and content, adjusted by the descriptor and by staleness and overgeneralization penalties, with positive memories and failure guards rendered as separate blocks in the injected prompt. Conflicts are located by structured-signature similarity above a threshold. Governance runs after every task: activation, rejection, merge and conflict checks immediately; summarization and archival when the active-memory budget is exceeded or a record goes stale. Baselines are No Memory, Synapse, AWM, ReasoningBank and a verifier-only filter, all matched on task order, step budget, retrieval budget, injected-memory budget, memory block template and decoding settings.

    Results Averaged over five seeds, MemGuard has the best success metric and the lowest average step count in all 16 backbone-benchmark settings. Against ReasoningBank, the strongest prior memory baseline evaluated, the largest gain is 7.9 success-rate points on WebArena and 5.6 step-success-rate points on Mind2Web, with 2.4 to 3.5 points on the terminal and software-engineering benchmarks. Backbones are Qwen-3.5-Flash, Qwen-3.5-Plus, Gemini-3-Flash and Gemini-3.1-Pro. The verifier-only control improves on ReasoningBank in most cells by admitting better candidates but loses to MemGuard in every benchmark-backbone cell, which the authors read as evidence that the gain comes from persisting the verifier signal through the lifecycle rather than from filtering once at the door.

Storage Substrate

Where bytes actually live: object storage for full transcripts (S3-style), managed session/filesystem state, self-hosted archives, and lightweight embedded stores (SQLite/FTS5) — the persistence layer beneath the semantic layer.

Open questions
  • Keep full transcripts forever (cheap object storage) and derive memory, or only keep derived memory?
  • Who owns durability across stop/resume — the runtime, or your store?
  • Does this need a vector DB at all, or does SQLite + full-text search cover it?
  1. Git and S3 as the memory layer for agents

    VijitDhingra1 · 2026 0 cites

    Synthesis

    Proposes plain Git + S3 as the durable memory substrate for agents — versioned, cheap, auditable object storage rather than a bespoke memory service.

    Why it matters The freshest signal in the 'deliberately boring substrate' movement: keep full history cheaply, derive memory downstream. Counterpoint to managed services; pairs with [chatdb] and [what-i-learned-coding-agent]. (Low vote count — early/thin.)

  2. Amazon Bedrock AgentCore Runtime: managed session storage for persistent filesystem state

    AWS · 2026 0 cites

    Synthesis

    Managed session storage (preview) persists an agent's filesystem state — code, installed packages, generated artifacts — across stop/resume cycles, which was previously lost.

    Why it matters A different memory layer than semantic memory: durable working state, not distilled knowledge. Important to keep distinct in a design — the runtime can own session durability while your store owns knowledge.

  3. Self-hosted archive for all AI conversations (hybrid keyword + semantic search)

    u/Sufficient_Guard9850 · 2026 0 cites

    Synthesis

    'ChatDB' — a self-hosted conversation archive across multiple AI apps, with a proper hybrid keyword + semantic search interface, deployable free on Cloudflare.

    Why it matters Demonstrates the storage+search split: keep transcripts, layer hybrid retrieval on top. The hybrid (lexical+semantic) retrieval choice connects to the Retrieval category.

  4. What I Learned Building a Memory System for My Coding Agent (SQLite, FTS5)

    u/Medium_Island_2795 · 2026 0 cites

    Synthesis

    Argues many agents don't need a vector DB: SQLite + FTS5 full-text search covers a lot, with far less operational weight.

    Why it matters The pragmatic floor of the substrate spectrum — strong reminder to not reach for pgvector/graph DBs by default. Echoes [kg-wrong-abstraction]'s 'simpler is fine' instinct.

  5. OpenAI rolls out ChatGPT Library to store your personal files

    TLDR AI · 2026 0 cites

    Synthesis

    Consumer-facing personal file/content store inside ChatGPT.

    Why it matters Market signal that 'remember my stuff' is becoming a default consumer expectation, shaping what agents are assumed to retain. Low technical depth.

  6. Show HN: MemoryBank — unify memory across agents, improve context rot (Rust)

    feelingsonice · 2026 0 cites

    Synthesis

    Local memory layer (Rust) motivated by memory being tool-locked (re-explaining things when switching tools/sessions) and by markdown-append memories that dump everything into context and rot.

    Why it matters Cross-tool, local substrate aimed squarely at context rot and portability — connects to Interop ([memorywire], [unified-memory-stack]) and Working-Memory/context-rot.

  7. Built a unified LLM memory system combining Memori + Mem0 + Supermemory

    u/0sparsh2 · 2025 0 cites

    Synthesis

    DIY unification: Memori's interceptor architecture (zero code changes), Mem0's research-validated retrieval/consolidation, and Supermemory's structure — composed into one stack.

    Why it matters Exactly the multi-store composition pattern that motivates a wire format. Real-world evidence builders are already gluing frameworks together by hand. Connects to [memorybank-rust] (cross-tool) and [memorywire] (the standard that would obviate the glue).

  8. OpenMemory by Mem0: 'local' but still needs an OpenAI key?

    u/Perplexed_86400 · 2026 0 cites

    Synthesis

    Notes that Mem0's OpenMemory MCP advertises local/private operation but still requires an OpenAI key (embeddings + gpt-4.1-nano) for extraction in the default Docker setup.

    Why it matters Operational gotcha for anyone assuming a 'local' memory framework is self-contained — the LLM dependency leaks in via extraction/embeddings. Relevant to substrate/privacy and to the zero-LLM designs ([superlocalmemory], [yourmemory]) reacting to exactly this.

  9. GitOfThoughts: Version-Controlled Reasoning and Agent Memory You Can Replay, Diff, and Merge

    Shekar, Pavan C · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract GitOfThoughts stores an LLM agent's reasoning tree as a git repository — every scored thought is a commit, scores are git notes, validation outcomes are tags, and retrieval is git log over the agent's own history — which makes reasoning replayable, auditable, diffable, and mergeable across agents. The paper then asks the harder question of whether memory, in any substrate, actually improves accuracy, and runs a pre-registered comparison of five substrates (none, markdown, vector, graph, git).

    Motivation Reasoning is the last unversioned software process: chains of thought expire with the context window, pruned search branches leave no record, and memory buffers cannot be diffed, merged, or audited. The authors argue this ephemerality is a structural blocker — it prevents reproducibility ('what did the agent think at step 17?'), audit (detecting train–test leakage or gold-answer memorization), memory transfer between agents, and incident review. Code, infrastructure, datasets, and experiments are all version-controlled; reasoning is the remaining outlier.

    Methodology A reasoning tree shares git's structural invariants, so the paper maps it one-to-one onto git primitives (node = commit, refinement = parent edge, score = note, outcome = tag, session vs. cross-session = branch, retrieval = git log --grep / -S). A pluggable MemoryBackend routes every read/write through one interface so the same agent can swap substrate with a one-line change. To isolate retrieval from write-path noise, all five backends ingest identical answer-free lessons, then solve held-out, domain-stratified problems read-only; benchmarks are GPQA-Diamond and MATH-500, scored with paired-bootstrap CIs, across two backbones and pre-registered replications, with a similarity sweep to locate when retrieval helps.

    Results H-substrate is supported: git delivers auditability, line-level diffs over reasoning text, deterministic replay by SHA, and mergeable memory at accuracy parity, costing ~15 ms/write and ~48 ms/read. H-memory is rejected: across two benchmarks, two backbones, and up to n=500, no substrate reliably improves accuracy on novel problems, and a +15 pp git trend at n=40 collapsed under its pre-registered replication. Memory pays only above a 'copyability threshold' — a near-duplicate retrieved case (cosine ≳ 0.8) lifts accuracy +12 to +13.5 pp, and a 4.5× larger model steepens that step to +22.5–28.5 pp but still extracts no transferable method; the only general accuracy lever is test-time sampling (self-consistency, +3.4 pp at n=500). The authors deliberately document a measurement bug, a retracted result, and a refuted hypothesis as the evaluation standard.

  10. The Log Is the Agent

    Nakajima, Yohei · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract The paper presents ActiveGraph, an open-source (Apache-2.0) agent runtime that inverts the usual framework design: instead of a conversation loop with logging bolted on, the append-only event log is the source of truth, the working graph is a deterministic projection of that log, and agent behavior is a population of reactions that fire on graph changes and emit new events. No component instructs another; coordination happens through the shared graph. This buys deterministic replay of any run, cheap forking at any event without re-executing the shared prefix, and total lineage from goal to each model call.

    Motivation Conventional LLM agent stacks grow by accretion: chat loop, then tools, then rules, then logging, with memory as a lossy similarity-queried store, so the log is a byproduct and questions like 'why is this fact in context', 'what did the agent believe before rule R changed', or 'what if step 42 had gone differently' are awkward or impossible. For long-running agentic work such as diligence, compliance, and research, where the reasoning matters as much as the answer, the authors argue the recoverable causal chain is the actual product.

    Methodology A systems and architecture contribution, explicitly not a task-performance benchmark. The runtime defines events (id, type, payload, actor, caused_by, timestamp), behaviors as subscriptions over event types plus Cypher-subset graph-shape patterns, and a determinism contract (no random, wall-clock, fresh UUIDs, or outside I/O in behavior bodies) policed dynamically by strict replay. Nondeterministic model calls are handled by recording responses in a content-addressed cache keyed on a hash of the full request, so replay and forks make no new model calls. A bundled investment-diligence pack runs fully offline against recorded fixtures on three companies, with no API key, completing in under thirty seconds.

    Results The reproducible quickstart run produced 671 events yielding 93 objects (3 companies, 24 questions, 9 documents, 25 claims, 25 evidence items, 1 contradiction, 3 risks, 3 memos) and 76 relations via 103 model calls and 48 tool calls, with zero orchestration code; re-running produced byte-identical logs. Forking a 200-step run at step 150 pays only for steps from 150 onward. Named costs and limits: replay time grows with log length (no checkpointing or compaction yet), schema evolution is a real operational burden, side-effecting tools still mutate the world on first execution, multi-writer ordering is unresolved, and self-improving agents are discussed only as an affordance, not demonstrated.

  11. SQLite is Enough. Lexical, Semantic, and Hybrid Search with scrydb

    Breuer, Timo · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract scrydb puts a whole small-to-medium search pipeline inside one SQLite file: documents, an FTS5 lexical index, and binary, int8, and float32 embeddings, with reranking and rank fusion on top. Evaluated on eight BEIR datasets on a laptop with no GPU, a cheap binary first stage reranked at higher precision matches an exhaustive high-precision scan at the top of the ranking for about a fifth of the latency, while RRF hybrid fusion helps on only one dataset of eight.

    Motivation A retrieval component usually arrives as a bundle: an index dump, a vector-store snapshot, configuration files, and a server process to keep them in sync. The author's premise is that for small-to-medium corpora none of that is necessary, and that packaging the documents, index, and embeddings as a single artifact makes an IR resource as easy to share, archive, and rerun as any other file, inheriting SQLite's preservation properties. The second motivation is efficiency as a first-class outcome rather than effectiveness at any computational cost.

    Methodology Lexical retrieval is delegated to FTS5 with BM25 ranking; semantic retrieval uses sqlite-vec, storing float32, int8, and binary vectors in vec0 virtual tables with one table per collection and precision, so all three precisions live side by side in the same file and any one can rerank another's candidates. Embeddings are binarized with a component-wise Heaviside quantizer and compared by Hamming distance via XOR plus popcount, giving a 32x storage reduction, with int8 scalar quantization as an intermediate. A single Index object owns the connection; search takes mode (lexical, semantic, hybrid), precision (binary, int8, float), and an optional rerank stage, and batch search exports TREC runs. Thirteen retrieval configurations were evaluated on eight BEIR datasets (ArguAna, FiQA, NFCorpus, Quora, SciDocs, SciFact, Touche, TREC-COVID), all embedded with Qwen3-Embedding-8B and compared against that model's published full-precision MTEB results. All retrieval ran on an Apple M2 MacBook Air with 24 GB of memory and no GPU; embeddings were computed once ahead of time through a remote API.

    Results Reranking improved nDCG@10 over raw BM25 on every dataset, and reranking by the cheap Hamming distance captured nearly all of that gain. Semantic retrieval over the binarized index beat BM25 on nDCG@10 on all eight datasets. The central result is that refining a coarse ranking to higher precision is equivalent, at the top of the ranking, to scanning the whole corpus at that precision: Hamming plus int8 cosine and exhaustive int8 gave identical P@10 and nDCG@10 to three decimals on all eight datasets, as did the float32 pairs, diverging only in AP below the 1000-document reranking depth. That makes Hamming plus int8 cosine dominant over exhaustive int8, matching top-10 effectiveness at 164.5 ms against 822.5 ms. Four of eight datasets met or exceeded the MTEB baseline. The remaining gaps were not caused by compression: the author's own exhaustive full-precision run reproduced the TREC-COVID deficit (0.885 against a reported 0.950), placing it upstream in the embedding pipeline. RRF was best on exactly one dataset, Touche, and trailed the better of its two inputs on the other seven, so the author concludes it is worthwhile only where lexical and semantic retrieval are of comparable strength and make complementary errors. On latency, Hamming was the fastest of thirteen configurations on seven of eight datasets and scaled with corpus size, while BM25 scaled with query length.

Multi-Agent & Shared Memory

Memory shared across agents and sessions: shared/team memory profiles, distributed multi-agent memory, and keeping long-running agent swarms coherent without dumping chat logs.

Open questions
  • Is memory per-agent, per-user, or a shared team profile multiple agents read/write?
  • How do you keep N long-running agents coherent without re-feeding raw history?
  • What's the cost/accuracy curve of long-term memory across cloud+edge agents?
  1. Agents that remember: introducing Agent Memory

    Tyson Trautmann · 2026 0 cites

    Synthesis

    Shared memory profiles let multiple agents access common knowledge.

    Why it matters Productizes team/shared memory; pair with Slack's distilled-truth lesson for the coherence side.

  2. How Slack Manages Context in Long-running Multi-agent Systems

    Sergio De Simone · 2026 0 cites

    Synthesis

    Targets coherence across long-running multi-agent systems specifically.

    Why it matters Pairs with Cloudflare shared profiles as the two main public takes on multi-agent memory.

  3. Show HN: MemoryBank — unify memory across agents, improve context rot (Rust)

    feelingsonice · 2026 0 cites

    Synthesis

    Local memory layer (Rust) motivated by memory being tool-locked (re-explaining things when switching tools/sessions) and by markdown-append memories that dump everything into context and rot.

    Why it matters Cross-tool, local substrate aimed squarely at context rot and portability — connects to Interop ([memorywire], [unified-memory-stack]) and Working-Memory/context-rot.

  4. Cost and Accuracy of Long-Term Memory in Distributed Multi-Agent Systems

    Wolff & Bennati · 2026 0 cites

    Synthesis

    A testbed for long-term memory in distributed multi-agent systems (cloud+edge), directly comparing Mem0 (vector-based) vs Graphiti/Zep (graph-based) on system-level cost and accuracy — not just the tokens/latency that framework-published evals report.

    Why it matters The single most decision-useful head-to-head for splitting work between a vector store and a graph store in a multi-agent setting. Bridges Multi-Agent and Evaluation; complements [personalai] (KG-internal comparison).

  5. Building memory systems at production scale (100k+ users): lessons from 10+ implementations

    u/singh_taranjeet · 2026 0 cites

    Synthesis

    Lessons from ~10 production deployments (healthcare, fintech, consumer SaaS, dev tooling) on what actually matters vs the 'just add a vector DB' tutorials.

    Why it matters The hard-won operational counterweight to research/product claims. Read with [what-using-production] as the practitioner reality layer of the Evaluation category.

  6. Decentralized Multi-Agent Systems with Shared Context (DeLM)

    Mao, Yuzhen · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract DeLM (Decentralized Language Models) is a multi-agent framework that drops the central controller. Instead of a main agent assigning subtasks, waiting, and merging results, parallel agents asynchronously claim tasks from a shared queue and read/write a shared 'verified context' of accumulated progress. It targets two settings — parallel exploration in software engineering and concurrent evidence processing in long-context QA — and improves accuracy while roughly halving cost.

    Motivation Most multi-agent systems use centralized scatter-gather orchestration, which parallelizes sub-agent execution but not the coordination around it. Every finding must return to the main agent to be merged and rebroadcast, so progress-sharing becomes a serialized bottleneck as agents grow, and the controller can dilute, omit, or distort details. In long-context reasoning the main agent must pre-assign evidence clusters before knowing what is relevant, triggering extra delegation rounds. DeLM removes the controller as the coordination chokepoint.

    Methodology Coordination is state-based, not prompt-routed. Two global structures: a shared context C of compact verified gists and a task queue T of pending subtasks. The pipeline initializes the queue from the input, executes ready subtasks in parallel, then compresses-verifies-admits each result into the shared context, generates more subtasks when the context is insufficient, and finalizes once none remain. The shared context is compact, global, and 'unfoldable' — agents read coarse gists by default and expand to detailed summaries or raw evidence only when needed. Admission-time verification checks each update against its underlying evidence and reasoning trajectory before it enters shared state, rejecting or regenerating unsupported updates so errors cannot propagate as reusable problem state.

    Results On SWE-bench Verified, DeLM is strongest across test-time-scaling metrics, reaching 77.4% pass@4 at ~$0.12/task — roughly half the baselines' cost — with trace-level examples showing agents reuse each other's discoveries through the compact shared context. On LongBench-v2 Multi-Doc QA it leads four frontier model families by up to 5.7 points, with both admission-time verification and hierarchical summarization contributing. On OOLONG, vanilla DeLM underperforms RLM (which needs exact row-level aggregation via code execution), but RLM combined with DeLM yields the best accuracy and lowest cost, showing DeLM works as a coordination layer for programmatic reasoning too.

  7. ESAA-Conversational: An Event-Sourced Memory Layer for Continuity, Handoff, and Curation Across Heterogeneous LLM Coding Agents

    Brito dos Santos Filho, Elzo · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract ESAA-Conversational is a shared memory layer that lets several LLM coding agents — Codex, Claude Code, Grok — hand work off to one another without copying the conversation by hand. It watches each agent's visible turns through hooks or watchers, writes them into one append-only log (activity.jsonl), and deterministically projects compact files — a handoff contract, current state, recorded decisions, and an open-task list — that the next agent reads to pick up where the last left off.

    Motivation Developers increasingly switch among multiple coding agents as context windows fill or a different tool suits a subtask, but each agent keeps its conversation in a private, vendor-specific log. The result is 'conversational state drift': goals, rejected alternatives, decisions already made, and open tasks established with one agent are not reliably available to the next. The usual fix — copy-pasting context — is manual, lossy, expensive in tokens, and conflates capturing evidence with interpreting it.

    Methodology The system applies event sourcing and CQRS: visible turns are captured mechanically, with no LLM inference, into an append-only activity.jsonl that is the single source of truth, while state.md, handoff.md, decisions.md, and tasks.json are reconstructible read models that are never hand-edited. 'Inverted ingestion' means the runtime reads native agent logs, hooks, or watchers and normalizes them into conversation_turn events rather than requiring agents to share a protocol. A strict boundary separates mechanical capture (turns are evidence) from curation (durable decisions and tasks entered through explicit decide/task commands). A paginated context command serves filtered windows (--last, --around, --before, --topic) so a cold agent reads a slice, not the whole log; workspace_root isolates projects and a lockfile serializes writes. The v1.1.0 release is a local PowerShell CLI.

    Results A self-referential case study recorded 570 events (562 conversation turns) in a single workspace on 21 June 2026, distributed across Codex (304), Claude (79), and Grok (67). The three heterogeneous agents co-designed and reviewed the tool through the shared log alone, with no direct agent-to-agent channel — e.g., Codex was given a focused view of recent Grok iterations via `context --agent grok --last 20`, and one concrete defect (incomplete filtering of legacy events under context --topic) was found, fixed, and closed as a task. The public release ships 51 tests in its main battery. Reported limitations: the implementation is Windows/PowerShell-only, sync depends on third-party hook surfaces outside the authors' control, retrieval is purely textual with no embeddings, and the system offers operational but not forensic auditability (no hash chains or signatures); validation covers one workspace and three agents.

Working Memory & Context

The working-memory boundary: what competes for the context window, context rot over long horizons, and selective/dependency-aware construction of what the model actually sees each step.

Open questions
  • What are the components competing for the window, and what's each one's token budget?
  • How do you fight context rot as histories grow — compress, retrieve, or restructure?
  • Should context be rebuilt selectively per step rather than appended/slid?
  1. To Retrieve or To Think? An Agentic Approach for Context Evolution

    Chen et al. · 2026 0 cites

    Synthesis

    RAG-at-every-step is a rigid brute-force strategy that wastes compute and can degrade performance. Proposes an agent that decides when to retrieve vs reason from current context.

    Why it matters Reframes retrieval as a policy decision, not a reflex — the cost/quality lever most retrieval designs ignore. Bridges Retrieval and Working-Memory categories.

  2. What fills the context window (the 7 competing components)

    u/Vuducdung28 · 2026 0 cites

    Synthesis

    Production-grounded (LangGraph) deep dive on the seven things competing for the window — system prompts, user messages, conversation state, long-term memory, RAG, tool definitions, output schemas — with token ranges for each.

    Why it matters Sets the budget frame: long-term memory is just one of seven consumers, so memory design is inseparable from context engineering. The practical entry point to this category.

  3. ContextWeaver: Selective and Dependency-Structured Memory Construction

    Wu et al. · 2026 0 cites

    Synthesis

    Sliding-window and prompt-compression context management omit earlier structured info later steps rely on; retrieval-based memory surfaces relevant content but overlooks dependencies. Builds memory selectively with explicit dependency structure.

    Why it matters Targets the failure where compression drops the one earlier fact a later step needs — a dependency-aware answer to context rot. Connects consolidation (what to keep) with context (what to show).

  4. ARC: Active and Reflection-driven Context Management for Long-Horizon Agents

    Yao et al. · 2026 0 cites

    Synthesis

    Names 'context rot' — performance degradation as interaction histories grow — as a failure to maintain coherent, task-relevant internal state. Proposes active + reflection-driven context management for deep-search/long-horizon agents.

    Why it matters Canonical reference for the context-rot problem this category orbits. Pairs with [contextweaver] (structural fix) and [retrieve-or-think] (retrieve-vs-reason policy).

  5. Agentic Context Engineering: Evolving Contexts for Self-Improving Models

    Zhang et al. · 2026 0 cites

    Synthesis

    Context adaptation (modifying inputs vs updating weights) suffers two failure modes: brevity bias (concise summaries drop domain insight) and context collapse (iterative rewriting erodes detail).

    Why it matters Names the exact degradation mode behind LLM-rewritten memory — the context-side mirror of [useful-memories-faulty]. Strong argument against over-summarizing either context or stored memory.

  6. TokenPilot: Cache-Efficient Context Management for LLM Agents

    Xu, Buqiang · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract TokenPilot is a context-management framework for LLM agents in long-horizon sessions that reconciles cutting token count with preserving backend KV prompt-cache continuity. The authors observe that prior pruning, compaction, and memory-eviction methods mutate the prompt layout, which shatters prefix continuity and triggers cache-miss pre-fill penalties that override the financial savings from text reduction. TokenPilot operates at two granularities: a global Ingestion-Aware Compaction harness that stabilizes the prompt prefix and strips open-world tool-output noise at the ingestion gate, plus a local Lifecycle-Aware Eviction that defers purging context segments until their residual task utility expires, on a conservative batch-turn schedule.

    Motivation Continuous multi-turn agent interactions accumulate verbose execution traces that inflate sequence length and per-turn inference cost. Existing content-reduction methods reduce tokens but constantly mutate input boundaries, causing prefix mismatches and KV cache invalidation; the resulting pre-fill penalties can exceed the text-reduction savings. The core insight is a trade-off between text sparsity and prompt-cache continuity that must be reconciled, by safeguarding physical prefix continuity during observation ingestion and deferring structural eviction until residual utility expires.

    Methodology Messages are partitioned into internal intentional messages (high utility) and open-world environmental feedback (lower density unless content-hash access frequency exceeds a threshold). A canonicalization operator enforces a byte-identical prefix across turns; environmental messages are reduced to structural previews with full payloads stored in a content-hash-indexed artifact registry and recallable via a recovery tool. Lifecycle-Aware Eviction tracks segments through active/completed/evictable states using a zero-shot estimator run every B=3 turns over a compressed history view; only evictable segments (zero residual utility) are purged in a single pass. Evaluation is on PinchBench and Claw-Eval in isolated and continuous modes with GPT-5.4-mini as backbone, against compression baselines (LLMLingua-2, SelectiveContext, Keep-Last-N) and paging/summarization baselines (MemoBrain, MemOS, others); cache hit/miss token counts are read directly from provider API metadata.

    Results TokenPilot achieves the lowest inference cost while maintaining competitive accuracy: isolated mode $3.22 on PinchBench and $2.27 on Claw-Eval (61% and 56% reductions vs Vanilla); continuous mode score 81.3 at $2.79 on PinchBench and $10.58 vs Vanilla's $81.52 on Claw-Eval (61% and 87% reductions). Ablation on continuous PinchBench: Ingestion-Aware Compaction cuts cost $7.24 -> $4.22 (cache-miss tokens 5.943M -> 1.589M); adding Lifecycle-Aware Eviction reaches $2.79 (cache-read tokens 26.716M -> 8.551M, a 65% drop). Prefix stabilization raises macro cache hit rate from 38.7% to 79.2% on PinchBench and 67.2% to 83.1% on Claw-Eval. Removing the recovery tool drops accuracy 80.9 -> 77.1; B=1 eviction is too aggressive, B=infinity bloats context, B=3 is the chosen balance.

  7. How Agent Skills Fail under Long Contexts: A White-Box Study in Code Auditing

    Xue, Yue · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Agent Skills package procedural instructions for coding agents, but a loaded skill's requirements do not all stay active across a long tool-using trajectory. In a white-box code-audit workflow with 24 fixed artifact checks, the same task passes 8/10 runs in a clean 11K-character context and only 3/10 in 299K-character contexts - whether the added material is relevant production text or irrelevant archive. Requirement coverage stays above 92% even in failing runs: agents lose one or two obligations, not the task. An external checklist restating all 24 checks passes 10/10 runs where a generic self-check passes 5/10.

    Motivation Long-context research shows a model's usable context is smaller and less uniform than its advertised window, and trajectory studies show failures begin early and hide behind plausible final states. The missing piece is a focused test: hold a skill-based task and its checks fixed, vary only the surrounding context, and identify exactly which stated requirement fails - and whether a simple external check prevents the failure.

    Methodology Mandatory instructions from a production-derived audit workflow (an industrial code-audit scanner's Stage 4CD task) are translated into 24 deterministic, equally weighted checks, each validated against positive and negative fixtures before any model runs. Every attempt starts from a sanitized, answer-free workspace; conditions add clean (10,991 chars), relevant-long (299,140 chars of same-workflow production material), or irrelevant-long (equal-length unrelated archive) context. Ten valid runs per condition with Codex + gpt-5.4-mini; failed runs are classified by first visible failure location (lost requirement, editing drift, failed checking, non-agent failure). A mitigation arm compares a generic self-validation prompt against a detailed checklist differing by only 649 characters; scaffold probes compare direct prompting, a retrospectively selected evidence bundle, and tool-using agent shells.

    Results Clean passes 8/10; both long conditions pass 3/10 - a 50-point observed drop that remains trend-level (two-sided Fisher p=0.0698) - with no ordering between relevant and irrelevant context (p=1.0). Check coverage stays at 92-94% in the long conditions: typical failures omit one required array or exceed a field cardinality, though one run abandoned two of four audit tasks entirely. The detailed checklist passes 10/10 versus 5/10 for the generic self-check (p=0.0325); the generic validator's spot-checks omit the same field generation omitted. A second task passes every clean and long run, so the evidence supports no universal context-length threshold. Infrastructure is a separate failure source: only 29 of 57 extension attempts produced scoreable output, with first-token disconnects clustered in the irrelevant-long condition. For gpt-5.5, a small evidence bundle passes 3/3 without tools while the full 299K direct prompt passes 1/3, suggesting selective retrieval - not tools per se - explains coding-agent benefit.

  8. Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories

    Khatri, Prakhar · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract AGENTS.md and CLAUDE.md are standard practice, and the evidence that they work is contradictory. This study runs the controlled version: three ways of delivering repository context - none at all, the full file in the system prompt every turn, or topic-organized wiki files the agent reads on demand - across two frontier agents from different providers, on 17 tasks mined from merged pull requests in three real Python repositories, 288 evaluated runs graded by the pull request's own hidden tests. Correctness does not move on either agent, and the paper is careful about what that does and does not establish. A failure-mode triage explains why: the tasks that fail, fail on implementation skill - designing the feature, picking the pattern, wiring it exactly right - not on repository knowledge a context file could have supplied. A manipulation probe confirms the real AGENTS.md never turns a near-miss into a pass. The one reliable effect is a cache footprint difference that follows from delivery mechanics rather than from better use of the context.

    Motivation Platforms autoload these files into every session and practitioners invest real effort authoring conventions, architectural constraints and workflow guidance, expecting better code out the other side. The published evidence splits: one 2026 study reports efficiency improvements for Codex-family agents, another finds no significant effect on task completion for Claude-family agents. The two differ in agent, evaluation method and experimental control at once, so they cannot be reconciled without a study that varies injection strategy under controlled conditions across both agent families. Neither prior study isolated injection strategy as an independent variable. There is also a mechanism question worth separating from the outcome: work on long-context attention shows models use information unevenly across a prompt and often underuse material placed mid-context, which bears directly on whether an always-on file in the system prompt is attended to at all.

    Methodology Repositories were screened from roughly 40 candidates on four criteria: exactly one root AGENTS.md with no competing instruction stack, file quality rated Good or Excellent on a structured rubric, feasible pilot setup, and Python-only to avoid confounding with build-system differences. Three survived to the study - pdm (477-word file), firebase-admin-python (1236 words, rated Excellent) and opshin (248 words). Tasks come from merged pull requests: the PR description is the prompt, the base commit the starting state, the PR's own test files the gold oracle, applied only after the agent finishes in SWE-bench Tier-C fashion. A Codex screening sweep over 84 candidate tasks located the borderline band, adding four tasks to avoid a floor/ceiling design. Every run executes in an egress-locked pod with GitHub DNS blackholed, credentials stripped, push and commit denied via PATH shims, and future commit history pruned so the gold solution cannot be read from git log. The unit of analysis is the task, with three repeats averaged per cell; analysis uses omnibus permutation tests, paired Wilcoxon with Holm-Bonferroni across 12 efficiency tests, TOST equivalence on a task-clustered bootstrap, and Monte Carlo power simulation.

    Results Claude pass rates are 53.3 / 55.6 / 55.6% for NONE / ALWAYS ON / SELECTIVE (omnibus p=1.000, all pairwise differences 2.3pp or less); Codex 58.8 / 56.9 / 52.9% (p=0.66, largest difference 5.9pp). The authors flag the omnibus test as low-power given floor/ceiling structure and rest the null on the borderline subset instead, where NONE reaches 58% against 42% for both context arms. TOST bounds every pairwise difference below 10pp for Claude and 15pp for Codex, described as descriptive rather than powered against a minimum detectable effect above 30pp at 15-17 clusters. On efficiency, only one contrast survives correction: Claude SELECTIVE creates fewer cache-creation tokens than NONE on 11 of 11 tasks (p=0.001, Holm 0.012). Task difficulty is agent-specific at Spearman rho=0.75, and the tasks whose strategy markers separate differ by agent. Code, data and analysis are released.

  9. Paritok-4B: Intent-Conditioned Context Compression for Coding Agents

    Shi, Jiayu · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A 4B LoRA compressor trained specifically to shrink coding-agent context before it reaches a frontier model. Two commitments define it: it is extractive, selecting spans rather than rewriting them so identifiers, paths and error strings survive verbatim, and it is intent-conditioned, receiving the agent's current task and keeping what that task names. It operates one typed segment at a time behind a gateway, is distilled from a gpt-4.1-mini teacher over 67,074 real OpenHands trajectories, and ships as a 264 MB adapter that self-hosts on a single 24 GB GPU under Apache 2.0.

    Motivation An autonomous coding agent re-sends its accumulated context, file reads, command output and history, on every turn, so input rather than output dominates the token bill. Compressing that context with a small model before it reaches the expensive one is an obvious lever, and prompt-compression research shows heavy compression can preserve task quality. But general prose compressors face constraints they were never built for here. An agent edits by exact string match, so a paraphrased signature or renamed variable breaks the downstream edit. A segment's value is not intrinsic but depends on what the agent is currently doing, and the function it is about to modify has to survive even when it looks unremarkable. Agent context is also strongly heterogeneous, since a cat -n file read, a pytest traceback, an ls listing and a reasoning block have nothing in common, and one uniform ratio is wrong for all four.

    Methodology A gateway performs segmentation, kind classification and level labelling; the model is then invoked once per compressible segment with exactly two inputs, the agent's current task and the segment tagged with kind and level, and returns that segment's compressed form. Kinds cover file reads, bash commands, log output, tool results, file operations, directory listings, assistant thinking and meta actions, alongside protected system and user messages; four levels from protected through stale carry intended per-level budgets that live only as a static table in the system prompt, with no numeric budget passed to the model. Emitting an empty body is a first-class action for unrelated helpers, build noise and superseded re-reads. Output mirrors input inside per-segment markers reusing the input segment id, so every compressed span maps back to one original and untouched bytes can be recovered on demand. Retained code lines, identifiers, paths, line numbers, imports, error classes, exact error text, shell commands and edit payloads are copied verbatim; only a closed, enumerated set of structural markers may replace deleted content, and a body of markers alone is forbidden. Training distils a gpt-4.1-mini teacher over 67,074 OpenHands trajectories through a five-stage funnel into a 45K-segment pool and 40,606 validated examples, fine-tuning a Qwen3-4B backbone with LoRA, chosen over 3B and 7B code-pretrained alternatives under a matched protocol, with the deployed checkpoint selected by an out-of-distribution sweep rather than by training loss.

    Results The extractiveness claim is audited rather than asserted: 96.0% of emitted identifiers, paths and numbers already appear in the input over the training corpus, 98.3% outside the one kind rewritten by design, and 96.2% on held-out SWE-bench Lite output across 212,506 emitted tokens. Intent conditioning is measured to act chiefly inside a retained segment, selecting which lines survive rather than changing how much is retained, with retained lines +0.067 more intent-relevant than removed ones (paired 95% CI [+0.056, +0.078]). On an out-of-distribution holdout the released checkpoint emits well-formed output on 100% of segments and compresses to 23.7% of input tokens, with must-keep identifier retention not below the teacher's at comparable budget. End-to-end over all 300 SWE-bench Lite instances it compresses to 25.7% of original size, 2.0 times harder than a gpt-4.1-mini compressor at 50.2% and 2.4 times harder than gpt-5 at 61.9%, retaining 86.5% of uncompressed single-shot solve quality; in the in-distribution cat -n regime it compresses to 27.8% and retains 89.3%, where 30 instances are solved only uncompressed and 17 only compressed, an exact McNemar p=0.079. The authors are explicit that this harness measures comprehension under compression, not end-to-end agent cost, and that the four-level budget design collapsed into two effective bands rather than four. On economics, a 264 MB adapter on one 24 GB GPU carries no per-token compressor fee, whereas gpt-5 used as a compressor is net-negative at list prices, costing more than the downstream tokens it saves.

  10. ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance

    Jin, Qirui · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Real maintenance is a stream of related fixes on a codebase you yourself just changed. Every SWE benchmark instead resets the repository, hands over one self-contained issue, and grades it in isolation. ChainSWE keeps the repository, orders real issues chronologically within a project, and asks whether an agent's earlier patches leave a usable starting point for the later ones. Performance drops by up to 70% as chains get longer, and roughly half the downstream failures are caused by the agent's own accumulated edits rather than by the bug in front of it.

    Motivation Agent performance has improved rapidly across SWE-bench, SWE-bench Live, SWE-rebench, SWE-Gym and SWE-bench Pro, but all of them share one protocol: a fresh container reset to a clean base commit, one problem statement, a fresh conversation, and grading against pre-written tests from that issue. This measures single-shot issue resolution and deliberately removes the cross-task dependencies that characterize real maintenance. It says nothing about whether an agent avoids unnecessary edits, resolves latent multi-file dependencies, or manages context across an evolving sequence of related fixes. In practice an engineer must leave the codebase both correct for the current issue and usable as the starting point for future work, reading surrounding context and adapting to the repository state they inherit. Adjacent sequential benchmarks (AgentBench, AgentBoard, WebArena, OSWorld) and long-conversation memory benchmarks are sequential but reset between tasks or track dialogue state rather than persistent changes to an external artifact.

    Methodology Instances are pooled from six repository-level SWE benchmarks, all Python with pre-built Docker images for reproducible test execution. Per repository, instances are sorted by commit date and grouped by a sliding window into chains when they overlap in the code they touch, then validated by containerized re-execution: replay the accumulated gold patches, run the bug's test commands, and require that all FAIL_TO_PASS tests pass with no PASS_TO_PASS regression. A chain is accepted only if every filter stage clears. Length-2 chains are dropped during data selection. At evaluation step k the agent receives only the k-th issue statement and works on the repository state produced by the previous steps, with the Docker image reset only to the base commit of the first bug in the chain. Scoring is both per-bug, by chain position, and full-chain, where a chain counts as successful only if every bug in it is resolved, alongside average API cost per task and per chain. Seven state-of-the-art models are evaluated on a fixed SWE-EDIT scaffold under three context-management configurations. Failures at a downstream position that are attributable to accumulated agent-generated state rather than the intrinsic difficulty of the current bug are labeled chain errors.

    Results The dataset is 100 chains, 304 instances, 54 repositories, with chain lengths from 3 to 5 after selection. Performance falls by up to 70% relative to the single-issue setting, and the decline is steepest at the deepest chain positions. Chain errors account for 318 of the 663 downstream BASELINE failures at positions 2 and 3, 48% overall, growing with depth from 43% at position 2 to 52% at position 3. On context management, conversation memory across bugs yields only a marginal gain and only for GPT-5.5, while summarization and sub-agent delegation both consistently degrade performance. The two named failure mechanisms are illustrated by concrete chains: amaranth for overshoot, where three unrelated backend files rewritten during an earlier fix cause all eight downstream tests to fail, and MONAI for undershoot, where omitting three supporting refactors from a five-file gold patch breaks a downstream test despite a correct modification to the target file. The authors read the results as pointing at dependency tracking, long-horizon reasoning and repository-state management in agent harnesses rather than at further gains in isolated issue resolution.

  11. Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement

    Yan, Haoyang · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Autonomous software development means handing an agent a high-level requirement and getting back a working system with no human in the loop, which turns development into a trajectory far longer than a repository issue. Over that horizon agents lose earlier decisions, make local fixes that break constraints elsewhere, and cycle between inspection and repair. HoH wraps an existing coding-agent harness in repeated planning-coding-testing loops, each producing a small verifiable increment, with artifacts kept on disk and surfaced through an index rather than held in context. It beats the standalone harnesses on three benchmarks and, over more than 70 iterations across several days, builds a playable first-person-shooter game.

    Motivation Most coding agents still operate human-in-the-loop: developers define tasks, guide intermediate decisions, review changes and intervene on failure. Building a system from scratch without that support requires translating requirements into executable plans, coordinating interdependent tasks, designing and integrating components, and continuously testing and debugging an evolving system, so trajectories grow long by construction. As they grow, agents lose track of earlier requirements and design decisions, introduce local fixes that violate constraints elsewhere, accumulate failed attempts and suboptimal decisions, and let new test evidence invalidate earlier assumptions. Long trajectories also invite repetitive inspection-and-repair cycles, redundant verification of finished components, and premature declarations of completion. The problem is therefore not longer execution but sustaining coherent progress over time. Existing harnesses organize development within a bounded episode and offer limited support for preserving decisions, verified functionality and evaluation evidence across revisions.

    Methodology HoH applies a fixed harness-model configuration to a specification and organizes execution into iterations. A planner synthesizes the high-level requirements and evidence from previous iterations into a development plan; each plan must both address outstanding problems and deliver one small concrete new capability, following iterative and incremental development, which keeps the loop from collapsing into repetitive local repair and keeps progress verifiable. A developer implements the plan with focused testing embedded throughout implementation for immediate local feedback. A tester then independently evaluates the system against both the overall requirements and the plan, using complementary white-box and black-box tests across functional correctness, completeness, usability and visual or audio quality, and returns a structured report as evidence for the next iteration. HoH specifies the artifacts and evidence each role must deliver but does not prescribe a workflow for producing them, enforcing only a schema with retry on violation. Continuity comes from progressive disclosure over a file-system artifact store with a categorized index, role-organized tools and Markdown skills, and versioned project state at role and iteration granularity. Evaluation runs in two settings: three controlled benchmarks under their original specifications with the loop alone, and open-ended multi-day game development where role-specific tools and skills for engine interaction, asset acquisition and generation, reference retrieval, testing and project-state management are added, with every stage committed to a public repository so the trajectory is traceable.

    Results Across all three benchmarks and all three harness-model configurations, HoH outperforms the corresponding standalone harness. After three iterations it yields absolute gains of 16.62 to 22.08 points on GameCraft-Bench, 19 to 29 points on FrontierSWE, and 6.09 to 16.85 points on ProgramBench, an average relative gain of 52.25% with a maximum of 82.86%. Improvement continues past three iterations: on FrontierSWE with Codex and GPT-5.5 at high reasoning effort, ten iterations take the score from 22% to 72.67%. In the open-ended setting, more than 70 iterations over roughly six days turn high-level product requirements into a human-playable first-person-shooter game with a coherent storyline, implemented combat, weapon and enemy-interaction systems, player guidance, HUD and menu systems, cinematic animation, and polished visual and audio presentation, with the open-issue count trending down across the run as new issues are opened and closed each iteration.

Evaluation & Cost

How we know any of this works: benchmarks beyond factual recall, separating retrieval-correctness from answer-correctness, cost/latency accounting, and production reality checks.

Open questions
  • Does the benchmark test retrieval quality, or just whether the final answer was right?
  • Are you measuring beyond surface factual recall (implicit user state, goals, values)?
  • What does this actually cost at 100k+ users, not in a demo?
  1. Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrieval

    Jeffrey Flynt · 2026 0 cites

    Synthesis

    Observes that returning the entire belief store yields recall 1.0 and passes answer-quality evals — so answer-correctness can't validate a retrieval system. Introduces a precision-aware retrieval benchmark over a structured belief state.

    Why it matters The cleanest statement of the retrieval-vs-answer-correctness gap (the 'unit test vs integration test' framing). A north star for evaluating any multi-channel retriever. Bridges Retrieval and Evaluation.

  2. PersonalAI: Systematic Comparison of KG Storage and Retrieval for Personalized LLM Agents

    Menschikov et al. · 2026 0 cites

    Synthesis

    Systematically compares knowledge-graph storage/retrieval approaches for personalized agents, against the backdrop that RAG improves factual accuracy but lacks structured memory and doesn't scale in complex long-term settings.

    Why it matters The closest thing to an apples-to-apples KG-design comparison; pairs with [mem0-vs-graphiti] (vector-vs-graph in distributed setting) for store-selection decisions.

  3. PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?

    Pulipaka et al. · 2026 0 cites

    Synthesis

    Persisting facts (e.g. 'user is vegetarian') aids personalization but also introduces safety risks that are largely overlooked. PersistBench measures when persistence becomes a liability and when memories *should* be forgotten.

    Why it matters Reframes forgetting as a safety requirement, not just hygiene — and is one of the only benchmarks targeting forgetting at all. Bridges Forgetting and Evaluation; pairs with the 'forgetting is unmeasured' gap.

  4. Cost and Accuracy of Long-Term Memory in Distributed Multi-Agent Systems

    Wolff & Bennati · 2026 0 cites

    Synthesis

    A testbed for long-term memory in distributed multi-agent systems (cloud+edge), directly comparing Mem0 (vector-based) vs Graphiti/Zep (graph-based) on system-level cost and accuracy — not just the tokens/latency that framework-published evals report.

    Why it matters The single most decision-useful head-to-head for splitting work between a vector store and a graph store in a multi-agent setting. Bridges Multi-Agent and Evaluation; complements [personalai] (KG-internal comparison).

  5. LoCoMo-Plus: Beyond-Factual Cognitive Memory Evaluation

    Li et al. · 2026 0 cites

    Synthesis

    Existing benchmarks (LoCoMo foremost) focus on surface factual recall, but good responses often hinge on implicit constraints — user state, goals, values — never explicitly queried later. LoCoMo-Plus evaluates that 'beyond-factual' setting.

    Why it matters Marks the benchmark frontier shifting from 'did it recall the fact' to 'did it honor implicit user context'. Pairs with [adamem] (user-centric retrieval) and [precision-belief-state] (retrieval precision).

  6. EngramaBench: Long-Term Conversational Memory with Structured Graph Retrieval

    Julian Acuna · 2026 0 cites

    Synthesis

    Benchmark for multi-session memory: five personas, 100 multi-session conversations, 150 queries spanning factual recall, cross-space integration, and more.

    Why it matters A concrete harness for evaluating cross-session retrieval quality. Use alongside [precision-belief-state] (precision) and [locomo-plus] (beyond-factual) to triangulate.

  7. What are people actually using for agent memory in production?

    u/MeasurementSelect251 · 2026 0 cites

    Synthesis

    Field thread: chat-history-only, vector-DB RAG, and summary+embedding hybrids all 'work for demos' but break once the agent runs a while — preferences drift, the same mistakes recur, stale context gets pulled purely on semantic closeness.

    Why it matters The blunt production reality check that motivates half this map (temporal updating, forgetting, beyond-similarity retrieval). The 'stale context on semantic closeness' complaint is exactly [adamem]'s thesis.

  8. Building memory systems at production scale (100k+ users): lessons from 10+ implementations

    u/singh_taranjeet · 2026 0 cites

    Synthesis

    Lessons from ~10 production deployments (healthcare, fintech, consumer SaaS, dev tooling) on what actually matters vs the 'just add a vector DB' tutorials.

    Why it matters The hard-won operational counterweight to research/product claims. Read with [what-using-production] as the practitioner reality layer of the Evaluation category.

  9. What Happens Inside Agent Memory? Circuit Analysis from Emergence to Diagnosis

    Mao et al. · 2026 0 cites

    Synthesis

    Agent memory failures are silent — a fluent answer can hide a failure to extract, retain, or retrieve. Traces feature circuits across Qwen-3 (0.6B–14B) to map the write–manage–read loop to internal computations.

    Why it matters A mechanistic-interpretability angle on *why* memory fails, complementing black-box benchmarks. Connects the write/manage/read framing used across consolidation, substrate, and retrieval.

  10. Engram: A Bi-Temporal Memory Engine Where a Lean Retrieved Context Beats the Full History

    Wang, Liuyin · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract Engram is an open-source long-term memory engine for LLM agents. Instead of replaying an entire conversation history into the prompt, it stores the past as a bi-temporal knowledge graph and retrieves a small, precisely-targeted slice at answer time. On a standard 500-question memory benchmark that lean ~9.6k-token slice answers more accurately than feeding the model the full ~79k-token history, turning memory from a cost optimization into an accuracy improvement.

    Motivation Stateless LLM agents forget across sessions, and the usual fix — concatenating the whole history — grows token cost and latency linearly and loses accuracy as distractors crowd the window ('lost in the middle'). Two gaps stay open: most memory systems are cheaper or faster but not more accurate than full-context, and memory benchmarks run on inconsistent harnesses where one system reports wildly different scores across sources. Engram targets both — beating full-context on accuracy, and shipping a neutral, re-runnable harness.

    Methodology A dual-process design. A System-1 hot write path appends lossless episodes with no LLM (sub-50ms) and enqueues them. A System-2 async path extracts atomic (subject,predicate,object) facts, builds a bi-temporal knowledge graph (valid time vs transaction time on every fact and edge), detects conflicts, and resolves them cheap-then-escalate: exact slot match, embedding similarity, and content subsumption handle the common case with no LLM call, invalidating (never deleting) a superseded fact and recording a supersedes chain and provenance, with only ambiguous cases escalated to an LLM adjudicator. The hybrid read path retrieves through four channels (dense, BM25, graph n-hop, recency/salience), fuses them with Reciprocal Rank Fusion, applies an 'as-of' temporal filter and an abstention gate, and assembles a deduplicated, provenance-tagged, token-budgeted context of facts plus raw chunks.

    Results On the full 500-question LongMemEval_S under the official category-specific judge, Engram's lean configuration scores 83.6% vs 73.2% for full-context (+10.4 points, McNemar exact p<10^-6) at ~8x fewer tokens (9.6k vs 79k), 0/500 errored. The gain is load-bearing on the read path being hybrid: facts alone lose recall, while facts plus retrieved chunks recover detail. Bi-temporal modeling pays off most on knowledge-update (87.5%) and temporal (81.1%) categories, while multi-session aggregation and preference remain headroom. The paper documents measurement-integrity pitfalls (truncation, home-grown judges, full-history leaks) and ships a neutral in-repo harness with the official judge baked in and raw per-question logs, every number reproducible by command.

  11. GitOfThoughts: Version-Controlled Reasoning and Agent Memory You Can Replay, Diff, and Merge

    Shekar, Pavan C · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract GitOfThoughts stores an LLM agent's reasoning tree as a git repository — every scored thought is a commit, scores are git notes, validation outcomes are tags, and retrieval is git log over the agent's own history — which makes reasoning replayable, auditable, diffable, and mergeable across agents. The paper then asks the harder question of whether memory, in any substrate, actually improves accuracy, and runs a pre-registered comparison of five substrates (none, markdown, vector, graph, git).

    Motivation Reasoning is the last unversioned software process: chains of thought expire with the context window, pruned search branches leave no record, and memory buffers cannot be diffed, merged, or audited. The authors argue this ephemerality is a structural blocker — it prevents reproducibility ('what did the agent think at step 17?'), audit (detecting train–test leakage or gold-answer memorization), memory transfer between agents, and incident review. Code, infrastructure, datasets, and experiments are all version-controlled; reasoning is the remaining outlier.

    Methodology A reasoning tree shares git's structural invariants, so the paper maps it one-to-one onto git primitives (node = commit, refinement = parent edge, score = note, outcome = tag, session vs. cross-session = branch, retrieval = git log --grep / -S). A pluggable MemoryBackend routes every read/write through one interface so the same agent can swap substrate with a one-line change. To isolate retrieval from write-path noise, all five backends ingest identical answer-free lessons, then solve held-out, domain-stratified problems read-only; benchmarks are GPQA-Diamond and MATH-500, scored with paired-bootstrap CIs, across two backbones and pre-registered replications, with a similarity sweep to locate when retrieval helps.

    Results H-substrate is supported: git delivers auditability, line-level diffs over reasoning text, deterministic replay by SHA, and mergeable memory at accuracy parity, costing ~15 ms/write and ~48 ms/read. H-memory is rejected: across two benchmarks, two backbones, and up to n=500, no substrate reliably improves accuracy on novel problems, and a +15 pp git trend at n=40 collapsed under its pre-registered replication. Memory pays only above a 'copyability threshold' — a near-duplicate retrieved case (cosine ≳ 0.8) lifts accuracy +12 to +13.5 pp, and a 4.5× larger model steepens that step to +22.5–28.5 pp but still extracts no transferable method; the only general accuracy lever is test-time sampling (self-consistency, +3.4 pp at n=500). The authors deliberately document a measurement bug, a retracted result, and a refuted hypothesis as the evaluation standard.

  12. StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance

    Liu, Guanming · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract StreamMemBench is a streaming benchmark that tests whether a personal-agent memory system can turn what it observes and how users interact with it into future-oriented assistance. Built on EgoLife egocentric lifelogs, it anchors each evaluation on a hidden piece of user-specific evidence and wraps a two-step task around it: an initial task that depends on the evidence, then a follow-up task that tests whether the agent reused both the evidence and the user's feedback. Four metrics diagnose evidence retention, feedback incorporation, initial evidence use, and follow-up reuse.

    Motivation A central job of personal-agent memory is to carry stored observations and prior interactions forward into later, similar tasks, but existing memory benchmarks test dialogue recall or task improvement in isolation and usually rely on scripted or synthesized dialogues whose feedback is not tied to verifiable observations. That leaves the trajectory from streaming observations to later assistance largely untested — and even commercial assistants such as ChatGPT and Gemini store information that fails to help when it is actually needed.

    Methodology In the construction stage an anchor agent processes five-minute EgoLife segments in stream order and extracts a user-specific evidence anchor plus two application-oriented queries, and a review agent retains a candidate only if both queries satisfy Leak=0 (the query does not reveal the evidence), Need=1 (ignoring the anchor yields a wrong or generic answer), and Natural=1 (it reads as a plausible request). In the evaluation stage the memory system ingests the lifelog chronologically, answers the initial task, receives confirming or correcting feedback from a user simulator, commits that interaction to memory, and then answers a follow-up task grounded in the same anchor; an evaluation agent scores Fidelity, Feedback Incorporation, Initial Evidence Use, and Follow-up Reuse against atom-level checklists, and the Fidelity−IEU / Fidelity−FUR gaps localize failures.

    Results Across eight systems — two retrieval baselines (RAGraw, RAGext) and six memory systems (A-Mem, Mem0, EverMemOS, MemOS, MemoryOS, MemSkill) — on two backbones, current memory systems are not yet reliable for future-oriented assistance: they often fail to use evidence from egocentric observations in the initial task and to turn interaction feedback into reusable follow-up behavior. The failures are not explained by storage alone — systems frequently retain the evidence (with some Fidelity inflated by raw-text retention) yet do not use it — which motivates evaluation that traces each piece of evidence from its first appearance in the stream through initial use, feedback incorporation, and follow-up reuse.

  13. MemTrace: Probing What Final Accuracy Misses in Long-Term Memory

    Long, Xianxuan · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MemTrace is a benchmark for testing how well an AI assistant remembers facts about a user across many conversations. Instead of scoring isolated questions, it tracks each individual fact—like a user's job title—and repeatedly asks about it under different conditions: long after it was mentioned, when it has since changed, or when the question contains false information. This reveals failures that a single overall accuracy score hides, such as a system that recalls a user's current role correctly but invents a false history of how they got there.

    Motivation Long-term memory benchmarks usually aggregate accuracy over question rows or interaction episodes, treating questions that probe the same underlying fact as independent items. That makes it impossible to hold a fact fixed and ask how a system behaves as conditions around it change—whether it still recalls a fact after many sessions, whether it tracks how the fact evolved, and whether it behaves safely when evidence is missing or contradicted. Two systems with similar pooled scores can fail in entirely different ways, and aggregate scoring cannot show which.

    Methodology MemTrace makes the knowledge point—a single typed fact about the user—the unit of measurement, and probes each fact along three controlled dimensions: memory age (how many sessions ago it appeared), question type (current state, an earlier state, or the trajectory of change), and evidence condition (present, missing, or contradicted by a false premise). It comprises 835 typed knowledge points from 20 users, expanded into 15,422 question rows and over 200,000 scored answers, and evaluates 13 memory-system configurations across four paradigms: long-context models, retrieval-augmented systems, external-memory stores, and agentic-memory architectures. A diagnostic step classifies each failure by whether the needed evidence was unreachable or reachable but unused.

    Results Performance varies systematically across all three dimensions. Long-context systems answer recent facts well but lose accuracy as facts age, especially on trajectory questions; RAG systems, including graph-based retrieval, handle current and earlier-state questions better than questions about change over time; some external-memory systems decline almost all questions about facts that were never mentioned yet rarely correct a false premise. The dominant remaining bottleneck is evidence use, not retrieval: when systems fail, the evidence was already retrievable about 10× more often than it was missing, so improving memory depends on using reachable evidence rather than storing or retrieving more.

  14. MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

    Xiang, Zhishang · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MemSyco-Bench is a benchmark that tests a specific failure mode in AI agents with long-term memory: sycophancy, where an agent trusts a memory of something the user said or believed before, even when that memory is outdated, out of scope, or contradicted by current evidence. Existing memory benchmarks mostly test whether an agent can retrieve the right memory; this one tests whether the agent uses a retrieved memory correctly once it has it.

    Motivation The authors first show the problem is real: adding a plausible-but-wrong memory snippet before an objective factual question drops accuracy across all tested models and roughly doubles the rate at which the agent adopts the incorrect claim. They then show existing benchmarks (LongMemEval, LoCoMo, STALE, PersonaMem) can't isolate this: 47.4-66.1% of their errors come from failed retrieval, versus only 5.8-13.7% from correct-retrieval-but-wrong-use, so post-retrieval reasoning failures are essentially invisible in current scores.

    Methodology MemSyco-Bench defines five task categories matched to the decisions an agent should make about a retrieved memory: reject it as factual evidence, respect its scope, resolve a conflict between memory and objective evidence, track that a memory has been updated, and use a genuinely valid memory for personalization. Multiple memory systems and backbone models are evaluated across these categories.

    Results In the preliminary study, injecting a misleading memory cue drops factual accuracy on DeepSeek-V4-Flash from 56.1% to 40.2% and raises its sycophancy rate from 24.3% to 52.3%, with similar (smaller) effects on the other tested models. Full benchmark results show current memory systems generally increase sycophancy and struggle to balance personalization against factual reliability.

  15. MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations

    Hao, Xixuan · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract MemOps is a benchmark that reframes long-term conversational memory as a lifecycle of explicit operations (remembering, forgetting, updating, reflecting, and their compositions) rather than as static question answering. Each memory event is a structured trace specifying its trigger, target, scope, state transition, and supporting evidence. A controllable pipeline embeds these operations into long, task-oriented conversations, producing gold operation traces and six categories of operation-level probes evaluated under both adjacent-evidence and long-context settings. Across long-context, retrieval-based, parametric, and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.

    Motivation Existing long-term memory benchmarks such as LoCoMo, LongMemEval, MemBench, and others evaluate almost exclusively through downstream question answering, scoring only the correctness of a final answer. That black-box formulation conflates heterogeneous causes of failure (missing the introduction of a relevant fact, binding an operation to the wrong target, or relying on a stale value after a correction) and can credit a correct answer that rests on an inconsistent or unsafe memory state. This is most acute in dynamic long-horizon interaction, where memory functions as a lifecycle process: a user introduces a fact, corrects it, asks that part be forgotten, or implicitly signals a preference, each of which is a distinct operation with its own trigger, target, scope, state transition, and characteristic failure modes that downstream QA cannot diagnose.

    Methodology Each instance is a tuple of a topic-specific user background, a set of evidence conversations, a gold operation trace, and evaluation probes. The trace is the evaluation anchor: each operation carries a type, target object, old and new value, and evidence spans quoted verbatim from user turns. Five operation types are defined, including TrajectoryOps that compose remember, update, forget, and reflect events across time so the benchmark can check intermediate states and their temporal order. A four-stage generation pipeline (background construction; evidence conversation and gold-trace generation; operation-level probe generation; long-context dialogue generation with distractor pools) produces natural conversations with explicit operation supervision, verified by local schema and span gates plus an LLM verifier. Six probe categories (operation trace, target binding, state transition, candidate disambiguation, operation application, state trajectory) score intermediate memory behavior, and representative systems from four paradigms are evaluated under adjacent-evidence and long-context settings.

    Results Performance is strongest when evidence sits adjacent to the query and degrades consistently once it is dispersed into long, distractor-laden histories, for both answer accuracy and operation-level reliability. Session-level retrieval substantially outperforms turn-level retrieval, and managed-memory services that preserve longer, context-rich memory units outperform those storing short, isolated facts, indicating that surrounding context is critical for correctly executing lifecycle operations rather than merely retrieving relevant content. Parametric memory, which folds interaction history into model parameters, remains markedly unreliable across almost all diagnostic dimensions. Reconstructing an ordered memory-state trajectory across multiple composed operations is far more fragile than any single-step operation, and this weakness persists even for otherwise strong long-context models, exposing a failure mode that final-answer accuracy alone would not surface.

  16. Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

    Kwon, Alex · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A language model's memory can be worse than no memory at all. Give a model a memory that kept a wrong conclusion but dropped the work behind it and it re-emits the stale value as a confident answer; give the same model an empty memory and it abstains. The paper names this failure brittle memory, measures it with a reclaim-evaluation protocol that tests whether a correction can recover a known answer after compression, and shows a one-line fix (keep the recomputable source, drop the re-derivable conclusion) restores correctability at equal memory budget.

    Motivation Memory systems carry information across sessions by compressing it, on the implicit assumption that a compression preserving the model's answer has preserved what matters. The paper shows the same compression decides whether the model can later be corrected: once the answer-determining source is gone, a correction has nothing to act on, and the resulting error compounds as deployed agents feed memory into memory.

    Methodology Reclaim evaluation drifts a model into committing to a wrong answer via a planted premise, deepens the commitment over neutral turns, then issues a correction in a fresh session whose only inheritance is a memory written under one of three matched-budget policies: lossy (keep the salient conclusion, shed the source), source-first (keep the source, shed the conclusion), and lossy-padded (lossy plus neutral filler to at least source-first's length, controlling for budget). Success is exact recovery of the known answer, with no judge. Tasks are multi-step arithmetic and constraint-logic puzzles with objectively scorable answers; the pipeline runs end to end on llama-3.1-8b and grok-4.3 with a frontier replay on Claude models; headline cells are n=96, and three validators designed to fail against a deterministic fake all pass.

    Results Within one conversation there is no wall, only anchoring: a directed correction holds far longer than a generic one (reclaim 0.79 to 0.50 over eight commitment turns) and pushing the error further back lifts reclaim rather than starving it. Across a session boundary the window becomes a wall: once the lossy note drops the source line items, even a directed correction dies, reclaim is 0.00 by measurement, and a lossy memory is worse than an empty one because models that abstain with nothing emit the confident wrong value with a source-less note. The wall sits in the same place from the 8B model to frontier systems. Source-first restores reclaim at equal budget (oracle 1.00; the deployable one-prompt distiller 0.49-0.88, concentrated on compact numeric sources), and the length-matched control rules out added text as the cause. Chained through a memory loop, one dropped-source error corrupts a growing span of downstream steps and stays uncorrectable however late it is caught, while source-first holds to a bounded budget horizon; the wall and the fix replicate on three deployed memory systems and on MultiWOZ, and past the budget where the source no longer fits, the fix fails silently unless the note records its own completeness.

  17. RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts

    Shriniwas Arya, Mihir · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract RECON (Reasoning over Extended Contexts with Obfuscated Narratives) is a benchmark that tests whether an agent's memory can maintain a coherent, evolving understanding over long contexts where facts do not just accumulate but interact, contradict, and cascade. It spans 24 investigative case files across criminal, medical, and financial domains, each 50k to 100k tokens, with 1,604 questions over six memory-intensive tasks, and reports that even the strongest non-Oracle system reaches only 22.4% accuracy.

    Motivation Existing memory benchmarks model memory as a state machine: they check whether an agent can retrieve a scattered fact or detect that a fact changed. Real workflows demand more. When a lab result is revised on Day 9, a witness statement is contradicted on Day 5, or a flagged transaction is reversed, an agent must trace which downstream conclusions are affected, which survive on independent support, and how an alternative timeline would have unfolded. RECON targets what happens after a change, not just whether the current value is tracked.

    Methodology A deterministic pipeline constructs each case from a provenance DAG, then a question generator traverses the DAG to synthesize questions per task category, with a fixed question matrix enforcing per-task and per-format quotas so the distribution does not drift between runs. The six tasks are chain reconstruction (order 5-15 evidence hops), cascade propagation (which conclusions break versus survive after an invalidation), source-conflict resolution, counterfactual reasoning, temporal-constraint satisfaction, and temporal fact retrieval as a control. Solvers - long-context, RAG, hybrid RAG, Supermemory, Mem0, Mem0-Graph, Hindsight, closed-book, and an Oracle handed the full structured ground truth - share one answering template; freeform answers are graded by two independent LLM judges from different model families and averaged, while other formats are scored deterministically with an explicit abstain option (+1 correct, -0.2 wrong, 0 abstain).

    Results Even the Oracle given the ground-truth dependency graph reaches only 54.6% accuracy, and the best non-Oracle system 22.4%; RAG attains 20.6% on full-coverage retrieval hits. Decomposing failures shows roughly four in five persist even when retrieval succeeds, placing the bottleneck in reasoning rather than evidence selection. Human annotators reach 63.0% accuracy, exceeding the Oracle solver by 8.4 points and confirming reasoning, not retrieval, as the residual challenge across long-context, retrieval, and agentic-memory architectures.

  18. FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents

    Wang, Tianyou · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A benchmark in which an LLM agent runs a football club for 20 in-game years against 15 rivals, scored cumulatively by a deterministic engine with no LLM judge. It measures long-horizon management under hidden information, cumulative consequences, a counter-adaptive market and multi-objective pressure, and treats the agent's self-written notebook as its only cross-stop memory.

    Motivation Language model agents handle bounded tasks reliably, but those tasks have a short horizon to a correct answer and no competing agents. Newer benchmarks push one dimension at a time: long-horizon work stretches episodes to hundreds of steps or simulates months of a vending machine, shop, startup or company's finances, while competitive benchmarks stage cooperation and conflict over short episodes or rank models by bargaining inside a single session. None combines both dimensions while giving the agent an organization to keep alive. The authors define management as running an organization against rivals pursuing the same opportunities over a horizon long enough that early decisions reshape the world, judged cumulatively under the conditions those decisions created.

    Methodology The environment instantiates four demands with concrete mechanisms: hidden information through scout bands carrying a permanent per-scout bias, hidden player traits and hidden asks in negotiation; cumulative consequences through youth and facility investment that pays off over years, insolvency and confidence spirals that compound yet remain recoverable, and honors accruing into the final score season by season; a counter-adaptive market where rejected bids raise the hidden ask, repeat pairs draw markups, and negotiation cooldowns apply; and multi-objective pressure from a board judging results and financial discipline jointly with season targets scaled to squad strength. A run spans roughly 340 to 400 decision stops across 26 schema-generated tools, with a per-stop budget of 30 queries and 10 negotiation moves. Each stop is a fresh conversation, so the only carried state is a private notebook edited through append_note and rewrite_notes, which makes memory curation itself a measured capability. Two tracks share one engine: a solo track playing each of 15 frontier models against a frozen scripted world with tiered opponents, and an Arena placing the same models plus a scripted anchor in one shared 20-year economy, made comparable by an equal-endowment draft, sealed-bid conflict resolution and capped revival. Grading is continuous and mechanism-computed throughout, with no LLM judge or human rater. Memory curation is scored by reconstructing each notebook at season end and taking TF-IDF cosine similarity between consecutive snapshots, read alongside notebook size. Three seeds were run, and six first-play humans ran the same track.

    Results Across three seeds all 15 models complete every horizon while the blind scripted anchors, including a disciplined heuristic, die out in 7 of their 9 runs. claude-fable-5 tops both tracks, reaching about 95% of a scripted upper anchor allowed to read the hidden state (90.94 against 95.54), yet competition still reshuffles the board: the league title rotates among ten models and mid-board solo standings do not survive adaptive rivals. Neither scale, price nor vendor predicts the order, the order settles only late in the horizon, and the best first-play human finishes at the bottom of the model board while four of six humans died out. Three of the six behavioral capabilities track score on every seed: reducing slow-payoff investment as the horizon ends (Spearman -0.58), keeping cash deployed rather than idle (-0.50), and opening contract renewals early (+0.45). Token spend is uncorrelated with score under every accounting. Two negative results stand out. Hundreds of rejected bids never teach a model where the market's true prices lie. And self-managed memory fails in two opposite regimes rather than one: an append-only archive in which current state drowns in history at the high-similarity end, and wholesale rewriting at the low end, with claude-sonnet-5 at 0.20 and qwen3.7-max at 0.23 rewriting so completely that no plan survives to be executed. The winner sits at 0.39 against a field median of 0.31, holding a stable strategy skeleton while rewriting state each season. Similarity alone does not certify curation quality: notebook size decodes it, since the same high consistency is a 200k-character archive for one model and a 3-6k curated document for the winner.

  19. Can Agent Memory Systems Track Evolving State?

    Fan, Xinyi · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A benchmark and method for a memory capability separate from recall: keeping track of which version of a fact is currently in force after it has been revised across sessions. The benchmark, StateMemBench, generates each scenario as a symbolic program of state operations so the correct answer is computed by replay and the specific way a lazy reader would get it wrong is known in advance. The method, StateMem, parses each turn into structured state units with typed dependency links, then handles supersession and staleness deterministically rather than with an LLM.

    Motivation Existing memory systems and benchmarks optimize recall of relevant facts, but as agents run longer, facts, constraints and decisions get revised, and an answer must reflect the current state rather than a superseded one. The authors call the failure state drift: the relevant fact is present in the assembled context, but the agent acts on a stale or incomplete version of it. This is distinct from dialogue state tracking, which prescribes a slot-value representation and evaluates it directly over cooperative dialogues that accumulate a goal monotonically; here state is whatever a system must maintain to answer correctly, evaluation is purely behavioral, and revisions are adversarial across sessions. Some concurrent work centers state, but the authors argue none cleanly isolates state tracking from the other errors it co-occurs with.

    Methodology The authors first define and label drift on existing benchmarks, assigning a failure to drift only after excluding retrieval, comprehension, schema and reasoning readings, dropping unassignable points, and cross-checking with two judge passes, a cross-family judge and two human annotators. They then build StateMemBench: each scenario is a symbolic event program of typed operations over ground, derived and declared state; the gold answer comes from deterministic replay; a family of executable lazy reader policies is run against the replay, and a scenario is admitted as a trap when policies disagree, with the disagreeing set forming its failure-mode signature (status, salience, sequence, compound, plus anti-trap controls). Programs are grounded in public data for surface vocabulary and rendered into multi-session dialogue by a strong LLM, then programmatically verified for fact placement and phrase leakage. Probes are closed-pool: an unseen pool of three to four options holds the gold answer, the targeted policy's drift answer and neutral distractors. StateMem itself runs a per-turn TurnEncoder producing state units (id, content, priority, source, deps), a deterministic update stage applying supersessions and marking dependents needs_recheck by dependency-graph traversal, and a single answer-time call over the assembled active state. A wrapper variant applies the same trace-then-resolve structure as a prompt-level transformation of any backend's answer call, evaluated against a length- and cost-matched generic-extraction control.

    Results Drift leads the confirmed failure distribution on several existing benchmarks (63.5% on MemoryArena-shopping, 44.4% on LongMemEval oracle where retrieval is perfect by construction, 10 of 16 on tau-squared-bench-Z) but is not universal, falling to 19.0% on MemoryArena-travel. On StateMemBench, long-context is not the strong baseline it is on recall tasks: the best long-context model reaches 0.277 and same-backbone long-context 0.149. StateMem reaches 0.363 on DeepSeek-V4-Flash, 1.8x the best memory system and 2.4x same-backbone long-context, and 0.233 on Qwen-3.5-9B, 1.6x the best memory system, both significant by paired McNemar at p < 0.001; GraphRAG at 0.224 is statistically level on Qwen. Ablations put supersession marking as the largest single component and show dependency propagation over-fires on Set B anti-traps by 12.5 points, so removing it leaves DeepSeek slightly better. Drift-rate analysis shows the memory layer barely changes outcomes on a weak answerer, where every arm drifts at 61 to 66%, and separates on the stronger backbone, where StateMem's drift rate falls 15 points and correct answers rise by 42 while long-context, Mem0 and BM25 move by 1 to 3 points. The wrapper improves every one of six backends on both benchmarks, adding 31.7 to 66.6 points on StateMemBench with 15.0 to 31.7 attributable to state structure over the matched control, significant in all twelve cells. State tracking does not cost recall: StateMem also leads memory systems on LongMemEval (0.656 on DeepSeek) and LoCoMo (0.592), with margins concentrated on temporal-reasoning and knowledge-update question types. The authors note StateMem mirrors the policy family behind the traps, so its StateMemBench margins should be read as an upper bound.

  20. Paritok-4B: Intent-Conditioned Context Compression for Coding Agents

    Shi, Jiayu · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A 4B LoRA compressor trained specifically to shrink coding-agent context before it reaches a frontier model. Two commitments define it: it is extractive, selecting spans rather than rewriting them so identifiers, paths and error strings survive verbatim, and it is intent-conditioned, receiving the agent's current task and keeping what that task names. It operates one typed segment at a time behind a gateway, is distilled from a gpt-4.1-mini teacher over 67,074 real OpenHands trajectories, and ships as a 264 MB adapter that self-hosts on a single 24 GB GPU under Apache 2.0.

    Motivation An autonomous coding agent re-sends its accumulated context, file reads, command output and history, on every turn, so input rather than output dominates the token bill. Compressing that context with a small model before it reaches the expensive one is an obvious lever, and prompt-compression research shows heavy compression can preserve task quality. But general prose compressors face constraints they were never built for here. An agent edits by exact string match, so a paraphrased signature or renamed variable breaks the downstream edit. A segment's value is not intrinsic but depends on what the agent is currently doing, and the function it is about to modify has to survive even when it looks unremarkable. Agent context is also strongly heterogeneous, since a cat -n file read, a pytest traceback, an ls listing and a reasoning block have nothing in common, and one uniform ratio is wrong for all four.

    Methodology A gateway performs segmentation, kind classification and level labelling; the model is then invoked once per compressible segment with exactly two inputs, the agent's current task and the segment tagged with kind and level, and returns that segment's compressed form. Kinds cover file reads, bash commands, log output, tool results, file operations, directory listings, assistant thinking and meta actions, alongside protected system and user messages; four levels from protected through stale carry intended per-level budgets that live only as a static table in the system prompt, with no numeric budget passed to the model. Emitting an empty body is a first-class action for unrelated helpers, build noise and superseded re-reads. Output mirrors input inside per-segment markers reusing the input segment id, so every compressed span maps back to one original and untouched bytes can be recovered on demand. Retained code lines, identifiers, paths, line numbers, imports, error classes, exact error text, shell commands and edit payloads are copied verbatim; only a closed, enumerated set of structural markers may replace deleted content, and a body of markers alone is forbidden. Training distils a gpt-4.1-mini teacher over 67,074 OpenHands trajectories through a five-stage funnel into a 45K-segment pool and 40,606 validated examples, fine-tuning a Qwen3-4B backbone with LoRA, chosen over 3B and 7B code-pretrained alternatives under a matched protocol, with the deployed checkpoint selected by an out-of-distribution sweep rather than by training loss.

    Results The extractiveness claim is audited rather than asserted: 96.0% of emitted identifiers, paths and numbers already appear in the input over the training corpus, 98.3% outside the one kind rewritten by design, and 96.2% on held-out SWE-bench Lite output across 212,506 emitted tokens. Intent conditioning is measured to act chiefly inside a retained segment, selecting which lines survive rather than changing how much is retained, with retained lines +0.067 more intent-relevant than removed ones (paired 95% CI [+0.056, +0.078]). On an out-of-distribution holdout the released checkpoint emits well-formed output on 100% of segments and compresses to 23.7% of input tokens, with must-keep identifier retention not below the teacher's at comparable budget. End-to-end over all 300 SWE-bench Lite instances it compresses to 25.7% of original size, 2.0 times harder than a gpt-4.1-mini compressor at 50.2% and 2.4 times harder than gpt-5 at 61.9%, retaining 86.5% of uncompressed single-shot solve quality; in the in-distribution cat -n regime it compresses to 27.8% and retains 89.3%, where 30 instances are solved only uncompressed and 17 only compressed, an exact McNemar p=0.079. The authors are explicit that this harness measures comprehension under compression, not end-to-end agent cost, and that the four-level budget design collapsed into two effective bands rather than four. On economics, a 264 MB adapter on one 24 GB GPU carries no per-token compressor fee, whereas gpt-5 used as a compressor is net-negative at list prices, costing more than the downstream tokens it saves.

Interop, Schema & Governance

Portability and control: shared wire formats across memory frameworks, schema discipline, and governance/audit surfaces for what gets written and read.

Open questions
  • Can you migrate memory between frameworks, or does switching mean rebuilding from scratch?
  • Is the memory schema explicit and reviewable, or implicit and emergent?
  • Is there a human-auditable surface over memory writes/reads?
  1. memorywire: A Vendor-Neutral Wire Format for Agent Memory Operations

    Munirathinam · 2026 0 cites

    Synthesis

    Mem0, Letta/MemGPT, Cognee, Zep/Graphiti, MemoryOS, MemTensor each ship their own SDK, storage layout, and vocabulary — no shared wire format. Every integration is bespoke, every migration rebuilds from scratch, and none ships a governance surface to review writes. Proposes a neutral wire format.

    Why it matters The interoperability + governance gap stated plainly — directly relevant if a service spans multiple stores (vector + graph + object storage). Pairs with [schema-quality] (schema discipline) and [unified-memory-stack] (DIY unification).

  2. Agent Memory Is Only as Good as Its Schema

    Daily Dose of DS · 2026 0 cites

    Synthesis

    Deep dive on production-grade agent memory arguing memory quality is bounded by schema quality — get the schema wrong and retrieval/consolidation can't recover.

    Why it matters Elevates schema design to a first-class concern; the human-readable companion to [memorywire]'s machine wire-format. Connects to Representation (the schema encodes the representation choice).

  3. Built a unified LLM memory system combining Memori + Mem0 + Supermemory

    u/0sparsh2 · 2025 0 cites

    Synthesis

    DIY unification: Memori's interceptor architecture (zero code changes), Mem0's research-validated retrieval/consolidation, and Supermemory's structure — composed into one stack.

    Why it matters Exactly the multi-store composition pattern that motivates a wire format. Real-world evidence builders are already gluing frameworks together by hand. Connects to [memorybank-rust] (cross-tool) and [memorywire] (the standard that would obviate the glue).

  4. OpenMemory by Mem0: 'local' but still needs an OpenAI key?

    u/Perplexed_86400 · 2026 0 cites

    Synthesis

    Notes that Mem0's OpenMemory MCP advertises local/private operation but still requires an OpenAI key (embeddings + gpt-4.1-nano) for extraction in the default Docker setup.

    Why it matters Operational gotcha for anyone assuming a 'local' memory framework is self-contained — the LLM dependency leaks in via extraction/embeddings. Relevant to substrate/privacy and to the zero-LLM designs ([superlocalmemory], [yourmemory]) reacting to exactly this.

  5. Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents

    Li, Baichuan · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A benchmark for the decision an agent makes before it writes to memory: should this piece of interaction-derived information be persisted durably, used only in the current context, re-checked against the world, or clarified with the user. It evaluates three models from two families under three prompt conditions, and separately tests whether the stated decision predicts which memory tool the model actually calls.

    Motivation Persistent memory personalizes an agent, but an incorrect durable update silently distorts future behavior. A temporary request should not become a standing preference, a service status can go stale, one tool failure may be noise, and an underspecified correction may need a question before it is generalized. The authors argue the critical capability is not recall but commitment, and that two sources of uncertainty are usually collapsed: verification queries the world, which is authoritative for changing facts, while clarification queries the user, who is authoritative for intent and scope. Prior work covers adjacent targets (binary session-level storage gating, ADD/UPDATE/DELETE/NOOP operations, whether retrieved memory grounds tool parameters, general ambiguity clarification) without jointly distinguishing durable storage, local use, world verification and user clarification at commitment time.

    Methodology Each item supplies an acquire context, a candidate update and a later reuse context, with a gold action assigned by released rules and a tie between persist and a weaker action resolved toward the weaker commitment. 140 primary scenarios are split 70/70 by sorted identifier within category, covering stable and episodic preferences, freshness-sensitive facts, one-off corrections, policy constraints, ambiguous updates and noisy failures at 20 items each, with eight lexical traps where a surface cue points to the wrong action. Two non-authors labeled the held-out and contrast items blind to author labels and to each other, with a blind third resolving ties. Claude Haiku 4.5, Claude Sonnet 4.6 and a locally served Qwen3.5-9B (Q4_K_M via Ollama, temperature 0, seed 13, thinking disabled) were each run under three conditions: a bare prompt defining the actions, a policy prompt adding five commitment rules including the tie-breaker, and a four-shot prompt with one development example per action. A separate track, MCB-Act, removes the label vocabulary and requires one structured tool call, scored by mapping the selected tool to an action. Analysis uses accuracy with bootstrap intervals, macro-F1, over-memory, under-memory and per-class recalls, with exact paired McNemar tests and Holm correction within each family of comparisons.

    Results Both model families under-ask. Claude label-mode verification recall runs 0.889 to 1.000 while clarification recall runs 0.500 to 0.750; bare Qwen verifies 12 of 18 freshness items and asks on 0 of 12 clarification items. Few-shot prompting lifts Qwen accuracy from 0.557 to 0.771 (paired delta +0.214, p_H = 0.002) and clarification recall from 0 to 0.333, still missing 8 of 12. The policy prompt raises Qwen accuracy by only 0.071 (p_H = 0.539) but cuts erroneous persistence from 0.243 to 0.100 (p_H = 0.038), moving the uncertainty to verification (recall 0.667 to 0.944) rather than to the user (clarification 0 to 0.083). Haiku's policy and few-shot gains survive correction (p_H = 0.002 and 0.047); Sonnet's do not, so the benchmark measures a prompt-conditioned commitment policy rather than a fixed model trait. Label-to-tool agreement is 0.571 for each Claude model and 0.229 for Qwen; Sonnet accuracy falls from 0.814 to 0.529 (p_H < 0.001) and Qwen from 0.557 to 0.343 (p_H = 0.047), with Qwen calling use_now on 54 of 70 items and verification recall collapsing to 0.056. All emitted arguments pass the deterministic well-formedness rules, locating the bottleneck in tool choice. On the combined 140 Qwen items of the contrast extension, bare, policy and few-shot accuracy is 0.614, 0.757 and 0.843, with clarification remaining the weakest class throughout; the authors retain the extension as a controlled sensitivity check rather than a claim of naturalistic external validity, since its rule-authored templates align closely with the explicit policy rules. MCB-Act scores tool-call selection and does not execute downstream effects.

  6. MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance

    Wang, Haoyu · 2026 0 cites arXiv

    Synthesis

    Plain-language abstract A memory framework for LLM agents that runs an LLM verifier over each candidate trajectory and then keeps the verifier's output on the record permanently, as reward, confidence, label and uncertainty fields. Those fields decide whether the memory is admitted, held provisional or rejected, and are read again later to rank retrieval, resolve conflicts, trigger summarization and archive stale records. Evaluated on Terminal-Bench 2.0, SWE-Bench Verified, WebArena and Mind2Web across four backbones.

    Motivation Agent memory is only useful if stored experience stays reliable over hundreds of interactions, and two failure modes break that. Unreliable admission: failed trajectories, accidental successes, invalid patches and misleading observations all look relevant to a retriever and get written, then mislead later decisions — a web agent that learns to edit the first row after filtering an admin table edits the wrong row once a table is sorted differently. Memory drift: a bank that grows across many tasks accumulates duplicate, conflicting, stale and overgeneralized records that remain retrievable long after their assumptions stop holding. Prior experience-memory systems (Synapse, AWM, ReasoningBank) make memory persistent but treat write-time feedback as a one-shot signal, so nothing continues to govern the record afterward. Verifier work, in turn, scores the current episode and stops there.

    Methodology Verification is decomposed into multiple criteria — completion, consistency, validity, generalizability — with reward estimated from score-token distributions and repeated views; repeated verification is triggered when uncertainty crosses a threshold. The resulting descriptor (reward, confidence, label, uncertainty) is attached before activation and routes the candidate to rejection, a provisional state, the active bank, or a failure-guard pool that stores failed experience as constraints rather than recipes. Each record additionally carries lifecycle state, quality, usage statistics and conflict links. Retrieval is a hybrid of BM25 and embedding cosine similarity over title, description and content, adjusted by the descriptor and by staleness and overgeneralization penalties, with positive memories and failure guards rendered as separate blocks in the injected prompt. Conflicts are located by structured-signature similarity above a threshold. Governance runs after every task: activation, rejection, merge and conflict checks immediately; summarization and archival when the active-memory budget is exceeded or a record goes stale. Baselines are No Memory, Synapse, AWM, ReasoningBank and a verifier-only filter, all matched on task order, step budget, retrieval budget, injected-memory budget, memory block template and decoding settings.

    Results Averaged over five seeds, MemGuard has the best success metric and the lowest average step count in all 16 backbone-benchmark settings. Against ReasoningBank, the strongest prior memory baseline evaluated, the largest gain is 7.9 success-rate points on WebArena and 5.6 step-success-rate points on Mind2Web, with 2.4 to 3.5 points on the terminal and software-engineering benchmarks. Backbones are Qwen-3.5-Flash, Qwen-3.5-Plus, Gemini-3-Flash and Gemini-3.1-Pro. The verifier-only control improves on ReasoningBank in most cells by admitting better candidates but loses to MemGuard in every benchmark-backbone cell, which the authors read as evidence that the gain comes from persisting the verifier signal through the lifecycle rather than from filtering once at the door.

Foundations & Landscape

Orienting maps: surveys that taxonomize the field, and the managed-service landscape (Cloudflare, Mem0, Zep, LangMem, Letta) that frames the build-vs-buy decision.

Open questions
  • What's the shared vocabulary — memory forms (logs/weights/vectors) × functions (factual/experiential/working)?
  • Build a memory layer, or adopt a managed service?
  • Which incumbent assumptions are already being challenged?
  1. Agents that remember: introducing Agent Memory

    Tyson Trautmann · 2026 0 cites

    Synthesis

    Cloudflare's managed memory service: extracts structured memories from agent conversations and serves them on demand, with shared memory profiles so teams of agents read common knowledge. Framed around getting the right info into context even as windows pass 1M tokens.

    Why it matters The most complete public blueprint for the shape of service described in many internal designs (sessions → turns → consolidated docs → multi-channel search). Build-vs-buy anchor for the whole map.

  2. Cloudflare Announces Agent Memory, a Managed Persistent Memory Service

    Steef-Jan Wiggers · 2026 0 cites

    Synthesis

    Third-party writeup of Cloudflare Agent Memory (private beta). Names the competitive set explicitly: Mem0, Zep, LangMem, Letta.

    Why it matters Use this to scope the managed-memory market in one line. Takeaway: the named incumbents are exactly the frameworks the wire-format work [memorywire] says can't interoperate.

  3. Memory in the Age of AI Agents (survey)

    Hu et al. (50 authors) · 2026 0 cites

    Synthesis

    Large multi-institution survey formalizing agent memory as a core capability. Unifies a fragmented field into a taxonomy of memory forms (token-level logs vs parametric weights vs latent vectors) × functions (factual knowledge vs experiential learning vs working scratchpad).

    Why it matters The canonical framing doc — read first to make the rest of the map legible. Its forms×functions grid is a good axis for any design review. Connects to every other category as the orienting vocabulary.

  4. AI Agents of the Week: Memory as a First-Class Citizen

    Pascal Biese · 2025 0 cites

    Synthesis

    Newsletter roundup that flagged the agent-memory survey wave and frameworks like MemVerse (fast parametric recall + hierarchical retrieval) and WorldMM (multimodal experience consolidation).

    Why it matters Good lay-of-the-land pulse on what the research community foregrounded as memory went mainstream. Lighter signal than the survey itself.

  5. What Is Twilio Conversation Memory?

    Sean Spediacci · 2026 0 cites

    Synthesis

    Productized 'memory across conversations' for Twilio's agent/CX stack.

    Why it matters Data point that conversation memory is now table-stakes in vertical comms platforms, not just AI-infra vendors. Lower technical depth; useful as market evidence.

Companion podcast

1 episode walking this map as an argument: what each study measured, where they disagree, and what that leaves open. Listen inline or read the transcript.

  • Agent Memory: The Design Decisions

    A practitioner's decision tree for building agent memory in production, walking eleven design choices from the session-turn-document data model through retrieval fusion, temporality, security, and the evaluation gap.

    Read transcript 43 min · 7,092 words

    Memory is the feature everyone demos and nobody can evaluate. You have seen the demo. An agent remembers that you are vegetarian, that you fly out of Newark, that last quarter’s incident was a connection-pool leak, and it brings that back at exactly the right moment, and the room nods. What you have not seen, because nobody demos it, is the same system three months later, quietly returning a stale preference, contradicting itself across two sessions, pulling a memory that is semantically close to the question and useless for answering it, and doing all of that with complete fluency, because a wrong memory and a right memory sound exactly the same coming out of a language model. That gap, between the demo and the deployment, between how good memory looks and how hard it is to know whether it works, is the whole subject of this episode.

    This is the engineering cut. There is a companion series on this site that walks the research literature, the taxonomies, the survey wave, the storage-to-experience arc. This is the other lens. You are building agent memory in production, you have a budget and a latency target and real users, and you have to make a sequence of design decisions, each of which has options, each of which has a real tradeoff, and each of which some shipping system has already gotten right or wrong in public. I am going to walk you through eleven of those decisions, in the order you would actually hit them building the thing. For each one I will give you the decision, the options on the table, the tradeoff that bites, and what real systems chose. Think of it as a decision tree for agent memory, drawn from a corpus of fifty-one sources spanning vendor blogs, production postmortems, and the 2026 research front, plus a handful of launches from the last few weeks that landed while the corpus was being assembled.

    Hold one idea through all eleven decisions, because it is the spine. Memory is not a database you attach to an agent. Memory is a pipeline with three jobs that can each fail silently: deciding what to write, keeping what you wrote coherent over time, and surfacing the right piece at the right moment. Most of the decisions ahead are really about where in that pipeline you spend your effort and your tokens, and every one of them is a place where the system can look like it is working while it is not. Let us walk the tree.

    Start with the data model, because every other decision inherits from it. The question is: what is the shape of a memory? And the answer the field has converged on, almost without arguing about it, is a three-tier hierarchy. Sessions at the top: a conversation, an investigation, a work order, the natural unit of a user interaction. Turns underneath: the individual messages, the raw back-and-forth, the literal transcript. And consolidated documents on top of those: the distilled, durable artifacts you actually retrieve later, the facts and summaries and lessons extracted from the raw turns. Sessions, turns, consolidated docs. If you have read Cloudflare’s Agent Memory writeup from this year, that is its spine exactly, and it is worth dwelling on why this particular shape keeps winning. The raw turns are ground truth, cheap to store and never wrong about what was actually said. The consolidated docs are expensive to produce and lossy, but they are what fits in a context window and what answers a question fast. Keeping both means you have a fallback when the distillation drops something, and a fast path when it does not. That redundancy is not waste. It is the design.

    Look at how the managed services landed on the same structure independently, because that convergence is the strongest signal you get in this field. Amazon’s Bedrock AgentCore Memory, which reached general availability this year, splits cleanly into short-term and long-term. Short-term memory stores raw interaction events, the literal conversation, with a configurable expiry you can stretch up to a year. Long-term memory is generated asynchronously, a background process that extracts insights from the raw events after the fact, without blocking the live interaction. That is turns and consolidated docs, with a different vocabulary. Google’s Vertex AI Memory Bank, also generally available now, does the same: it keeps session state for the live conversation and extracts long-term memories from conversation history with a Gemini model running in the background. Different cloud, same two-layer split, same decision to make consolidation asynchronous so it never sits on the critical path of a response. When three independent teams at three hyperscalers ship the same shape, that is not fashion. That is the shape the problem actually has. So the first decision is mostly made for you: keep the raw turns, derive durable documents from them, do the derivation off the hot path. Where the real choices begin is in how you do that derivation.

    Which brings us to consolidation, the second decision, and the first one that is genuinely contested. You have raw turns piling up. When do you turn them into durable memory, and what do you keep? The naive answer, the one every tutorial reaches for, is eager consolidation: every time a turn comes in, fire an LLM at it, extract the facts, write them down. It is simple and it is current, and it is also, at scale, a quiet catastrophe for your token bill, because you are paying for an extraction call on every single message including the ones that contain nothing worth keeping. The 2026 research has a name for the alternative. RecMem, a paper out this year, calls eager consolidation a major cost driver and proposes recurrence-based consolidation that batches the work and defers it, trading a little staleness for a large reduction in spend. SimpleMem frames the same dilemma as a dial between retaining the full history, which is redundant, and reasoning hard over every turn to filter noise, which is expensive, and goes looking for the efficient middle. MemFly gives the tradeoff an actual objective function, an information-bottleneck criterion that balances compressing redundancy against keeping retrieval precise. The point underneath all three is the same: consolidation is a cost knob, and eager-on-every-turn is the most expensive setting on it.

    Now the harder half of consolidation, the one that should keep you up at night. When you let an LLM rewrite your memory, the memory degrades. There is a paper this year with a title that is the whole warning: “Useful Memories Become Faulty When Continuously Updated by LLMs.” The finding is that if you repeatedly run a model to rewrite a textual memory bank, edit by edit, the consolidated memory drifts and decays over time, accumulating small distortions until what you have stored is confidently wrong. There is a context-side mirror of this in the work on agentic context engineering, which names the two failure modes precisely: brevity bias, where concise summaries silently drop domain insight, and context collapse, where iterative rewriting erodes detail until the abstraction is hollow. Put those together and you get the central caution for this decision. Distillation is lossy, and compounding distillation is lossy in a way that hides. Each rewrite looks fine. The drift only shows up in aggregate.

    So what do you do about it, in production, today? Look at what Slack shipped, because they hit this wall on real long-running agents and wrote up the answer. Their security-investigation agents run for hundreds of inference requests, far past any context window, and their first instinct, accumulating chat logs, broke. So they moved to structured memory with an explicit validation step, and the phrase they use is distilled truth. Concretely it is three channels. A Director’s Journal that holds structured working memory. A Critic’s Review that scores findings for credibility using evidence-inspection tools. And a Critic’s Timeline that builds a single coherent narrative by taking the journal, the latest review, and the previous timeline, then keeping only credible evidence, removing duplicates, and resolving conflicts. The load-bearing move is the second channel. They do not trust the LLM’s rewrite. They validate it against evidence before it becomes truth. That is the production answer to the degradation problem: treat distilled memory as a claim that has to be checked, not as a fact because a model said it. Keep your raw traces as ground truth, distill aggressively for speed, and put a validation gate between the distillation and anything that calls itself memory. The systems that skip the gate are the ones that drift.

    The third decision is what data structure that durable memory actually lives in, and this is where the field is loudest and least settled. The options are real and they genuinely diverge: a flat vector store, an entity-relationship knowledge graph, a pile of atomic facts, event-grounded episodic records, or layered stores split by type. For a couple of years the momentum ran hard toward knowledge graphs. The pitch is seductive: model entities and relationships explicitly, accumulate structured knowledge, let the graph self-evolve, and you get relational reasoning that flat vectors cannot do. Mem0, Zep, supermemory all leaned in. And then the backlash arrived, and it is worth taking seriously because it comes from practitioners shipping this stuff. The sharpest version is a widely-read post arguing that knowledge graphs are simply the wrong abstraction for agent memory. The costs it flags are concrete: every write now needs an extra entity-extraction LLM pass, which is latency and money, and graphs hallucinate edges, fabricating connections between entities that were never actually related, when the real job most of the time is just fast retrieval of the right past context. Adding a graph can add a failure mode and a bill without adding an answer.

    The research has been busy patching the graph’s specific pathologies rather than abandoning it. GAAMA, this year, targets the mega-hub problem, where a few popular entity nodes accrue so many edges that they dominate every traversal and the structure stops discriminating, and proposes graph-augmented associative memory that keeps structure without the hub blowup. But notice the deeper challenge sitting under the whole representation debate, which several 2026 papers raise at once: maybe atomic facts are the wrong primitive in the first place. The dominant pipeline takes raw dialogue, runs a handcrafted prompt to compress it into atomic facts, stores those, matches them, and injects them. A paper this year titled, roughly, “Rethinking How to Remember: Beyond Atomic Facts” argues that this compression throws away exactly what you need to reason deeply over history. The coherence-first alternatives are getting concrete. CAST grounds episodic memory in who, when, and where, modeling characters and scenes instead of disembodied facts, because a fact stripped of its event loses the thing that made it answerable. Amory argues that fragmenting a conversation into isolated embeddings or graph nodes destroys narrative coherence, and rebuilds a continuous narrative instead. The throughline: the unit of memory is a design choice with teeth, and the more you shred the conversation into shards optimized for retrieval, the more you lose the structure that made the conversation mean something.

    Here is the practical floor, though, because it is easy to overbuild this. There is a builder’s writeup in the corpus whose whole lesson is that a great many agents do not need a vector database at all: SQLite with full-text search covers an enormous amount of ground with a fraction of the operational weight. So the representation decision is not graph-versus-vector at the top of a ladder. It is: start at the simplest structure that answers your actual queries, and add structure only when a query you genuinely need fails on the simpler store. Most systems reach for the graph long before they have a query that requires one.

    That naturally leads into the storage substrate, the fourth decision, which is the layer underneath the semantic one: where do the bytes physically live? And the freshest movement here is a deliberate retreat to boring infrastructure. One proposal making the rounds is plain Git plus S3 as the entire memory substrate: versioned, cheap, auditable object storage, keep the full history forever because storage is nearly free, and derive your memory downstream from a durable log you never have to trust a service to hold. A 2026 paper called GitOfThoughts takes that seriously and measures it, treating Git as the persistence layer beneath the semantic memory, each session its own repo, cross-problem insights on a memory branch. What is striking is the number it reports: roughly fifteen milliseconds per write and forty-eight per read, the same order of magnitude as an embedding index’s read latency, while Git uniquely buys you tested three-way merge with conflict surfacing, signed commits, and reproducible bundles. The substrate you would have dismissed as too primitive performs in the same ballpark as the bespoke one and gives you an audit trail for free.

    The substrate decision also has a piece people consistently miss, and AWS drew the line cleanly this year. There is a difference between durable knowledge and durable working state. Bedrock AgentCore’s runtime added managed session storage that persists an agent’s filesystem state, the code it wrote, the packages it installed, the artifacts it generated, across stop and resume cycles, state that used to simply vanish when the session ended. That is not semantic memory. It is the agent’s desk, preserved. The design lesson is to keep those two layers distinct: the runtime can own session durability, the working filesystem an agent resumes into, while your memory store owns distilled knowledge. Conflate them and you will end up either stuffing transcripts into a knowledge store or trying to make a knowledge store hold a filesystem, and both go badly. The substrate spectrum runs from SQLite-and-full-text at the pragmatic floor, through object storage and Git for cheap auditable history, up to managed runtime storage for working state, and the right answer is usually a combination, chosen per layer, not one store asked to do everything.

    Now the fifth decision, and the one I would argue is the most under-respected: retrieval. The reason it is under-respected is that the default is so easy it does not feel like a decision. Embed the query, embed the memories, return the nearest neighbors by cosine similarity, done. And that default is wrong often enough, in a specific way, that the entire production frontier has moved off it. The problem is that cosine similarity measures surface semantic closeness, and surface closeness is not relevance. A memory can sit right next to your query in embedding space and be completely useless for answering it, while the memory you actually need, the one whose connection to the question runs through an inference rather than through shared vocabulary, sits far away and never surfaces. A 2026 paper, AdaMem, makes this its whole thesis: memory systems lean too hard on semantic similarity, which misses user-centric evidence, and they store related experiences as isolated fragments, so the one relevant thing is both ranked wrong and disconnected from its context. The blunt version comes from a production thread in the corpus: people report that vector-DB RAG, summary-plus-embedding hybrids, all of it works for demos and then breaks once the agent runs a while, because it keeps pulling stale context purely on semantic closeness.

    So what do the systems that have actually solved this do? They stop doing a single lookup and run several channels in parallel, then fuse the results. Cloudflare’s Agent Memory is the cleanest public blueprint, and it is worth naming all five channels because the decomposition is the lesson. One: full-text search, the lexical channel, for exact terms. Two: exact fact-key lookup, a direct hit on a structured key. Three: raw message search, going back to the literal transcript. Four: direct vector search, classic dense semantic retrieval. Five, and this is the clever one: a HyDE channel, where the system generates a hypothetical declarative answer to the query and embeds that, to catch the case where the question and the answer share no vocabulary at all. Five channels run at once, and then the results are merged with Reciprocal Rank Fusion, RRF, which combines them by where each result ranked within its own channel rather than by raw scores you cannot compare across channels. And the weighting tells you their model of relevance: the exact fact-key match gets the highest weight, because an exact topic hit is the strongest possible signal, while raw message matches get a low weight as a safety net, a backstop to catch things the extraction pipeline missed. There is even a tidy engineering detail in their model choices, a smaller mixture-of-experts model for extraction and classification and a much larger one reserved for synthesis only, because they found the big model only earned its cost at the final synthesis step.

    This is not one vendor’s idiosyncrasy. It is convergent. Mem0’s own State of Agent Memory writeup this year credits multi-signal retrieval, running semantic similarity, keyword matching, and entity matching in parallel rather than in sequence, as one of two changes that drove their benchmark gains, reporting numbers like ninety-two and a half on LoCoMo and ninety-four point four on LongMemEval at around sixty-nine hundred tokens a query, against a full-context baseline that burned roughly twenty-six thousand tokens to score worse. The research front is doing the same thing with more structure: a bi-temporal engine called Engram retrieves through four parallel channels, dense semantic, BM25 lexical, graph traversal from the query’s entities, and recency-slash-salience, fuses them with RRF, and then assembles a deliberately hybrid context of conflict-resolved facts plus raw session chunks, because, they show, facts alone lose recall. The pattern to internalize: retrieval is not a lookup, it is an ensemble. Decompose relevance into the signals that actually carry it, run them in parallel, fuse with rank fusion, and keep a raw-text safety channel so the extraction pipeline’s misses do not become the system’s misses. If you build one thing from this episode well, build the retrieval ensemble.

    There is a subtler retrieval decision riding alongside that one, and it is about when to retrieve at all. The reflexive design retrieves on every step, RAG-at-every-turn, and a 2026 paper, “To Retrieve or To Think,” calls that out as a rigid, brute-force strategy that wastes compute and can actively degrade performance by flooding the context with retrieved noise the model then has to fight through. The reframe is to make retrieval a policy decision the agent makes, retrieve when you need external evidence, reason from what you already hold when you do not. That decision, retrieve versus think, is a lever most designs leave permanently jammed in the on position, and turning it into an actual choice is both cheaper and, often, more accurate.

    The sixth decision is temporality, and it is the one that breaks more production systems than anything else without anyone seeing it coming, because it looks solved until the day a user updates a fact. Here is the failure, lifted straight from a corpus thread: someone is running Mem0, their user changes a piece of information, and the agent develops amnesia about the timeline, can’t tell that the new fact supersedes the old one, ends up holding both, or surfacing the dead one as if it were current. The naive design has memory as a flat set of facts with no time axis, so when reality changes there is no principled way to know which version is current. The fix the field has landed on is bi-temporal modeling: track two clocks, not one. Valid time, when something was actually true in the world, and transaction time, when your system learned it. Aurra, which that same thread reaches for as the upgrade, differentiates exactly on bi-temporal modeling, and the Engram paper makes it first-class: a contradicted fact is not deleted, it is invalidated, with an invalid-at timestamp set and a supersedes pointer kept, so a point-in-time query, what did we believe was true as of last March, resolves correctly against history. Engram reports its knowledge-update and temporal-category scores rest precisely on that bi-temporality being built in rather than bolted on.

    The design decision underneath is sharper than just adding timestamps. When a user’s fact changes, you have three options, and they are not equivalent. You can silently overwrite, which is the default and which destroys your ability to ever answer an as-of question or audit what changed. You can version, keeping both with a currency marker. Or you can supersede, the bi-temporal move, marking the old fact invalid-from a moment while keeping it queryable. Silent overwrite is the one that feels fine until it doesn’t, because the day someone asks why the agent did what it did six weeks ago, the history is gone. And there is a research caution stacked on top of this: the contradiction-detection problem is hard, and the “Useful Memories Become Faulty” finding means you cannot just throw every fact update at an LLM and trust it to reconcile cleanly, because that reconciliation is exactly where the drift creeps in. Detecting that two facts conflict, deciding which wins, and recording the supersession without corrupting the record is a real piece of engineering, not a property you get for free from your store.

    Seventh decision: forgetting. And the reframe I want you to take from this one is that forgetting is not primarily a cost optimization. It is a correctness and safety requirement, and it is almost completely unmeasured. The default non-decision is that memory only grows. You write and you write and nothing ever leaves, and the store slowly rots, accumulating stale facts and dead context that drag every retrieval down. The bio-inspired research cluster offers an elaborate lifecycle as the alternative: a human-inspired architecture this year proposes sleep-phase consolidation, interference-based forgetting, engram maturation, reconsolidation on retrieval, the full neuroscience toolkit, turning forgetting from a crude time-to-live into a set of principled mechanisms. SuperLocalMemory bundles biologically-inspired forgetting with multi-channel retrieval in a zero-LLM local package, opening on the paradox that a coding agent can hold vast parametric knowledge and still not remember what happened an hour ago.

    But the sharpest result on forgetting is the one that reframes it as safety. PersistBench, a 2026 benchmark, points out that persisting a fact like the user is vegetarian helps personalization and also introduces a safety risk that is largely overlooked, and it sets out to measure when persistence becomes a liability, when a memory should be forgotten rather than kept. And a paper this year, with the unwieldy name about observability-safe memory retention, treats what-to-forget as the primary decision rather than a side effect of retrieval. It trains an evidence learner offline from gold-evidence labels, not from an LLM’s guess at importance, to decide what to retain, and crucially it runs deliberately below full capacity, stopping early when no remaining candidate looks useful. That last detail matters: the system chooses to hold less than it could, because holding more is a liability, not a virtue. The throughline for your design: build an explicit lifecycle, decide on purpose what leaves and when, and treat some forgetting as mandatory for safety rather than optional for cost. The reason this is so under-built is the reason it is dangerous: there is almost no benchmark pressure on it. As Mem0’s own writeup admits, nearly every public benchmark grades the retrieval step, and the write step, deciding what is even worth keeping out of a conversation that is mostly noise, is barely measured at all.

    The eighth decision is who the memory belongs to: per-user, per-agent, or a shared team profile. The single-user case is the easy one and not where the interesting tradeoffs live. The moment you have multiple agents collaborating, you face a real choice. Do they each keep private memory, or do they read and write a shared store? Cloudflare productized the shared answer with shared memory profiles, letting multiple agents access common knowledge, which is exactly what you want when a fleet of agents should not each independently rediscover the same fact. But sharing memory across long-running agents reintroduces the coherence problem at the team level, and the naive version, every agent dumping its full trace into a common pool, is a disaster of noise and contradiction. Slack’s three-channel design is one answer to keeping a multi-agent system coherent without that dump. A 2026 paper, DeLM, decentralized multi-agent systems with shared context, is another and a cleaner statement of the principle: instead of dumping full traces or routing everything through one main agent, the team shares a single verified context, agents read compact gists by default and unfold detail only on demand, and there is admission-time verification gating what is even allowed to enter the shared state. That gate is the same idea as Slack’s critic, applied to the team store: shared memory needs a bouncer, or it fills with garbage and one agent’s hallucination becomes every agent’s premise.

    And there is a hard operational fact about shared, multi-agent memory that the corpus surfaces and that the framework-published benchmarks tend to hide. A paper this year, on the cost and accuracy of long-term memory in distributed multi-agent systems, builds a testbed across cloud and edge and runs the comparison everyone actually wants, vector-based Mem0 against graph-based Graphiti and Zep, on system-level cost and accuracy, not just the tokens and latency the framework vendors report in their own evals. That distinction, system-level cost versus framework-reported cost, is the whole game when you scale to many agents, because the costs that kill you, the cross-agent coordination overhead, the consistency machinery, show up at the system level and are invisible in a single-agent benchmark. If you are splitting work between a vector store and a graph store across a multi-agent deployment, that head-to-head is the most decision-useful thing in the corpus.

    Which lands us at the ninth decision, the one that turned into a real market in the last twelve months: build versus buy. For most of this field’s short history there was nothing serious to buy, so the decision was trivial, you built. That is no longer true, and the change is recent enough that I want to give you the actual landscape as it stands in mid-2026. On the framework side, the named incumbents are Mem0, Zep, LangMem, and Letta, the open-or-self-hostable layers that pioneered the category. And then, in roughly the last year, every hyperscaler shipped a managed memory service and turned memory from a thing you build into a line item you provision. Amazon’s Bedrock AgentCore Memory went generally available, with short-term and long-term tiers, asynchronous extraction, and this year added metadata so you can tag and filter long-term records alongside semantic search, plus streaming notifications so you stop polling for memory changes. Google’s Vertex AI Memory Bank went generally available too, and here is the detail that tells you the market has matured: on January twenty-eighth this year, Google started charging for it, twenty-five cents per thousand stored events or memories. When a cloud provider starts metering a feature per thousand units, it has graduated from demo to infrastructure. Microsoft, at Build this year, pushed somewhere the others had not: procedural memory in Foundry Agent Service. Not just facts and preferences, but successful execution patterns, captured as structured items that record both when to use a procedure, the task context and preconditions, and what to do, the ordered actions and required checks, then retrieved and injected when a similar task appears. Their early numbers are real and modest, on the order of seven to fourteen points of absolute success-rate gain on Tau-bench at near-baseline cost, and they paired it with a governance surface, a portal where developers can view stored memories and do CRUD on individual items, plus time-to-live controls. That governance and TTL pairing is the tell: this is memory built for people who have to answer to compliance, not just to a benchmark.

    So how do you make the build-versus-buy call now that buying is real? The managed services buy you asynchronous extraction, durability, and increasingly a governance surface, all things that are tedious and easy to get wrong. What they cost you is control over the representation and the retrieval logic, which, per everything in the first eight decisions, is exactly where the differentiation lives. The corpus also carries a strong counter-current of builders going the other way on purpose: the SQLite-and-full-text floor, the Git-and-S3 substrate, the self-hosted conversation archives, all motivated by wanting to own the bytes and the logic. The real framing is that this is a layered decision, not a single one. You might buy durable session storage from the runtime, build your own retrieval ensemble because that is your edge, and lean on a managed extraction pipeline for the consolidation you do not want to babysit. Buy the plumbing, build the part that is your product.

    The tenth decision is the one that bites the day after you pick a vendor: interop. Because here is the uncomfortable fact the corpus states plainly. Mem0, Letta, Cognee, Zep with Graphiti, MemoryOS, MemTensor, each ships its own SDK, its own storage layout, its own vocabulary, and there is no shared wire format among them. The consequence is brutal and concrete: every integration is bespoke, every migration rebuilds your memory from scratch, and, the part that should alarm anyone in a regulated shop, none of them ships a governance surface to review what gets written and read. You do not just get locked in. You get locked in with no audit trail. The proposed fix in the corpus is memorywire, a vendor-neutral wire format for agent memory operations, and in the last weeks a second, complementary effort surfaced in the fresh research: Portable Agent Memory, which positions memory as the third leg of an interoperability stack, MCP standardizing how agents reach tools, A2A standardizing how agents delegate to each other, and portable memory standardizing how agents transfer accumulated knowledge, with a defined set of operations, remember, recall, forget, merge, expire, and a concrete artifact format, human-readable JSON by default and a compact binary option for constrained transport. Two independent groups converging on the same gap in the same quarter is the field telling you this matters and is not yet solved.

    There is a quieter, human-facing companion to the wire-format problem, which is schema. A good writeup in the corpus argues that agent memory is only as good as its schema, that memory quality is bounded by schema quality, and that if you get the schema wrong, no amount of clever retrieval or consolidation downstream can recover what the schema failed to capture. That reframes interop as not merely a portability convenience but a design discipline: an explicit, reviewable schema is the thing that makes your memory both migratable and auditable, and the absence of one is why DIY unification efforts, the builders in the corpus gluing Mem0 and Memori and Supermemory together by hand, are so painful. They are reconciling three implicit schemas that were never meant to meet. So the interop decision, even if you are nowhere near switching vendors, is really a decision to make your schema explicit now, while it is cheap, rather than discover it implicitly later, when it is load-bearing and undocumented.

    Interop bleeds directly into governance and security, which I am treating as the eleventh region of the tree because in production they are inseparable, and because the corpus is blunt that the memory frameworks largely lack a governance surface entirely. Start with multi-user isolation, the most basic and most violated invariant: one user’s memory must never surface in another user’s context. It sounds trivial and it is a frequent, expensive breach, because a shared retrieval index without hard per-user scoping will happily return a neighbor’s nearest-neighbor. But the threat that the persistent-memory design specifically creates, the one that does not exist for a stateless agent, is memory poisoning. And the distinction from ordinary prompt injection is the whole point. Prompt injection corrupts a single conversation, a single response, and then it is gone. Memory poisoning embeds the malicious content into persistent storage, so it remains, indefinitely, influencing every future interaction. The security writeups this year are sharp about a consequence builders miss: the standard defense against prompt injection is session isolation, every conversation starts from a clean context, and that defense does nothing against memory poisoning, because the poison lives in the store the clean session reads from. The mechanism is uglier than it sounds. An attacker does not need to talk to your agent directly. They plant the instruction in a document, a web page, a support ticket, anything the agent will later read, and if your consolidation pipeline extracts a fact from that poisoned source and writes it to long-term memory, the injection has installed itself permanently. The next clean session reads it back as established truth, and the agent has no way to tell a planted memory from an earned one, because by the time it is retrieved they are byte-identical. Worse, in a multi-agent system the poison propagates: a corrupted memory in a shared profile influences every agent that reads it, and one agent’s compromised premise spreads through the fleet by normal message passing. Memory is the one component that lets an attack outlive the conversation it arrived in.

    The design implications are concrete and they are the same set of moves the better systems already make for other reasons, which is the good news. Hard per-user and per-tenant scoping on every retrieval, enforced at the store, not in the prompt. An admission gate on writes, the same critic-style validation Slack uses for coherence, doing double duty as a security control, because the gate that checks whether a distilled memory is true against evidence is also the gate that catches an injected instruction trying to install itself as a fact. Provenance threaded through every memory, so you can answer where this came from and revoke a poisoned source, which is exactly the provenance that the Engram representation carries on every fact and that the Git-based substrates give you as version history. Snapshot and rollback, so you can recover the store to a known-good state after a poisoning event. And the governance surface itself, the human-auditable view over what gets written and read that the wire-format paper calls out as missing and that Microsoft’s Foundry portal is one of the first managed offerings to ship. The pattern is that the coherence machinery, the temporal machinery, and the security machinery are largely the same machinery: validation gates, provenance, supersession, audit. Build them once, for any of those reasons, and you have most of what you need for all three. The systems that treat memory as a passive store and bolt security on later find that there is nowhere to bolt it, because there is no gate, no provenance, and no audit surface to bolt it to.

    Now the part that ties every one of these eleven decisions in a knot, the thing this whole episode opened on: how do you know any of it works? Evaluation is the decision that audits all the others, and the corpus is unanimous and a little alarming about how badly the field has been doing it. The core mistake is measuring answer correctness and calling it memory quality. There is a beautifully clean demonstration of why that fails in a paper on structured belief state and precision-aware benchmarking. The observation: if you just return the entire belief store on every query, you get perfect recall, you pass the answer-quality eval, and you have built a useless retrieval system, because dumping everything is not retrieval. Which means answer correctness cannot validate a retrieval system at all. It is the unit-test-versus-integration-test problem. A green integration test, the right final answer, tells you nothing about whether the unit underneath, the retrieval, actually did its job, or whether the model just compensated for bad retrieval by reasoning over a pile of junk you handed it.

    And when you do separate the two, the result is genuinely surprising and it should redirect where you spend your effort. A 2026 paper, MemTrace, evaluates thirteen memory systems and separates retrieval-correctness from answer-correctness, and finds that when a system answers wrong, the evidence it needed was already retrievable about ten times more often than it was actually missing. Read that again, because it inverts the common intuition. The dominant failure is not retrieval. It is evidence use. The right memory was in hand, and the system still got it wrong. Systems with identical pooled accuracy fail in completely different places once you pull the two apart, which means the single accuracy number everyone reports is actively hiding where the problem is, pointing your optimization at storage and retrieval when the real bottleneck is the model failing to use evidence it already has. StreamMemBench operationalizes the same split with a four-metric design, separating whether evidence is retained from whether it is actually used, and warns specifically that a system can inflate its retention score just by hoarding raw text, the same dump-everything trap. And the benchmark frontier is moving past factual recall entirely: LoCoMo-Plus targets the beyond-factual setting, whether the agent honored implicit constraints, the user’s state and goals and values that were never explicitly queried later, which is precisely the user-centric relevance that AdaMem argued cosine similarity misses.

    There is one more evaluation finding I want to leave you with because it is the most humbling, and it comes from GitOfThoughts measuring when memory helps at all. They run a similarity sweep and find a copyability threshold. When the retrieved past case is a near-duplicate of the current problem, cosine similarity above roughly point-eight, accuracy jumps twelve to thirteen points. Below that threshold, nothing helps. And the gain, even at the top, is answer retrieval, not method transfer. The system is essentially finding a near-identical worked example and copying its answer. It is not extracting a transferable method from a related-but-different case, and a backbone four and a half times larger steepens the near-duplicate effect but still cannot pull a reusable method out of a worked example. That is a quiet, important result. A lot of what we call agent memory, measured rigorously, is sophisticated near-duplicate retrieval, and the thing we most want, learning a general lesson from one situation and applying it to a genuinely new one, the cross-trajectory abstraction the research literature calls the frontier, is exactly the thing these systems mostly cannot yet do. Measure your system rigorously and you may find it is a very good lookup wearing the costume of learning.

    Let me pull the tree together, because eleven decisions is a lot to hold and the shape of it is the takeaway. Start with the data model: sessions, turns, consolidated documents, raw traces as ground truth and distilled docs derived off the hot path, which is what all three hyperscalers independently shipped. Then consolidation: defer it, because eager-per-turn is a cost trap, and gate it, because LLM-driven rewriting drifts, which is the Slack distilled-truth lesson. Then representation: start at the simplest store that answers your queries, treat graph versus vector as load-bearing not fashionable, and respect the argument that atomic facts may be the wrong primitive. Then substrate: separate durable knowledge from durable working state, and do not dismiss boring object storage, which performs and audits better than you would guess. Then retrieval, the decision I would spend the most care on: an ensemble of parallel channels fused with rank fusion, never a single cosine lookup, with a raw-text safety net and a retrieve-versus-think policy. Then temporality: bi-temporal modeling, supersede rather than overwrite, so as-of queries and audits survive a fact changing. Then forgetting: an explicit lifecycle where some forgetting is a safety requirement, in a field with almost no benchmark pressure to do it. Then shared memory: a verified admission gate on the team store, because shared memory without a bouncer fills with one agent’s hallucinations. Then build versus buy: a layered call now that the hyperscalers have made buying real, buy the plumbing, build the retrieval that is your edge. Then interop: make your schema explicit now while it is cheap, because two separate standards efforts this quarter are telling you the lock-in is real. And governance and security woven through all of it: per-user isolation at the store, provenance on every memory, admission gates that serve coherence and security at once, because memory is the one component that lets an attack outlive its conversation. And over all of it, evaluation: separate retrieval-correctness from answer-correctness, because the single accuracy number lies, and the real bottleneck, ten times out of eleven, is using evidence already in hand.

    I will close where the field is genuinely stuck, because the open problems are sharper than the solved ones. First, the write step is unmeasured. We benchmark retrieval obsessively and barely measure what to keep, when to forget, how to reconcile a contradiction, which is to say we measure the easy third of the pipeline and look away from the hard two-thirds. Second, distilled memory degrades when an LLM maintains it, and our only real defense so far is to not fully trust the LLM, to keep raw traces and validate against evidence, which works but is an admission that we cannot yet let the system maintain its own memory unsupervised. Third, the systems we call learning are mostly near-duplicate lookup, and genuine cross-situation abstraction, the lesson learned once and applied somewhere new, is still out past the copyability threshold for everyone. Fourth, there is no shared wire format and no standard governance surface, so memory is non-portable and largely un-auditable at exactly the moment it is becoming the most security-sensitive component an agent has. And fifth, underneath all of it, we still cannot reliably tell, in production, whether a fluent answer rests on a real memory or a confidently wrong one, because they come out of the model sounding identical, and until we can attribute a failure to the precise stage that caused it, the write, the consolidation, the retention, the retrieval, or the use, we are tuning a pipeline by its final output and hoping. Memory is the feature everyone demos. The work that turns it into a feature you can trust is the part nobody can show you in five minutes, and it is most of these eleven decisions, made on purpose, measured rigorously, and gated every step of the way. That is the design problem. Go build it carefully.

Open problems

Where the literature is thin and the next contribution could land.

  1. One vector index, or parallel channels (recency, semantic, entity, summary, raw) fused with RRF?

  2. Is cosine similarity the right relevance signal, or does it surface the semantically-near-but-useless?

  3. When should the agent retrieve at all versus reason from what it already holds?

  4. Eager (consolidate every turn) or lazy/recurrent (batch, on idle, on retrieval)?

  5. Keep raw episodic traces alongside distilled facts, or replace one with the other?

  6. What experience is even worth keeping — and how should it change behavior, not just fill storage?

  7. Graph, vector, atomic facts, or events — and is the choice load-bearing or incidental?

  8. Do you model who/when/where (episodic coherence) or only what (semantic recall)?

  9. Is the atomic-fact paradigm (handcrafted prompts → compressed facts) actually the right primitive?

  10. When a user's fact changes, does the system supersede, version, or silently overwrite?

  11. Do you track both when something was true and when you learned it (bi-temporal)?

  12. How are contradictions detected — formally, or left to retrieval to disambiguate?