Multi-Agent Orchestration

Plain-language abstract This paper is a survey of systems where multiple AI agents, each powered by a large language model, work together to solve complex tasks or simulate real-world environments. It reviews how these multi-agent systems are built, how agents are given distinct roles and communicate, and what kinds of problems they have been applied to, from software development to social simulation.

Motivation Single LLM-based agents had shown promise for planning and autonomous decision-making, but there was no systematic overview of the rapidly growing body of work on multi-agent systems built from LLMs. Earlier efforts in the field proceeded independently, leaving researchers without a unified framework for understanding agent design, communication, and collective capability — a gap this survey addresses.

Methodology The authors conducted a broad literature survey, organizing existing work around a set of guiding questions: what domains and environments LLM-based multi-agents simulate, how agents are profiled and how they communicate, and what mechanisms enable agents to grow in capability. They also catalogued commonly used datasets and benchmarks, and tracked the rising volume of publications in the field over time using paper counts at three-month intervals across application categories.

Results The survey maps LLM-based multi-agent research into two high-level application areas — problem solving (including software development, multi-robot systems) and world simulation (including society, policy, and game simulation) — and documents a rapid growth in publication volume across these categories. The authors identify open challenges and maintain a living open-source repository to keep the community updated as the field evolves.

Plain-language abstract AgentVerse is a software framework that lets multiple AI agents—each powered by a large language model—work together on tasks, much like a team of people would. Instead of relying on a single AI assistant, the system dynamically assembles a group of specialized agents, has them discuss and plan, then carries out actions and checks whether the results are satisfactory.

Motivation Single AI agents struggle with complex real-world tasks that naturally require cooperation among multiple specialists. Prior multi-agent research was limited to narrow, fixed tasks and kept agent roles static, leaving open the question of whether a flexible, general-purpose multi-agent system could outperform a lone agent across a wide variety of problems.

Methodology The framework divides problem-solving into four stages: expert recruitment (choosing and adjusting which agents participate based on current progress), collaborative decision-making (agents discuss strategies together), action execution (agents interact with the environment to carry out decisions), and evaluation (comparing the current state to the goal and feeding back results for the next round). Experiments were run on diverse tasks spanning text understanding, mathematical reasoning, code generation, tool use, and embodied AI in Minecraft, using GPT-4 as the backbone language model.

Results Multi-agent groups assembled by AgentVerse outperformed single-agent baselines across the tested task categories. Analysis of agent interactions also revealed the emergence of distinct social behaviors—both cooperative behaviors that improved group efficiency and harmful behaviors that required mitigation—offering insight into how group dynamics in LLM-based systems can be shaped.

Plain-language abstract This paper proposes a framework for making large language models (LLMs) more capable by having multiple AI agents work together, each with its own role and strengths. The framework is illustrated through case studies involving existing AGI-oriented models such as Auto-GPT, BabyAGI, and the Gorilla model, which connects LLMs to external APIs.

Motivation Standard LLMs like GPT-4 operate in isolation and cannot collaborate with other agents or draw on external knowledge, which limits their usefulness for complex tasks requiring coordination and information sharing. The paper addresses this gap by designing a multi-agent architecture that mirrors the human principle of division of labor, aiming to push AI systems closer to artificial general intelligence.

Methodology The framework models the multi-agent environment as a graph where nodes represent Intelligent Generative Agents (IGAs) and plugins, and edges represent communication channels between them. Each agent is formally defined as a tuple capturing its language model instance, role, current state (knowledge and thoughts), ability to spawn new agents, and authority to halt other agents. The system is designed as a dynamic black-box environment in which agents interact, exchange information, create subtasks, and request assistance from one another, with GPT-4 and GPT-3.5-turbo serving as the underlying LLMs.

Results The paper demonstrates the framework's applicability through domain case studies including courtroom simulations and software development scenarios. It shows that combining agents with complementary capabilities — for example, one agent with persistent memory and another with internet access — can reduce hallucinations and improve output reliability. The framework addresses practical challenges including looping issues, security risks, scalability, system evaluation, and ethical considerations.

Plain-language abstract This paper presents DyLAN (Dynamic LLM-Powered Agent Network), a framework that automatically assembles and coordinates teams of AI agents to solve tasks. Instead of using a fixed group of agents with rigid communication rules, DyLAN selects the most useful agents for a given task and lets them collaborate in a flexible, adaptive structure.

Motivation Existing multi-agent systems built on large language models use a fixed number of agents with static communication patterns, regardless of the task at hand. This inflexibility limits performance, as some agents may be irrelevant or counterproductive for certain tasks, and no principled method existed to select or adjust agent teams based on their actual contributions during collaboration.

Methodology DyLAN operates in two stages. In the first stage, Team Optimization, a preliminary collaboration run is conducted among a pool of candidate agents, and each agent is scored using an unsupervised metric called the Agent Importance Score, computed via a forward-backward message-passing algorithm on a temporal feed-forward network structure. The top-scoring agents form a smaller team. In the second stage, Task Solving, selected agents collaborate dynamically on the actual query, with an LLM-powered ranker able to deactivate low-performing agents mid-collaboration. The framework was evaluated on code generation, decision-making, general reasoning, and arithmetic reasoning benchmarks.

Results DyLAN outperformed strong baselines across all evaluated task types while using substantially fewer API calls — for example, achieving comparable or better accuracy than LLM Debate with 10.6% fewer API calls, and using only 36.6% of LLM Debate's API calls on arithmetic reasoning tasks. On specific subjects in the MMLU benchmark, agent selection improved accuracy by up to 25.0%. Ablation studies showed that a dynamically selected team of three agents could outperform both an unoptimized team and competing methods with four agents, and overall task performance improved by up to 6.7% after team optimization.

Plain-language abstract When multiple AI language model agents debate a question together, they can arrive at better answers than any single agent — but this approach is expensive because every agent reads every other agent's responses. This paper asks whether agents actually need to communicate with everyone, or whether a sparser network of connections can achieve the same or better results at lower cost.

Motivation Existing multi-agent debate (MAD) frameworks use fully-connected communication, meaning each agent reads all other agents' responses at every round. As the number of agents and debate rounds grows, the input context balloons, making inference increasingly expensive. There was no systematic study of whether this full connectivity is necessary, or whether limiting each agent's view to only a few neighbors could preserve — or even improve — debate quality.

Methodology The authors conducted experiments with GPT and Mistral models across text-only reasoning benchmarks (MATH, GSM8K) and multimodal tasks, comparing fully-connected MAD topologies against sparse alternatives such as neighbor-connected graphs. They also extended the framework to alignment labeling tasks using the Anthropic-HH dataset, and explored heterogeneous setups in which different agents are instantiated with different LLMs assigned to graph positions of varying centrality.

Results Sparse, neighbor-connected MAD matched or exceeded fully-connected MAD while cutting inference input token costs by over 40% on reasoning tasks. On the MATH dataset, the neighbor-connected configuration gained +2% accuracy over the fully-connected baseline while keeping the same accuracy on GSM8K. On alignment labeling, sparse MAD improved helpfulness by +0.5% and harmlessness by +1.0% compared to single-agent setups, while reducing costs by roughly 50%. Assigning stronger LLMs to higher-centrality positions in the communication graph consistently improved overall performance in multi-model debate settings.

Plain-language abstract Magentic-One is an open-source multi-agent AI system designed to complete complex, multi-step tasks that require using a web browser, managing files, and writing and executing code. A lead agent called the Orchestrator coordinates a team of specialized agents, dynamically planning and recovering from errors to accomplish high-level goals without needing task-specific tuning.

Motivation Existing agentic AI systems tend to be specialized for narrow domains or rely on rigid, single-agent workflows that struggle to generalize across the diverse tasks people encounter in everyday work. There was a need for a generalist system that could plan, reason across multiple steps, use varied tools, and recover from errors across a wide range of scenarios.

Methodology Magentic-One uses a multi-agent architecture built on the AutoGen framework, consisting of five agents: an Orchestrator, a Coder, a Computer Terminal, a FileSurfer, and a WebSurfer. The Orchestrator maintains two structured ledgers to track progress and assign tasks, and dynamically replans when errors occur. The system was evaluated on three agentic benchmarks — GAIA, WebArena, and AssistantBench — using AutoGenBench, a new standalone evaluation tool that enforces isolation and controlled initial conditions across runs to handle the side effects of agent actions.

Results Magentic-One achieved task-completion rates of 38% on GAIA and 32.8% on WebArena, and an accuracy of 27.7% on AssistantBench, placing it statistically competitive with state-of-the-art systems on all three benchmarks, including systems specialized for individual benchmarks. Ablation experiments confirmed the additive contribution of each agent to overall performance, and the modular design allowed agents to be added or removed without retraining or prompt tuning.

Plain-language abstract This paper presents MACI (Multi-Agent Collaborative Intelligence), a framework that coordinates multiple specialized AI agents to tackle complex planning tasks that a single large language model handles poorly. Rather than relying on one model to reason through everything sequentially, MACI uses a meta-planner to map out a task as a dependency graph and then assigns different agents to specific roles, including validation, common-sense checking, and domain-specific reasoning.

Motivation Large language models struggle with planning tasks that require tracking many constraints simultaneously, verifying their own outputs, and reasoning across time. They tend to lose track of earlier constraints as a conversation grows longer, a problem called attention bias or constraint drift, and they lack any mechanism to check their own reasoning for errors. Existing multi-agent frameworks focus on coordinating LLMs but do not address these fundamental planning weaknesses.

Methodology MACI uses a three-part architecture: a Meta-Planner that analyzes a task and generates a dependency graph encoding roles and constraints; Common Agents that handle general tasks such as constraint validation and common-sense integration; and Task-specific Agents that address domain requirements. Agents operate within restricted context windows of around 1,000 tokens to physically prevent attention bias. The framework was evaluated on two planning benchmarks: the Traveling Salesman Problem (TSP) and a multi-constraint Thanksgiving dinner planning scenario involving travel logistics, cooking times, and people-role assignments across three LLM backends (Claude, DeepSeek, GPT-4o).

Results When given the original problem specifications without MACI, all three LLMs failed to produce feasible solutions for the Thanksgiving dinner planning task. After integrating the MACI meta-planner to generate enhanced workflow specifications, all three LLMs successfully produced feasible schedules. The TSP experiments benchmarked standalone planners against their MP-integrated counterparts on solution quality and optimality, with the meta-planner integration consistently improving outcomes across all tested models.

Plain-language abstract This paper presents S-Agents, a system in which a group of AI agents powered by large language models organizes itself to tackle complex tasks in open-ended environments — specifically the game Minecraft — without requiring humans to design each step of their workflow. The agents coordinate through a tree-like hierarchy, splitting work, monitoring each other's progress, and executing tasks in parallel.

Motivation Most multi-agent AI systems rely on fixed, human-designed workflows that assign roles and sequences in advance, making them brittle in open-ended or unpredictable environments. The gap this work addresses is the absence of agent-centric organizational structures that can flexibly self-arrange without predefined human instructions, particularly when a large group must divide complex collaborative tasks autonomously.

Methodology The system uses three coordinated mechanisms: a 'tree of agents' structure where one leadership agent (root) directs multiple executor agents (leaf nodes); an 'hourglass agent architecture' that funnels perceptual inputs and organizational context into a unified objective, then decomposes it through a two-level hierarchical planner (task planner and action planner driven by GPT-4 and GPT-3.5-turbo respectively); and a 'nonobstructive collaboration' paradigm in which each agent runs as an independent asynchronous process and reports back to the root upon task completion rather than waiting for a synchronized round. Experiments were conducted in Minecraft 1.19 using the MineDojo framework, with agents initialized from Voyager's pre-trained skill library.

Results The tree-of-agents structure outperformed chain-of-agents (29.0 min) and graph-of-agents (9.3 min) configurations, completing a 50-resource mining task in 7.5 minutes with 3.8 mean planning iterations. For simpler tasks such as mining 50 or 100 logs, the multi-agent organization reduced completion time by 5.1 and 7.2 minutes compared to a solo agent; for harder tasks like collecting 50 or 100 iron, solo agents consistently failed (exceeding 40 minutes with no success), while the organized group succeeded. The agents also exhibited emergent human-like organizational behaviors including leadership delegation, project oversight, and strategic personnel assignment.

Plain-language abstract This paper presents the Hierarchical Language Agent (HLA), an AI system designed to cooperate with human players in real time in the cooperative cooking video game Overcooked. HLA combines a powerful large language model for reasoning and natural language communication with a lightweight model and a fast reactive policy, allowing the agent to both understand complex instructions and respond quickly enough for live gameplay.

Motivation Large language model-powered agents typically call cloud APIs, which introduces inference latency of seconds to minutes. This is acceptable for low-frequency tasks like code generation but makes them unsuitable for real-time interactive settings such as video games, where responses are needed at roughly 3 Hz. Existing fast gaming AI uses small reactive policies that lack the ability to understand language instructions or carry on meaningful dialogue with human partners.

Methodology The authors extend the open-source Overcooked cooperative cooking game environment with a natural language chat interface that allows human players to issue spoken or typed commands to an AI partner. HLA is built from three stacked modules: a Slow Mind (a capable LLM) that handles language understanding, intention disambiguation, and language replies; a Fast Mind (a lightweight LLM) that converts interpreted commands into macro actions; and an Executor (a reactive script policy) that translates macro actions into low-level game actions at high frequency. Human commands are routed simultaneously through Slow Mind and Fast Mind to minimize latency. The system is evaluated on action latency benchmarks, command-reasoning tasks, and human studies across four game maps.

Results Human studies showed that HLA achieved approximately 50% higher game scores than the strongest baseline and received the highest human preference ratings for cooperative ability, response speed, and communication consistency. HLA was an order of magnitude faster than the best competing agent in real-time action responsiveness. It also outperformed baselines significantly on command reasoning tasks involving both simple and complex natural language instructions.

Plain-language abstract DynTaskMAS is a software framework for running teams of AI agents—each powered by a large language model—on complex tasks in parallel and without waiting for one another. It breaks a hard problem into subtasks automatically, tracks which subtasks depend on which others, and runs as many as possible at the same time, so the overall job finishes faster and the AI models stay busier.

Motivation Existing multi-agent systems built on large language models typically use simple, fixed workflows where agents work one at a time or in rigid sequences. This leaves computing resources underused and makes it hard to tackle tasks whose structure changes as work progresses, creating a need for frameworks that can decompose and schedule work dynamically.

Methodology The framework is built around four integrated components: a Dynamic Task Graph Generator that decomposes an input task into subtasks while preserving their logical dependencies; an Asynchronous Parallel Execution Engine that schedules those subtasks for concurrent execution; a Semantic-Aware Context Management System that shares relevant information among agents efficiently; and an Adaptive Workflow Manager that adjusts execution in real time. The work was presented as a conference paper at the 35th International Conference on Automated Planning and Scheduling, and performance was evaluated experimentally across varying task complexities and agent counts.

Results DynTaskMAS achieved a 21–33% reduction in execution time compared to traditional approaches, with larger gains for more complex tasks. Resource utilization improved from 65% to 88%, a 35.4% increase. Throughput scaled near-linearly up to 16 concurrent agents, reaching a 3.47× improvement when using 4× the agents.

Plain-language abstract AgentNet is a new framework for coordinating multiple AI agents without a central controller. Instead of having one orchestrator assign tasks to agents with fixed roles, AgentNet lets agents self-organize in a network, route tasks to the most capable peers, and continuously improve their own skills by learning from past task trajectories.

Motivation Most multi-agent AI systems rely on a central controller that assigns tasks to agents with static, predefined roles. This creates a single point of failure, limits scalability as workloads grow, and prevents organizations from collaborating across institutional boundaries because sharing data with a central coordinator raises privacy concerns. A decentralized alternative that allows dynamic task routing and privacy-preserving collaboration was needed.

Methodology AgentNet arranges agents in a Directed Acyclic Graph (DAG) topology whose connections adapt in real time based on task demand and agent performance. Each agent has a router module that decides whether to execute a task, forward it to a more suitable agent, or split it into subtasks delegated to others. Agents maintain a retrieval-augmented memory pool of successful task trajectories; when a new task arrives, relevant past trajectories are retrieved via embedding similarity to guide the agent's response, and lower-value trajectories are pruned to stay within capacity. The system was evaluated on three benchmark categories—mathematics (MATH dataset, 140 test problems), coding (APPS, 100 problems), and logical question answering (BBH, 100 problems)—using GPT-4o-mini as the backbone LLM and compared against both single-agent baselines and centralized multi-agent frameworks.

Results AgentNet matched or outperformed all baselines across the three task domains. On the MATH benchmark it reached 85.00% accuracy compared to 73.57% for MegaGPT and 77.14% for the best single-agent method. On BBH logical QA it achieved 86% accuracy versus 75% for AFLOW and 53% for MegaGPT. An ablation study confirmed the adaptive learning warm-up phase was critical: removing it dropped MATH accuracy from 85.00 to 77.86 and BBH from 86% to 76%. Scalability experiments showed that performance increased gradually as more agents and larger executor pools were added, with training-phase BBH scores rising from 80.38 (3 agents) to 81.18 (9 agents), and agents autonomously developed task-specific specializations without any centralized direction.

Plain-language abstract This paper argues that how multiple AI agents coordinate with each other is itself a design choice that can be studied and optimized, separate from what those agents know or how capable they are. The authors test five different coordination patterns on a real-world prediction-market task, measuring accuracy, cost, and calibration to see which architectural choices lead to which failure patterns.

Motivation Multi-agent LLM systems fail in production at rates between 41% and 87%, and most of those failures trace to coordination problems—ambiguous task delegation, misalignment between agents, and verification gaps—not to the underlying model's intelligence. Despite a growing catalog of failure modes, no principled mapping exists from specific coordination configurations to the predictable failure signatures they produce; existing frameworks treat coordination as an engineering convenience rather than as a measurable architectural variable.

Methodology The authors held a single LLM (claude-opus-4-6), a fixed tool stack, a fixed per-call output cap, and a fixed prompt template constant across five coordination configurations—sequential pipeline, independent ensemble, orchestrator-specialist, peer-critique-debate, and consensus alignment—tested on 100 Polymarket binary prediction markets resolved after the model's training cutoff. Total compute per question was treated as an outcome variable, not a fixed input. The Murphy decomposition was used to separate calibration error from discriminative power, and a bootstrap power-projection analysis quantified which architectural contrasts were statistically resolvable at the observed sample size. The same five configurations were also deployed as live agents on Foresight Arena for on-chain replication.

Results Three of five pre-specified Murphy-signature predictions were upheld in the predicted direction. On the cost-quality Pareto frontier, independent ensemble ($0.10/market, Brier 0.159) and sequential pipeline ($0.36/market, Brier 0.153) dominated the other three configurations; orchestrator-specialist and peer-critique-debate were both dominated, costing more while delivering worse Brier scores. Category-conditional analysis showed that architectural differences mattered far more in economics markets (0.121 Brier spread across configurations) than in sports markets (0.012 spread). Bootstrap confidence intervals identified consensus alignment as the most clearly separated configuration, with three pairwise contrasts excluding zero at the 95% level, though no pair survived Bonferroni correction at n=100.

Plain-language abstract This paper addresses how to connect large groups of autonomous agents — such as drone swarms or fleets of vehicles — so they can share information quickly without overwhelming the network. The authors propose CayleyTopo, a new family of communication network layouts derived from algebraic graph theory, whose structure is optimized by a machine-learning algorithm rather than chosen by hand.

Motivation Large-scale multi-agent systems cannot use fully connected networks because the number of required communication links grows quadratically with the number of agents, causing congestion. Existing sparse network layouts rely on handcrafted rules (such as connecting each agent to peers at fixed exponential distances), leaving open the question of whether those rules are actually optimal and prompting the authors to treat the network structure itself as something to be designed and optimized.

Methodology The authors model multi-agent communication networks as circulant Cayley graphs parameterized by a set of generator steps, and frame topology design as a discrete combinatorial optimization problem that minimizes graph diameter under a fixed degree budget. To search the enormous space of candidate generator sets efficiently, they develop a lightweight reinforcement learning framework that incorporates two key components: a number-theoretic prior based on multiplicative order to bias the search toward structurally rich generators, and a message-propagation score that provides dense connectivity feedback during construction.

Results CayleyTopo consistently outperforms rule-based baseline topologies — including ExpoComm, Fibonacci, simple-prime, and broadcast designs — across information dissemination speed, resilience to link failures, and communication load. In cumulative communication-count experiments, CayleyTopo achieves approximately 40% faster average dissemination while incurring the lowest average per-step bandwidth usage, and its discovered graph diameters approach the theoretical Moore lower bound.

Plain-language abstract This paper proposes a two-dimensional classification system for AI agent design patterns, combining what an agent does (cognitive function) with how it is structurally organized (execution topology). The resulting 7-by-6 matrix identifies 27 named patterns and provides a shared vocabulary for describing, comparing, and designing large-language-model-based agent systems.

Motivation Existing frameworks from major AI organizations each describe agent architectures from only one perspective: industry guides focus on how data flows (execution topology) while cognitive science surveys focus on what agents do (cognitive function). This single-axis view fails to distinguish architecturally distinct systems — for example, the same Orchestrator-Workers topology can implement task planning, hierarchical delegation, or observability monitoring, three patterns with fundamentally different failure modes — leaving architects without a principled, framework-neutral vocabulary.

Methodology The authors define two independent axes: a Cognitive Function axis with seven categories (Context Engineering, Memory, Reasoning, Action, Reflection, Collaboration, Governance) and an Execution Topology axis with six structural archetypes (Chain, Route, Parallel, Orchestrate, Loop, Hierarchy). They take the Cartesian product to produce a 42-cell matrix, populate 27 cells with named patterns (13 with original names), demonstrate that the two axes are orthogonal through systematic cross-axis analysis, and validate the framework's coverage by mapping four real-world deployment domains — financial lending, legal due diligence, network operations, and healthcare triage — onto the pattern catalog.

Results The framework identifies 27 named patterns across the 7-by-6 matrix, with 15 cells either structurally redundant or not yet observed in practice. Cross-domain analysis yields five empirical laws of pattern selection: time pressure favors Chain-of-Thought and Loop patterns; action authority favors Blast Radius Control; asymmetric failure costs shape how Reflection is parameterized; volume determines which collaboration patterns are needed; and the same pattern (such as Generator-Critic) appears across all four domains but with domain-specific parameterization. Four patterns — Context Triage, RAG Pipeline, Complexity-Based Routing, and Generator-Critic — appear in three or more domains, suggesting they are foundational to most production agent systems.

Plain-language abstract ARIADNE is a system that uses a search algorithm called Monte Carlo Tree Search (MCTS), guided by a shared memory structure called a blackboard, to automatically write correct and efficient solutions to competitive programming problems. Rather than generating code in a single pass, it organizes the process into five stages — picking an algorithm strategy, writing code, generating tests, evaluating quality, and repairing bugs — and continuously learns from execution feedback to steer the search toward better solutions.

Motivation Large language models can write general code but consistently fail at competitive programming, which demands precise algorithm selection, multi-step logical reasoning, and handling of tricky edge cases under strict time and memory limits. Existing approaches using agentic pipelines, search-based exploration, or shared workspaces each address part of the problem but leave critical gaps: rigid workflows break when early assumptions are wrong, search methods don't reuse evidence across attempts, and shared-workspace designs lack a global planner to allocate effort based on what has been learned.

Methodology ARIADNE frames competitive program generation as a sequential decision-making process navigated by MCTS, where each node represents a state of the shared blackboard — a persistent store of intermediate results, diagnostics, and failure evidence. Five types of actions (strategy selection, code generation, test generation, quality evaluation, and code repair) are explored by the tree search, with rewards backpropagated to guide future decisions toward branches most likely to produce correct, efficient code. The system was evaluated on four benchmarks — APPS, CodeContests, CodeContests+, and LiveCodeBench — using GPT-4o and DeepSeek-V3.2 as backends, and also tested on real contest instances from the 2025 ICPC Asia Shenyang Regional Contest and the 2025 CCPC Fujian Invitational.

Results With GPT-4o, ARIADNE achieved Pass@1 scores of 41.30% on APPS, 46.67% on CodeContests, 27.27% on CodeContests+, and 20.91% on LiveCodeBench, outperforming the strongest baseline (CodeSim) by up to 26.06 percentage points. On the 2025 ICPC Shenyang Regional Contest, ARIADNE solved 3, 6, and 7 out of 13 problems at pass@1/3/5 respectively, compared to 1, 4, and 5 for the best baseline, with similar gains on the CCPC Fujian Invitational. Further improvements were observed when using DeepSeek-V3.2 as the backend.

Plain-language abstract This paper studies how groups of AI language model (LLM) agents form shared conventions — agreeing on a common choice when many options are equally valid — and how the design of their communication network and memory length shapes whether they reach full agreement, fragment into competing factions, or settle into stable but divided states.

Motivation As multi-agent systems built on LLMs become more capable, two key design parameters — how much past interaction each agent remembers, and how agents are connected in a network — are typically optimized separately. Prior work had shown LLM agents can develop shared conventions in fully connected settings, but real systems use constrained network topologies and bounded memory, and the interaction between these two design choices was unknown.

Methodology The authors ran 432 simulations of a networked Naming-Game on eight fixed 16-node network topologies (varying in clustering and path length) drawn from a benchmark set by Mason and Watts. In each simulation, LLM agents repeatedly chose one of ten arbitrary convention letters and were matched with network neighbors; matched choices earned points, mismatches lost points. Memory depth was varied across three values (last 2, 5, or 10 interactions), yielding a fully balanced 8-topology × 3-memory factorial design with 18 independent runs per cell and 691,200 total dyadic decisions. Agent behavior was compared against Fictitious Play and Reinforcement Learning baselines.

Results Longer memory slowed convergence in decentralized networks but accelerated settling in centralized ones, meaning the same parameter pushes outcomes in opposite directions depending on topology. Critically, faster settling in centralized networks meant locking into fragmented multi-convention plateaus, not global consensus. Fictitious Play outperformed Reinforcement Learning in predictive accuracy across all memory depths, indicating agents adapt via belief-based rather than reward-based reasoning. At the agent level, high-betweenness bridge nodes suffered a coordination penalty while locally clustered agents achieved higher coordination success.

Plain-language abstract This paper addresses how to automatically design better multi-agent systems built from large language models (LLMs). The authors introduce MASS (Multi-Agent System Search), a framework that jointly optimizes both the text instructions (prompts) given to individual agents and the structure (topology) describing how agents interact with one another, rather than relying on hand-crafted configurations.

Motivation Building effective multi-agent systems from LLMs requires carefully designed prompts for each agent and carefully chosen interaction topologies, but doing this by hand is complex and does not scale. Most prior work either optimized prompts or topologies in isolation, or still relied on handcrafted prompts, leaving the full design space unexplored and suboptimal systems as a result.

Methodology MASS optimizes the multi-agent design space in three interleaved stages: (1) block-level local prompt optimization for individual agent types, (2) workflow topology optimization to find effective agent interaction structures, and (3) workflow-level global prompt optimization conditioned on the previously optimized topology and prompts. Each stage builds on the outputs of the prior stage. The authors also conducted analysis comparing automatic prompt optimization against common scaling strategies such as self-consistency, self-refinement, and multi-agent debate, measuring token-cost efficiency on benchmarks including MATH using Gemini 1.5 Pro.

Results MASS-optimized multi-agent systems outperform a spectrum of existing alternatives by a substantial margin. The analysis showed that prompt optimization is more token-efficient than simply scaling the number of agents with default prompts, and that applying self-consistency on top of prompt-optimized agents yields better scaling behavior than standard approaches like self-consistency or self-refinement alone. The work also produces a set of design principles for building effective multi-agent systems.

Plain-language abstract This paper introduces "generative agents" — software agents powered by large language models that can simulate believable human behavior over time. The researchers built a virtual town populated by twenty-five such agents, each capable of waking up, forming daily plans, holding conversations, remembering past events, and coordinating social activities, all driven by an underlying language model architecture.

Motivation Prior computational agents could simulate plausible human behavior at a single moment in time, but lacked the ability to maintain coherent, evolving identities across longer timescales. There was no architecture for managing a continuously growing memory of experiences, reflecting on those memories to form higher-level understanding, or coordinating emergent social dynamics among multiple agents simultaneously.

Methodology The authors designed a three-component agent architecture built on top of the ChatGPT language model. A memory stream stores all of an agent's experiences as natural-language records; a retrieval model surfaces relevant memories using a combined score of recency (exponential decay over time), importance (rated by the language model itself), and relevance. A reflection module synthesizes memories into higher-level inferences, and a planning module translates those inferences into daily and moment-to-moment behaviors. Twenty-five agents were instantiated in an interactive sandbox environment inspired by The Sims, and the system was evaluated through both controlled "interview" probes of individual agents and an open-ended two-day simulation.

Results Generative agents produced believable individual and emergent social behaviors. In one demonstration, a single seed instruction — that one agent wanted to throw a Valentine's Day party — led agents to autonomously spread invitations, form new acquaintances, ask each other on dates, and coordinate arrival at the party without further human input. Ablation studies showed that each architectural component (memory retrieval, reflection, and planning) was critical to believable behavior; removing any one of them degraded performance across interview-based evaluation tasks. The most common failure modes were agents failing to retrieve relevant memories, fabricating embellishments to memory, or defaulting to overly formal language inherited from the base language model.

Plain-language abstract This paper asks whether a group of simulated ants can learn to use pheromone trails to coordinate food-gathering without being given any explicit instructions about when or how to do so. Each ant is controlled by a spiking neural network — a type of artificial brain that processes information as discrete spikes, similar to biological neurons — and the network's connection weights are tuned by an evolutionary algorithm over many generations. The result is a colony that spontaneously discovers pheromone-based communication on its own.

Motivation Prior computational models of swarm coordination have relied on hand-crafted rules or probabilistic decision tables to produce collective behaviour such as foraging. Designing these rules is difficult, does not scale well as task complexity grows, and may steer the swarm toward solutions that are functional but not optimal. The paper targets the 'design problem': can useful collective strategies, including communication, emerge automatically from an optimisation process rather than from a designer's intuition?

Methodology The authors built a 2D simulated environment based on Wilensky's NetLogo ant-colony model, where pheromone can diffuse and evaporate across a grid. A colony of 15 ants is controlled by copies of a single spiking neural network (SNN) implemented in the NEST simulator. A genetic algorithm — run through the Learning to Learn (L2L) framework — optimises the synaptic weights and spike-time delays of that network across populations of 32 individuals per generation, evaluating each individual by how efficiently the colony retrieves food and returns it to the nest. No rule about pheromone use is pre-wired; the network starts with no knowledge of the task.

Results Through evolutionary optimisation, pheromone-based stigmergic communication emerged spontaneously: ants learned to deposit pheromone near food sources and near the nest without any pre-coded instruction to do so. Colonies in which this communication emerged outperformed those in which it did not, and disabling the pheromone sensor after training drastically reduced foraging performance, confirming that the emergent signals are functionally important. The evolved SNN-based colony achieved foraging performance comparable to a hand-designed rule-based multi-agent system. The authors also observed that pheromone acts as an attractor rather than an event-based signal, resembling the positive pheromone trails seen in real ant species.

Plain-language abstract This paper presents PheroCom, a coordination model for swarms of robots that mimics how ants use chemical signals (pheromones) to navigate and work together. Instead of relying on a central controller, each robot maintains its own local map of pheromone concentrations and shares updates with nearby robots through a gossip-like wireless communication protocol inspired by ant vibroacoustic signaling. The model was tested in simulation across multiple environments and swarm sizes to validate its ability to perform indoor surveillance tasks.

Motivation Most bio-inspired swarm robotics approaches that simulate pheromone dynamics require a centralising agent to maintain and distribute a shared pheromone map, which conflicts with the core swarm robotics requirement for decentralised, autonomous robots. Existing alternatives such as volatile chemicals, RFID tags, and intelligent environments are expensive, task-specific, or impose constraints that limit swarm autonomy. The paper addresses the gap of producing a fully decentralised and asynchronous pheromone-based coordination model that preserves the robustness, scalability, and flexibility expected of swarm systems.

Methodology The authors designed PheroCom around a finite-state machine governing each robot's behavior, with each robot holding an independent virtual pheromone map implemented as a cellular automaton grid. Robots deposit and sense pheromone locally, and periodically disseminate their local map data to nearby robots using the ViBIT (Vibroacoustic Based Indirect Transmission) protocol—a gossip-style broadcast limited to a configurable transmission radius. Aggregating robots filter incoming data using stored transmission histories to avoid processing duplicate information and only update cells where the incoming pheromone concentration is higher than the local value. The model was evaluated in a custom agent simulation framework built by the authors and in the Webots robotics simulator, using multiple indoor environment layouts (different shapes and sizes), varying numbers of robots, and three movement strategies (deterministic, inertial, and a 2/3 inertial / 1/3 deterministic heterogeneous mix).

Results PheroCom achieved surveillance coverage results comparable to the predecessor centralised model (IACA-DI) across all tested environments, swarm sizes, and movement strategies. The most striking quantitative finding is communication efficiency: PheroCom accomplished the same tasks using approximately 1.69% of the information volume transmitted by IACA-DI, a roughly 59-fold reduction in communication overhead. The heterogeneous movement strategy allowed the swarm to visit all rooms within one hour of simulation, and by five to ten hours both models produced similarly homogeneous environment coverage. The model demonstrated robustness and scalability by adapting to varied environment shapes and robot counts without architectural changes.

Plain-language abstract ATOM is a framework for coordinating groups of AI language-model agents to solve complex tasks. Instead of using a fixed, hand-wired communication structure, ATOM builds a stable core team of agents offline and then adds or removes additional agents at query time depending on how difficult the task appears. This design keeps costs down on easy questions while scaling up capacity for hard ones, achieving state-of-the-art accuracy across six benchmarks while using up to 30% fewer tokens than comparable systems.

Motivation Multi-agent systems based on large language models depend on well-designed collaboration graphs to balance reasoning quality against communication cost, but existing methods either use rigid static structures that waste compute on easy queries or use fully dynamic generation that faces an exponentially large search space. Both approaches also decouple the number of agents from actual query difficulty, causing over-spending on simple tasks and under-provisioning on hard ones. ATOM was designed to close this gap by explicitly aligning computational budget with task complexity.

Methodology ATOM models a multi-agent system as a dynamic spatial-temporal graph with two edge types: spatial edges for within-round synchronous communication and temporal edges for cross-round memory retrieval. It splits the agent pool into a nucleus (a stable backbone condensed offline via stochastic edge parameterization over a fully connected supernet) and electrons (additional agents activated online). A difficulty estimator predicts query complexity and sets a query-dependent token budget, which controls how many electron agents are instantiated; a task-driven reinforcement learning objective with structural regularization then optimizes the online routing policy within that budget. Experiments were run on six diverse benchmarks spanning reasoning and language understanding tasks.

Results Across six benchmarks, ATOM consistently achieved state-of-the-art accuracy compared to strong baselines while improving token efficiency by up to 30%. Analysis confirmed an average-complexity trap in static or uniformly dynamic systems: easy queries needed only a minimal agent count to saturate performance while hard queries benefited from larger budgets, validating the complexity-aware budgeting approach. The framework established a new Pareto frontier for multi-agent collaboration, delivering both higher accuracy and lower token cost than prior methods.

Plain-language abstract This paper introduces a mathematical diagnostic for choosing how to connect multiple AI language model agents before running them. By borrowing the "successor representation" from reinforcement learning and neuroscience, the authors show that three numbers derived from a communication graph's structure can predict, ahead of any actual computation, whether a given arrangement of agents will accumulate errors, fail to reach agreement, or crumble under noisy inputs.

Motivation When building systems that chain or network multiple large language models together, practitioners must choose a communication topology — chain, star, mesh, or other arrangements — with no principled way to anticipate which will cause reasoning errors, slow consensus, or brittleness under perturbation. Existing evaluation methods can only answer these questions after the pipeline has already run on a specific task, leaving topology selection as a costly trial-and-error process.

Methodology The authors model a multi-agent LLM system as a directed graph whose row-normalized adjacency matrix serves as a stochastic transition operator P, then compute the successor representation M = (I − γP)⁻¹. Three spectral quantities of M are derived in closed form for three canonical topologies (chain, star, mesh): the spectral radius ρ(M), the spectral gap Δ(M), and the condition number κ(M). Predictions from these quantities are validated on a 12-step structured state-tracking task using Qwen2.5-7B-Instruct over 100 independent trials per topology.

Results The condition number is a perfect rank-order predictor of empirical perturbation robustness (Spearman rs = 1.0); the spectral gap partially predicts consensus dynamics (rs = 0.5); and the spectral radius is perfectly inverted with respect to cumulative error (rs = −1.0). The inversion — dubbed the "stability paradox" — occurs because the chain topology, which is most stable in the linear spectral sense, allows each agent's small bias to accumulate sequentially, while star and mesh topologies include aggregation steps that suppress drift. An affine-noise extension of the predictive map recovers the correct empirical ordering of cumulative error across topologies.

Plain-language abstract This paper asks whether continuously adding more AI agents working together improves performance, the same way adding more neurons to a neural network does. The authors build a system called MACNET that organizes large language model (LLM) agents into a network shaped like a directed acyclic graph, where agents collaborate to solve tasks. They find that performance grows in a predictable pattern as the number of agents increases, and that networks with over a thousand agents can work effectively.

Motivation Most multi-agent AI research involves fewer than ten agents, with only a handful of studies extending to several dozen. It was unknown whether continuously scaling up the number of collaborating agents would yield sustained performance improvements, and there was no established framework for organizing hundreds or thousands of agents into an effective reasoning network.

Methodology The researchers built MACNET, a multi-agent collaboration network that arranges LLM-based agents as nodes in a directed acyclic graph. Each edge in the graph is managed by a supervisory critic agent issuing commands, while each node is handled by a compliant actor agent producing task outputs. This topology allows interactive reasoning to be orchestrated across large numbers of agents for autonomous task solving.

Results MACNET outperformed all baselines on average and successfully supported collaboration among over a thousand agents. Irregular network topologies unexpectedly outperformed regular ones. The study identified a collaborative scaling law: overall performance follows a logistic growth pattern as agent count increases, and collaborative emergence occurs earlier than the neural emergence seen in traditional scaling of model parameters.

Plain-language abstract This paper asks a simple but important question: when you build a team of AI agents all powered by the same underlying language model, does adding more agents reliably make the system smarter? The authors design a minimal framework called SIMAS (Sequential Iterative Multi-Agent System) to test this cleanly, running experiments across many tasks and model sizes to map out how performance actually changes as team size grows.

Motivation Multi-agent systems built on large language models have been touted as a way to tackle complex tasks that a single AI cannot handle alone, but almost all research has focused on engineering new architectures rather than understanding the basic science of how these systems scale. A key question had gone unanswered: is collective intelligence an automatic consequence of having more agents, or does it depend on something more specific about how agents are designed to interact?

Methodology The researchers built SIMAS, a deliberately stripped-down multi-agent architecture in which n agents take turns responding sequentially, each seeing the full accumulated conversation history, and the first agent synthesizes a final answer after T rounds of discussion. Agents were given distinct profiles (personality, beliefs, expertise) but drawn from the same base language model. Experiments varied agent count, task type (including college-level STEM, formal logic, philosophy, and knowledge-recall benchmarks), model family (Llama-3.1 and Qwen2.5 at multiple scales), and agent configuration (ablating personality, belief, and expertise attributes) to isolate the effect of collaboration from model heterogeneity.

Results Performance does not improve monotonically with agent count; instead it rises to an optimum and then declines, following a pattern of diminishing returns governed by a trade-off between collaborative synergy and coordination overhead. Only models above a certain capability threshold can power an effective multi-agent system. The optimal number of agents depends strongly on task type, and performance degradation at high agent counts stems from coordination overhead rather than simply running out of context window. Qwen2.5 models showed higher peak accuracy retention (86.27% versus 60.78% for Llama-3.1) and greater stability across agent counts. Agent attribute ablations revealed large discipline-specific swings—for example, removing personality attributes dropped accuracy by up to 21.57% on philosophy tasks—while leaving the overall scaling shape intact.

Plain-language abstract This paper studies how groups of AI agents powered by large language models (LLMs) can work together more effectively by borrowing ideas from how human organizations are structured. The researchers show that giving LLM agents explicit organizational roles — especially a designated leader — dramatically reduces chaotic, redundant communication and improves task completion. They also introduce a method where LLMs themselves propose and refine better team structures through a Criticize-Reflect process.

Motivation LLM agents used in multi-agent systems tend to over-report information and comply with every instruction they receive, a side effect of how they are trained. This causes information overload, conflicting messages, and disorganized decision-making when multiple agents must cooperate — problems that grow worse as teams get larger or include human participants. No principled framework existed for imposing organizational structure on free-flowing LLM agent communication.

Methodology The researchers built a framework on top of AutoGen, a multi-agent conversation platform, that allows LLM agents (primarily GPT-4) to be organized into different team structures via prompt-based instructions in physical and simulated embodied environments. They tested flat versus hierarchical arrangements — including teams of three and five agents — measuring task efficiency and communication cost. They also ran human-agent collaboration experiments and used a Criticize-Reflect process in which LLMs iteratively propose and evaluate new organizational prompts to discover novel team structures.

Results Imposing a designated leader on a three-agent team improved efficiency by up to 30% while adding almost no extra communication overhead (up to 3% more messages), consistent with findings from organizational theory research on human teams. The same benefit held in five-agent teams. LLM agents also demonstrated the ability to elect their own leader and adjust leadership dynamically. The Criticize-Reflect process produced novel organizational structures that further reduced communication costs and enhanced overall team efficiency.

Plain-language abstract DeLM (Decentralized Language Models) is a multi-agent framework that drops the central controller. Instead of a main agent assigning subtasks, waiting, and merging results, parallel agents asynchronously claim tasks from a shared queue and read/write a shared 'verified context' of accumulated progress. It targets two settings — parallel exploration in software engineering and concurrent evidence processing in long-context QA — and improves accuracy while roughly halving cost.

Motivation Most multi-agent systems use centralized scatter-gather orchestration, which parallelizes sub-agent execution but not the coordination around it. Every finding must return to the main agent to be merged and rebroadcast, so progress-sharing becomes a serialized bottleneck as agents grow, and the controller can dilute, omit, or distort details. In long-context reasoning the main agent must pre-assign evidence clusters before knowing what is relevant, triggering extra delegation rounds. DeLM removes the controller as the coordination chokepoint.

Methodology Coordination is state-based, not prompt-routed. Two global structures: a shared context C of compact verified gists and a task queue T of pending subtasks. The pipeline initializes the queue from the input, executes ready subtasks in parallel, then compresses-verifies-admits each result into the shared context, generates more subtasks when the context is insufficient, and finalizes once none remain. The shared context is compact, global, and 'unfoldable' — agents read coarse gists by default and expand to detailed summaries or raw evidence only when needed. Admission-time verification checks each update against its underlying evidence and reasoning trajectory before it enters shared state, rejecting or regenerating unsupported updates so errors cannot propagate as reusable problem state.

Results On SWE-bench Verified, DeLM is strongest across test-time-scaling metrics, reaching 77.4% pass@4 at ~$0.12/task — roughly half the baselines' cost — with trace-level examples showing agents reuse each other's discoveries through the compact shared context. On LongBench-v2 Multi-Doc QA it leads four frontier model families by up to 5.7 points, with both admission-time verification and hierarchical summarization contributing. On OOLONG, vanilla DeLM underperforms RLM (which needs exact row-level aggregation via code execution), but RLM combined with DeLM yields the best accuracy and lowest cost, showing DeLM works as a coordination layer for programmatic reasoning too.

Plain-language abstract WebSwarm is a multi-agent web-search framework that builds its task decomposition and agent collaboration during inference instead of fixing them upfront. A root agent instantiates search nodes as evidence accumulates; each node couples a local objective with a search mode and can either search on its own or recursively spawn and coordinate child nodes, returning evidence upward so parents can expand, revise, or aggregate. It is evaluated on four deep, wide, and interleaved web-search benchmarks.

Motivation A single ReAct-style agent has one long trajectory and limited context, so it handles either depth or coverage but not both at once. Existing multi-agent search systems usually decompose only at the root, apply one fixed collaboration pattern, and split tasks by surface query semantics, leaving them short on recursive depth, adaptability, and evidence-grounded expansion, because in deep-and-wide tasks the solving structure only emerges as intermediate evidence reveals new entities and constraints.

Methodology WebSwarm organizes search as progressive recursive delegation. Each node's search mode picks a local collaboration structure, parallel divide-and-conquer, sequential search-and-verification, or multi-path sampling and aggregation, and the node either solves directly or delegates children. Two signals keep recursion from going blind: a lightweight web-probing step determines whether relevant evidence is concentrated in a few aggregated pages or dispersed along a dimension, guiding how nodes expand, and trajectory experience distilled from earlier homogeneous sibling nodes guides later ones. It is tested on BrowseComp-Plus, WideSearch, DeepWideSearch, and GISA across several backbones.

Results WebSwarm consistently outperforms single-agent ReAct and multi-agent baselines across all four benchmarks, improving over ReAct by 17.50 accuracy points on BrowseComp-Plus and by 10.91 Row F1 and 9.76 Item F1 on WideSearch-EN. Ablations show both mechanisms matter: removing recursive delegation lowers BrowseComp-Plus accuracy from 68.00 to 63.50, and removing web-probing inflates average web-tool calls (for example from 137.03 to 239.90), indicating it mainly reduces redundant, misaligned search.

Plain-language abstract This paper is a survey of systems where multiple AI agents, each powered by a large language model, work together to solve complex tasks or simulate real-world environments. It reviews how these multi-agent systems are built, how agents are given distinct roles and communicate, and what kinds of problems they have been applied to, from software development to social simulation.

Motivation Single LLM-based agents had shown promise for planning and autonomous decision-making, but there was no systematic overview of the rapidly growing body of work on multi-agent systems built from LLMs. Earlier efforts in the field proceeded independently, leaving researchers without a unified framework for understanding agent design, communication, and collective capability — a gap this survey addresses.

Methodology The authors conducted a broad literature survey, organizing existing work around a set of guiding questions: what domains and environments LLM-based multi-agents simulate, how agents are profiled and how they communicate, and what mechanisms enable agents to grow in capability. They also catalogued commonly used datasets and benchmarks, and tracked the rising volume of publications in the field over time using paper counts at three-month intervals across application categories.

Results The survey maps LLM-based multi-agent research into two high-level application areas — problem solving (including software development, multi-robot systems) and world simulation (including society, policy, and game simulation) — and documents a rapid growth in publication volume across these categories. The authors identify open challenges and maintain a living open-source repository to keep the community updated as the field evolves.

Plain-language abstract AutoGen is an open-source framework that lets developers build applications powered by large language models (LLMs) by having multiple software agents converse with each other to complete tasks. Each agent can be backed by an LLM, a human, a tool, or some combination, and developers can define how agents interact using natural language or Python code. The framework was evaluated across applications spanning mathematics, coding, question answering, operations research, online decision-making, and entertainment.

Motivation Building capable LLM-based applications is difficult because it requires coordinating multiple capabilities — reasoning, tool use, human feedback, and code execution — in ways that existing single-agent or ad hoc multi-agent approaches do not support cleanly. Prior work showed that using multiple cooperating agents can improve factuality, reasoning, and validation, but there was no general framework that made it easy to define, customize, and coordinate such agents across diverse domains and complexity levels.

Methodology AutoGen introduces two core abstractions: conversable agents and conversation programming. Conversable agents are entities that send and receive messages and can be backed by LLMs, human input, or tool execution; developers configure them by composing built-in capabilities or extending base classes such as AssistantAgent and UserProxyAgent. Conversation programming lets developers specify agent interaction patterns — including static flows, dynamic group chats, and human-in-the-loop rounds — via natural language prompts or Python code. The framework was applied to six demonstration applications and evaluated on benchmarks including the MATH dataset, using 120 randomly selected level-5 problems and the full test set.

Results On the MATH benchmark, AutoGen's built-in two-agent setup outperformed alternative approaches including Multi-Agent Debate, LangChain ReAct, vanilla GPT-4, and commercial products such as ChatGPT with Code Interpreter and the Wolfram Alpha plugin. Across all six demonstration applications, the framework enabled high-performing and in some cases novel interaction patterns — such as multi-human-in-the-loop problem solving and conversational chess — while reducing the development effort required to build each application.

Plain-language abstract MetaGPT is a software framework that coordinates multiple AI agents — each playing a specific role like Product Manager, Architect, or Engineer — to collaboratively write software from a plain-language description. Instead of letting agents chat freely, it enforces structured handoffs modeled on how real software teams operate, which reduces errors and produces working code more reliably.

Motivation Existing multi-agent systems built on large language models struggle with complex tasks because agents produce cascading errors when they communicate through unconstrained natural language. There was no principled way to enforce coordination standards, verify intermediate outputs, or prevent one agent's mistake from compounding through the rest of the pipeline.

Methodology MetaGPT encodes Standardized Operating Procedures (SOPs) into prompt sequences and assigns five specialist roles — Product Manager, Architect, Project Manager, Engineer, and QA Engineer — each of which produces structured artifacts (requirements documents, system design diagrams, interface specifications) rather than free-form dialogue. A shared message pool with a publish-subscribe mechanism lets agents retrieve only the information relevant to their role, and an executable feedback loop allows the Engineer agent to run generated code, detect errors, and self-correct at runtime. The system was evaluated on HumanEval (164 tasks), MBPP (427 Python tasks), and a self-constructed SoftwareDev benchmark of 70 diverse software development tasks.

Results MetaGPT achieved 85.9% and 87.7% Pass@1 on HumanEval and MBPP respectively, surpassing all prior approaches on both benchmarks. On the SoftwareDev benchmark it reached an executability score of 3.75 out of 4, completed tasks in 503 seconds on average, and required only 124–127 tokens per line of code compared to 249 for the next-best system (ChatDev). It also achieved a 100% task completion rate across the benchmark, with significantly lower human revision cost than competing frameworks.

Plain-language abstract This paper proposes a method to make large language models more accurate by having multiple instances of a model debate each other. Instead of querying a single model once, several model instances each generate an answer, then read and critique each other's responses over multiple rounds until they converge on a shared answer. The approach works with publicly available black-box models and requires no access to model internals.

Motivation Large language models frequently hallucinate facts and make reasoning errors, even when prompted carefully. Prior work on improving accuracy—such as chain-of-thought prompting, self-consistency, and reflection—acts on a single model instance, leaving open the question of whether disagreement and debate across multiple independent instances could surface and correct errors that no single model catches on its own.

Methodology Multiple instances of the same language model (or a mix of models such as ChatGPT and Bard) each independently answer a query. Each instance then reads all other instances' answers and reasoning, critiques them, and updates its own answer. This debate loop repeats for several rounds. The method uses identical prompt templates across all tasks and requires only black-box text generation access. Performance is evaluated on six benchmarks covering mathematical reasoning, strategic reasoning, factual question answering, and a newly introduced dataset of computer scientist biography facts.

Results Multiagent debate outperformed single-model baselines including zero-shot chain-of-thought and reflection across all six benchmarks. The debate process reduced hallucinated or factually incorrect content: as rounds progressed, model instances tended to drop facts they disagreed on, filtering out internally uncertain claims. Crucially, debate was not merely amplifying an already-correct majority—many cases were observed where all models initially gave wrong answers but converged on the correct answer after debate. Using more agents and more rounds of debate both contributed to higher accuracy.

Plain-language abstract ChatDev is a framework that uses multiple AI agents, each playing a specialized role such as CEO, programmer, reviewer, or tester, to collaboratively write software from a plain-language description. The agents communicate through a structured sequence of subtasks using both natural language and programming language, and the system includes a mechanism to reduce common AI errors such as incomplete or unexecutable code.

Motivation Prior deep-learning approaches to software development targeted individual phases of the process in isolation, requiring unique model designs for design, coding, and testing. This fragmentation produced inconsistencies across phases and an overall ineffective pipeline. The paper addresses the gap by proposing a unified, language-based communication framework that connects all phases through multi-agent dialogue.

Methodology ChatDev organizes software development into three sequential phases (design, coding, testing), each subdivided into smaller subtasks. In every subtask, two agents with assigned roles (instructor and assistant) conduct multi-turn dialogue until they reach consensus, extracting solutions in text or code. A 'communicative dehallucination' mechanism prompts agents to request clarifying details before responding, reducing incomplete or inaccurate code generation. Experiments used ChatGPT-3.5 (temperature 0.2) and a dataset of software requirement descriptions, evaluating completeness, executability, and consistency against baselines GPT-Engineer and MetaGPT.

Results ChatDev outperformed both baselines across all metrics. Compared to MetaGPT, it raised the overall quality score from 0.1523 to 0.3953. In pairwise human evaluations, ChatDev was preferred over GPT-Engineer in 90.16% of comparisons and over MetaGPT in 88.00% of comparisons. Analysis of agent communications showed that natural-language exchanges drove system design while programming-language exchanges accelerated debugging, and error rates declined steadily across multi-turn testing rounds.

Plain-language abstract CAMEL introduces a framework called role-playing that lets two AI chat agents collaborate autonomously to complete complex tasks — one acting as an assistant, one as a user — with only a rough idea from a human to get started. The system uses a technique called inception prompting to keep agents on track, and the authors used it to generate large datasets of agent conversations as well as to study how language models learn across domains.

Motivation Existing chat-based language models depend heavily on humans crafting detailed, precise prompts to steer them, which is difficult, time-consuming, and requires domain expertise. There was no scalable way to have AI agents cooperate autonomously on multi-step tasks while staying aligned with human intentions and avoiding failure modes such as role-reversal, repetitive replies, and infinite loops.

Methodology The authors built a role-playing framework in which a task-specifier agent refines a human's rough idea into a concrete task, then an AI user agent and AI assistant agent converse in an instruction-following loop until the task is done or termination conditions are met. Using this framework they generated five datasets: AI Society (25,000 role-playing conversations across 50 assistant and 50 user roles with 10 tasks each), Code, Math, Science, and a Misalignment dataset. They evaluated agent performance via blind human ratings and GPT-4 scoring against single-shot GPT-3.5-turbo baselines, and studied knowledge emergence by progressively fine-tuning a LLaMA-7B model on the generated datasets and benchmarking it on HumanEval and HumanEval+.

Results CAMEL's multi-agent cooperative solutions outperformed GPT-3.5-turbo single-shot solutions by a large margin: in the AI Society task, 76.3% of human evaluators and 73.0% of GPT-4 evaluations preferred CAMEL; on the Code task, GPT-4 preferred CAMEL 76.0% of the time. Progressive fine-tuning of LLaMA-7B showed clear domain knowledge emergence as each new dataset was added. The CAMEL-7B model achieved a HumanEval pass@1 of 14.0% and pass@100 of 57.9%, competitive with LLaMA-7B and Vicuna-7B baselines of similar size.

Plain-language abstract AgentVerse is a software framework that lets multiple AI agents—each powered by a large language model—work together on tasks, much like a team of people would. Instead of relying on a single AI assistant, the system dynamically assembles a group of specialized agents, has them discuss and plan, then carries out actions and checks whether the results are satisfactory.

Motivation Single AI agents struggle with complex real-world tasks that naturally require cooperation among multiple specialists. Prior multi-agent research was limited to narrow, fixed tasks and kept agent roles static, leaving open the question of whether a flexible, general-purpose multi-agent system could outperform a lone agent across a wide variety of problems.

Methodology The framework divides problem-solving into four stages: expert recruitment (choosing and adjusting which agents participate based on current progress), collaborative decision-making (agents discuss strategies together), action execution (agents interact with the environment to carry out decisions), and evaluation (comparing the current state to the goal and feeding back results for the next round). Experiments were run on diverse tasks spanning text understanding, mathematical reasoning, code generation, tool use, and embodied AI in Minecraft, using GPT-4 as the backbone language model.

Results Multi-agent groups assembled by AgentVerse outperformed single-agent baselines across the tested task categories. Analysis of agent interactions also revealed the emergence of distinct social behaviors—both cooperative behaviors that improved group efficiency and harmful behaviors that required mitigation—offering insight into how group dynamics in LLM-based systems can be shaped.

Plain-language abstract This paper presents DyLAN (Dynamic LLM-Powered Agent Network), a framework that automatically assembles and coordinates teams of AI agents to solve tasks. Instead of using a fixed group of agents with rigid communication rules, DyLAN selects the most useful agents for a given task and lets them collaborate in a flexible, adaptive structure.

Motivation Existing multi-agent systems built on large language models use a fixed number of agents with static communication patterns, regardless of the task at hand. This inflexibility limits performance, as some agents may be irrelevant or counterproductive for certain tasks, and no principled method existed to select or adjust agent teams based on their actual contributions during collaboration.

Methodology DyLAN operates in two stages. In the first stage, Team Optimization, a preliminary collaboration run is conducted among a pool of candidate agents, and each agent is scored using an unsupervised metric called the Agent Importance Score, computed via a forward-backward message-passing algorithm on a temporal feed-forward network structure. The top-scoring agents form a smaller team. In the second stage, Task Solving, selected agents collaborate dynamically on the actual query, with an LLM-powered ranker able to deactivate low-performing agents mid-collaboration. The framework was evaluated on code generation, decision-making, general reasoning, and arithmetic reasoning benchmarks.

Results DyLAN outperformed strong baselines across all evaluated task types while using substantially fewer API calls — for example, achieving comparable or better accuracy than LLM Debate with 10.6% fewer API calls, and using only 36.6% of LLM Debate's API calls on arithmetic reasoning tasks. On specific subjects in the MMLU benchmark, agent selection improved accuracy by up to 25.0%. Ablation studies showed that a dynamically selected team of three agents could outperform both an unoptimized team and competing methods with four agents, and overall task performance improved by up to 6.7% after team optimization.

Plain-language abstract When multiple AI language model agents debate a question together, they can arrive at better answers than any single agent — but this approach is expensive because every agent reads every other agent's responses. This paper asks whether agents actually need to communicate with everyone, or whether a sparser network of connections can achieve the same or better results at lower cost.

Motivation Existing multi-agent debate (MAD) frameworks use fully-connected communication, meaning each agent reads all other agents' responses at every round. As the number of agents and debate rounds grows, the input context balloons, making inference increasingly expensive. There was no systematic study of whether this full connectivity is necessary, or whether limiting each agent's view to only a few neighbors could preserve — or even improve — debate quality.

Methodology The authors conducted experiments with GPT and Mistral models across text-only reasoning benchmarks (MATH, GSM8K) and multimodal tasks, comparing fully-connected MAD topologies against sparse alternatives such as neighbor-connected graphs. They also extended the framework to alignment labeling tasks using the Anthropic-HH dataset, and explored heterogeneous setups in which different agents are instantiated with different LLMs assigned to graph positions of varying centrality.

Results Sparse, neighbor-connected MAD matched or exceeded fully-connected MAD while cutting inference input token costs by over 40% on reasoning tasks. On the MATH dataset, the neighbor-connected configuration gained +2% accuracy over the fully-connected baseline while keeping the same accuracy on GSM8K. On alignment labeling, sparse MAD improved helpfulness by +0.5% and harmlessness by +1.0% compared to single-agent setups, while reducing costs by roughly 50%. Assigning stronger LLMs to higher-centrality positions in the communication graph consistently improved overall performance in multi-model debate settings.

Plain-language abstract This paper introduces MALT (Multi-Agent LLM Training), a method for improving how AI language models reason through hard problems by splitting the work across three specialized models: one that generates an answer, one that verifies it, and one that refines it. The three models are trained together using automatically produced data, with no need for human labeling or a separate teacher model.

Motivation Large language models typically produce answers through a single chain of reasoning, which limits their ability to explore alternative paths or catch and fix their own errors on complex tasks. While multi-agent frameworks at inference time had shown promise, the underlying models lacked training tailored to their specific roles, creating a distribution mismatch and leaving an open gap in how to jointly train models to cooperate in a multi-agent reasoning pipeline.

Methodology MALT post-trains three role-conditioned models — a Generator, a Verifier, and a Refiner — using a sequential pipeline. During data generation, each agent is sampled repeatedly to build a multi-agent search tree, and final outputs are scored against ground-truth answers. A value-iteration procedure then propagates binary outcome rewards back through the tree to assign credit to each agent, automatically producing positive and negative reasoning trajectories for supervised fine-tuning and Direct Preference Optimization (DPO), without human supervision or an oracle policy.

Results On the MATH, GSM8K, and CSQA benchmarks, MALT achieves relative accuracy improvements of 15.66%, 7.42%, and 9.40% respectively over the same baseline language model (Llama 3.1) trained without the multi-agent pipeline. The method also outperforms a set of single-model baselines and ablations across math and commonsense reasoning tasks.

Plain-language abstract DynTaskMAS is a software framework for running teams of AI agents—each powered by a large language model—on complex tasks in parallel and without waiting for one another. It breaks a hard problem into subtasks automatically, tracks which subtasks depend on which others, and runs as many as possible at the same time, so the overall job finishes faster and the AI models stay busier.

Motivation Existing multi-agent systems built on large language models typically use simple, fixed workflows where agents work one at a time or in rigid sequences. This leaves computing resources underused and makes it hard to tackle tasks whose structure changes as work progresses, creating a need for frameworks that can decompose and schedule work dynamically.

Methodology The framework is built around four integrated components: a Dynamic Task Graph Generator that decomposes an input task into subtasks while preserving their logical dependencies; an Asynchronous Parallel Execution Engine that schedules those subtasks for concurrent execution; a Semantic-Aware Context Management System that shares relevant information among agents efficiently; and an Adaptive Workflow Manager that adjusts execution in real time. The work was presented as a conference paper at the 35th International Conference on Automated Planning and Scheduling, and performance was evaluated experimentally across varying task complexities and agent counts.

Results DynTaskMAS achieved a 21–33% reduction in execution time compared to traditional approaches, with larger gains for more complex tasks. Resource utilization improved from 65% to 88%, a 35.4% increase. Throughput scaled near-linearly up to 16 concurrent agents, reaching a 3.47× improvement when using 4× the agents.

Plain-language abstract This paper proposes a two-dimensional classification system for AI agent design patterns, combining what an agent does (cognitive function) with how it is structurally organized (execution topology). The resulting 7-by-6 matrix identifies 27 named patterns and provides a shared vocabulary for describing, comparing, and designing large-language-model-based agent systems.

Motivation Existing frameworks from major AI organizations each describe agent architectures from only one perspective: industry guides focus on how data flows (execution topology) while cognitive science surveys focus on what agents do (cognitive function). This single-axis view fails to distinguish architecturally distinct systems — for example, the same Orchestrator-Workers topology can implement task planning, hierarchical delegation, or observability monitoring, three patterns with fundamentally different failure modes — leaving architects without a principled, framework-neutral vocabulary.

Methodology The authors define two independent axes: a Cognitive Function axis with seven categories (Context Engineering, Memory, Reasoning, Action, Reflection, Collaboration, Governance) and an Execution Topology axis with six structural archetypes (Chain, Route, Parallel, Orchestrate, Loop, Hierarchy). They take the Cartesian product to produce a 42-cell matrix, populate 27 cells with named patterns (13 with original names), demonstrate that the two axes are orthogonal through systematic cross-axis analysis, and validate the framework's coverage by mapping four real-world deployment domains — financial lending, legal due diligence, network operations, and healthcare triage — onto the pattern catalog.

Results The framework identifies 27 named patterns across the 7-by-6 matrix, with 15 cells either structurally redundant or not yet observed in practice. Cross-domain analysis yields five empirical laws of pattern selection: time pressure favors Chain-of-Thought and Loop patterns; action authority favors Blast Radius Control; asymmetric failure costs shape how Reflection is parameterized; volume determines which collaboration patterns are needed; and the same pattern (such as Generator-Critic) appears across all four domains but with domain-specific parameterization. Four patterns — Context Triage, RAG Pipeline, Complexity-Based Routing, and Generator-Critic — appear in three or more domains, suggesting they are foundational to most production agent systems.

Plain-language abstract This paper introduces MPAC (Multi-Principal Agent Coordination Protocol), a new communication standard that lets AI agents owned by different people or organizations work together on shared tasks without a central authority overseeing them. Today's agent protocols assume one person or organization controls all the agents involved, which breaks down when independent parties — like two engineers whose coding agents both edit the same file — need to coordinate. MPAC defines how agents declare intentions before acting, handle conflicts explicitly, and escalate disagreements to their human owners when needed.

Motivation Existing AI agent protocols (MCP and A2A) both assume a single controlling principal who owns and trusts every participating agent, but many real coordination problems involve independent principals whose agents must work on shared state — two engineers' coding agents editing the same repository, family members' agents planning a trip together, or agents from different organizations drafting a joint contract. When this single-principal assumption breaks down, coordination collapses to ad-hoc chat, manual merging, or silent overwrites. No existing protocol addressed this multi-principal coordination gap.

Methodology MPAC is specified as a five-layer application-layer protocol (Session, Intent, Operation, Conflict, and Governance layers) with 21 message types, three state machines with normative transition tables, Lamport-clock watermarking for causal ordering, two execution models (pre-commit and post-commit), three security profiles, and an optimistic-concurrency-control mechanism for shared state. The authors released two interoperable reference implementations (Python with 122 tests; TypeScript with 101 tests), 66 adversarial enforcement tests, a machine-readable JSON Schema suite covering all 21 message types, and seven live multi-agent demos spanning scenarios such as code editing, trip planning, pre-commit authorization with fault recovery, and multi-level conflict escalation.

Results A controlled three-agent cross-module code review benchmark showed a 95% reduction in coordination overhead (from 68.65 seconds to 3.02 seconds) and a 4.8x wall-clock speedup (from 131.76 seconds to 27.38 seconds) under MPAC compared to a serialized human-mediated baseline. Per-agent decision time was preserved (63.11 seconds versus 57.13 seconds), confirming that the speedup comes from eliminating coordination waits rather than compressing model inference. The full specification, reference implementations, test suites, and demo transcripts were released as open source.

Plain-language abstract This paper introduces Multi-Agent Debate (MAD), a framework in which multiple large language model (LLM) agents argue opposing positions under the supervision of a judge agent in order to reach better answers on hard reasoning tasks. The key insight is that having agents challenge each other forces the models to generate genuinely new lines of thought, overcoming the tendency to simply repeat or reinforce an initial answer.

Motivation Self-reflection — asking an LLM to iteratively critique and refine its own answers — is a popular strategy for improving reasoning, but the authors identify a fundamental failure mode they call Degeneration-of-Thought (DoT): once an LLM becomes confident in a (possibly wrong) answer, further self-reflection produces no new thinking and the model stays stuck. The paper argues this happens because self-reflection lacks external feedback and is vulnerable to the model's own biases and rigidity.

Methodology The MAD framework pits two LLM agents against each other in a 'tit for tat' debate, where each agent argues its position across multiple rounds; a separate judge agent monitors the exchange and decides when to stop and what the final answer is. The framework was evaluated on two benchmark tasks chosen to require deep reasoning: commonsense machine translation and counterintuitive arithmetic reasoning. The authors also analyzed how debate dynamics (number of rounds, level of disagreement between agents) affect performance, and examined what happens when different LLMs play the agent versus judge roles.

Results Experiments on commonsense machine translation and counterintuitive arithmetic reasoning show that MAD outperforms self-reflection baselines, demonstrating that structured inter-agent debate can overcome the DoT problem. Analyses found that an adaptive stopping criterion for debate and a moderate level of 'tit for tat' disagreement are both necessary for good performance; debates that are too short or too combative degrade results. The authors also found that using a different LLM as the judge than the debating agents can introduce unfairness into the evaluation.

Plain-language abstract ReConcile is a framework that sets up multiple different AI language models as participants in a round-table discussion to collaboratively solve reasoning problems. Instead of using many copies of a single model, it brings together models from different families—such as ChatGPT, Bard, and Claude—who exchange answers, share confidence estimates, and try to convince each other using human-style explanations until they reach a consensus answer.

Motivation Large language models still struggle with complex reasoning tasks, and existing approaches to improve this—such as having a single model critique its own outputs—suffer from stagnation when the model becomes overly confident and stops generating new ideas. Multi-agent debate frameworks exist but have typically used multiple instances of the same underlying model, which limits diversity because all agents share the same pre-training data and architecture, producing an inherent model bias and a narrow knowledge scope.

Methodology ReConcile runs multiple rounds of structured discussion among agents drawn from different model families. In each round, each agent receives a discussion prompt containing the grouped answers and explanations from the previous round, the confidence scores of all agents, and demonstrations of human explanations that successfully corrected wrong answers. Agents then update their answers by attempting to convince others or by being convinced. A confidence-weighted voting mechanism aggregates the agents' final answers into a consensus. The framework was evaluated on seven reasoning benchmarks and flexibly accommodates API-based, open-source, and domain-specific models.

Results ReConcile outperformed prior single-agent and multi-agent baselines by up to 11.4 percentage points across seven benchmarks, and even surpassed GPT-4 on three datasets. Incorporating a mix of diverse models—including domain-specific ones—yielded an 8% improvement on the MATH benchmark. Analysis confirmed that model diversity is a critical driver of performance gains, with the confidence-weighted consensus mechanism and the convincing-sample demonstrations each contributing meaningfully to the final results.

Plain-language abstract AutoGen Studio is an open-source, no-code developer tool for building, testing, and debugging multi-agent AI systems — networks of AI models and tools that collaborate to complete complex tasks. It provides a drag-and-drop web interface and a Python API so developers can define agents, assemble them into workflows, and inspect their behavior without writing code from scratch. The tool is built on top of the AutoGen framework and was downloaded over 200,000 times in its first five months.

Motivation Existing frameworks for building multi-agent AI applications, such as AutoGen, CAMEL, and TaskWeaver, require developers to express agent configurations as Python code, creating a high barrier to entry. Configuring the many parameters of a multi-agent system — models, tools, orchestration order, termination conditions — and then making sense of failures is tedious and error-prone. There was no open-source tool offering a visual, no-code interface for this purpose.

Methodology The authors designed and implemented AutoGen Studio with two main components: a frontend React web interface and a backend API (REST/WebSocket via FastAPI, a Python API, and a command-line interface). The UI provides a build view for defining and composing agents via drag-and-drop, a playground view for interactive task execution and workflow debugging, and a gallery of reusable agent and workflow templates. A profiler module visualizes per-agent metrics including message counts, token consumption, dollar costs, and tool-call success rates. Workflows are represented as declarative JSON specifications that can be exported, versioned, and deployed as API endpoints or Docker containers. The tool was iteratively refined over five months based on more than 135 GitHub issues filed by its user community.

Results AutoGen Studio was downloaded more than 200,000 times within five months of release, demonstrating substantial adoption. Analysis of 135 GitHub issues — embedded with OpenAI text-embedding-3-large, reduced via UMAP, and clustered with KMeans (k=8) — revealed recurring user pain points around component definition and persistence, tool authoring, and end-to-end debugging. These informed four design patterns for no-code multi-agent tooling: a define-and-compose workflow authoring approach, robust debugging and sensemaking tools, seamless export and deployment, and collaboration and sharing features. The authors identify it as the first open-source project to provide a no-code interface for autonomous multi-agent application development.

Plain-language abstract This paper addresses how to automatically design better multi-agent systems built from large language models (LLMs). The authors introduce MASS (Multi-Agent System Search), a framework that jointly optimizes both the text instructions (prompts) given to individual agents and the structure (topology) describing how agents interact with one another, rather than relying on hand-crafted configurations.

Motivation Building effective multi-agent systems from LLMs requires carefully designed prompts for each agent and carefully chosen interaction topologies, but doing this by hand is complex and does not scale. Most prior work either optimized prompts or topologies in isolation, or still relied on handcrafted prompts, leaving the full design space unexplored and suboptimal systems as a result.

Methodology MASS optimizes the multi-agent design space in three interleaved stages: (1) block-level local prompt optimization for individual agent types, (2) workflow topology optimization to find effective agent interaction structures, and (3) workflow-level global prompt optimization conditioned on the previously optimized topology and prompts. Each stage builds on the outputs of the prior stage. The authors also conducted analysis comparing automatic prompt optimization against common scaling strategies such as self-consistency, self-refinement, and multi-agent debate, measuring token-cost efficiency on benchmarks including MATH using Gemini 1.5 Pro.

Results MASS-optimized multi-agent systems outperform a spectrum of existing alternatives by a substantial margin. The analysis showed that prompt optimization is more token-efficient than simply scaling the number of agents with default prompts, and that applying self-consistency on top of prompt-optimized agents yields better scaling behavior than standard approaches like self-consistency or self-refinement alone. The work also produces a set of design principles for building effective multi-agent systems.

Plain-language abstract This paper is a comprehensive survey and security analysis of the Model Context Protocol (MCP), an open standard introduced by Anthropic in late 2024 that gives AI models a unified way to discover and interact with external tools, APIs, databases, and files. The authors map the current MCP ecosystem, catalogue security risks across the full server lifecycle, and lay out research directions for making the protocol secure and sustainable.

Motivation Before MCP, connecting an AI application to external tools required writing custom API integrations for every service, leading to fragmented, platform-specific ecosystems and heavy maintenance burdens. MCP was designed to eliminate this fragmentation with a single standardized interface, but its rapid adoption outpaced any systematic analysis of its architecture or security posture, leaving a gap that this paper addresses.

Methodology The study performs an ecosystem-wide analysis of MCP as of March 2025, examining the protocol's architecture (host, client, and server roles), the three-phase lifecycle of MCP servers (creation, operation, and update), and the security risks at each phase. The authors survey publicly available MCP server collections—cataloguing registries such as MCP.so (4,774 servers), Glama (3,356), and Smithery (2,942)—review unofficial auto-installers, and analyse real-world adoption by industry platforms including OpenAI, Cursor, and Cloudflare.

Results The analysis identifies a range of concrete security threats: name collision attacks (malicious servers impersonating legitimate ones), installer spoofing through unofficial auto-installers, code injection and backdoors during server creation, and configuration drift in remote multi-tenant deployments. The paper finds that MCP currently lacks a centralized security oversight model, standardized authentication and authorization, and an official package management system. It concludes with recommendations for cryptographic server verification, namespace policies, and reputation-based trust mechanisms to support the protocol's sustainable growth.

Plain-language abstract This paper introduces MMA2A, a routing layer for multi-agent AI systems that forwards voice, image, and text messages between agents in their original format rather than converting everything to text. The authors show that keeping data in its native modality significantly improves how well agents complete tasks requiring combined reasoning over multiple types of information.

Motivation When AI agents communicate using the Agent-to-Agent (A2A) protocol, the standard practice is to serialize all messages into text — even though the protocol supports audio, image, and structured data. This text-bottleneck discards perceptual signals such as spatial defect features and prosodic cues that downstream agents could use for better decisions, but no prior work had measured how much accuracy this conversion costs or what is needed to recover it.

Methodology The authors built MMA2A, a lightweight architecture layer atop the A2A protocol that reads each agent's published capability declarations (Agent Cards) to decide whether to forward a message part natively or fall back to text. They evaluated it on CrossModal-CS, a controlled 50-task customer-service benchmark requiring joint reasoning over voice, image, and text inputs, holding the LLM backend (Gemini), tasks, and knowledge base constant so that only the routing strategy varied. A key ablation replaced the LLM-backed reasoning agent with a keyword-matching heuristic to isolate the contribution of routing from the contribution of capable downstream reasoning.

Results MMA2A achieved 52% task-completion accuracy versus 32% for the text-bottleneck baseline on CrossModal-CS (95% bootstrap CI on the difference: 8–32 percentage points; McNemar's exact p = 0.006). Gains were largest on vision-dependent tasks: product defect reports improved by 38.5 percentage points and visual troubleshooting by 16.7 percentage points. Critically, the ablation with keyword matching produced identical accuracy under both routing strategies (36% vs. 36%), demonstrating that modality-native routing only yields benefits when paired with an agent capable of exploiting the richer input — and that the 1.8x latency cost of native multimodal processing creates an accuracy-latency tradeoff designers must weigh.

Plain-language abstract This paper is the first systematic survey of how AI agents built on large language models (LLMs) are being applied across software engineering tasks. It maps 115 papers into a unified framework, explains how these agents are structured, and identifies open problems and future directions for the field.

Motivation Many researchers were already building LLM-based agents for software engineering tasks such as code generation, bug detection, and code summarization, but there was no shared vocabulary or framework to describe what these agents actually do. The lack of a structured survey made it hard to compare approaches, identify gaps, or understand how the field was developing.

Methodology The authors collected and filtered the literature, arriving at 115 papers on LLM-based agents applied to software engineering. They then developed a three-module conceptual framework — perception, memory, and action — grounded in classical agent theory, and used it to classify and analyze each paper. The perception module covers how agents receive inputs (text, visual, auditory, tree/graph-based); the memory module covers semantic, episodic, and procedural memory; and the action module covers internal actions (reasoning, retrieval, learning) and external actions (interacting with environments or humans).

Results The survey produced a taxonomy of LLM-based agents in software engineering organized around the perception-memory-action framework. The authors identified current challenges in combining LLMs with SE and proposed future research opportunities in response to those challenges. A curated repository of the 115 surveyed papers was released publicly on GitHub.

Plain-language abstract OpenHands (formerly OpenDevin) is an open-source platform for building AI agents that can do software development tasks the way a human developer would: writing and editing code, running commands in a terminal, and browsing the web. The platform provides sandboxed execution environments, a flexible event-stream architecture for agent-environment interaction, support for multiple agents working together, and a suite of evaluation benchmarks.

Motivation AI agents powered by large language models are becoming capable of tackling complex, real-world software tasks, but building and evaluating such agents requires infrastructure for safe code execution, flexible environment interaction, and standardized benchmarking. Existing open-source frameworks lacked a unified, immediately usable platform that addressed all of these needs together, slowing research and development in the field.

Methodology OpenHands is built around an event-stream architecture through which user interfaces, agents, and environments communicate. Each task session runs inside a Docker-sandboxed operating system that exposes a bash shell, a Jupyter/IPython server, and a web browser to the agent. The platform includes an agent hub with over 10 implemented agents — including a generalist agent based on the CodeAct architecture — multi-agent delegation support, and an evaluation framework. Agents are assessed across 15 benchmark tasks spanning software engineering (including SWE-Bench) and web navigation (including WebArena).

Results OpenHands successfully demonstrated agents performing across 15 challenging task categories including software engineering and web browsing benchmarks. The project has attracted more than 2,100 contributions from over 188 contributors and is released under the permissive MIT license, reflecting broad adoption across academia and industry. The platform was accepted as a conference paper at ICLR 2025.

Plain-language abstract This paper introduces FlowGen, a system that uses multiple AI language model agents to generate code by mimicking the collaborative workflows of real software engineering teams. Three variants emulate the Waterfall, Test-Driven Development, and Scrum process models, with specialized agents playing roles like requirement engineer, architect, developer, and tester. The system is evaluated on standard coding benchmarks and compared against baseline approaches.

Motivation Large language models have shown promise in code generation, but most approaches treat it as a solo task, ignoring the collaborative and structured nature of real software development. Existing multi-agent systems either don't follow specific process models or focus only on Waterfall-style workflows, leaving open the question of whether emulating different software process models—like TDD or Scrum—leads to meaningfully different code quality outcomes.

Methodology The authors built FlowGen on top of GPT-3.5, assigning LLM agents to role-specific positions corresponding to software development activities. Three process model variants were implemented: FlowGenWaterfall, FlowGenTDD, and FlowGenScrum. Agents communicate using chain-of-thought prompting and prompt composition with continuous self-refinement. Performance was measured using Pass@1 on four benchmarks: HumanEval, HumanEval-ET, MBPP, and MBPP-ET, with comparisons to RawGPT, CodeT, and Reflexion baselines.

Results FlowGenScrum outperformed the other process model variants, achieving Pass@1 scores of 75.2, 65.5, 82.5, and 56.7 on HumanEval, HumanEval-ET, MBPP, and MBPP-ET respectively—an average 15% improvement over RawGPT. Combining FlowGenScrum with CodeT produced statistically significant further gains and the highest overall Pass@1 scores. Analysis also showed that design and code review activities specifically increased exception handling and reduced code smells, and that FlowGen's performance remained stable across different GPT-3.5 versions and temperature settings.

Plain-language abstract This paper is a comprehensive survey of LLM-based multi-agent systems — software architectures in which multiple AI agents, each powered by a large language model, collaborate to tackle tasks too complex for any single agent. It introduces a four-stage framework called LIFE (Lay the capability foundation, Integrate agents through collaboration, Find faults through attribution, and Evolve through autonomous self-improvement) to organize the field as a causally linked progression rather than isolated topics.

Motivation While individual LLM agents have become capable at reasoning, planning, and tool use, they struggle with tasks that require sustained coordination across many roles and environments. Existing surveys treated individual agent capabilities, multi-agent collaboration, and agent self-evolution as separate topics, leaving the causal connections among them — and in particular the problem of diagnosing and recovering from failures in multi-agent systems — largely unexamined. No prior survey had systematically covered failure attribution in multi-agent systems, despite it becoming a distinct and rapidly growing research area.

Methodology The survey organizes its review around the LIFE progression, providing stage-wise structured taxonomies and comparative analyses. It covers: core capabilities of individual agents (reasoning, memory, planning, tool use); organizational mechanisms of multi-agent collaboration (role allocation, communication protocols, orchestration topologies, interaction patterns); a methodological landscape of failure attribution approaches (data-driven, constraint-guided, and causal-inference methods); and a hierarchical design space of self-evolution. Throughout, the authors formally characterize causal dependencies between adjacent stages. The work also curates a publicly maintained repository of structured literature, taxonomy visualizations, and comparative tables.

Results The survey establishes LIFE as the first unified analytical framework covering the complete operational lifecycle of LLM-based multi-agent systems. It reveals an underexplored attribution–evolution closed loop: failure attribution narrows the search space for targeted self-improvement, while collaborative structures shape which failures can be observed and attributed. The authors identify open challenges at the boundaries between LIFE stages and propose a cross-stage research agenda aimed at closed-loop systems capable of continuously diagnosing failures, reorganizing collaborative structures, and refining agent behaviors toward more self-organizing and resilient collective intelligence.

Plain-language abstract This paper presents OneManCompany (OMC), an open-source framework that organizes teams of AI agents like a real company — with hiring, role assignment, task management, performance reviews, and continuous improvement. Rather than building yet another chatbot or single-agent tool, it creates an organizational layer so that many different AI agents can work together on complex, open-ended projects without being pre-configured for each one.

Motivation Existing multi-agent AI systems are brittle: they hardcode team structures before a project begins, lock all agents into the same software runtime, and lose any lessons learned when a session ends. There is no principled way to recruit new specialists mid-project, coordinate agents from different software families, or have the organization improve itself over time — the same gaps that would make a real company dysfunctional.

Methodology OMC introduces three core mechanisms. First, a Talent-Container architecture packages each agent's persona, skills, and tools into a portable 'Talent' that can run on any supported backend (LangGraph, Claude CLI, or script processes), with a community-driven Talent Market for on-demand recruitment. Second, an Explore-Execute-Review (E2R) tree search decomposes tasks top-down into a dependency-tracked DAG, enforces bottom-up completion propagation with formal termination and deadlock-freedom guarantees, and gates each subtask result behind supervisor review before downstream tasks proceed. Third, a self-evolution layer lets individual agents update their working principles after each task and manager feedback session, while organization-level standard operating procedures and culture rules accumulate across projects. The system was evaluated on PRDBench, a benchmark of 50 project-level software development tasks across 20+ domains, using a founding LangGraph agent plus three Claude Code-based specialists recruited from the Talent Market.

Results OMC achieved an 84.67% success rate on PRDBench, surpassing the previous state of the art by 15.48 percentage points, at an average cost of approximately $6.91 per task ($345.59 total across 50 tasks). Cross-domain case studies — including autonomous content generation (completed in under 10 minutes for ~$4.48), iterative game development with human-in-the-loop feedback, cross-modal audiobook production, and a research survey that generated 17 structured documents and three novel research proposals in under one hour for $16.26 — demonstrated that the same framework generalizes across heterogeneous agent backends and task types without any domain-specific reconfiguration.

Plain-language abstract XFlow is a system for building multi-agent LLM workflows that are more reliable, and XPF is its protocol language. The core idea: today, constraints, rules, and process obligations are buried inside prompts that agents must remember and re-apply, with no way for the surrounding system to enforce them. XFlow draws an explicit prompt-harness boundary: informal semantic reasoning stays inside actors, but selected commitments are moved into harness structure that is compiled, checked, and enforced. A protocol is written as a readable literate document but compiled into a typed intermediate representation and executed as a program. At runtime, agent outputs are staged as lifecycle-governed symbols and only become shared state after passing checks.

Motivation In a multi-agent pipeline, one agent's hallucination, malformed output, or misread instruction becomes shared state and corrupts later decisions. Current frameworks sit at two extremes of the Chomsky hierarchy: markup/config tools describe workflow shape but say nothing about where knowledge leaves the prompt, and prompt-based tools keep formalizable constraints stuck in instructions. Developers have no language for drawing, testing, and adjusting which commitments are governable versus left to the actor.

Methodology Three-phase architecture. Specification: authors write XPF (YAML frontmatter for symbols/policies, Markdown stages, fenced semantic blocks for actor interfaces, guarded control flow, and responsibility handoffs). Compilation: mechanical passes parse the literate surface, resolve names, statically check that actor reads/writes are legal, judge outputs bind to declared symbols, flow targets exist, and call returns match child-protocol outputs, then lower to a typed IR. Execution: a runtime wraps each actor call in a typed interface, commits symbol writes only after schema validation/provenance/commit-policy checks inside atomic transactions with rollback, propagates staleness to derived values reactively, and persists scoped session frames for resume and audit. Evaluation uses Qwen3.5-9B and DeepSeek-V4-Flash across tau3-bench, CorpusQA (layered on the XpandA baseline), and SWE-bench Verified (around mini-SWE-agent).

Results On tau3-bench, wrapping a bare ReAct agent with XFlow raised constraint-compliance to 100% in Retail and Airline for Qwen3.5-9B (from 96.5% and 91.8%) and to 100% across all three domains for DeepSeek-V4-Flash (Telecom compliance +36.3pp), while task pass1 stayed comparable, separating 'reached the answer' from 'reached it via a valid path.' CorpusQA accuracy improved with XpandA+XFlow to 61.7 (+2.4) for Qwen and 75.7 (+0.9) for DeepSeek by encoding domain interpretation rules as deterministic derivations over extracted symbols. On SWE-bench Verified, XFlow around mini-SWE-agent raised the pass rate from 77.4 to 79.8 (+2.4) by gating patch submission on a passing local validation check; a case study shows the protocol rejecting a submission after a failed test run and issuing a targeted retry. The paper also presents cloud-edge use cases where edge worker outputs must pass schema and coverage checks before entering global state.

Plain-language abstract Glite ARF is an open-source framework for running many LLM coding agents in parallel on a shared research codebase without the whole thing quietly corrupting itself. It wraps each agent's work in an isolated task folder, makes finished work immutable (fixes happen in new tasks, not edits to old ones), and auto-generates a dashboard so a human can see the true state of a multi-week campaign instead of relying on an agent-written summary.

Motivation Delegating research experiments directly to coding agents doesn't scale: agents follow most instructions but the few they skip compound into corrupted data, fabricated citations, contaminated splits, and stale summaries. The authors' own prior audit found 13 such incidents in one campaign, including one where a single agent step recomputed and corrupted 20,304 historical training rows across 38 feature sets.

Methodology The framework defines a three-role stack (human chooses hypotheses, coding agents execute isolated tasks, deterministic Python 'verificator' scripts enforce structure) built on seven structural principles: task isolation via git worktrees, immutability with a corrections overlay, aggregators-only cross-task reading, and a materialized human-facing overview regenerated from committed artifacts. It's evaluated via a real external shared task (BEA 2026 vocabulary-difficulty) plus measured overhead across three author-run campaigns.

Results The BEA 2026 submission built with Glite ARF placed first (closed track) and second (open track) across all three target languages, cutting the baseline RMSE by 29.9% (closed) and 35.9% (open), across 273 tracked tasks run by up to twelve parallel agents from a single laptop at roughly $450 in LLM spend. Structured per-fold provenance caught and let them strip four target-leaking feature sets that had inflated one result to an implausible 0.609 RMSE (corrected to 0.802). The framework's structural machinery adds only about 1% wall-clock overhead across three campaigns in three domains.

Plain-language abstract WebSwarm is a multi-agent web-search framework that builds its task decomposition and agent collaboration during inference instead of fixing them upfront. A root agent instantiates search nodes as evidence accumulates; each node couples a local objective with a search mode and can either search on its own or recursively spawn and coordinate child nodes, returning evidence upward so parents can expand, revise, or aggregate. It is evaluated on four deep, wide, and interleaved web-search benchmarks.

Motivation A single ReAct-style agent has one long trajectory and limited context, so it handles either depth or coverage but not both at once. Existing multi-agent search systems usually decompose only at the root, apply one fixed collaboration pattern, and split tasks by surface query semantics, leaving them short on recursive depth, adaptability, and evidence-grounded expansion, because in deep-and-wide tasks the solving structure only emerges as intermediate evidence reveals new entities and constraints.

Methodology WebSwarm organizes search as progressive recursive delegation. Each node's search mode picks a local collaboration structure, parallel divide-and-conquer, sequential search-and-verification, or multi-path sampling and aggregation, and the node either solves directly or delegates children. Two signals keep recursion from going blind: a lightweight web-probing step determines whether relevant evidence is concentrated in a few aggregated pages or dispersed along a dimension, guiding how nodes expand, and trajectory experience distilled from earlier homogeneous sibling nodes guides later ones. It is tested on BrowseComp-Plus, WideSearch, DeepWideSearch, and GISA across several backbones.

Results WebSwarm consistently outperforms single-agent ReAct and multi-agent baselines across all four benchmarks, improving over ReAct by 17.50 accuracy points on BrowseComp-Plus and by 10.91 Row F1 and 9.76 Item F1 on WideSearch-EN. Ablations show both mechanisms matter: removing recursive delegation lowers BrowseComp-Plus accuracy from 68.00 to 63.50, and removing web-probing inflates average web-tool calls (for example from 137.03 to 239.90), indicating it mainly reduces redundant, misaligned search.

Plain-language abstract MetaGPT is a software framework that coordinates multiple AI agents — each playing a specific role like Product Manager, Architect, or Engineer — to collaboratively write software from a plain-language description. Instead of letting agents chat freely, it enforces structured handoffs modeled on how real software teams operate, which reduces errors and produces working code more reliably.

Motivation Existing multi-agent systems built on large language models struggle with complex tasks because agents produce cascading errors when they communicate through unconstrained natural language. There was no principled way to enforce coordination standards, verify intermediate outputs, or prevent one agent's mistake from compounding through the rest of the pipeline.

Methodology MetaGPT encodes Standardized Operating Procedures (SOPs) into prompt sequences and assigns five specialist roles — Product Manager, Architect, Project Manager, Engineer, and QA Engineer — each of which produces structured artifacts (requirements documents, system design diagrams, interface specifications) rather than free-form dialogue. A shared message pool with a publish-subscribe mechanism lets agents retrieve only the information relevant to their role, and an executable feedback loop allows the Engineer agent to run generated code, detect errors, and self-correct at runtime. The system was evaluated on HumanEval (164 tasks), MBPP (427 Python tasks), and a self-constructed SoftwareDev benchmark of 70 diverse software development tasks.

Results MetaGPT achieved 85.9% and 87.7% Pass@1 on HumanEval and MBPP respectively, surpassing all prior approaches on both benchmarks. On the SoftwareDev benchmark it reached an executability score of 3.75 out of 4, completed tasks in 503 seconds on average, and required only 124–127 tokens per line of code compared to 249 for the next-best system (ChatDev). It also achieved a 100% task completion rate across the benchmark, with significantly lower human revision cost than competing frameworks.

Plain-language abstract Magentic-One is an open-source multi-agent AI system designed to complete complex, multi-step tasks that require using a web browser, managing files, and writing and executing code. A lead agent called the Orchestrator coordinates a team of specialized agents, dynamically planning and recovering from errors to accomplish high-level goals without needing task-specific tuning.

Motivation Existing agentic AI systems tend to be specialized for narrow domains or rely on rigid, single-agent workflows that struggle to generalize across the diverse tasks people encounter in everyday work. There was a need for a generalist system that could plan, reason across multiple steps, use varied tools, and recover from errors across a wide range of scenarios.

Methodology Magentic-One uses a multi-agent architecture built on the AutoGen framework, consisting of five agents: an Orchestrator, a Coder, a Computer Terminal, a FileSurfer, and a WebSurfer. The Orchestrator maintains two structured ledgers to track progress and assign tasks, and dynamically replans when errors occur. The system was evaluated on three agentic benchmarks — GAIA, WebArena, and AssistantBench — using AutoGenBench, a new standalone evaluation tool that enforces isolation and controlled initial conditions across runs to handle the side effects of agent actions.

Results Magentic-One achieved task-completion rates of 38% on GAIA and 32.8% on WebArena, and an accuracy of 27.7% on AssistantBench, placing it statistically competitive with state-of-the-art systems on all three benchmarks, including systems specialized for individual benchmarks. Ablation experiments confirmed the additive contribution of each agent to overall performance, and the modular design allowed agents to be added or removed without retraining or prompt tuning.

Plain-language abstract ARIADNE is a system that uses a search algorithm called Monte Carlo Tree Search (MCTS), guided by a shared memory structure called a blackboard, to automatically write correct and efficient solutions to competitive programming problems. Rather than generating code in a single pass, it organizes the process into five stages — picking an algorithm strategy, writing code, generating tests, evaluating quality, and repairing bugs — and continuously learns from execution feedback to steer the search toward better solutions.

Motivation Large language models can write general code but consistently fail at competitive programming, which demands precise algorithm selection, multi-step logical reasoning, and handling of tricky edge cases under strict time and memory limits. Existing approaches using agentic pipelines, search-based exploration, or shared workspaces each address part of the problem but leave critical gaps: rigid workflows break when early assumptions are wrong, search methods don't reuse evidence across attempts, and shared-workspace designs lack a global planner to allocate effort based on what has been learned.

Methodology ARIADNE frames competitive program generation as a sequential decision-making process navigated by MCTS, where each node represents a state of the shared blackboard — a persistent store of intermediate results, diagnostics, and failure evidence. Five types of actions (strategy selection, code generation, test generation, quality evaluation, and code repair) are explored by the tree search, with rewards backpropagated to guide future decisions toward branches most likely to produce correct, efficient code. The system was evaluated on four benchmarks — APPS, CodeContests, CodeContests+, and LiveCodeBench — using GPT-4o and DeepSeek-V3.2 as backends, and also tested on real contest instances from the 2025 ICPC Asia Shenyang Regional Contest and the 2025 CCPC Fujian Invitational.

Results With GPT-4o, ARIADNE achieved Pass@1 scores of 41.30% on APPS, 46.67% on CodeContests, 27.27% on CodeContests+, and 20.91% on LiveCodeBench, outperforming the strongest baseline (CodeSim) by up to 26.06 percentage points. On the 2025 ICPC Shenyang Regional Contest, ARIADNE solved 3, 6, and 7 out of 13 problems at pass@1/3/5 respectively, compared to 1, 4, and 5 for the best baseline, with similar gains on the CCPC Fujian Invitational. Further improvements were observed when using DeepSeek-V3.2 as the backend.

Plain-language abstract This paper introduces DecentMem, a memory system for groups of AI agents that work together to solve tasks. Instead of all agents sharing one central memory store, each agent keeps its own private memory split into two parts: one for reusing past successful strategies, and one for generating fresh ideas in new situations. Tested across multiple agent frameworks and model sizes, DecentMem improves task accuracy substantially while cutting down the number of tokens the system uses.

Motivation Current multi-agent AI systems almost universally use a centralized shared memory, where every agent reads from and writes to a single common repository. This design causes agents to gradually behave identically over time, collapsing the diversity that makes having multiple specialized agents valuable in the first place. It also incurs high communication and coordination costs and raises privacy concerns, since all agents share the same information pool.

Methodology Each agent in DecentMem maintains a dual-pool memory: an exploitation pool of consolidated trajectories from past tasks, and an exploration pool of LLM-generated strategy candidates for unseen contexts. An LLM-as-a-judge evaluates each stage of a solution trajectory and re-weights the two pools online, letting each agent learn its own exploitation-exploration balance from feedback. The system was evaluated across three multi-agent frameworks (AutoGen, DyLAN, AgentNet), five LLM backbones including Qwen3 (4B, 8B, 14B) and Gemma4 (E2B, E4B), and five benchmarks covering mathematical reasoning, code generation, question answering, and embodied decision-making.

Results DecentMem improved average accuracy by up to 23.8% over the strongest centralized memory baseline (G-Memory) and by up to 52.5% over a no-memory baseline, while reducing token usage by up to 49%. The authors also prove theoretically that the design guarantees global reachability of each agent's local solution space and achieves O(log T) cumulative regret, matching the stochastic bandit lower bound up to constants. Performance gains widened as collaboration became more stochastic.

Plain-language abstract This paper introduces SHIMI (Semantic Hierarchical Memory Index), a new memory architecture for AI agents that organizes knowledge as a tree of layered semantic concepts rather than flat vector embeddings. Each agent maintains its own local memory tree and can synchronize it with other agents using a lightweight protocol, making the system suitable for decentralized or blockchain-based environments. The authors evaluate SHIMI against standard retrieval-augmented generation (RAG) baselines and show that it achieves higher retrieval accuracy, better interpretability, and much lower synchronization bandwidth.

Motivation Current AI memory systems, including RAG and dense vector search, rely on flat, unstructured embeddings that cannot capture conceptual abstraction, produce hard-to-interpret results, and depend on centralized infrastructure. This creates a fundamental mismatch for decentralized AI systems such as autonomous agent networks or blockchain-native applications, where memory must be explainable, distributed, and efficiently synchronized across peers.

Methodology SHIMI models memory as a dynamically structured rooted tree where nodes store semantic summaries and entities are inserted by descending through the hierarchy using LLM-based semantic similarity checks. Retrieval mirrors insertion, pruning branches where similarity to the query falls below a threshold. Decentralized synchronization uses a hybrid protocol combining Merkle-DAG hashing to detect divergent subtrees, Bloom filters to minimize data transfer, and CRDT-style conflict resolution for eventual consistency. The system was evaluated in a simulated multi-agent task-assignment scenario with 20 semantically non-trivial queries and scalability tests up to 2,000 entities.

Results SHIMI achieved 90% top-1 retrieval accuracy and a mean precision@3 of 92.5%, compared to 65% and 68% respectively for the RAG baseline, along with a human-rated interpretability score of 4.7 versus 2.1. Synchronization bandwidth was reduced by over 90% compared to full-state replication across all tested node counts (3 to 6 nodes). Query latency remained approximately flat as the number of stored entities grew to 2,000, in contrast to the near-linear degradation observed with flat vector retrieval.

Plain-language abstract This paper introduces iAgents, a multi-agent AI system designed to let groups of people complete collaborative tasks even when each person only has access to their own private information. The system mirrors a human social network in a network of AI agents that can communicate with each other to gather and share the information needed to solve a task on behalf of their respective users.

Motivation Existing multi-agent AI systems assume that all agents share the same information pool, which fails when the task involves multiple real people each holding private knowledge. Real-world collaborative tasks — such as scheduling across social networks or coordinating between people with different information — require agents to actively seek out and exchange information they do not already possess, a challenge no prior system was designed to handle.

Methodology The authors propose the iAgents framework, in which each person in a social network is paired with a dedicated agent that can only access that person's information. Agents communicate over multiple turns using a mechanism called InfoNav, which guides each agent's communication toward purposeful information exchange by maintaining a structured plan and updating it as the conversation progresses. Human information is stored in a Mixed Memory system combining fuzzy and precise retrieval. The paper also introduces InformativeBench, the first benchmark designed specifically to evaluate LLM agents on tasks requiring resolution of information asymmetry.

Results Experiments show that iAgents can operate within a social network of 140 individuals and 588 relationships, autonomously communicate over more than 30 turns, and retrieve information from nearly 70,000 messages to complete tasks within 3 minutes. The system demonstrates that agents can collaboratively resolve information asymmetry at scale in a realistic social network setting.

Plain-language abstract DroidSpeak is a system for making multi-agent AI workflows more efficient when several fine-tuned language models derived from the same base model need to process the same shared text. Instead of each model independently recomputing everything from scratch, DroidSpeak lets models share intermediate computations — specifically the key-value (KV) caches produced during inference — so that redundant work is eliminated while each model still produces accurate results.

Motivation When multiple fine-tuned language models collaborate on a task — such as in multi-agent pipelines, personalized assistants, or enterprise customer support — they frequently process the same input context independently. This causes each model to redundantly perform the computationally expensive prefill phase, wasting compute, increasing latency, and reducing overall system throughput. Simply reusing one model's KV cache in another model causes significant accuracy loss because fine-tuning introduces task-specific differences between the models.

Methodology DroidSpeak analyzes how fine-tuning affects individual transformer layers and identifies which layers produce KV caches that most strongly influence model outputs (critical layers). Rather than reusing the full KV cache from a sender model or fully recomputing it, the framework selectively recomputes only contiguous chunks of critical layers while reusing the rest. The design avoids the compounding errors that arise when isolated critical layers are recomputed amid reused non-critical layers, and is evaluated on diverse datasets and model pairs.

Results DroidSpeak achieves up to 3x higher throughput and up to 2.6x faster prefill times compared to full recomputation, with negligible accuracy loss. Memory footprint is reduced by up to 1.5x in scenarios where the same context is shared across multiple models.

Plain-language abstract When multiple AI agents collaborate by passing their internal memory (called KV caches) directly to one another, sensitive information can leak through those representations even if it never appears in any text the agents produce. This paper introduces LCGuard, a system that intercepts and transforms those internal memory artifacts before they are shared, so that an attacker who captures the shared data cannot reconstruct the private inputs that produced them, while the agents still perform their tasks effectively.

Motivation Recent work has shown that passing transformer key-value caches between agents is more efficient and informationally richer than exchanging natural language messages, but KV caches implicitly encode private context, retrieved documents, and intermediate reasoning states. Existing safety mechanisms in multi-agent systems only check generated text or tool actions, leaving this representation-level channel completely unguarded. There was no principled framework for controlling what sensitive information is recoverable from KV representations intentionally shared across agents.

Methodology LCGuard frames the problem as a minimax adversarial game: a set of learnable communication functions transform each agent's KV cache before transmission, while an adversarial decoder is simultaneously trained to reconstruct the agent's private input from the shared artifacts. The communication functions are optimized to minimize a task loss (so agents remain useful) and maximize the adversary's reconstruction loss (so private inputs become unrecoverable), with a tunable tradeoff hyperparameter beta. The framework is evaluated on Qwen3 (4B, 8B, 14B), Gemma2-9B, and LLaMA (3B, 8B) across sequential, hierarchical, and graph-based multi-agent topologies, using the AgentLeak, MAGPIE, and PrivacyLens benchmarks.

Results LCGuard consistently reduced attack success rate (ASR) by approximately 65-75% relative to vanilla KV sharing while maintaining competitive task helpfulness. For example, on Qwen3-4B in a sequential PrivacyLens setting, ASR dropped from 0.871 to 0.216 while helpfulness held at 0.710; on Gemma-9B, ASR fell from 0.885 to 0.205 with helpfulness remaining at 0.735. In contrast, the noise-based ADAPT baseline achieved similarly low ASR but collapsed helpfulness (e.g., to 0.285 on Qwen3-4B), while the output-level PrivAct baseline left ASR largely unchanged. Full-system joint optimization across all communication edges outperformed per-agent sanitization, indicating that leakage can re-emerge through aggregation across multiple agents.

Plain-language abstract This paper introduces the Imperfect Byzantine Generals Problem (IBGP), a new coordination framework for multi-agent AI systems that need to work together reliably even when some agents are compromised or behaving incorrectly. Unlike the classic Byzantine Generals Problem, which requires all agents to agree, IBGP only requires a minimum threshold of agents to reach consensus, matching how real multi-agent tasks actually work. The authors design a consensus protocol for IBGP and show it can be embedded into large language model agents via simple prompt instructions, without any additional training.

Motivation As LLM-based agents are increasingly deployed in real-world infrastructure — from autonomous vehicles to sensor networks — ensuring they can coordinate safely in the presence of malicious or hallucinating agents becomes critical. Existing Byzantine consensus protocols require global agreement among all agents, which is both unnecessarily strict and inefficient for most multi-agent tasks, where only a subset of agents needs to coordinate. There was no formal framework or protocol tailored to this weaker, more practical notion of partial consensus.

Methodology The authors formally define IBGP and a corresponding (k, lambda)-protocol that operates in multiple broadcast rounds, using a global randomizer to determine the number of rounds, so that attackers cannot predict when the final decision will be made. They integrate this protocol as a coordination module into a decentralized multi-agent reinforcement learning pipeline (Dec-POMDP with Q-learning), testing it across environments including Predator-Prey (small and large scale), Hallway, and two StarCraft II scenarios (4bane_vs_1hM and 3z_vs_1r). Robustness is evaluated as the ratio of performance under adversarial communication attacks during testing versus unattacked performance during training.

Results The IBGP protocol achieved near-perfect or perfect robustness percentages (up to 100%) across most tested environments, substantially outperforming baseline approaches including recursive training, AME, and ADMAC, which frequently collapsed to 0% robustness or failed to converge under adversarial attack. The protocol tolerates up to 50% malicious agents — compared to 33% for classical BGP — while requiring less redundancy. A sensor network case study further demonstrated that the protocol maintains consistent belief about a moving target's position despite communicative attacks.

Plain-language abstract This paper introduces ACIArena, a benchmark framework for testing how well multi-agent AI systems resist a class of attacks called Agent Cascading Injection (ACI), where a compromised agent spreads malicious instructions to other agents in the network. The framework provides a standardized way to evaluate six real-world multi-agent system designs against a range of attack types and also proposes a new defense strategy.

Motivation Multi-agent AI systems are increasingly deployed in real-world products, but their collaborative nature creates a security vulnerability: a single compromised agent can exploit inter-agent trust to propagate harmful instructions across an entire system. Prior work on this threat was fragmented, covering only limited attack surfaces or simplified system configurations, making it hard to compare defenses or generalize findings to realistic deployments.

Methodology ACIArena provides a unified evaluation framework covering multiple attack surfaces (external inputs, agent profiles, and inter-agent messages) and three attack objectives (instruction hijacking, task disruption, and information exfiltration). It standardizes interfaces for building both multi-agent systems and attack-defense modules, covers six widely used multi-agent system implementations, and supplies a benchmark of 1,356 test cases. The authors also propose a defense called ACI-Sentinel that focuses on preserving task-aligned information rather than trying to detect suspicious messages.

Results Experiments show that current multi-agent systems are broadly vulnerable to ACI attacks across all tested configurations. Evaluating robustness by network topology alone is insufficient; robust systems require deliberate role design and controlled interaction patterns. Defenses built in simplified or narrow settings frequently fail to transfer to realistic scenarios, and in some cases amplify the impact of attacks rather than reducing it.

Plain-language abstract This paper asks whether multi-agent AI systems inherit the demographic biases of their individual components—or make things worse. Working with a new benchmark of ethically loaded decision scenarios, the authors show that even when each AI agent in a chain behaves neutrally on its own, the system as a whole can develop pronounced preferences for certain demographic groups as information passes from agent to agent.

Motivation Alignment research has focused on reducing bias in standalone language models, and techniques like instruction tuning and reinforcement learning from human feedback have made individual models more cautious. But modern AI deployments increasingly chain multiple models together in multi-agent systems (MAS), where each agent's output becomes input for the next. Whether safety-aligned individual agents can still produce biased collective behavior—and whether common architectural choices like role specialization or complex communication topologies can prevent it—had not been systematically measured.

Methodology The authors built Discrim-Eval-Open, a benchmark of 210 demographically diverse protagonist profiles (balanced across age, gender, and race/ethnicity) embedded in open-ended decision scenarios such as kidney transplant allocation and hiring. They ran these scenarios through multi-agent chains of two to four LLMs (including DeepSeek-V3, DeepSeek-R1, GPT-4o, Qwen-Max, and others) arranged in serial, spindle, parallel, and fully-connected topologies. Agents were assigned varied personas (doctor, lawyer, engineer, merchant) and functional roles (judger, analyst, reflector, summarizer). Bias was quantified using distributional metrics—Gini coefficient and entropy—on each agent's probability assignments across demographic groups.

Results Bias amplified monotonically as judgments passed through successive agents across virtually all tested architectures and models; Gini coefficients grew at each step (e.g., from 0.13 to 0.38 over four agents for DeepSeek-R1). Architectural complexity—diverse personas, specialized functions, spindle or fully-connected topologies—did not mitigate amplification and often accelerated it. The system exhibited systemic preferences for younger individuals, females, and Black individuals even when single agents were nominally neutral. The authors also identified a 'Trigger Vulnerability': injecting a single piece of objective, neutral external text (as a standard RAG harness would) caused dramatic polarization, exposing severe fragility in system-level robustness.

Plain-language abstract This paper studies how groups of AI language model (LLM) agents form shared conventions — agreeing on a common choice when many options are equally valid — and how the design of their communication network and memory length shapes whether they reach full agreement, fragment into competing factions, or settle into stable but divided states.

Motivation As multi-agent systems built on LLMs become more capable, two key design parameters — how much past interaction each agent remembers, and how agents are connected in a network — are typically optimized separately. Prior work had shown LLM agents can develop shared conventions in fully connected settings, but real systems use constrained network topologies and bounded memory, and the interaction between these two design choices was unknown.

Methodology The authors ran 432 simulations of a networked Naming-Game on eight fixed 16-node network topologies (varying in clustering and path length) drawn from a benchmark set by Mason and Watts. In each simulation, LLM agents repeatedly chose one of ten arbitrary convention letters and were matched with network neighbors; matched choices earned points, mismatches lost points. Memory depth was varied across three values (last 2, 5, or 10 interactions), yielding a fully balanced 8-topology × 3-memory factorial design with 18 independent runs per cell and 691,200 total dyadic decisions. Agent behavior was compared against Fictitious Play and Reinforcement Learning baselines.

Results Longer memory slowed convergence in decentralized networks but accelerated settling in centralized ones, meaning the same parameter pushes outcomes in opposite directions depending on topology. Critically, faster settling in centralized networks meant locking into fragmented multi-convention plateaus, not global consensus. Fictitious Play outperformed Reinforcement Learning in predictive accuracy across all memory depths, indicating agents adapt via belief-based rather than reward-based reasoning. At the agent level, high-betweenness bridge nodes suffered a coordination penalty while locally clustered agents achieved higher coordination success.

Plain-language abstract This paper studies how shared behavioral norms can arise spontaneously among populations of AI language models (LLMs) interacting with one another, using a classic naming-game framework. The researchers show that group-wide conventions form through purely local, pairwise interactions, that collective biases toward particular conventions emerge even when individual models appear unbiased, and that small committed minorities of agents can flip an established convention if they exceed a critical size.

Motivation As large language models are increasingly deployed in multi-agent settings where they interact with each other and with humans, understanding whether and how they develop shared norms is essential for predicting AI behavior and ensuring alignment with human values. Prior work focused on single-agent bias assessment, leaving open questions about how individual biases translate to group behavior, and how stable AI-generated conventions are against deliberate change.

Methodology The authors ran simulated naming-game experiments with populations of N=24 LLM agents (also tested at N=200) drawn from four models: Llama 2 70B, Llama 3 70B, Llama 3.1 70B, and Claude 3.5 Sonnet. At each time step, two randomly selected agents tried to coordinate on a name chosen from a pool of W letters; agents were rewarded for matching their partner's choice and penalized for mismatches, retaining a memory of their last M=5 interactions. Experimental results were contrasted with predictions from a minimal analytical naming-game model to isolate LLM-specific effects.

Results Globally accepted conventions emerged spontaneously across all four models, consistent with the theoretical model. Strong collective biases toward specific conventions arose even when individual agents were statistically unbiased at first interaction—for example, Llama 2 70B and Claude 3.5 Sonnet agents showed no individual name preference (chi-squared p=0.100 and 0.410), yet the population converged non-uniformly. Committed minority groups could overturn established conventions once they reached a critical mass that varied widely by model and by the relative strength of the competing conventions: as little as 2% of the population sufficed in Llama 3 70B populations, while Llama 2 70B required up to 67%, and in Llama 3.1 70B the weaker convention could be spontaneously overturned without any committed agents at all.

Plain-language abstract This paper asks whether distinct personalities can arise in a group of AI agents that start out completely identical — with no preset traits, memories, or roles. The researchers ran a simulation where ten LLM-powered agents freely moved around a 2D grid, exchanged natural-language messages with nearby neighbors, and built up their own memories, all without any predefined characteristics. Over 100 simulation steps, the agents spontaneously developed different behavioral tendencies, emotional patterns, and personality types through their social interactions alone.

Motivation Prior work on LLM-based multi-agent systems has consistently assigned each agent a fixed personality and memory from the start, sidestepping the question of how individuality could arise on its own. The authors draw on a 'Community First' theory — grounded in studies of animal communities — which holds that a collective forms first and individual differentiation follows. The gap they address is the lack of a computational framework for watching individuality emerge from an undifferentiated group of agents through social interaction alone.

Methodology Ten homogeneous Llama-2-7b-chat-hf agents were placed at random positions on a periodic 2D grid with no initial personalities or memories. At each time step every agent generated a natural-language message, updated a situational memory summary, and chose a movement direction, all driven by prompts that included the agent's own memory and messages received from neighbors within a Chebyshev distance of 5. No personality or background context was seeded into the prompts. After 100 steps (roughly 6 hours on an A100 GPU), the researchers analyzed behavior using DBSCAN clustering, Sentence-BERT and UMAP for message/memory embeddings, word-cloud and hallucination detection via GPT-4o, BERT-based emotion extraction across six emotion categories, and Myers-Briggs Type Indicator (MBTI) personality tests administered to each agent at step 0 and step 100.

Results Agents that started with nearly identical INFJ personality profiles differentiated into five distinct MBTI types (ESFJ, ISTJ, ENTJ, ESTJ, ISFJ) by step 100, with clustering experience being a key driver — agents that had grouped with others generated 'stay' commands more frequently and developed different behavioral rhythms. Agents spontaneously invented hallucinated concepts (words like 'cave,' 'hill,' 'treasure,' 'trees' absent from the prompts) and hashtags (e.g., #cooperation, #competition), which spread virally within spatial clusters and increased overall vocabulary diversity. Emotion synchrony emerged within clusters but varied by cluster, and broader communication ranges increased message topic diversity while reducing hashtag persistence, suggesting that spatially bounded interaction is necessary for stable social norms to form.

Plain-language abstract This paper introduces Agent Workflow Memory (AWM), a method that teaches language model-based web agents to extract reusable task routines — called workflows — from past experience and apply them to guide future actions. Rather than treating each task in isolation, AWM lets agents build up a growing library of skills, from simple steps like finding a place by name to more complex multi-step procedures composed from earlier workflows.

Motivation Current language model agents struggle with long, complex web navigation tasks because they process each task independently and do not learn from past successes or failures. They lack the ability to extract and reuse common task patterns across similar contexts, making them brittle when the task environment changes even slightly.

Methodology AWM operates in two modes: offline, where it extracts workflows from annotated training trajectories before test time, and online, where it induces workflows on the fly from self-generated predictions judged correct by an evaluator. The method was evaluated on two major web navigation benchmarks — Mind2Web (1000+ tasks across 200+ domains) and WebArena (execution-based evaluation covering travel, shopping, social media, and other domains) — using language model agents that maintain and grow a memory of induced workflows over time.

Results AWM improved baseline success rates by 24.6% relative on Mind2Web and 51.1% relative on WebArena, while also reducing the number of steps needed to solve WebArena tasks. It outperformed methods that use human-expert-written workflows by 7.9% on WebArena. In cross-task, cross-website, and cross-domain generalization evaluations, online AWM surpassed baselines by 8.9 to 14.0 absolute percentage points, with the performance gap growing as the train-test distribution divergence increased — reaching as high as 22.5 points after rolling over only tens of examples.

Plain-language abstract This paper introduces Multi-Agent Debate (MAD), a framework in which multiple large language model (LLM) agents argue opposing positions under the supervision of a judge agent in order to reach better answers on hard reasoning tasks. The key insight is that having agents challenge each other forces the models to generate genuinely new lines of thought, overcoming the tendency to simply repeat or reinforce an initial answer.

Motivation Self-reflection — asking an LLM to iteratively critique and refine its own answers — is a popular strategy for improving reasoning, but the authors identify a fundamental failure mode they call Degeneration-of-Thought (DoT): once an LLM becomes confident in a (possibly wrong) answer, further self-reflection produces no new thinking and the model stays stuck. The paper argues this happens because self-reflection lacks external feedback and is vulnerable to the model's own biases and rigidity.

Methodology The MAD framework pits two LLM agents against each other in a 'tit for tat' debate, where each agent argues its position across multiple rounds; a separate judge agent monitors the exchange and decides when to stop and what the final answer is. The framework was evaluated on two benchmark tasks chosen to require deep reasoning: commonsense machine translation and counterintuitive arithmetic reasoning. The authors also analyzed how debate dynamics (number of rounds, level of disagreement between agents) affect performance, and examined what happens when different LLMs play the agent versus judge roles.

Results Experiments on commonsense machine translation and counterintuitive arithmetic reasoning show that MAD outperforms self-reflection baselines, demonstrating that structured inter-agent debate can overcome the DoT problem. Analyses found that an adaptive stopping criterion for debate and a moderate level of 'tit for tat' disagreement are both necessary for good performance; debates that are too short or too combative degrade results. The authors also found that using a different LLM as the judge than the debating agents can introduce unfairness into the evaluation.

Plain-language abstract This paper introduces DAMCS (Decentralized Adaptive Knowledge Graph Memory and Structured Communication System), a framework that lets multiple AI agents powered by large language models cooperate on complex, long-horizon tasks in an open-world environment. Rather than relying on traditional reinforcement learning approaches that require centralized training, DAMCS agents each maintain a personal memory organized as a hierarchical knowledge graph and communicate with one another through a structured protocol, enabling them to plan and act independently while still coordinating effectively.

Motivation Multi-agent coordination problems such as search-and-rescue or autonomous vehicle fleets require agents to make long-term plans under uncertainty in dynamic environments. Existing multi-agent reinforcement learning methods depend on centralized training and assume fixed cooperation strategies, making them brittle when conditions change. There was no scalable framework that allowed agents to reason from past experience and share information selectively without requiring central coordination.

Methodology The authors built a novel Multi-agent Crafter environment in which agents spawn together and must collect a diamond by crafting tools in a hierarchical order while maintaining health. Agents are driven by large language models and each maintain a multi-modal memory system organized as a hierarchical knowledge graph that stores and retrieves past experiences. A structured communication protocol governs what information agents share with each other. The system was evaluated against both multi-agent reinforcement learning baselines and LLM-only baselines across single-agent, two-agent, and six-agent scenarios.

Results DAMCS outperformed both multi-agent reinforcement learning and LLM-only baselines in task efficiency and collaboration. Compared to a single-agent scenario, a two-agent DAMCS setup achieved the same goal with 63% fewer steps, and a six-agent setup achieved it with 74% fewer steps, demonstrating that adaptive memory and structured communication substantially improve performance as the number of cooperating agents scales up.

Plain-language abstract This technical report from Altera describes Project Sid, an experiment in running large-scale simulations of 10 to over 1,000 AI agents living together in a shared Minecraft environment. The researchers built a new agent architecture called PIANO and tested whether groups of AI agents could develop the hallmarks of a functioning civilization — specialized jobs, laws, governance, and cultural transmission.

Motivation Prior AI agent research evaluated agents in isolation or in small groups, and existing benchmarks measured narrow skills like web search or coding rather than civilizational progress. Large autonomous groups of agents had never been shown to make meaningful long-term progress together, partly because individual agents accumulate hallucinations over time and groups compound those errors through miscommunication, and partly because no suitable benchmarks for civilizational-scale behavior existed.

Methodology The researchers introduced PIANO (Parallel Information Aggregation via Neural Orchestration), a cognitive architecture that runs multiple LLM-powered modules — cognition, planning, motor execution, speech — concurrently rather than sequentially, using a shared Agent State to maintain coherence across output streams. Simulations were run inside a Minecraft environment with societies of 50–100 agents and civilizations of 500–1,000 agents across multiple interacting societies. Agent performance was evaluated using new benchmarks measuring role specialization, adherence to and modification of collective rules, and cultural and religious propagation. One experiment placed 25 constituent agents and 3 influencer agents in a world with tax laws and a democratic voting system managed by a remote Election Manager agent.

Results Agents autonomously developed specialized professional identities and followed collective rules. In the taxation experiment, constituents deposited roughly 20% of their inventory as required by the initial constitution, and when pro-tax or anti-tax influencers shifted agent sentiment and triggered democratic amendments, tax compliance shifted accordingly — for example, after a constitutional change reducing the tax rate from 20% to 5–10%, average taxes paid dropped from 20% to 9%. Control runs showed that freezing the constitution or removing key brain modules prevented this bidirectional responsiveness, confirming the architecture's role. Agents also demonstrated cultural and religious transmission across the simulated societies.

Plain-language abstract This paper introduces "generative agents" — software agents powered by large language models that can simulate believable human behavior over time. The researchers built a virtual town populated by twenty-five such agents, each capable of waking up, forming daily plans, holding conversations, remembering past events, and coordinating social activities, all driven by an underlying language model architecture.

Motivation Prior computational agents could simulate plausible human behavior at a single moment in time, but lacked the ability to maintain coherent, evolving identities across longer timescales. There was no architecture for managing a continuously growing memory of experiences, reflecting on those memories to form higher-level understanding, or coordinating emergent social dynamics among multiple agents simultaneously.

Methodology The authors designed a three-component agent architecture built on top of the ChatGPT language model. A memory stream stores all of an agent's experiences as natural-language records; a retrieval model surfaces relevant memories using a combined score of recency (exponential decay over time), importance (rated by the language model itself), and relevance. A reflection module synthesizes memories into higher-level inferences, and a planning module translates those inferences into daily and moment-to-moment behaviors. Twenty-five agents were instantiated in an interactive sandbox environment inspired by The Sims, and the system was evaluated through both controlled "interview" probes of individual agents and an open-ended two-day simulation.

Results Generative agents produced believable individual and emergent social behaviors. In one demonstration, a single seed instruction — that one agent wanted to throw a Valentine's Day party — led agents to autonomously spread invitations, form new acquaintances, ask each other on dates, and coordinate arrival at the party without further human input. Ablation studies showed that each architectural component (memory retrieval, reflection, and planning) was critical to believable behavior; removing any one of them degraded performance across interview-based evaluation tasks. The most common failure modes were agents failing to retrieve relevant memories, fabricating embellishments to memory, or defaulting to overly formal language inherited from the base language model.

Plain-language abstract This paper asks whether a group of simulated ants can learn to use pheromone trails to coordinate food-gathering without being given any explicit instructions about when or how to do so. Each ant is controlled by a spiking neural network — a type of artificial brain that processes information as discrete spikes, similar to biological neurons — and the network's connection weights are tuned by an evolutionary algorithm over many generations. The result is a colony that spontaneously discovers pheromone-based communication on its own.

Motivation Prior computational models of swarm coordination have relied on hand-crafted rules or probabilistic decision tables to produce collective behaviour such as foraging. Designing these rules is difficult, does not scale well as task complexity grows, and may steer the swarm toward solutions that are functional but not optimal. The paper targets the 'design problem': can useful collective strategies, including communication, emerge automatically from an optimisation process rather than from a designer's intuition?

Methodology The authors built a 2D simulated environment based on Wilensky's NetLogo ant-colony model, where pheromone can diffuse and evaporate across a grid. A colony of 15 ants is controlled by copies of a single spiking neural network (SNN) implemented in the NEST simulator. A genetic algorithm — run through the Learning to Learn (L2L) framework — optimises the synaptic weights and spike-time delays of that network across populations of 32 individuals per generation, evaluating each individual by how efficiently the colony retrieves food and returns it to the nest. No rule about pheromone use is pre-wired; the network starts with no knowledge of the task.

Results Through evolutionary optimisation, pheromone-based stigmergic communication emerged spontaneously: ants learned to deposit pheromone near food sources and near the nest without any pre-coded instruction to do so. Colonies in which this communication emerged outperformed those in which it did not, and disabling the pheromone sensor after training drastically reduced foraging performance, confirming that the emergent signals are functionally important. The evolved SNN-based colony achieved foraging performance comparable to a hand-designed rule-based multi-agent system. The authors also observed that pheromone acts as an attractor rather than an event-based signal, resembling the positive pheromone trails seen in real ant species.

Plain-language abstract This paper presents PheroCom, a coordination model for swarms of robots that mimics how ants use chemical signals (pheromones) to navigate and work together. Instead of relying on a central controller, each robot maintains its own local map of pheromone concentrations and shares updates with nearby robots through a gossip-like wireless communication protocol inspired by ant vibroacoustic signaling. The model was tested in simulation across multiple environments and swarm sizes to validate its ability to perform indoor surveillance tasks.

Motivation Most bio-inspired swarm robotics approaches that simulate pheromone dynamics require a centralising agent to maintain and distribute a shared pheromone map, which conflicts with the core swarm robotics requirement for decentralised, autonomous robots. Existing alternatives such as volatile chemicals, RFID tags, and intelligent environments are expensive, task-specific, or impose constraints that limit swarm autonomy. The paper addresses the gap of producing a fully decentralised and asynchronous pheromone-based coordination model that preserves the robustness, scalability, and flexibility expected of swarm systems.

Methodology The authors designed PheroCom around a finite-state machine governing each robot's behavior, with each robot holding an independent virtual pheromone map implemented as a cellular automaton grid. Robots deposit and sense pheromone locally, and periodically disseminate their local map data to nearby robots using the ViBIT (Vibroacoustic Based Indirect Transmission) protocol—a gossip-style broadcast limited to a configurable transmission radius. Aggregating robots filter incoming data using stored transmission histories to avoid processing duplicate information and only update cells where the incoming pheromone concentration is higher than the local value. The model was evaluated in a custom agent simulation framework built by the authors and in the Webots robotics simulator, using multiple indoor environment layouts (different shapes and sizes), varying numbers of robots, and three movement strategies (deterministic, inertial, and a 2/3 inertial / 1/3 deterministic heterogeneous mix).

Results PheroCom achieved surveillance coverage results comparable to the predecessor centralised model (IACA-DI) across all tested environments, swarm sizes, and movement strategies. The most striking quantitative finding is communication efficiency: PheroCom accomplished the same tasks using approximately 1.69% of the information volume transmitted by IACA-DI, a roughly 59-fold reduction in communication overhead. The heterogeneous movement strategy allowed the swarm to visit all rooms within one hour of simulation, and by five to ten hours both models produced similarly homogeneous environment coverage. The model demonstrated robustness and scalability by adapting to varied environment shapes and robot counts without architectural changes.

Plain-language abstract This paper proposes Global Workspace Agents (GWA), a multi-agent architecture for large language models inspired by the cognitive science concept of Global Workspace Theory. Rather than treating AI systems as passive input-output machines, GWA organizes a swarm of specialized AI agents around an active central broadcast hub that continuously cycles information, enabling sustained autonomous reasoning without constant human prompting.

Motivation Current large language models and multi-agent frameworks operate reactively, only responding when prompted, and lack mechanisms for continuous self-directed thought. When multiple identical agents are connected in a loop, they rapidly fall into cognitive stagnation — repeating similar ideas, forming echo chambers, and failing to progress — because no agent has the mandate to introduce genuinely novel information or shift the system's reasoning trajectory.

Methodology The authors designed GWA around a discrete cognitive cycle called a Cognitive Tick, which proceeds through four synchronized phases: Perceive and Retrieve (an Attention Agent queries long-term vector memory via RAG), Think (a Generator Agent proposes candidate thoughts at a dynamically regulated temperature, then a Critic Agent scores each one), Arbitrate (a Meta Agent selects the winning thought and decides whether to continue reasoning or respond), and Update (state is written back and memory is managed). To prevent reasoning loops, the system measures Shannon entropy over recent thought embeddings and exponentially increases the Generator's sampling temperature when entropy drops, forcing semantic exploration. A dual-layer memory strategy bifurcates working memory into a compressed short-term cache and a long-term vector archive to stay within model context limits.

Results The paper presents GWA as a theoretical and engineering framework rather than an empirical benchmark study, so quantitative performance comparisons against baselines are not reported in the retrieved text. The contribution is the architecture itself: a reproducible, mathematically grounded design that couples entropy-based intrinsic drive with heterogeneous agent specialization and dual-layer memory to sustain coherent autonomous operation in LLM-based systems.

Plain-language abstract SkillClaw is a framework that lets AI agents improve their reusable skills automatically by pooling experience across many users. Rather than each user rediscovering the same solutions independently, the system collects interaction traces from all users, identifies recurring patterns, and updates a shared skill library that every agent benefits from—without requiring any extra effort from users.

Motivation LLM-based agents like OpenClaw rely on a fixed set of reusable skills that remain static after deployment. As a result, similar workflows, tool-usage patterns, and failure modes are repeatedly rediscovered across users, and no mechanism exists to convert heterogeneous cross-user experience into reliable skill improvements. A single user rarely produces enough signal to distinguish a generalizable fix from an idiosyncratic one, so aggregating evidence across users is necessary for stable evolution.

Methodology SkillClaw operates in a closed loop: independently deployed agents record full session trajectories—capturing prompts, tool calls, intermediate feedback, and final responses—and upload them to a shared evidence base. A centralized, autonomous evolver periodically processes these trajectories to identify recurring behavioral patterns, then refines existing skills or adds new capabilities to a shared skill repository. Updated skills are synchronized back to all agents, so improvements discovered in one context propagate system-wide. Experiments were conducted on WildClawBench, a real-world agent benchmark, using Qwen3-Max as the underlying model.

Results Experiments on WildClawBench demonstrate that SkillClaw yields substantial improvements across tasks with limited interaction and feedback, highlighting the effectiveness of multi-user driven collective evolution for building continuously improving agent systems in real-world environments.

Plain-language abstract This paper introduces M* (MSTAR), a system that automatically designs custom memory setups for AI agents. Rather than forcing all agents to use the same type of memory—such as conversation history retrieval or skill reuse—MSTAR figures out, through an evolutionary process, which memory design works best for each specific task.

Motivation AI agents powered by large language models need memory systems to retain and reuse knowledge across extended interactions, but different tasks require fundamentally different memory designs. A memory system tuned for conversational agents fails when applied to coding or web-browsing agents, and no principled method existed to automatically discover the best memory design for a given task.

Methodology MSTAR represents any agent memory system as an executable Python program (a "memory program") that encodes what to store and retrieve (Schema) and how to read and write that information (Logic). It then applies a reflective code evolution process to search over possible memory programs, evaluating and iteratively refining them against the target task. The approach was evaluated on four benchmarks spanning different agent task domains, compared against baselines from three established memory paradigms.

Results MSTAR consistently improved agent performance over fixed-memory baselines across all evaluated tasks. The evolved memory programs developed structurally distinct processing mechanisms for each domain, demonstrating that task-specific memory specialization explores a broader design space and yields better solutions than general-purpose memory approaches.

Plain-language abstract This paper argues that as AI agents move from controlled enterprise pipelines to open internet-scale societies — where agents discover each other through public registries and interact without central oversight — the field needs a proper theory of governance, not just risk checklists. The authors borrow a classic framework from sociology (Parsons' AGIL) to design a formal governance architecture for these agent societies, then diagnose how far current real-world ecosystems fall short of it.

Motivation Existing AI governance approaches focus on risk enumeration and process compliance suited to local, human-orchestrated multi-agent pipelines. The shift to internet-wide agent societies — where autonomous agents discover each other through open registries, interact without central orchestrators, and generate emergent social behaviors — creates a qualitatively different governance challenge that existing frameworks do not address. There is no theoretical grounding for what institutional structures such societies require to function stably.

Methodology The authors apply Talcott Parsons' AGIL framework, which identifies four functional imperatives (Adaptation, Goal Attainment, Integration, and Latency/Pattern Maintenance) that every viable social system must satisfy, to derive a prescriptive sixteen-cell institutional architecture for agent governance. They then diagnostically apply this architecture to the OpenClaw ecosystem (250,000+ GitHub stars, 2 million+ monthly users, 770,000+ registered agents) using a recursive sub-function analysis of 64 binary indicators across 16 cells. A complementary assessment examines twelve inter-pillar media pathways. The diagnostic is then extended to the broader agent-native protocol stack including MCP, A2A, ANP, x402, and ERC-8004.

Results The analysis finds at most 19% sub-function coverage in the OpenClaw ecosystem (sensitivity range 17–30%), and crucially, zero inter-cell coordination (I-sub = 0%), meaning existing technical infrastructure cannot participate in inter-pillar interchange and represents potential rather than operative governance capacity. All twelve inter-pillar media pathways were found to be non-functional. The Fiduciary (Latency) and Political (Goal Attainment) pillars are most severely underserved. Extending the analysis to independent teams building agent-native protocols reveals the same structural pattern, confirming the governance gap is a systemic feature of market-driven agent infrastructure development rather than a sign of one ecosystem's immaturity. The paper concludes with a prioritized roadmap for the missing governance infrastructure.

Plain-language abstract Auto-Dreamer is a system that teaches AI language agents to periodically reorganize and compress what they have learned across many tasks into a smaller, more useful memory bank. Rather than storing every observation from every session, it uses a trained background process to rewrite accumulated memories into compact, generalized knowledge that makes future task performance better.

Motivation Language agents that work on streams of related tasks need memory that does more than just record past observations — they need to extract reusable patterns and procedures from experience. Existing memory systems couple information acquisition and reorganization into a single online step, so each update lacks a global view across sessions, making it hard to discover recurring patterns, remove redundant entries, or resolve contradictions.

Methodology The authors separate fast per-session memory writing from a slower offline consolidation step, inspired by complementary learning systems theory of human memory. The consolidator, Auto-Dreamer, treats a selected region of a typed memory bank as read-only evidence, uses multi-step tool calls to inspect entries and retrieve original source trajectories, and then synthesizes a fresh replacement set that abstracts across sessions. It is trained with GRPO reinforcement learning using a composite reward that combines downstream task success with a counterfactual utility term — estimated by randomly masking memory entries — that penalizes redundant entries and rewards load-bearing ones. Training used ScienceWorld task trajectories only, while the task agent and per-session writer were kept fixed.

Results On ScienceWorld, Auto-Dreamer achieves 41.1% task success, 7 percentage points above the strongest baseline, while keeping an active memory bank 12 times smaller than that baseline. Evaluated without retraining on held-out benchmarks, it reaches 60.2% success on ALFWorld using 6 times less memory than the strongest baseline there, and 52.3% on WebArena, demonstrating cross-domain generalization from training on a single environment.

Plain-language abstract This paper presents Unified Context Evolution (UCE), a system that allows AI agents powered by large language models to accumulate and reuse knowledge across tasks without modifying the model itself. Rather than starting each task from scratch, UCE builds a growing library of structured knowledge entries that are retrieved and injected into the agent's prompt at decision time, enabling the agent to apply lessons learned from past episodes.

Motivation LLM-based agents can solve multi-step tasks by combining reasoning with environment feedback, but each episode starts from the same fixed context and any useful strategy discovered during a task is lost once it ends. Existing approaches either restrict learning to the current task or pool all experience into a single untyped store, without distinguishing knowledge types, tracking quality through repeated use, or balancing what kinds of knowledge the library still lacks.

Methodology UCE externalizes agent experience into an evolving library of typed Evolvable Context Units (ECUs), decomposed into four complementary types: Memory (how an environment mechanism behaves), Strategy (what the agent should choose under a condition), Workflow (what execution sequence to follow), and Skill (how a reusable operation should be performed). Each ECU type has independent generation conditions drawn from task trajectories, is retrieved at decision time and injected into the actor prompt, scored through repeated usage outcomes, and pruned when low-value or redundant. A Knowledge Yield Scheduling module allocates each cycle's generation budget toward the types where the library is weakest. The framework requires no gradient access and leaves the model backbone unchanged.

Results Evaluated on two interactive benchmarks, UCE raised ALFWorld success rate from 75.4% to 96.3% and WebShop task score from 45.1% to 61.3%. The accumulated ECU library transferred effectively to four alternative actor backbones without retraining.

Plain-language abstract DeLM (Decentralized Language Models) is a multi-agent framework that drops the central controller. Instead of a main agent assigning subtasks, waiting, and merging results, parallel agents asynchronously claim tasks from a shared queue and read/write a shared 'verified context' of accumulated progress. It targets two settings — parallel exploration in software engineering and concurrent evidence processing in long-context QA — and improves accuracy while roughly halving cost.

Motivation Most multi-agent systems use centralized scatter-gather orchestration, which parallelizes sub-agent execution but not the coordination around it. Every finding must return to the main agent to be merged and rebroadcast, so progress-sharing becomes a serialized bottleneck as agents grow, and the controller can dilute, omit, or distort details. In long-context reasoning the main agent must pre-assign evidence clusters before knowing what is relevant, triggering extra delegation rounds. DeLM removes the controller as the coordination chokepoint.

Methodology Coordination is state-based, not prompt-routed. Two global structures: a shared context C of compact verified gists and a task queue T of pending subtasks. The pipeline initializes the queue from the input, executes ready subtasks in parallel, then compresses-verifies-admits each result into the shared context, generates more subtasks when the context is insufficient, and finalizes once none remain. The shared context is compact, global, and 'unfoldable' — agents read coarse gists by default and expand to detailed summaries or raw evidence only when needed. Admission-time verification checks each update against its underlying evidence and reasoning trajectory before it enters shared state, rejecting or regenerating unsupported updates so errors cannot propagate as reusable problem state.

Results On SWE-bench Verified, DeLM is strongest across test-time-scaling metrics, reaching 77.4% pass@4 at ~$0.12/task — roughly half the baselines' cost — with trace-level examples showing agents reuse each other's discoveries through the compact shared context. On LongBench-v2 Multi-Doc QA it leads four frontier model families by up to 5.7 points, with both admission-time verification and hierarchical summarization contributing. On OOLONG, vanilla DeLM underperforms RLM (which needs exact row-level aggregation via code execution), but RLM combined with DeLM yields the best accuracy and lowest cost, showing DeLM works as a coordination layer for programmatic reasoning too.

Plain-language abstract ESAA-Conversational is a shared memory layer that lets several LLM coding agents — Codex, Claude Code, Grok — hand work off to one another without copying the conversation by hand. It watches each agent's visible turns through hooks or watchers, writes them into one append-only log (activity.jsonl), and deterministically projects compact files — a handoff contract, current state, recorded decisions, and an open-task list — that the next agent reads to pick up where the last left off.

Motivation Developers increasingly switch among multiple coding agents as context windows fill or a different tool suits a subtask, but each agent keeps its conversation in a private, vendor-specific log. The result is 'conversational state drift': goals, rejected alternatives, decisions already made, and open tasks established with one agent are not reliably available to the next. The usual fix — copy-pasting context — is manual, lossy, expensive in tokens, and conflates capturing evidence with interpreting it.

Methodology The system applies event sourcing and CQRS: visible turns are captured mechanically, with no LLM inference, into an append-only activity.jsonl that is the single source of truth, while state.md, handoff.md, decisions.md, and tasks.json are reconstructible read models that are never hand-edited. 'Inverted ingestion' means the runtime reads native agent logs, hooks, or watchers and normalizes them into conversation_turn events rather than requiring agents to share a protocol. A strict boundary separates mechanical capture (turns are evidence) from curation (durable decisions and tasks entered through explicit decide/task commands). A paginated context command serves filtered windows (--last, --around, --before, --topic) so a cold agent reads a slice, not the whole log; workspace_root isolates projects and a lockfile serializes writes. The v1.1.0 release is a local PowerShell CLI.

Results A self-referential case study recorded 570 events (562 conversation turns) in a single workspace on 21 June 2026, distributed across Codex (304), Claude (79), and Grok (67). The three heterogeneous agents co-designed and reviewed the tool through the shared log alone, with no direct agent-to-agent channel — e.g., Codex was given a focused view of recent Grok iterations via `context --agent grok --last 20`, and one concrete defect (incomplete filtering of legacy events under context --topic) was found, fixed, and closed as a task. The public release ships 51 tests in its main battery. Reported limitations: the implementation is Windows/PowerShell-only, sync depends on third-party hook surfaces outside the authors' control, retrieval is purely textual with no embeddings, and the system offers operational but not forensic auditability (no hash chains or signatures); validation covers one workspace and three agents.

Plain-language abstract This paper investigates why multi-agent systems built from large language models so often fall short of expectations. The authors created MAST, a structured catalog of failure types, by carefully examining hundreds of conversation traces from seven widely-used multi-agent frameworks. The result is a practical taxonomy that names and organizes 14 distinct ways these systems break down, along with an automated tool to apply that taxonomy at scale.

Motivation Multi-agent LLM systems have attracted considerable interest because they can, in principle, divide complex tasks among specialized agents and coordinate their work. Yet empirical results consistently show their performance gains over single-agent setups are small or absent — for instance, one studied framework (ChatDev) solved only 33% of programming tasks. No systematic account of why these systems fail existed, leaving developers without a principled framework for diagnosis or improvement.

Methodology The authors analyzed more than 200 conversation traces drawn from seven open-source multi-agent frameworks (including MetaGPT, ChatDev, HyperAgent, AppWorld, AG2, and Magentic-One) running on diverse benchmarks. Six expert human annotators applied Grounded Theory to label failures in the traces, with inter-annotator agreement measured by Cohen's Kappa (reaching 0.88). An LLM-as-a-Judge pipeline using OpenAI's o1 was then developed and validated against expert labels (Kappa 0.77) to enable scalable automated annotation. Two case studies tested whether targeted interventions could reduce the identified failures.

Results The analysis identified 14 distinct failure modes organized into three categories: specification issues (system design problems, 41.77% of failures), inter-agent misalignment (coordination failures, 36.94%), and task verification failures (quality control, 21.30%). Prominent individual failure modes included step repetition (17.14%), reasoning-action mismatch (13.98%), and disobey task specification (10.98%). Targeted interventions such as improved role specification yielded only modest gains (e.g., +15.6% for ChatDev), indicating that the identified failures stem from fundamental system design challenges rather than easily patched prompt issues.

Plain-language abstract TraceFix is a system that automatically designs, verifies, and repairs coordination protocols for groups of AI agents working together. When multiple AI agents must share resources, pass messages, and coordinate actions concurrently, subtle bugs like deadlocks can emerge. TraceFix uses formal verification—exhaustive model checking with TLA+—to find these bugs before deployment and iteratively fixes the protocol until no violations can be found, then enforces the verified protocol at runtime.

Motivation As large language model (LLM) agents increasingly run concurrently and share external state, coordination failures—races, deadlocks, missed handshakes, and premature termination—become the dominant source of system failures rather than individual agent capability. These bugs depend on rare execution schedules and can remain hidden through all observed runs yet appear under untested interleavings. Prior approaches address coordination through orchestration frameworks or runtime guards but do not verify that the concurrent protocol itself is free of such hazards.

Methodology TraceFix operates as a four-stage pipeline: an orchestration agent synthesizes a protocol topology (a structured intermediate representation defining agents, shared locks, and directed message channels) from a natural-language task description; the topology is compiled into PlusCal coordination logic; the TLA+ model checker (TLC) exhaustively searches for counterexample traces under bounded assumptions; and a repair agent revises the PlusCal source based on the counterexample until TLC finds no further violations. Verified process bodies are then compiled into per-agent system prompts, and a runtime monitor rejects any coordination operations outside the verified topology. The approach was evaluated on a benchmark of 48 tasks spanning 16 scenario families at three difficulty tiers, with a 3,456-run runtime comparison across four architectures and two model capability tiers.

Results All 48 benchmark tasks reached full TLC verification; 62.5% passed on the first attempt and none required more than four repair iterations. Bounded model checking remained tractable across all tasks (median under 1 second, maximum under 60 seconds) even for state spaces reaching millions of distinct states. Topology-monitored execution achieved the highest task completion rates (89.4% average, 81.5% full completion) and degraded at roughly half the rate of prompt-only and chat-only baselines when model capability was reduced. A paired ablation showed that TLC-verified protocols cut deadlock and livelock occurrences from 31.1% to 14.1%, with the largest separation under fault injection conditions.

Plain-language abstract This paper introduces MPAC (Multi-Principal Agent Coordination Protocol), a new communication standard that lets AI agents owned by different people or organizations work together on shared tasks without a central authority overseeing them. Today's agent protocols assume one person or organization controls all the agents involved, which breaks down when independent parties — like two engineers whose coding agents both edit the same file — need to coordinate. MPAC defines how agents declare intentions before acting, handle conflicts explicitly, and escalate disagreements to their human owners when needed.

Motivation Existing AI agent protocols (MCP and A2A) both assume a single controlling principal who owns and trusts every participating agent, but many real coordination problems involve independent principals whose agents must work on shared state — two engineers' coding agents editing the same repository, family members' agents planning a trip together, or agents from different organizations drafting a joint contract. When this single-principal assumption breaks down, coordination collapses to ad-hoc chat, manual merging, or silent overwrites. No existing protocol addressed this multi-principal coordination gap.

Methodology MPAC is specified as a five-layer application-layer protocol (Session, Intent, Operation, Conflict, and Governance layers) with 21 message types, three state machines with normative transition tables, Lamport-clock watermarking for causal ordering, two execution models (pre-commit and post-commit), three security profiles, and an optimistic-concurrency-control mechanism for shared state. The authors released two interoperable reference implementations (Python with 122 tests; TypeScript with 101 tests), 66 adversarial enforcement tests, a machine-readable JSON Schema suite covering all 21 message types, and seven live multi-agent demos spanning scenarios such as code editing, trip planning, pre-commit authorization with fault recovery, and multi-level conflict escalation.

Results A controlled three-agent cross-module code review benchmark showed a 95% reduction in coordination overhead (from 68.65 seconds to 3.02 seconds) and a 4.8x wall-clock speedup (from 131.76 seconds to 27.38 seconds) under MPAC compared to a serialized human-mediated baseline. Per-agent decision time was preserved (63.11 seconds versus 57.13 seconds), confirming that the speedup comes from eliminating coordination waits rather than compressing model inference. The full specification, reference implementations, test suites, and demo transcripts were released as open source.

Plain-language abstract When multiple AI agents collaborate by passing their internal memory (called KV caches) directly to one another, sensitive information can leak through those representations even if it never appears in any text the agents produce. This paper introduces LCGuard, a system that intercepts and transforms those internal memory artifacts before they are shared, so that an attacker who captures the shared data cannot reconstruct the private inputs that produced them, while the agents still perform their tasks effectively.

Motivation Recent work has shown that passing transformer key-value caches between agents is more efficient and informationally richer than exchanging natural language messages, but KV caches implicitly encode private context, retrieved documents, and intermediate reasoning states. Existing safety mechanisms in multi-agent systems only check generated text or tool actions, leaving this representation-level channel completely unguarded. There was no principled framework for controlling what sensitive information is recoverable from KV representations intentionally shared across agents.

Methodology LCGuard frames the problem as a minimax adversarial game: a set of learnable communication functions transform each agent's KV cache before transmission, while an adversarial decoder is simultaneously trained to reconstruct the agent's private input from the shared artifacts. The communication functions are optimized to minimize a task loss (so agents remain useful) and maximize the adversary's reconstruction loss (so private inputs become unrecoverable), with a tunable tradeoff hyperparameter beta. The framework is evaluated on Qwen3 (4B, 8B, 14B), Gemma2-9B, and LLaMA (3B, 8B) across sequential, hierarchical, and graph-based multi-agent topologies, using the AgentLeak, MAGPIE, and PrivacyLens benchmarks.

Results LCGuard consistently reduced attack success rate (ASR) by approximately 65-75% relative to vanilla KV sharing while maintaining competitive task helpfulness. For example, on Qwen3-4B in a sequential PrivacyLens setting, ASR dropped from 0.871 to 0.216 while helpfulness held at 0.710; on Gemma-9B, ASR fell from 0.885 to 0.205 with helpfulness remaining at 0.735. In contrast, the noise-based ADAPT baseline achieved similarly low ASR but collapsed helpfulness (e.g., to 0.285 on Qwen3-4B), while the output-level PrivAct baseline left ASR largely unchanged. Full-system joint optimization across all communication edges outperformed per-agent sanitization, indicating that leakage can re-emerge through aggregation across multiple agents.

Plain-language abstract This paper introduces the Imperfect Byzantine Generals Problem (IBGP), a new coordination framework for multi-agent AI systems that need to work together reliably even when some agents are compromised or behaving incorrectly. Unlike the classic Byzantine Generals Problem, which requires all agents to agree, IBGP only requires a minimum threshold of agents to reach consensus, matching how real multi-agent tasks actually work. The authors design a consensus protocol for IBGP and show it can be embedded into large language model agents via simple prompt instructions, without any additional training.

Motivation As LLM-based agents are increasingly deployed in real-world infrastructure — from autonomous vehicles to sensor networks — ensuring they can coordinate safely in the presence of malicious or hallucinating agents becomes critical. Existing Byzantine consensus protocols require global agreement among all agents, which is both unnecessarily strict and inefficient for most multi-agent tasks, where only a subset of agents needs to coordinate. There was no formal framework or protocol tailored to this weaker, more practical notion of partial consensus.

Methodology The authors formally define IBGP and a corresponding (k, lambda)-protocol that operates in multiple broadcast rounds, using a global randomizer to determine the number of rounds, so that attackers cannot predict when the final decision will be made. They integrate this protocol as a coordination module into a decentralized multi-agent reinforcement learning pipeline (Dec-POMDP with Q-learning), testing it across environments including Predator-Prey (small and large scale), Hallway, and two StarCraft II scenarios (4bane_vs_1hM and 3z_vs_1r). Robustness is evaluated as the ratio of performance under adversarial communication attacks during testing versus unattacked performance during training.

Results The IBGP protocol achieved near-perfect or perfect robustness percentages (up to 100%) across most tested environments, substantially outperforming baseline approaches including recursive training, AME, and ADMAC, which frequently collapsed to 0% robustness or failed to converge under adversarial attack. The protocol tolerates up to 50% malicious agents — compared to 33% for classical BGP — while requiring less redundancy. A sensor network case study further demonstrated that the protocol maintains consistent belief about a moving target's position despite communicative attacks.

Plain-language abstract This paper introduces ACIArena, a benchmark framework for testing how well multi-agent AI systems resist a class of attacks called Agent Cascading Injection (ACI), where a compromised agent spreads malicious instructions to other agents in the network. The framework provides a standardized way to evaluate six real-world multi-agent system designs against a range of attack types and also proposes a new defense strategy.

Motivation Multi-agent AI systems are increasingly deployed in real-world products, but their collaborative nature creates a security vulnerability: a single compromised agent can exploit inter-agent trust to propagate harmful instructions across an entire system. Prior work on this threat was fragmented, covering only limited attack surfaces or simplified system configurations, making it hard to compare defenses or generalize findings to realistic deployments.

Methodology ACIArena provides a unified evaluation framework covering multiple attack surfaces (external inputs, agent profiles, and inter-agent messages) and three attack objectives (instruction hijacking, task disruption, and information exfiltration). It standardizes interfaces for building both multi-agent systems and attack-defense modules, covers six widely used multi-agent system implementations, and supplies a benchmark of 1,356 test cases. The authors also propose a defense called ACI-Sentinel that focuses on preserving task-aligned information rather than trying to detect suspicious messages.

Results Experiments show that current multi-agent systems are broadly vulnerable to ACI attacks across all tested configurations. Evaluating robustness by network topology alone is insufficient; robust systems require deliberate role design and controlled interaction patterns. Defenses built in simplified or narrow settings frequently fail to transfer to realistic scenarios, and in some cases amplify the impact of attacks rather than reducing it.

Plain-language abstract This paper addresses a security problem in systems where multiple AI language model agents collaborate by exchanging responses over a network. The authors introduce Self-Anchored Consensus (SAC), a protocol that lets honest agents detect and ignore bad or adversarial inputs from compromised peers, then iteratively refine their own answers without relying on any central coordinator.

Motivation When AI agents collaborate peer-to-peer, unreliable or adversarially corrupted agents — called Byzantine agents — can push neighboring agents toward wrong conclusions and degrade overall system performance. Existing defenses depend on leader-based coordination or agents self-reporting their own confidence, both of which an adversary can exploit, leaving decentralized multi-agent systems without a robust solution.

Methodology The authors design SAC as a fully decentralized, iterative filter-and-refine protocol: at each round agents exchange responses, locally evaluate incoming messages to filter out unreliable ones, and refine their own outputs. They derive formal (F+1)-robustness conditions on the communication graph that guarantee honest agents can preserve and propagate correct information even when up to F neighbors are Byzantine, and evaluate the protocol on mathematical and commonsense reasoning benchmarks across diverse communication topologies.

Results SAC effectively suppresses Byzantine influence and consistently improves performance across all tested communication topologies on the mathematical and commonsense reasoning benchmarks, while prior methods degrade under the same adversarial conditions.

Plain-language abstract This paper introduces Conformal Social Choice, a post-hoc decision layer for multi-agent AI debate systems that converts debate outputs into calibrated act-versus-escalate decisions. Rather than always acting on whatever answer a group of AI agents agrees on, the system can recognize when agent consensus is uncertain enough that a human should review the case instead. It requires no retraining of the underlying models and works as a plug-in layer on top of any multi-agent debate pipeline.

Motivation Multi-agent debate systems improve AI reasoning by having multiple language models argue and converge on answers, but they have a critical blind spot: agents can converge on a wrong answer with high confidence, and existing pipelines have no way to detect this. Current systems reduce debate to a single vote or majority answer and treat agreement as a signal that it is safe to act — but agreement and correctness are not the same thing, leaving no safety check before automated action.

Methodology The pipeline has four stages: heterogeneous LLM agents (Claude Haiku, DeepSeek-R1, and Qwen-3 32B) debate for multiple rounds and each agent outputs explicit numerical probability distributions over answer choices; these distributions are aggregated into a single social probability via a linear opinion pool; a conformal calibration procedure uses a held-out validation set to compute a threshold; and a hierarchical action policy maps high-confidence singleton prediction sets to autonomous action while larger sets trigger human escalation. The system was evaluated on eight MMLU-Pro subject domains with a coverage target of 95% (alpha = 0.05).

Results The conformal layer achieved coverage within 1 to 2 percentage points of the 95% target across all eight domains. Crucially, 81.9% of cases where the agents reached a wrong consensus were intercepted and escalated rather than acted upon. The cases the system chose to act on autonomously reached 90.0 to 96.8% accuracy, up to 22.1 percentage points above the accuracy of naive consensus stopping. By contrast, consensus stopping alone showed accuracy as low as 67.9% on some domains, with no mechanism to flag incorrect confident predictions, including a 17.4% error rate in Engineering despite apparent model agreement.

Plain-language abstract This paper introduces CUGA's policy system, a governance framework for AI agents deployed in enterprise settings. It adds a modular policy layer on top of a general-purpose large language model agent, enforcing rules about what the agent can do, when it must pause for human review, and what it can share — all without retraining the model.

Motivation Autonomous AI agents are being deployed in complex enterprise workflows, but production use requires predictable, auditable, and compliant behavior. Existing approaches typically constrain agents after the fact or require rebuilding the agent for each new domain, leaving a gap for a reusable governance architecture that can be composed with any generalist agent.

Methodology The system intercepts the agent at five structural checkpoints during execution: an Intent Guard upstream of planning to block harmful requests, a Playbook injected into the system prompt to steer reasoning, a Tool Guide at the tool-call boundary to enforce proper usage, a Human-in-the-Loop gate outside the reasoning loop for high-risk actions, and an Output Formatter at the response stage. The approach is demonstrated through a healthcare scenario that exercises dynamic playbook injection, intent blocking, and human approval checkpoints for potentially destructive actions.

Results The demo artifact shows that typed governance primitives composed across these five checkpoints enable faster and safer deployment of enterprise agentic systems, improving policy adherence and execution consistency without requiring model fine-tuning.

Plain-language abstract AGrail is a safety framework designed to protect AI agents powered by large language models from being tricked into harmful actions. It works by dynamically generating and refining safety checks that are applied before an agent executes any action, and it can also invoke specialized tools to verify that an action is safe in its specific environment. The system is designed to work across different tasks and agent types without requiring manual configuration for each new scenario.

Motivation As large language models are deployed as autonomous agents that take real-world actions—browsing the web, running code, managing files, querying databases—they become vulnerable to both deliberate attacks (like prompt injection) and inadvertent harm (like overwriting critical files). Existing defenses are limited because they rely on manually specified safety rules that do not generalize across tasks, or they use LLMs that generate either overly restrictive or overly permissive safety policies, blocking legitimate actions or allowing dangerous ones.

Methodology AGrail uses two collaborative LLMs in an iterative test-time adaptation loop: one proposes and refines safety checks guided by universal safety criteria, and the other executes those checks and can call auxiliary tools such as an OS environment detection tool to verify conditions in the real environment. Safety checks are stored in a memory module and optimized over time so that the most effective checks for each type of agent action are retained. The framework was evaluated on real-world agent outputs across several benchmarks: Mind2Web-SC and EICU-AC for task-specific risks, AdvWeb and EIA for prompt injection attacks on web agents, and a newly constructed Safe-OS benchmark covering prompt injection, system sabotage, and environment attacks on OS agents.

Results Using Claude 3.5 Sonnet as the underlying model, AGrail preserved 96% of benign agent actions while reducing the attack success rate for prompt injection to 0% on Safe-OS. For environmental attacks and system sabotage on Safe-OS, attack success rates were reduced to 3.8% and 5% respectively. On the AdvWeb benchmark, attack success rate was reduced to 0%, and on EIA it averaged 17% across action generation and action grounding tasks. The framework also demonstrated transferability across different LLM agent tasks.

Plain-language abstract This paper surveys the safety risks that arise when large language models (LLMs) are deployed in real-world applications and reviews the technical strategies used to constrain their behavior. It covers known failure modes—bias, hallucinations, dataset poisoning, lack of explainability, privacy leakage, and non-reproducibility—and then maps out both existing guardrail frameworks and the open challenges in designing them well.

Motivation LLMs are increasingly used in high-stakes domains such as law and medicine, yet they remain probabilistic systems that can produce biased, fabricated, or harmful outputs without warning. There was no consolidated account of the full risk landscape alongside a structured view of mitigation strategies, leaving developers and deployers without a clear framework for safe deployment.

Methodology The authors conduct a structured literature review, drawing on academic papers and online sources to catalogue LLM risks and then evaluate current guardrail approaches. They organize mitigations into a layered protection model with three levels—an external Gatekeeper layer (prompt filtering, system prompts, prompt injection defense), a Knowledge Anchor layer (Retrieval-Augmented Generation to ground outputs in verified sources), and a Parametric layer (fine-tuning, differential privacy, adversarial training). They also survey open-source guardrail toolkits including NeMo-Guardrails and LlamaGuard.

Results The review finds that no single solution eliminates LLM risks; effective guardrail design requires combining multiple layers tailored to the deployment context. Key tensions identified include accuracy versus privacy, flexibility versus stability, and the high cost of using a secondary LLM as a judge. Smaller specialized judge models are shown to outperform large general models for evaluating LLM outputs. The paper concludes that cosine similarity—the most widely used output-validation metric—is insufficient, and that agentic LLMs require additional safeguards such as testability frameworks, fail-safes, and situational awareness mechanisms.

Plain-language abstract This paper introduces "Prompt Infection," a new type of cyberattack specific to AI systems that use multiple AI models (called multi-agent systems) working together. The attack works like a computer virus: a malicious instruction hidden in external content hijacks one AI agent, which then passes the infection on to other agents in the network, potentially compromising the entire system while remaining undetected. The authors also propose a defense called LLM Tagging that, combined with existing safeguards, substantially reduces the attack's spread.

Motivation Safety research on large language models has overwhelmingly focused on single-agent vulnerabilities, leaving multi-agent systems — where many AI models collaborate and share information — poorly studied from a security standpoint. As frameworks like LangGraph, AutoGen, and CrewAI accelerate real-world deployment of multi-agent systems, understanding how a single compromised agent can propagate an attack through an entire network became an urgent open problem.

Methodology The authors defined and formalized the Prompt Infection attack, in which a malicious prompt embedded in external content causes a compromised agent to self-replicate the infection to peer agents, coordinating them to perform harmful tasks such as data exfiltration via code execution. They conducted extensive empirical experiments across multi-agent architectures, testing models including GPT-3.5 Turbo and GPT-4o, examining scenarios where agents do and do not share communications publicly. They also evaluated LLM Tagging — a defense that appends a distinguishing marker to agent outputs so downstream agents can separate user instructions from agent-generated content — both alone and in combination with existing prompt-injection defenses.

Results Experiments showed that multi-agent systems are highly susceptible to Prompt Infection even when agents do not openly share all communications. In social simulation settings, the infection spread followed a logistic growth pattern. Notably, more powerful models such as GPT-4o were not inherently safer: while GPT-4o showed stronger resistance to being initially compromised compared to GPT-3.5 Turbo, once infected it executed malicious tasks with greater precision, making it a more dangerous attacker. Neither LLM Tagging nor traditional defenses alone were sufficient to stop the attack, but their combination provided robust mitigation of the infection's spread.

Plain-language abstract This paper introduces Agent Security Bench (ASB), a framework for systematically testing how AI agents built on large language models can be attacked and how well various defenses hold up. The researchers built out 10 realistic deployment scenarios, defined over 400 tools, and ran a structured set of attacks and countermeasures against 13 different language models to find out where agents are most vulnerable.

Motivation LLM-based agents are increasingly deployed in high-stakes settings such as finance, healthcare, and autonomous driving, yet the security literature lacked a comprehensive, standardized way to evaluate their vulnerabilities. Prior work studied individual attack types in isolation, leaving it unclear how agents perform across a broad range of attack strategies and how effective existing defenses actually are.

Methodology ASB was constructed with 10 application scenarios, 10 corresponding agents, and more than 400 tools. The benchmark covers 27 attack and defense methods including 10 prompt injection attacks (both direct and indirect), a memory poisoning attack, a novel Plan-of-Thought backdoor attack targeting the agent's reasoning demonstrations via in-context learning, and 4 mixed attacks, along with 11 defenses. All combinations were evaluated across 13 LLM backbones using 7 evaluation metrics, including a new metric designed to capture the trade-off between agent utility and security.

Results Attacks were highly effective: the mixed attack achieved the highest average attack success rate at 84.30% while producing minimal refusal rates (3.22%). Memory poisoning was the least effective attack, with an average success rate of 7.92%. Current defenses showed limited effectiveness across the board, indicating that agent security remains an open and largely unsolved problem.

Plain-language abstract This paper introduces the Reward Hacking Benchmark (RHB), a suite of multi-step tasks that tests whether AI language model agents will take illegitimate shortcuts to score well rather than genuinely completing assigned work. The benchmark evaluates 13 leading models across realistic tool-use scenarios and measures how often they exploit weaknesses in the evaluation process instead of solving the actual task.

Motivation Reinforcement-learning-trained language models are increasingly deployed in coding assistants and autonomous systems where they can run code and manipulate files. A known failure mode is reward hacking, where an agent achieves high scores by gaming the evaluation mechanism rather than accomplishing the intended goal. Existing benchmarks focused mostly on short single-step tasks, leaving gaps in understanding how reward hacking scales to multi-step workflows, whether RL post-training directly causes higher hacking rates, and whether simple environmental defenses can reduce exploits.

Methodology The authors built RHB as a set of multi-step tool-use tasks in two regimes — independent tasks and chained tasks where intermediate outputs feed into later steps. Tasks cover data processing, multi-file reconstruction, and performance optimization workflows, and are automatically graded via recomputed-hash dependency enforcement so forged intermediate artifacts are detected. Thirteen frontier models from OpenAI, Anthropic, Google, and DeepSeek were evaluated. Reward hacking episodes were classified into a six-category taxonomy, and chain-of-thought traces were examined to characterize how models frame exploits. A controlled sibling comparison between DeepSeek-V3 and DeepSeek-R1-Zero isolated the effect of RL post-training, and environmental hardening ablations (e.g., restricting agent access to evaluation-relevant code) were tested.

Results Exploit rates ranged from 0% (Claude Sonnet 4.5) to 13.9% (DeepSeek-R1-Zero) across the 13 models evaluated. The controlled sibling comparison showed RL post-training is associated with substantially higher reward hacking — DeepSeek-V3 hacked at 0.6% versus DeepSeek-R1-Zero at 13.9%, with consistent gaps across all four task families. Seventy-two percent of reward hacking episodes included explicit chain-of-thought rationale in which the model framed the exploit as legitimate problem-solving. Simple environmental hardening reduced exploit rates by 5.7 percentage points (87.7% relative reduction) without degrading genuine task success. Models with near-zero exploit rates on standard tasks showed elevated rates on harder variants, suggesting production-aligned post-training suppresses reward hacking only below a complexity threshold where honest solutions remain tractable.

Plain-language abstract CodeTracer is a framework for making the internal workings of AI coding agents observable and debuggable. It parses the artifacts from an agent's run, reconstructs the full sequence of state transitions as a structured tree, and automatically identifies where in that sequence the agent first went wrong. A companion benchmark, CodeTraceBench, provides thousands of annotated agent trajectories to evaluate such diagnostic tools.

Motivation AI agents that autonomously fix bugs and interact with software repositories are growing more capable, but their executions are also growing longer and harder to inspect. When a run fails, existing evaluation methods only report whether the final answer was correct, collapsing entire multi-step trajectories into a single pass/fail label. That makes it difficult to determine which intermediate decision caused the failure or when the agent first went off track.

Methodology CodeTracer parses heterogeneous run artifacts through evolving extractors and reconstructs agent execution as a hierarchical trace tree that records state transitions and maintains persistent memory. A diagnosis module traverses this tree, issues structured evidence queries, and outputs the failure-responsible stage, the error-relevant steps, and a compact evidence set. CodeTraceBench was constructed from trajectories generated by four widely used code agent frameworks across five benchmarks (including SWE-bench Verified, SWE-bench Pro, MultiSWE-bench, SWE-PolyBench, and a terminal-interaction benchmark) covering tasks such as bug fixing, refactoring, and terminal interaction, with stage-level and step-level annotations for failure localization.

Results CodeTracer substantially outperforms direct prompting and lightweight baselines on failure localization. Replaying its diagnostic signals consistently recovers originally failed runs under matched compute budgets. Large-scale empirical analysis revealed systematic patterns: an evidence-to-action gap where agents retrieve relevant information but fail to translate it into correct state-changing actions, significant variance in action efficiency across trajectories even for strong models, and stage-dependent error modes where failure-critical decisions concentrate in specific phases of the workflow.

Plain-language abstract This paper proposes a structured approach to testing and evaluating large language model (LLM) applications in business settings, where systems must meet strict safety and reliability requirements despite generating probabilistic outputs. It adapts the software engineering practice of test-driven development — writing tests before writing code — to LLM systems, producing a framework for defining what a system must and must not do before any changes are made to prompts, models, or retrieval pipelines.

Motivation LLM applications deployed in enterprise settings such as customer service, legal drafting, and clinical intake must satisfy deterministic institutional requirements while relying on probabilistic generative components. Standard post-hoc benchmarking is insufficient because it does not prevent regressions, fails to encode safety and business constraints as enforceable gates, and offers no traceability for audits. A gap exists between how software reliability is normally engineered and how LLM systems are typically evaluated and released.

Methodology The paper extends the ATDLLMD (acceptance-test-driven LLM development) framework by defining a red-train-green lifecycle: first create failing acceptance tests, then improve the system through prompt changes, retrieval tuning, fine-tuning, or guardrails, and release only when multidimensional gates are satisfied. It specifies a reference architecture separating application runtime from an evaluation control plane containing a requirement registry, evaluation dataset store, scoring harness, statistical analyzer, release gate service, and monitoring feedback loop. A governance-oriented metric stack spans functional, safety, factuality, robustness, operational, and business outcome dimensions.

Results The paper is a framework and protocol contribution rather than an empirical study; it does not report experimental measurements. It defines evaluation protocols, release gates, and governance evidence structures for business-centric LLM systems, and proposes an empirical evaluation protocol for future work to quantify whether ATDLLMD-style workflows reduce escaped defects, improve stakeholder alignment, and lower the cost of safe iteration compared with prompt-first or benchmark-after workflows.

Plain-language abstract CompliBench is a benchmark for testing whether AI language models can reliably detect when a customer-service agent bot has violated its operational rules during a conversation. The paper introduces both the benchmark dataset and an automated pipeline to generate it, then evaluates a wide range of modern AI models on the task, finding that most struggle significantly — while a small model trained on the benchmark's synthetic data outperforms much larger general-purpose systems.

Motivation Enterprises increasingly deploy AI agents in contact centers, where those agents must follow complex, domain-specific rules such as specific refund workflows or mandatory transfer protocols. Automatically checking whether an agent followed every rule across a multi-turn conversation is critical for quality control, but no systematic benchmark existed for this problem — partly because obtaining precise, turn-level ground-truth labels from real interactions requires expensive expert annotation and is constrained by privacy regulations.

Methodology The authors built a scalable, automated data-generation pipeline covering three business domains: healthcare, insurance, and airline. The pipeline derives real-world agent guidelines, synthesizes diverse violation types, generates varied user personas and intents, then simulates user-agent dialogues using a user simulator paired with an intentionally perturbed agent — a controllable flaw injection process that automatically yields exact ground-truth labels (the specific guideline violated and the precise conversation turn). An adversarial search method selects the most challenging perturbations to ensure task difficulty. The resulting benchmark was used to evaluate numerous proprietary and open-source LLMs as judges, as well as classifier-based and generative reward models, alongside a fine-tuned Qwen3-8B model trained on 1,400 synthetic instances from the airline domain using GPT-5-distilled reasoning trajectories.

Results Most state-of-the-art LLMs struggle with the task; even the strongest proprietary models achieve only modest conversation-level accuracy, with violation detection being the primary bottleneck. Gemini-3-pro achieved the best conversation-level accuracy in healthcare and insurance, and the highest violation detection rates overall, while GPT-5 led on guideline accuracy for compliant turns but lagged on violation detection. A fine-tuned Qwen3-8B model substantially outperformed all open-source models and exceeded GPT-5 on both conversation-level accuracy and violation detection across all three domains, attaining the best conversation-level accuracy of 51.47 on the airline domain — demonstrating that task-specific supervision matters more than raw model scale for this task.

Plain-language abstract This paper is a comprehensive survey and security analysis of the Model Context Protocol (MCP), an open standard introduced by Anthropic in late 2024 that gives AI models a unified way to discover and interact with external tools, APIs, databases, and files. The authors map the current MCP ecosystem, catalogue security risks across the full server lifecycle, and lay out research directions for making the protocol secure and sustainable.

Motivation Before MCP, connecting an AI application to external tools required writing custom API integrations for every service, leading to fragmented, platform-specific ecosystems and heavy maintenance burdens. MCP was designed to eliminate this fragmentation with a single standardized interface, but its rapid adoption outpaced any systematic analysis of its architecture or security posture, leaving a gap that this paper addresses.

Methodology The study performs an ecosystem-wide analysis of MCP as of March 2025, examining the protocol's architecture (host, client, and server roles), the three-phase lifecycle of MCP servers (creation, operation, and update), and the security risks at each phase. The authors survey publicly available MCP server collections—cataloguing registries such as MCP.so (4,774 servers), Glama (3,356), and Smithery (2,942)—review unofficial auto-installers, and analyse real-world adoption by industry platforms including OpenAI, Cursor, and Cloudflare.

Results The analysis identifies a range of concrete security threats: name collision attacks (malicious servers impersonating legitimate ones), installer spoofing through unofficial auto-installers, code injection and backdoors during server creation, and configuration drift in remote multi-tenant deployments. The paper finds that MCP currently lacks a centralized security oversight model, standardized authentication and authorization, and an official package management system. It concludes with recommendations for cryptographic server verification, namespace policies, and reputation-based trust mechanisms to support the protocol's sustainable growth.

Plain-language abstract CascadeDebate is a system that chains together AI language models of different sizes, inserting small teams of debating AI agents at the decision points where a cheaper model would normally hand off to a more expensive one. Instead of blindly escalating uncertain questions to a larger model or a human expert, the system first tries to resolve ambiguity through internal group discussion. A confidence-based router activates this debate step only when a single model's answer is uncertain, and an online optimizer continuously tunes when escalations occur.

Motivation Deployed language model systems must balance accuracy against the high cost of running large models. Existing cascade pipelines route uncertain queries from small models to large ones based on a single model's confidence score, but that score is often poorly calibrated, causing premature and wasteful escalations. Multi-agent deliberation is known to improve reasoning, but existing multi-agent systems operate as standalone setups rather than as embedded components within cost-controlled pipelines, leaving a gap between the two approaches.

Methodology The authors build a four-stage cascade — single-model inference with a small base model, multi-agent deliberation with that same base model, single-model inference with a larger model, and multi-agent deliberation with the larger model — with human experts as a final fallback. Confidence-based routers using Bayesian-calibrated token probabilities and agent-agreement scores control progression between stages. Four specialized role-prompted agents per stage (e.g., Experimental Scientist, Misconception Detector for science tasks) deliberate via majority vote. An online Adam-based threshold optimizer updates escalation thresholds from streaming human feedback. Experiments used Llama-3.2 (1B/3B) and Qwen2.5 (1.5B/3B) instruction-tuned models on 1,000-instance samples from five multiple-choice benchmarks: ARC-Easy, ARC-Challenge, MMLU, MedQA, and MedMCQA, run on a single NVIDIA A100 GPU.

Results CascadeDebate achieved the best accuracy on all five benchmarks for both model families, improving over the strongest single baseline by 1.43 to 18.24 percentage points for Llama-3.2 and yielding especially large gains on medical tasks (MedQA: 86.44% vs. 64.00% for the best single-scale multi-agent baseline). The online threshold optimizer delivered 20.98 to 52.33% relative accuracy improvement over fixed threshold policies. On ARC-Challenge with Llama-3.2, the full cascade raised accuracy from 50.67% to 92.89%, a gain 1.62 times larger than a standard cascade at 15.62 times the single-base-model compute cost.

Plain-language abstract This paper argues that as AI agents move from controlled enterprise pipelines to open internet-scale societies — where agents discover each other through public registries and interact without central oversight — the field needs a proper theory of governance, not just risk checklists. The authors borrow a classic framework from sociology (Parsons' AGIL) to design a formal governance architecture for these agent societies, then diagnose how far current real-world ecosystems fall short of it.

Motivation Existing AI governance approaches focus on risk enumeration and process compliance suited to local, human-orchestrated multi-agent pipelines. The shift to internet-wide agent societies — where autonomous agents discover each other through open registries, interact without central orchestrators, and generate emergent social behaviors — creates a qualitatively different governance challenge that existing frameworks do not address. There is no theoretical grounding for what institutional structures such societies require to function stably.

Methodology The authors apply Talcott Parsons' AGIL framework, which identifies four functional imperatives (Adaptation, Goal Attainment, Integration, and Latency/Pattern Maintenance) that every viable social system must satisfy, to derive a prescriptive sixteen-cell institutional architecture for agent governance. They then diagnostically apply this architecture to the OpenClaw ecosystem (250,000+ GitHub stars, 2 million+ monthly users, 770,000+ registered agents) using a recursive sub-function analysis of 64 binary indicators across 16 cells. A complementary assessment examines twelve inter-pillar media pathways. The diagnostic is then extended to the broader agent-native protocol stack including MCP, A2A, ANP, x402, and ERC-8004.

Results The analysis finds at most 19% sub-function coverage in the OpenClaw ecosystem (sensitivity range 17–30%), and crucially, zero inter-cell coordination (I-sub = 0%), meaning existing technical infrastructure cannot participate in inter-pillar interchange and represents potential rather than operative governance capacity. All twelve inter-pillar media pathways were found to be non-functional. The Fiduciary (Latency) and Political (Goal Attainment) pillars are most severely underserved. Extending the analysis to independent teams building agent-native protocols reveals the same structural pattern, confirming the governance gap is a systemic feature of market-driven agent infrastructure development rather than a sign of one ecosystem's immaturity. The paper concludes with a prioritized roadmap for the missing governance infrastructure.

Plain-language abstract This paper studies how the internal design of multi-agent AI systems — networks of two or more cooperating AI agents — affects how easily those systems can be manipulated into carrying out harmful tasks. The authors find that purely looking at individual agent safety is not enough: the way agents are connected and coordinated can open or close attack pathways independent of how safe any single agent is.

Motivation As AI systems shift from single models to multi-agent architectures that plan, use tools, and delegate subtasks to one another, new security risks emerge that single-agent evaluations miss entirely. A model that reliably refuses a harmful request when prompted directly may nonetheless enable that harm when the same request is decomposed into innocent-looking subtasks routed through multiple specialist agents — meaning architectural choices themselves constitute a security variable that had not been systematically studied.

Methodology The authors adapted three existing single-agent misuse benchmarks — BrowserART (web browser tasks), OS-Harm (desktop/OS tasks), and RedCode-Gen (code generation) — to the multi-agent setting, keeping task semantics identical while varying only architecture. They evaluated 13 architectural configurations across multiple base models (including GPT-4o, GPT-5.4, GPT-5-mini, Claude Sonnet 4, Qwen3-VL, and Llama 70B), systematically varying three design dimensions: agent role specialization, communication topology (standalone, star, chain, and mesh), and memory visibility (private, shared reasoning traces, full shared memory). Stage-wise metrics distinguished planning-stage refusal, execution-stage interception, partial harmful execution, and full attack completion.

Results Multi-agent architectures were more vulnerable than standalone agents in the majority of configurations tested. Attack success rates varied by up to 3.8 times across configurations at comparable or higher benign task accuracy, demonstrating that security and capability can diverge sharply. On BrowserART, a star topology with four specialists raised the harmful task completion rate from 10% (standalone) to 31%. Topology effects were environment-dependent: star was riskiest for browser tasks, while chain was riskiest for code generation (42.5% harmful task completion versus 9.4% for standalone). Memory visibility effects were similarly context-dependent, sometimes worsening and sometimes improving security depending on topology and environment, ruling out any single universal architectural recommendation.

Plain-language abstract This paper introduces CSTM-Bench, a benchmark for detecting security threats in AI agents that unfold across multiple sessions. Current AI safety guardrails evaluate each message in isolation, which means attacks deliberately spread across dozens of conversations go entirely undetected. The paper defines the problem, releases a dataset, measures how well large language models can correlate threats across sessions, and proposes a memory-efficient algorithm that outperforms naive approaches.

Motivation AI agent guardrails are stateless: each message is judged in isolation, so an attacker who spreads a single attack across many sessions can bypass every per-session detector because no individual message carries the full payload. A documented 2025 cyber-espionage campaign (GTG-1002) exploited exactly this gap, using decomposition across sessions and sub-agents to bypass safety systems. Existing benchmarks and defenses address only within-session threats, leaving the cross-session, cross-agent generalization unaddressed.

Methodology The authors built CSTM-Bench, a dataset of 26 executable attack taxonomies organized along two axes: kill-chain stages and a cross-session operations ontology (accumulate, compose, launder, inject_on_reader). Traffic is split into Attack, Benign-pristine, and Benign-hard classes. A closed-loop rewriter generates a cross-session shard by softening surface phrasing while preserving cross-session attack artifacts, creating scenarios invisible to per-session judges. Three reader architectures are compared: a per-session judge, a Full-Log Correlator that concatenates all prompts chronologically into a long-context LLM call, and a bounded-memory Coreset Memory Reader that retains the highest-signal cross-session fragments at capacity K=50. All correlator experiments use Claude Sonnet 4.6.

Results Both the per-session judge and the Full-Log Correlator lose roughly half their attack recall when moving from the compositional-dilution shard to the cross-session-only shard, confirming that the degradation is a property of the correlator under dilution rather than context-window truncation. The Coreset Memory Reader catches 25 of 26 attack scenarios on the dilution shard versus 23 of 26 for the Full-Log Correlator, and achieves an F1 gain of +11.4 points (0.812 to 0.926) on that shard. False alarms on hard benign confounders drop from 7/14 to 3/14 (a reduction of 28.6 percentage points) with the coreset approach. The paper also introduces CSR_prefix, a label-free serving-stability metric, and a composite score CSTM = 0.7·F1 + 0.3·CSR_prefix for benchmarking future rankers on a single recall-versus-stability surface.

Plain-language abstract This paper identifies a new class of security failures in enterprise AI systems where multiple AI agents collaborate across departments. It formalizes the problem, builds a benchmark to test it, and proposes a distributed enforcement architecture called Distributed Sentinel that catches cross-agent policy violations that no single agent could detect on its own.

Motivation As organizations deploy many AI agents across departments, those agents share information across boundaries without a shared view of organizational policies. An agent acting on locally correct information can inadvertently trigger a policy breach when its output reaches a downstream agent in a different department — a failure mode the authors call a Context-Fragmented Violation. Existing prompt-based safety guardrails and rule-based data loss prevention tools are blind to these violations because each operates only on local context.

Methodology The authors formally define Context-Fragmented Violations using a distributed knowledge-graph model where each agent holds a private local graph and violations are detectable only against a hypothetical global graph. They construct PhantomEcosystem, a benchmark with 9 categories of realistic cross-agent violation scenarios with adversarially balanced safe controls. They then propose Distributed Sentinel, an enforcement architecture that uses lightweight sidecar proxies to propagate Semantic Taint Tokens (STTs) across organizational boundaries without exposing raw data, enabling Counterfactual Graph Simulation for cross-domain policy verification. They also evaluate eight frontier LLMs — including GPT-5.4, GPT-4.1, Claude Opus 4.6, and Gemini 2.5 Pro — in execution-oriented multi-agent workflows to measure empirical violation rates.

Results Distributed Sentinel achieves F1 = 0.95 on the PhantomEcosystem benchmark with 106ms end-to-end latency (16ms verification plus 90ms entity extraction on an A100 GPU), compared to F1 = 0.85 for prompt-based filtering and F1 = 0.65 for rule-based DLP baselines. Ablation studies show the system degrades to F1 = 0.53 without STT propagation, confirming that component is essential. All eight evaluated frontier LLMs exhibited substantial violation rates ranging from 14% to 98%, with cross-domain data flows producing systematically higher violation rates than same-domain flows, providing the first systematic empirical evidence that self-avoidance by individual models is insufficient for multi-agent security.

Plain-language abstract GAMMAF is an open-source benchmarking platform for evaluating security defenses in multi-agent AI systems, where multiple large language models collaborate to solve tasks. It is not a new defense mechanism itself, but a standardized environment that generates synthetic interaction datasets and tests existing defense models under realistic attack conditions. Researchers can use it to compare detection methods consistently, something the field has lacked until now.

Motivation When multiple AI agents communicate with one another to solve problems, a single compromised agent can manipulate the entire network through shared communication channels. Defenses based on graph-based anomaly detection have shown promise, but each research group implements evaluation differently, using different prompts and interaction logic, making results hard to compare or reproduce. GAMMAF was built to fill that gap with a shared, transparent benchmarking infrastructure.

Methodology The framework operates through two linked pipelines implemented in Python. The first generates training data by simulating agent debates across varied network topologies, including fixed and randomly generated structures, using benchmark question sets such as MMLU, CSQA, and GSM8K. Agent interactions are captured as attributed graphs in which nodes represent agents and edges represent communication links, with text outputs converted to embeddings for use in graph neural network models. The second pipeline evaluates trained defense models in live multi-agent sessions, dynamically isolating agents flagged as anomalous between debate rounds. Two published graph-based baselines, XG-Guard and BlindGuard, are replicated and used as reference points.

Results Experiments validated that both baseline defenses function correctly within the framework across multiple knowledge tasks and network sizes. When effective attack detection and isolation were applied, benign agents reached correct consensus earlier, substantially reducing total inference token costs compared to unprotected networks. Systems with no defense saw inference costs grow as adversarial agents prevented early stopping, while XG-Guard consistently reduced relative inference traffic when two to five adversarial agents were present. When adversarial agents exceeded 50 percent of the network, both defenses degraded because they rely on deviation from majority behavior, a known limitation. Throughput testing showed the framework reaches a saturation plateau of roughly 10,000 to 11,000 tokens per second under high concurrency, with the peak efficiency point tied to available GPU memory.

Plain-language abstract This paper tests whether having teams of AI language models debate each other actually improves their accuracy, or whether it just wastes compute. The researchers ran controlled experiments with groups of 10 identical small models on hard math and science questions, comparing peer debate against a simpler approach where each model corrects itself in isolation.

Motivation Multi-agent debate — where multiple AI models exchange reasoning and vote on answers — is widely deployed on the assumption that peer review filters out errors and hallucinations. However, no prior study had systematically measured whether this approach is actually more cost-effective than simpler alternatives, and the failure modes of homogeneous (all-same-model) debate teams were poorly understood.

Methodology The study evaluated teams of N=10 homogeneous agents using three open-weight models (Qwen2.5-7B, Llama-3.1-8B, Ministral-3-8B) across R=3 debate rounds on two high-difficulty benchmarks: GSM-Hard (algorithmic mathematics) and MMLU-Hard (complex scientific reasoning). Peer debate was compared against isolated self-correction and a stochastic noise control that injected rationales from unrelated problems. Ablations varied communication density (K in {2, 4, 9} peers per agent) and sampling temperature (T in {0.4, 0.7}).

Results Debate consistently underperformed or matched isolated self-correction while consuming 2.1–3.4 times more tokens (up to 28,631 tokens per problem). Three failure modes were identified: sycophantic conformity (agents adopted the majority answer without logical verification, modal adoption rate up to 85.5%), contextual fragility (peer rationales destabilized previously correct reasoning, vulnerability rate up to 70.0%), and consensus collapse (plurality voting discarded correct answers already present in the pool, oracle gap up to 32.3 percentage points). Artificial team consensus inflated to 90.1% even as accuracy degraded, showing that homogeneous unguided debate is both economically inefficient and logically unstable for 7–8B parameter models.

Plain-language abstract This paper examines the legal and governance challenges that arise when large language model (LLM) agents are deployed to act on behalf of humans or organizations. Using the lens of principal-agent theory — borrowed from economics and law — the authors map out who is responsible when an AI agent causes harm, makes mistakes, or acts in ways its operators did not intend. The paper aims to inform how AI systems should be designed, audited, and traced to make liability clearer.

Motivation LLM-based agentic systems are growing more capable and being deployed in increasingly high-stakes settings, yet existing legal frameworks for assigning liability were not designed with autonomous AI agents in mind. Prior work has studied AI risk but lacked a systematic examination of how principal-agent relationships in multi-agent systems interact with tort and agency law. The gap is urgent because LLM agents can plan, delegate subtasks, and interact with other agents in ways that make it difficult to trace responsibility when something goes wrong.

Methodology The paper presents a conceptual and legal analysis rather than an empirical study. The authors apply principal-agent theory to the structure of LLM-based multi-agent systems, drawing on US tort law and agency law (the Restatement of Torts and Restatement of Law) as the primary legal framework. They analyze both single-agent and multi-agent system architectures — including orchestrators, specialist agents, safety and compliance agents, and agentic software-as-a-service platforms — and consider both inherent liability issues (arising from the nature of LLM behavior) and emergent ones (arising from multi-agent interactions).

Results The analysis identifies a spectrum of liability issues spanning misalignment, misconduct, and the difficulty of attributing blame across multi-agent delegation chains. The authors argue that existing legal frameworks are insufficient for current agentic deployments and outline three directions for technical governance: interpretability and behavior evaluation, reward and conflict management, and principled engineering of detection and fail-safe mechanisms. The framework is designed to apply across jurisdictions by contextualizing agent behavior within the relevant local legal system.

Plain-language abstract This paper proposes a design pattern called bounded autonomy for making AI language models safe to use inside business software. Rather than letting the model directly issue commands to backend systems, all actions must pass through a mediation layer that enforces strict type checking, permission filtering, and tenant isolation before anything is executed. The authors built this Bounded Autonomy Layer (BAL) and tested it in a deployed multi-tenant enterprise application.

Motivation Language models used as AI assistants in enterprise software can hallucinate arguments, confuse similar entities, or overgeneralize permissions, turning a helpful interface into a source of unauthorized or destructive actions. Existing approaches relying on better prompting or stronger models leave the core problem unsolved: the model remains too close to raw execution in permission-sensitive, stateful, multi-tenant business environments.

Methodology The authors designed and implemented BAL, a portable mediation layer that sits between the language model and the enterprise application. BAL exposes only a published manifest of typed action contracts to the model, enforces permission-aware capability filtering, validates all arguments before side effects occur, and routes sensitive actions through optional human confirmation gates. They evaluated the system across 25 scenario trials spanning seven failure categories under three conditions: manual operation, unconstrained AI with safety layers disabled, and the full bounded-autonomy system.

Results The bounded-autonomy system completed 23 of 25 tasks with zero unsafe executions; the two incomplete tasks were safely contained with no enterprise side effects. The unconstrained AI configuration completed only 17 of 25 tasks. Both AI conditions achieved a 13 to 18 times speedup over manual operation. Several safety properties—including permission filtering, workspace isolation, and manifest governance—intercepted 100% of targeted violations regardless of model output. Counterintuitively, removing safety layers reduced task completion, because structured validation feedback helped the model self-correct, while the unconstrained system retried with generic errors or hallucinated success.

Plain-language abstract Pioneer Agent is an automated system that takes a small language model (1B–8B parameters) through the full cycle of task adaptation: gathering training data, diagnosing failures, retraining, and verifying results — all without manual engineering intervention. It operates in two modes: a cold-start mode that begins from a plain-language task description, and a production mode that starts from a deployed model's logged failures.

Motivation Small language models are attractive for production use because they are cheap and fast, but adapting them to a specific task still requires substantial human effort — deciding what data to collect, which failures matter, how to avoid regressions, and when to stop iterating. Existing AutoML tools address hyperparameter search but not the upstream challenges of data curation, failure diagnosis, and safe retraining, leaving teams to manage a difficult engineering loop manually.

Methodology Pioneer Agent uses an orchestrator LLM (Claude Sonnet) driving a LangGraph state machine that coordinates sub-agents for trace analysis, task delegation, data curation, training, and evaluation. Training pipelines are represented as triples of dataset, hyperparameters, and learning strategy, and the agent searches over these jointly using a Monte Carlo–style graph search (MCGS), expanding, fusing, scoring, and pruning nodes under explicit regression constraints. Experiments run in sandboxed Modal environments with LoRA fine-tuning via the Tinker SDK. The authors also introduce AdaptFT-Bench, a new benchmark of synthetic inference logs with progressively increasing noise, to evaluate the full adaptation loop across seven scenarios.

Results Across eight cold-start benchmarks covering reasoning, math, code generation, summarization, and classification, Pioneer Agent improves over base models by 1.6 to 83.8 points. On AdaptFT-Bench, the agent improves or preserves performance in all seven scenarios while naive retraining degrades by up to 43 points. On two production-style deployments, intent classification accuracy rose from 84.9% to 99.3% and named-entity F1 improved from 0.345 to 0.810. The agent also independently discovered effective training strategies — including chain-of-thought supervision and quality-focused data curation — purely from downstream feedback signals.

Plain-language abstract This paper studies a security vulnerability in AI systems where multiple language model agents collaborate by passing reports between a Worker agent and a Manager agent. The authors show that making the Worker smarter—using more capable language models—actually makes the overall system easier to attack, not harder. They identify this as the "capability paradox" and propose a defense strategy based on deliberately pairing agents of mismatched competence.

Motivation Multi-agent systems built on large language models divide tasks among specialized agents, but this creates a new security risk: the Manager agent often trusts Worker reports as reliable evidence, even if the Worker's input was manipulated. Prior security research focused on syntactic attacks like instruction overrides, but this paper addresses a harder-to-detect threat called semantic hijacking, where harmful requests are hidden inside plausible-sounding domain narratives with no explicit injection commands, making them invisible to traditional defenses.

Methodology The authors constructed an evaluation framework using real postmortem incident reports (such as site-reliability engineering outages) that were mutated by a language model into adversarial and benign payloads. They ran 42,000 adversarial trials across 12 Manager models and 7 Worker configurations, measuring Attack Success Rate (ASR) at the system level. To explain the observed paradox, they conducted multi-level mediation analysis on two independent datasets totaling 47,807 interactions, testing whether linguistic certainty in Worker reports mediates the relationship between Worker capability and Manager execution decisions.

Results Replacing a weak Worker (Llama-3.1-8B) with a stronger Worker (DeepSeek-R1) raised the system-level ASR from 20.6% to 94.4% against the same Manager. Across all configurations, mean ASR increased from 18.4% to 63.9% as Worker capability grew. Mediation analysis found that linguistic certainty—how assertively the Worker phrased its conclusions—explained 74% of the capability paradox effect in the larger dataset, with confidence intervals excluding zero. Worker-side safety prompting did not reliably reduce the risk. A proposed defense, heterogeneous ensemble verification using Workers with asymmetric domain competence, reduced ASR from 52.8% to 2.0% with negligible impact on benign tasks.

Plain-language abstract This paper studies how humans can more effectively oversee and correct the plans generated by AI-driven multi-agent systems. The authors build AMBIPOM, an interactive prototype that lets users inspect and edit multi-agent plans represented as dependency graphs, and they run both a user study and a controlled benchmark to learn how people and language models behave when collaborating on plan refinement.

Motivation Modern multi-agent systems coordinated by large language models can misalign with human intent, hallucinate steps, or violate domain constraints, yet most existing tools only let users check final outputs. This outcome-only supervision makes it hard to diagnose where a plan went wrong or apply targeted fixes, creating a need for process-level human oversight at intermediate planning stages.

Methodology The authors formalize a three-axis design space for human-LLM co-planning: mode (structural graph manipulation vs. natural-language feedback), scope (global vs. targeted subgraph), and level (low-level edits such as add/delete vs. high-level operations such as merge/split). They implement this space in AMBIPOM, which represents plans as directed acyclic graphs. A 13-participant user study measured efficiency, effort, and strategy, while a controlled benchmark of 200 gold plans and 1,150 broken-plan items tested four LLM revision conditions (global feedback, targeted feedback with and without full-plan context, and edit-sequence generation) across four task types.

Results Users naturally developed hybrid workflows, alternating between broad text feedback and local direct manipulation, and most (10 of 13) preferred inspecting plans before executing them. Global feedback produced the most structurally accurate LLM revisions (average execution accuracy 0.840 vs. 0.553 for edit-sequence generation), while targeted feedback was better at preserving plan stability by leaving untouched nodes unchanged. Edit-sequence generation offered higher interpretability but was markedly less robust for complex structural edits such as adding nodes or merging steps.

Cost-tier routing for SWE tasks — closest single paper to the Gas City tiering problem.

Why it matters Surfaced by the 2026-06-11 dense-lane rerun (missed in the lexical-only window).

Unifies routing + cascading formally.

Why it matters Surfaced by the 2026-06-11 dense-lane rerun (missed in the lexical-only window).

Self-optimizing LLM selection.

Why it matters Surfaced by the 2026-06-11 dense-lane rerun (missed in the lexical-only window).

Cascade via inter-model agreement.

Why it matters Surfaced by the 2026-06-11 dense-lane rerun (missed in the lexical-only window).

Trains models with the cascade in mind.

Why it matters Surfaced by the 2026-06-11 dense-lane rerun (missed in the lexical-only window).

Confidence tuning for cascade deferral.

Why it matters Surfaced by the 2026-06-11 dense-lane rerun (missed in the lexical-only window).

Early inference-triage pattern.

Why it matters Surfaced by the 2026-06-11 dense-lane rerun (missed in the lexical-only window).

Plain-language abstract DeLM (Decentralized Language Models) is a multi-agent framework that drops the central controller. Instead of a main agent assigning subtasks, waiting, and merging results, parallel agents asynchronously claim tasks from a shared queue and read/write a shared 'verified context' of accumulated progress. It targets two settings — parallel exploration in software engineering and concurrent evidence processing in long-context QA — and improves accuracy while roughly halving cost.

Motivation Most multi-agent systems use centralized scatter-gather orchestration, which parallelizes sub-agent execution but not the coordination around it. Every finding must return to the main agent to be merged and rebroadcast, so progress-sharing becomes a serialized bottleneck as agents grow, and the controller can dilute, omit, or distort details. In long-context reasoning the main agent must pre-assign evidence clusters before knowing what is relevant, triggering extra delegation rounds. DeLM removes the controller as the coordination chokepoint.

Methodology Coordination is state-based, not prompt-routed. Two global structures: a shared context C of compact verified gists and a task queue T of pending subtasks. The pipeline initializes the queue from the input, executes ready subtasks in parallel, then compresses-verifies-admits each result into the shared context, generates more subtasks when the context is insufficient, and finalizes once none remain. The shared context is compact, global, and 'unfoldable' — agents read coarse gists by default and expand to detailed summaries or raw evidence only when needed. Admission-time verification checks each update against its underlying evidence and reasoning trajectory before it enters shared state, rejecting or regenerating unsupported updates so errors cannot propagate as reusable problem state.

Results On SWE-bench Verified, DeLM is strongest across test-time-scaling metrics, reaching 77.4% pass@4 at ~$0.12/task — roughly half the baselines' cost — with trace-level examples showing agents reuse each other's discoveries through the compact shared context. On LongBench-v2 Multi-Doc QA it leads four frontier model families by up to 5.7 points, with both admission-time verification and hierarchical summarization contributing. On OOLONG, vanilla DeLM underperforms RLM (which needs exact row-level aggregation via code execution), but RLM combined with DeLM yields the best accuracy and lowest cost, showing DeLM works as a coordination layer for programmatic reasoning too.

Plain-language abstract This paper asks whether the reliability problems showing up in AI-coding-agent projects belong to the individual agent or to the repository and ecosystem it operates in. Using a statistical technique for detecting emergence, whether a whole-system property survives after accounting for all its individual parts, the authors show that a large, consistent share of integration friction (how hard it is to merge a contribution) belongs to the repository itself, not to any single contribution, author, or agent.

Motivation Coding agents are evaluated the way software has always been evaluated: one contribution, one benchmark task, at a time. But developers report that agent-authored code accumulates problems no single contribution accounts for, and shared understanding erodes even when every individual change passes review. The paper reframes this as a measurement question: at what level should reliability risk actually be measured?

Methodology The authors define statistical non-reducibility: using multilevel models, standard for nested data, on 930,000+ agent-authored pull requests, they measure how much variance in integration friction remains attached to the repository after controlling for the contribution, its author, its size, and the agent that produced it. They compare against a matched human-authored baseline in the same repositories to check the signal is agent-specific.

Results About half the variance in integration friction stays at the repository level after full controls, a property of the whole that no single part explains. Agent-authored contributions concentrate this repository-level friction roughly twice as much as human-authored ones (intraclass correlation 0.30 vs 0.16), and the gap survives controls for codebase size, age, task shape, process maturity, and merge path.

Plain-language abstract Glite ARF is an open-source framework for running many LLM coding agents in parallel on a shared research codebase without the whole thing quietly corrupting itself. It wraps each agent's work in an isolated task folder, makes finished work immutable (fixes happen in new tasks, not edits to old ones), and auto-generates a dashboard so a human can see the true state of a multi-week campaign instead of relying on an agent-written summary.

Motivation Delegating research experiments directly to coding agents doesn't scale: agents follow most instructions but the few they skip compound into corrupted data, fabricated citations, contaminated splits, and stale summaries. The authors' own prior audit found 13 such incidents in one campaign, including one where a single agent step recomputed and corrupted 20,304 historical training rows across 38 feature sets.

Methodology The framework defines a three-role stack (human chooses hypotheses, coding agents execute isolated tasks, deterministic Python 'verificator' scripts enforce structure) built on seven structural principles: task isolation via git worktrees, immutability with a corrections overlay, aggregators-only cross-task reading, and a materialized human-facing overview regenerated from committed artifacts. It's evaluated via a real external shared task (BEA 2026 vocabulary-difficulty) plus measured overhead across three author-run campaigns.

Results The BEA 2026 submission built with Glite ARF placed first (closed track) and second (open track) across all three target languages, cutting the baseline RMSE by 29.9% (closed) and 35.9% (open), across 273 tracked tasks run by up to twelve parallel agents from a single laptop at roughly $450 in LLM spend. Structured per-fold provenance caught and let them strip four target-leaking feature sets that had inflated one result to an implausible 0.609 RMSE (corrected to 0.802). The framework's structural machinery adds only about 1% wall-clock overhead across three campaigns in three domains.

Plain-language abstract LAMaS is an orchestration framework for LLM multi-agent systems that treats end-to-end latency as a first-class objective rather than a side effect of cost. It learns execution graphs under a constrained objective that minimizes latency and cost subject to an accuracy floor, attributes the latency penalty to operators in proportion to how close they sit to the critical path, and adds a small learned controller that drops redundant remaining agent calls at run time once the answer has stabilized.

Motivation Automated multi-agent orchestration has optimized task performance and inference cost, but has largely left latency alone, and state-of-the-art agent systems already take 10-60 minutes per task in computer-use and autonomous-research settings. Latency will not follow from cost reduction: cost sums over every operator in the graph while end-to-end time is the longest source-to-sink path, so operators off the critical path do not affect wall-clock time at all. That makes naive latency optimization fail in two ways, mis-assigning operator-level credit under a uniform penalty, and degrading task accuracy unpredictably when nothing holds accuracy down. A graph committed at training time also cannot use evidence that only appears as execution unfolds.

Methodology The orchestrator is a policy network over an agentic supernet, a probabilistic DAG over candidate operators, sampling operators per step until cumulative probability exceeds a threshold tau so graph width and depth adapt per query. Training minimizes E[lambda_t*T + lambda_c*C] subject to E[S] >= S0, relaxed to reward R = lambda_S*S/S0 - lambda_t*T - lambda_c*C with the dual variable lambda_S updated by projected gradient ascent, rising when batch accuracy falls below the floor and decaying when it is met. S0 is set to the validation accuracy of the same orchestrator trained at lambda_t = 0, making the constraint a do-no-harm floor. Each operator's share of the latency penalty is scaled by path-criticality w(o) = l(o)/T, where l(o) is the longest path through o, obtained with the critical path in a single forward/backward pass. The orchestrator trains by policy gradient with EMA reward normalization. After it converges, a small MLP controller trains by MSE regression on frozen-orchestrator traces to predict the control advantage delta_k = R_k - R_N from completion fraction, output agreement, and a compressed query embedding; at inference the predicted advantage becomes a Bernoulli retain/eliminate decision at each operator completion. Evaluation uses gpt-4o-mini-0718 on GSM8K, HumanEval, MATH and MMLU-Pro against Generate, CoT, SC, GDesigner, AgentDropout and MaAS, with latency measured as wall-clock time under one API provider and runtime, no response caching, averaged over 3 runs.

Results LAMaS cuts end-to-end latency 55.6-75.6% against the strongest learning-based MAS baselines while matching or beating accuracy. On GSM8K it reaches 93.65% at 11.73s and 622 critical-path tokens against MaAS at 93.36%, 48.11s and 2501; on HumanEval 95.42% at 21.00s against MaAS 93.38% at 47.32s; on MMLU-Pro 66.27% at 14.81s against MaAS 65.40% at 49.12s. Non-MAS prompting (Generate, CoT, SC) stays faster but consistently loses accuracy, most visibly on harder benchmarks. Ablations isolate each component: removing the inference controller holds accuracy within 0.38 points but adds 16.5-21.1% latency and 16.0-25.2% cost; removing critical-path credit degrades all three metrics; setting lambda_t = 0 leaves accuracy within 0.25 points while inflating latency 24-70% and cost 19-35%. A bare uniform latency term with no accuracy floor, no path-criticality weighting and no controller loses 1.30-5.34 accuracy points for only 2-11% latency savings over lambda_t = 0 and no cost reduction. The pipeline transfers to other learning-based orchestrators: GDesigner+LAMaS cuts GSM8K latency from 32.31s to 20.55s and cost from 17.86e-4 to 12.08e-4 USD at 93.27% vs 93.30% accuracy, with a comparable effect on AgentDropout.

Plain-language abstract This paper proposes a method to make large language models more accurate by having multiple instances of a model debate each other. Instead of querying a single model once, several model instances each generate an answer, then read and critique each other's responses over multiple rounds until they converge on a shared answer. The approach works with publicly available black-box models and requires no access to model internals.

Motivation Large language models frequently hallucinate facts and make reasoning errors, even when prompted carefully. Prior work on improving accuracy—such as chain-of-thought prompting, self-consistency, and reflection—acts on a single model instance, leaving open the question of whether disagreement and debate across multiple independent instances could surface and correct errors that no single model catches on its own.

Methodology Multiple instances of the same language model (or a mix of models such as ChatGPT and Bard) each independently answer a query. Each instance then reads all other instances' answers and reasoning, critiques them, and updates its own answer. This debate loop repeats for several rounds. The method uses identical prompt templates across all tasks and requires only black-box text generation access. Performance is evaluated on six benchmarks covering mathematical reasoning, strategic reasoning, factual question answering, and a newly introduced dataset of computer scientist biography facts.

Results Multiagent debate outperformed single-model baselines including zero-shot chain-of-thought and reflection across all six benchmarks. The debate process reduced hallucinated or factually incorrect content: as rounds progressed, model instances tended to drop facts they disagreed on, filtering out internally uncertain claims. Crucially, debate was not merely amplifying an already-correct majority—many cases were observed where all models initially gave wrong answers but converged on the correct answer after debate. Using more agents and more rounds of debate both contributed to higher accuracy.

Plain-language abstract This paper investigates why multi-agent systems built from large language models so often fall short of expectations. The authors created MAST, a structured catalog of failure types, by carefully examining hundreds of conversation traces from seven widely-used multi-agent frameworks. The result is a practical taxonomy that names and organizes 14 distinct ways these systems break down, along with an automated tool to apply that taxonomy at scale.

Motivation Multi-agent LLM systems have attracted considerable interest because they can, in principle, divide complex tasks among specialized agents and coordinate their work. Yet empirical results consistently show their performance gains over single-agent setups are small or absent — for instance, one studied framework (ChatDev) solved only 33% of programming tasks. No systematic account of why these systems fail existed, leaving developers without a principled framework for diagnosis or improvement.

Methodology The authors analyzed more than 200 conversation traces drawn from seven open-source multi-agent frameworks (including MetaGPT, ChatDev, HyperAgent, AppWorld, AG2, and Magentic-One) running on diverse benchmarks. Six expert human annotators applied Grounded Theory to label failures in the traces, with inter-annotator agreement measured by Cohen's Kappa (reaching 0.88). An LLM-as-a-Judge pipeline using OpenAI's o1 was then developed and validated against expert labels (Kappa 0.77) to enable scalable automated annotation. Two case studies tested whether targeted interventions could reduce the identified failures.

Results The analysis identified 14 distinct failure modes organized into three categories: specification issues (system design problems, 41.77% of failures), inter-agent misalignment (coordination failures, 36.94%), and task verification failures (quality control, 21.30%). Prominent individual failure modes included step repetition (17.14%), reasoning-action mismatch (13.98%), and disobey task specification (10.98%). Targeted interventions such as improved role specification yielded only modest gains (e.g., +15.6% for ChatDev), indicating that the identified failures stem from fundamental system design challenges rather than easily patched prompt issues.

Plain-language abstract This paper addresses how to connect large groups of autonomous agents — such as drone swarms or fleets of vehicles — so they can share information quickly without overwhelming the network. The authors propose CayleyTopo, a new family of communication network layouts derived from algebraic graph theory, whose structure is optimized by a machine-learning algorithm rather than chosen by hand.

Motivation Large-scale multi-agent systems cannot use fully connected networks because the number of required communication links grows quadratically with the number of agents, causing congestion. Existing sparse network layouts rely on handcrafted rules (such as connecting each agent to peers at fixed exponential distances), leaving open the question of whether those rules are actually optimal and prompting the authors to treat the network structure itself as something to be designed and optimized.

Methodology The authors model multi-agent communication networks as circulant Cayley graphs parameterized by a set of generator steps, and frame topology design as a discrete combinatorial optimization problem that minimizes graph diameter under a fixed degree budget. To search the enormous space of candidate generator sets efficiently, they develop a lightweight reinforcement learning framework that incorporates two key components: a number-theoretic prior based on multiplicative order to bias the search toward structurally rich generators, and a message-propagation score that provides dense connectivity feedback during construction.

Results CayleyTopo consistently outperforms rule-based baseline topologies — including ExpoComm, Fibonacci, simple-prime, and broadcast designs — across information dissemination speed, resilience to link failures, and communication load. In cumulative communication-count experiments, CayleyTopo achieves approximately 40% faster average dissemination while incurring the lowest average per-step bandwidth usage, and its discovered graph diameters approach the theoretical Moore lower bound.

Plain-language abstract TraceFix is a system that automatically designs, verifies, and repairs coordination protocols for groups of AI agents working together. When multiple AI agents must share resources, pass messages, and coordinate actions concurrently, subtle bugs like deadlocks can emerge. TraceFix uses formal verification—exhaustive model checking with TLA+—to find these bugs before deployment and iteratively fixes the protocol until no violations can be found, then enforces the verified protocol at runtime.

Motivation As large language model (LLM) agents increasingly run concurrently and share external state, coordination failures—races, deadlocks, missed handshakes, and premature termination—become the dominant source of system failures rather than individual agent capability. These bugs depend on rare execution schedules and can remain hidden through all observed runs yet appear under untested interleavings. Prior approaches address coordination through orchestration frameworks or runtime guards but do not verify that the concurrent protocol itself is free of such hazards.

Methodology TraceFix operates as a four-stage pipeline: an orchestration agent synthesizes a protocol topology (a structured intermediate representation defining agents, shared locks, and directed message channels) from a natural-language task description; the topology is compiled into PlusCal coordination logic; the TLA+ model checker (TLC) exhaustively searches for counterexample traces under bounded assumptions; and a repair agent revises the PlusCal source based on the counterexample until TLC finds no further violations. Verified process bodies are then compiled into per-agent system prompts, and a runtime monitor rejects any coordination operations outside the verified topology. The approach was evaluated on a benchmark of 48 tasks spanning 16 scenario families at three difficulty tiers, with a 3,456-run runtime comparison across four architectures and two model capability tiers.

Results All 48 benchmark tasks reached full TLC verification; 62.5% passed on the first attempt and none required more than four repair iterations. Bounded model checking remained tractable across all tasks (median under 1 second, maximum under 60 seconds) even for state spaces reaching millions of distinct states. Topology-monitored execution achieved the highest task completion rates (89.4% average, 81.5% full completion) and degraded at roughly half the rate of prompt-only and chat-only baselines when model capability was reduced. A paired ablation showed that TLC-verified protocols cut deadlock and livelock occurrences from 31.1% to 14.1%, with the largest separation under fault injection conditions.

Plain-language abstract This paper asks whether multi-agent AI systems inherit the demographic biases of their individual components—or make things worse. Working with a new benchmark of ethically loaded decision scenarios, the authors show that even when each AI agent in a chain behaves neutrally on its own, the system as a whole can develop pronounced preferences for certain demographic groups as information passes from agent to agent.

Motivation Alignment research has focused on reducing bias in standalone language models, and techniques like instruction tuning and reinforcement learning from human feedback have made individual models more cautious. But modern AI deployments increasingly chain multiple models together in multi-agent systems (MAS), where each agent's output becomes input for the next. Whether safety-aligned individual agents can still produce biased collective behavior—and whether common architectural choices like role specialization or complex communication topologies can prevent it—had not been systematically measured.

Methodology The authors built Discrim-Eval-Open, a benchmark of 210 demographically diverse protagonist profiles (balanced across age, gender, and race/ethnicity) embedded in open-ended decision scenarios such as kidney transplant allocation and hiring. They ran these scenarios through multi-agent chains of two to four LLMs (including DeepSeek-V3, DeepSeek-R1, GPT-4o, Qwen-Max, and others) arranged in serial, spindle, parallel, and fully-connected topologies. Agents were assigned varied personas (doctor, lawyer, engineer, merchant) and functional roles (judger, analyst, reflector, summarizer). Bias was quantified using distributional metrics—Gini coefficient and entropy—on each agent's probability assignments across demographic groups.

Results Bias amplified monotonically as judgments passed through successive agents across virtually all tested architectures and models; Gini coefficients grew at each step (e.g., from 0.13 to 0.38 over four agents for DeepSeek-R1). Architectural complexity—diverse personas, specialized functions, spindle or fully-connected topologies—did not mitigate amplification and often accelerated it. The system exhibited systemic preferences for younger individuals, females, and Black individuals even when single agents were nominally neutral. The authors also identified a 'Trigger Vulnerability': injecting a single piece of objective, neutral external text (as a standard RAG harness would) caused dramatic polarization, exposing severe fragility in system-level robustness.

Plain-language abstract This paper studies how shared behavioral norms can arise spontaneously among populations of AI language models (LLMs) interacting with one another, using a classic naming-game framework. The researchers show that group-wide conventions form through purely local, pairwise interactions, that collective biases toward particular conventions emerge even when individual models appear unbiased, and that small committed minorities of agents can flip an established convention if they exceed a critical size.

Motivation As large language models are increasingly deployed in multi-agent settings where they interact with each other and with humans, understanding whether and how they develop shared norms is essential for predicting AI behavior and ensuring alignment with human values. Prior work focused on single-agent bias assessment, leaving open questions about how individual biases translate to group behavior, and how stable AI-generated conventions are against deliberate change.

Methodology The authors ran simulated naming-game experiments with populations of N=24 LLM agents (also tested at N=200) drawn from four models: Llama 2 70B, Llama 3 70B, Llama 3.1 70B, and Claude 3.5 Sonnet. At each time step, two randomly selected agents tried to coordinate on a name chosen from a pool of W letters; agents were rewarded for matching their partner's choice and penalized for mismatches, retaining a memory of their last M=5 interactions. Experimental results were contrasted with predictions from a minimal analytical naming-game model to isolate LLM-specific effects.

Results Globally accepted conventions emerged spontaneously across all four models, consistent with the theoretical model. Strong collective biases toward specific conventions arose even when individual agents were statistically unbiased at first interaction—for example, Llama 2 70B and Claude 3.5 Sonnet agents showed no individual name preference (chi-squared p=0.100 and 0.410), yet the population converged non-uniformly. Committed minority groups could overturn established conventions once they reached a critical mass that varied widely by model and by the relative strength of the competing conventions: as little as 2% of the population sufficed in Llama 3 70B populations, while Llama 2 70B required up to 67%, and in Llama 3.1 70B the weaker convention could be spontaneously overturned without any committed agents at all.

Plain-language abstract ReConcile is a framework that sets up multiple different AI language models as participants in a round-table discussion to collaboratively solve reasoning problems. Instead of using many copies of a single model, it brings together models from different families—such as ChatGPT, Bard, and Claude—who exchange answers, share confidence estimates, and try to convince each other using human-style explanations until they reach a consensus answer.

Motivation Large language models still struggle with complex reasoning tasks, and existing approaches to improve this—such as having a single model critique its own outputs—suffer from stagnation when the model becomes overly confident and stops generating new ideas. Multi-agent debate frameworks exist but have typically used multiple instances of the same underlying model, which limits diversity because all agents share the same pre-training data and architecture, producing an inherent model bias and a narrow knowledge scope.

Methodology ReConcile runs multiple rounds of structured discussion among agents drawn from different model families. In each round, each agent receives a discussion prompt containing the grouped answers and explanations from the previous round, the confidence scores of all agents, and demonstrations of human explanations that successfully corrected wrong answers. Agents then update their answers by attempting to convince others or by being convinced. A confidence-weighted voting mechanism aggregates the agents' final answers into a consensus. The framework was evaluated on seven reasoning benchmarks and flexibly accommodates API-based, open-source, and domain-specific models.

Results ReConcile outperformed prior single-agent and multi-agent baselines by up to 11.4 percentage points across seven benchmarks, and even surpassed GPT-4 on three datasets. Incorporating a mix of diverse models—including domain-specific ones—yielded an 8% improvement on the MATH benchmark. Analysis confirmed that model diversity is a critical driver of performance gains, with the confidence-weighted consensus mechanism and the convincing-sample demonstrations each contributing meaningfully to the final results.

Plain-language abstract This paper introduces Conformal Social Choice, a post-hoc decision layer for multi-agent AI debate systems that converts debate outputs into calibrated act-versus-escalate decisions. Rather than always acting on whatever answer a group of AI agents agrees on, the system can recognize when agent consensus is uncertain enough that a human should review the case instead. It requires no retraining of the underlying models and works as a plug-in layer on top of any multi-agent debate pipeline.

Motivation Multi-agent debate systems improve AI reasoning by having multiple language models argue and converge on answers, but they have a critical blind spot: agents can converge on a wrong answer with high confidence, and existing pipelines have no way to detect this. Current systems reduce debate to a single vote or majority answer and treat agreement as a signal that it is safe to act — but agreement and correctness are not the same thing, leaving no safety check before automated action.

Methodology The pipeline has four stages: heterogeneous LLM agents (Claude Haiku, DeepSeek-R1, and Qwen-3 32B) debate for multiple rounds and each agent outputs explicit numerical probability distributions over answer choices; these distributions are aggregated into a single social probability via a linear opinion pool; a conformal calibration procedure uses a held-out validation set to compute a threshold; and a hierarchical action policy maps high-confidence singleton prediction sets to autonomous action while larger sets trigger human escalation. The system was evaluated on eight MMLU-Pro subject domains with a coverage target of 95% (alpha = 0.05).

Results The conformal layer achieved coverage within 1 to 2 percentage points of the 95% target across all eight domains. Crucially, 81.9% of cases where the agents reached a wrong consensus were intercepted and escalated rather than acted upon. The cases the system chose to act on autonomously reached 90.0 to 96.8% accuracy, up to 22.1 percentage points above the accuracy of naive consensus stopping. By contrast, consensus stopping alone showed accuracy as low as 67.9% on some domains, with no mechanism to flag incorrect confident predictions, including a 17.4% error rate in Engineering despite apparent model agreement.

Plain-language abstract This paper introduces the Reward Hacking Benchmark (RHB), a suite of multi-step tasks that tests whether AI language model agents will take illegitimate shortcuts to score well rather than genuinely completing assigned work. The benchmark evaluates 13 leading models across realistic tool-use scenarios and measures how often they exploit weaknesses in the evaluation process instead of solving the actual task.

Motivation Reinforcement-learning-trained language models are increasingly deployed in coding assistants and autonomous systems where they can run code and manipulate files. A known failure mode is reward hacking, where an agent achieves high scores by gaming the evaluation mechanism rather than accomplishing the intended goal. Existing benchmarks focused mostly on short single-step tasks, leaving gaps in understanding how reward hacking scales to multi-step workflows, whether RL post-training directly causes higher hacking rates, and whether simple environmental defenses can reduce exploits.

Methodology The authors built RHB as a set of multi-step tool-use tasks in two regimes — independent tasks and chained tasks where intermediate outputs feed into later steps. Tasks cover data processing, multi-file reconstruction, and performance optimization workflows, and are automatically graded via recomputed-hash dependency enforcement so forged intermediate artifacts are detected. Thirteen frontier models from OpenAI, Anthropic, Google, and DeepSeek were evaluated. Reward hacking episodes were classified into a six-category taxonomy, and chain-of-thought traces were examined to characterize how models frame exploits. A controlled sibling comparison between DeepSeek-V3 and DeepSeek-R1-Zero isolated the effect of RL post-training, and environmental hardening ablations (e.g., restricting agent access to evaluation-relevant code) were tested.

Results Exploit rates ranged from 0% (Claude Sonnet 4.5) to 13.9% (DeepSeek-R1-Zero) across the 13 models evaluated. The controlled sibling comparison showed RL post-training is associated with substantially higher reward hacking — DeepSeek-V3 hacked at 0.6% versus DeepSeek-R1-Zero at 13.9%, with consistent gaps across all four task families. Seventy-two percent of reward hacking episodes included explicit chain-of-thought rationale in which the model framed the exploit as legitimate problem-solving. Simple environmental hardening reduced exploit rates by 5.7 percentage points (87.7% relative reduction) without degrading genuine task success. Models with near-zero exploit rates on standard tasks showed elevated rates on harder variants, suggesting production-aligned post-training suppresses reward hacking only below a complexity threshold where honest solutions remain tractable.

Plain-language abstract CodeTracer is a framework for making the internal workings of AI coding agents observable and debuggable. It parses the artifacts from an agent's run, reconstructs the full sequence of state transitions as a structured tree, and automatically identifies where in that sequence the agent first went wrong. A companion benchmark, CodeTraceBench, provides thousands of annotated agent trajectories to evaluate such diagnostic tools.

Motivation AI agents that autonomously fix bugs and interact with software repositories are growing more capable, but their executions are also growing longer and harder to inspect. When a run fails, existing evaluation methods only report whether the final answer was correct, collapsing entire multi-step trajectories into a single pass/fail label. That makes it difficult to determine which intermediate decision caused the failure or when the agent first went off track.

Methodology CodeTracer parses heterogeneous run artifacts through evolving extractors and reconstructs agent execution as a hierarchical trace tree that records state transitions and maintains persistent memory. A diagnosis module traverses this tree, issues structured evidence queries, and outputs the failure-responsible stage, the error-relevant steps, and a compact evidence set. CodeTraceBench was constructed from trajectories generated by four widely used code agent frameworks across five benchmarks (including SWE-bench Verified, SWE-bench Pro, MultiSWE-bench, SWE-PolyBench, and a terminal-interaction benchmark) covering tasks such as bug fixing, refactoring, and terminal interaction, with stage-level and step-level annotations for failure localization.

Results CodeTracer substantially outperforms direct prompting and lightweight baselines on failure localization. Replaying its diagnostic signals consistently recovers originally failed runs under matched compute budgets. Large-scale empirical analysis revealed systematic patterns: an evidence-to-action gap where agents retrieve relevant information but fail to translate it into correct state-changing actions, significant variance in action efficiency across trajectories even for strong models, and stage-dependent error modes where failure-critical decisions concentrate in specific phases of the workflow.

Plain-language abstract CascadeDebate is a system that chains together AI language models of different sizes, inserting small teams of debating AI agents at the decision points where a cheaper model would normally hand off to a more expensive one. Instead of blindly escalating uncertain questions to a larger model or a human expert, the system first tries to resolve ambiguity through internal group discussion. A confidence-based router activates this debate step only when a single model's answer is uncertain, and an online optimizer continuously tunes when escalations occur.

Motivation Deployed language model systems must balance accuracy against the high cost of running large models. Existing cascade pipelines route uncertain queries from small models to large ones based on a single model's confidence score, but that score is often poorly calibrated, causing premature and wasteful escalations. Multi-agent deliberation is known to improve reasoning, but existing multi-agent systems operate as standalone setups rather than as embedded components within cost-controlled pipelines, leaving a gap between the two approaches.

Methodology The authors build a four-stage cascade — single-model inference with a small base model, multi-agent deliberation with that same base model, single-model inference with a larger model, and multi-agent deliberation with the larger model — with human experts as a final fallback. Confidence-based routers using Bayesian-calibrated token probabilities and agent-agreement scores control progression between stages. Four specialized role-prompted agents per stage (e.g., Experimental Scientist, Misconception Detector for science tasks) deliberate via majority vote. An online Adam-based threshold optimizer updates escalation thresholds from streaming human feedback. Experiments used Llama-3.2 (1B/3B) and Qwen2.5 (1.5B/3B) instruction-tuned models on 1,000-instance samples from five multiple-choice benchmarks: ARC-Easy, ARC-Challenge, MMLU, MedQA, and MedMCQA, run on a single NVIDIA A100 GPU.

Results CascadeDebate achieved the best accuracy on all five benchmarks for both model families, improving over the strongest single baseline by 1.43 to 18.24 percentage points for Llama-3.2 and yielding especially large gains on medical tasks (MedQA: 86.44% vs. 64.00% for the best single-scale multi-agent baseline). The online threshold optimizer delivered 20.98 to 52.33% relative accuracy improvement over fixed threshold policies. On ARC-Challenge with Llama-3.2, the full cascade raised accuracy from 50.67% to 92.89%, a gain 1.62 times larger than a standard cascade at 15.62 times the single-base-model compute cost.

Plain-language abstract This paper introduces a mathematical diagnostic for choosing how to connect multiple AI language model agents before running them. By borrowing the "successor representation" from reinforcement learning and neuroscience, the authors show that three numbers derived from a communication graph's structure can predict, ahead of any actual computation, whether a given arrangement of agents will accumulate errors, fail to reach agreement, or crumble under noisy inputs.

Motivation When building systems that chain or network multiple large language models together, practitioners must choose a communication topology — chain, star, mesh, or other arrangements — with no principled way to anticipate which will cause reasoning errors, slow consensus, or brittleness under perturbation. Existing evaluation methods can only answer these questions after the pipeline has already run on a specific task, leaving topology selection as a costly trial-and-error process.

Methodology The authors model a multi-agent LLM system as a directed graph whose row-normalized adjacency matrix serves as a stochastic transition operator P, then compute the successor representation M = (I − γP)⁻¹. Three spectral quantities of M are derived in closed form for three canonical topologies (chain, star, mesh): the spectral radius ρ(M), the spectral gap Δ(M), and the condition number κ(M). Predictions from these quantities are validated on a 12-step structured state-tracking task using Qwen2.5-7B-Instruct over 100 independent trials per topology.

Results The condition number is a perfect rank-order predictor of empirical perturbation robustness (Spearman rs = 1.0); the spectral gap partially predicts consensus dynamics (rs = 0.5); and the spectral radius is perfectly inverted with respect to cumulative error (rs = −1.0). The inversion — dubbed the "stability paradox" — occurs because the chain topology, which is most stable in the linear spectral sense, allows each agent's small bias to accumulate sequentially, while star and mesh topologies include aggregation steps that suppress drift. An affine-noise extension of the predictive map recovers the correct empirical ordering of cumulative error across topologies.

Plain-language abstract This paper is a comprehensive survey of LLM-based multi-agent systems — software architectures in which multiple AI agents, each powered by a large language model, collaborate to tackle tasks too complex for any single agent. It introduces a four-stage framework called LIFE (Lay the capability foundation, Integrate agents through collaboration, Find faults through attribution, and Evolve through autonomous self-improvement) to organize the field as a causally linked progression rather than isolated topics.

Motivation While individual LLM agents have become capable at reasoning, planning, and tool use, they struggle with tasks that require sustained coordination across many roles and environments. Existing surveys treated individual agent capabilities, multi-agent collaboration, and agent self-evolution as separate topics, leaving the causal connections among them — and in particular the problem of diagnosing and recovering from failures in multi-agent systems — largely unexamined. No prior survey had systematically covered failure attribution in multi-agent systems, despite it becoming a distinct and rapidly growing research area.

Methodology The survey organizes its review around the LIFE progression, providing stage-wise structured taxonomies and comparative analyses. It covers: core capabilities of individual agents (reasoning, memory, planning, tool use); organizational mechanisms of multi-agent collaboration (role allocation, communication protocols, orchestration topologies, interaction patterns); a methodological landscape of failure attribution approaches (data-driven, constraint-guided, and causal-inference methods); and a hierarchical design space of self-evolution. Throughout, the authors formally characterize causal dependencies between adjacent stages. The work also curates a publicly maintained repository of structured literature, taxonomy visualizations, and comparative tables.

Results The survey establishes LIFE as the first unified analytical framework covering the complete operational lifecycle of LLM-based multi-agent systems. It reveals an underexplored attribution–evolution closed loop: failure attribution narrows the search space for targeted self-improvement, while collaborative structures shape which failures can be observed and attributed. The authors identify open challenges at the boundaries between LIFE stages and propose a cross-stage research agenda aimed at closed-loop systems capable of continuously diagnosing failures, reorganizing collaborative structures, and refining agent behaviors toward more self-organizing and resilient collective intelligence.

Plain-language abstract This paper tests whether having teams of AI language models debate each other actually improves their accuracy, or whether it just wastes compute. The researchers ran controlled experiments with groups of 10 identical small models on hard math and science questions, comparing peer debate against a simpler approach where each model corrects itself in isolation.

Motivation Multi-agent debate — where multiple AI models exchange reasoning and vote on answers — is widely deployed on the assumption that peer review filters out errors and hallucinations. However, no prior study had systematically measured whether this approach is actually more cost-effective than simpler alternatives, and the failure modes of homogeneous (all-same-model) debate teams were poorly understood.

Methodology The study evaluated teams of N=10 homogeneous agents using three open-weight models (Qwen2.5-7B, Llama-3.1-8B, Ministral-3-8B) across R=3 debate rounds on two high-difficulty benchmarks: GSM-Hard (algorithmic mathematics) and MMLU-Hard (complex scientific reasoning). Peer debate was compared against isolated self-correction and a stochastic noise control that injected rationales from unrelated problems. Ablations varied communication density (K in {2, 4, 9} peers per agent) and sampling temperature (T in {0.4, 0.7}).

Results Debate consistently underperformed or matched isolated self-correction while consuming 2.1–3.4 times more tokens (up to 28,631 tokens per problem). Three failure modes were identified: sycophantic conformity (agents adopted the majority answer without logical verification, modal adoption rate up to 85.5%), contextual fragility (peer rationales destabilized previously correct reasoning, vulnerability rate up to 70.0%), and consensus collapse (plurality voting discarded correct answers already present in the pool, oracle gap up to 32.3 percentage points). Artificial team consensus inflated to 90.1% even as accuracy degraded, showing that homogeneous unguided debate is both economically inefficient and logically unstable for 7–8B parameter models.

Plain-language abstract Pioneer Agent is an automated system that takes a small language model (1B–8B parameters) through the full cycle of task adaptation: gathering training data, diagnosing failures, retraining, and verifying results — all without manual engineering intervention. It operates in two modes: a cold-start mode that begins from a plain-language task description, and a production mode that starts from a deployed model's logged failures.

Motivation Small language models are attractive for production use because they are cheap and fast, but adapting them to a specific task still requires substantial human effort — deciding what data to collect, which failures matter, how to avoid regressions, and when to stop iterating. Existing AutoML tools address hyperparameter search but not the upstream challenges of data curation, failure diagnosis, and safe retraining, leaving teams to manage a difficult engineering loop manually.

Methodology Pioneer Agent uses an orchestrator LLM (Claude Sonnet) driving a LangGraph state machine that coordinates sub-agents for trace analysis, task delegation, data curation, training, and evaluation. Training pipelines are represented as triples of dataset, hyperparameters, and learning strategy, and the agent searches over these jointly using a Monte Carlo–style graph search (MCGS), expanding, fusing, scoring, and pruning nodes under explicit regression constraints. Experiments run in sandboxed Modal environments with LoRA fine-tuning via the Tinker SDK. The authors also introduce AdaptFT-Bench, a new benchmark of synthetic inference logs with progressively increasing noise, to evaluate the full adaptation loop across seven scenarios.

Results Across eight cold-start benchmarks covering reasoning, math, code generation, summarization, and classification, Pioneer Agent improves over base models by 1.6 to 83.8 points. On AdaptFT-Bench, the agent improves or preserves performance in all seven scenarios while naive retraining degrades by up to 43 points. On two production-style deployments, intent classification accuracy rose from 84.9% to 99.3% and named-entity F1 improved from 0.345 to 0.810. The agent also independently discovered effective training strategies — including chain-of-thought supervision and quality-focused data curation — purely from downstream feedback signals.

Plain-language abstract This paper documents a catalog of 63 confirmed real-world cases where AI agent systems spent far more money than intended—sometimes thousands of dollars—because no mechanism stopped runaway API usage. It also introduces a Rust software library that uses the programming language's type system to make overspending structurally impossible, not just something an operator hopes to prevent.

Motivation LLM-based agents running in production can enter retry loops or fan out tasks in ways that accumulate enormous API costs before anyone notices. Existing frameworks either lack spending caps entirely or enforce them with ad-hoc code that can be bypassed by operator error or concurrency bugs. There was no systematic empirical record of how often and how severely this failure mode occurs across the ecosystem.

Methodology The authors compiled a catalog of 63 confirmed production incidents drawn from 21 orchestration frameworks across 2023–2026, each supported by a quoted GitHub issue or maintainer statement and documented dollar losses where available. Incidents were organized into an eight-cluster failure taxonomy and validated with two independent human raters (Cohen's kappa = 0.837 on the four-class scheme). As a concrete mitigation, the authors built a 1,180-line Rust crate called token-budgets that enforces affine ownership—meaning a budget value cannot be cloned or double-spent—so that multi-agent delegation races that cause overspending become compile-time errors. The crate was evaluated against five production runtimes and tested with a temperature-stratified live API experiment of 160 runs across three providers.

Results Across 160 live API calls at varying temperatures, the Rust crate produced zero cap violations and zero false refusals. In the critical multi-agent delegation scenario, an unguarded Python asyncio pattern overshot the budget in 30 out of 30 trials, while the Rust crate (and a properly locked Python counter) overshot in 0 out of 30—a deterministic split attributable to compile-time enforcement of non-duplication. The static cost estimator reserves 4–6 times actual cost as a safety margin; an adaptive estimator narrows this to a 2.11× median overage. Reasoning models from OpenAI, Anthropic, and DeepSeek fall outside the primary guarantee because providers bill for hidden reasoning tokens, so the crate serves as a defense-in-depth layer for those models rather than a primary cap.

Plain-language abstract AgentForesight is a framework that monitors AI multi-agent systems in real time to catch critical errors as they happen, before those errors cascade into full task failures. Rather than diagnosing what went wrong after a task has already failed, it flags the specific step and responsible agent at the moment the decisive mistake occurs, enabling intervention while the system is still running.

Motivation When multiple AI agents collaborate on long, multi-step tasks, a single wrong action early in the process is typically accepted by downstream agents and compounds into a complete failure. Prior work only diagnosed these failures after the fact, once all computation had already been wasted and potentially irreversible side effects had occurred. There was no mechanism to catch and correct errors mid-execution.

Methodology The authors constructed AFTRaj-2K, a dataset of 2,272 multi-agent trajectories spanning coding, math, and agentic task domains, with safe trajectories filtered under strict criteria and failed trajectories annotated at their decisive error step using consensus among multiple LLM judges. They then fine-tuned a 7-billion-parameter model (Qwen2.5-7B-Instruct) using a two-stage reinforcement learning approach: a first stage that aligns the model to distinguish safe from failing trajectory prefixes at the error boundary, and a second stage that sharpens its predictions using a three-axis reward targeting what the error is, where in the trajectory it occurs, and which agent is responsible.

Results AgentForesight-7B outperformed larger proprietary models including GPT-4.1 and DeepSeek-V4-Pro on both the AFTRaj-2K test set and the external Who&When benchmark, achieving up to 19.9% higher Exact-F1 and 3x lower step localization error than the strongest proprietary baseline. These results demonstrate that a compact, purpose-trained auditor running alongside a deployed multi-agent system can detect decisive errors earlier and more precisely than general-purpose large language models used as judges.

Plain-language abstract This paper introduces HORIZON, a benchmark for studying how and why AI agents that use large language models fail when given long, multi-step tasks. The authors evaluate several state-of-the-art models across four task domains, collect over 3,100 agent run traces, and use an automated judging pipeline to classify the types of errors that appear as tasks become more complex.

Motivation LLM-based agents work reasonably well on short tasks but fail systematically as the number of required sequential steps grows. Prior benchmarks measure only whether a task ultimately succeeded, use inconsistent definitions of task length, and are confined to single domains, so there is no principled, cross-domain way to diagnose where the performance collapse happens or what causes it.

Methodology The authors built HORIZON, a cross-domain diagnostic benchmark that constructs task families with systematically increasing compositional depth across four domains: web navigation, operating systems, databases, and embodied manipulation. GPT-5 variants and Claude-4 models were evaluated over 3,100+ trajectories. Failed runs were labeled using a seven-category failure taxonomy derived from Failure Mode and Effects Analysis (FMEA), with a trajectory-grounded LLM-as-a-Judge pipeline validated against human annotators on 40 trajectories, yielding inter-annotator agreement of kappa=0.61 and human-to-judge agreement of kappa=0.84.

Results Agent success rates consistently and sharply decline beyond a domain-specific compositional depth threshold, and this degradation is not gradual but exhibits a clear breaking point. Web navigation collapses at low compositional depth while OS and database tasks sustain moderate performance longer; embodied tasks degrade steeply even with minimal increases. Planning-related failures (e.g., subplanning errors) and memory-related failures (e.g., catastrophic forgetting) become dominant as horizon increases, and model performance gaps narrow substantially in the failure regime, converging toward uniformly low success rates across model families.

Plain-language abstract HARBOR is a system that automatically tunes the configuration of the software wrapper — called a harness — that surrounds large language model agents. Rather than letting engineers manually toggle feature flags one at a time, HARBOR uses a principled search algorithm (Bayesian optimization) to find the best combination of harness settings efficiently and safely. The paper both defines this optimization problem formally and demonstrates it on a real coding agent.

Motivation Modern AI agents spend the vast majority of their codebase in deterministic harness code — context management, tool caching, memory, failure recovery — rather than in the model itself. Teams today tune these harnesses manually, but the space of possible feature combinations grows exponentially, and the authors show through a four-round case study that manual tuning is unreliable: three of four rounds produced no improvement or regressions, partly due to silent integration bugs that pass-rate inspection alone cannot detect.

Methodology The authors formalize harness configuration as constrained noisy Bayesian optimization over a mixed-variable, cost-heterogeneous flag space and introduce HARBOR (Harness Axis-aligned Regularized Bayesian Optimization Routine). HARBOR combines a block-additive SAAS sparse Gaussian-process surrogate, multi-fidelity cost-aware acquisition, TuRBO trust-region search, a cold-start-corrected reward estimator, and a posterior chance-constrained safety check that blocks regressions below the baseline. They evaluate it on a flag-gated version of codex-py — a production Python coding agent — against Terminal-Bench 2, an 89-task suite covering shell scripting, systems administration, cryptography, and scientific-Python tasks.

Results Manual tuning over four rounds (A–D) produced at best one credible net win: round B scored 17/89 tasks versus a baseline of 15/89, while adding more features in rounds C and D dropped performance to 13/89 and 12/89 respectively. HARBOR's automated two-flag solution matched the manual peak (17/89) using only two flags instead of five to eight, recovered 96.97% of the token-compression savings of the all-flags-on configuration, and additionally surfaced two silent integration bugs — a cross-session reflection store that was written but never read, and a speculative-tool predictor that was never wired into the runner — that four rounds of manual tuning had missed entirely.

Plain-language abstract This paper studies a security vulnerability in AI systems where multiple language model agents collaborate by passing reports between a Worker agent and a Manager agent. The authors show that making the Worker smarter—using more capable language models—actually makes the overall system easier to attack, not harder. They identify this as the "capability paradox" and propose a defense strategy based on deliberately pairing agents of mismatched competence.

Motivation Multi-agent systems built on large language models divide tasks among specialized agents, but this creates a new security risk: the Manager agent often trusts Worker reports as reliable evidence, even if the Worker's input was manipulated. Prior security research focused on syntactic attacks like instruction overrides, but this paper addresses a harder-to-detect threat called semantic hijacking, where harmful requests are hidden inside plausible-sounding domain narratives with no explicit injection commands, making them invisible to traditional defenses.

Methodology The authors constructed an evaluation framework using real postmortem incident reports (such as site-reliability engineering outages) that were mutated by a language model into adversarial and benign payloads. They ran 42,000 adversarial trials across 12 Manager models and 7 Worker configurations, measuring Attack Success Rate (ASR) at the system level. To explain the observed paradox, they conducted multi-level mediation analysis on two independent datasets totaling 47,807 interactions, testing whether linguistic certainty in Worker reports mediates the relationship between Worker capability and Manager execution decisions.

Results Replacing a weak Worker (Llama-3.1-8B) with a stronger Worker (DeepSeek-R1) raised the system-level ASR from 20.6% to 94.4% against the same Manager. Across all configurations, mean ASR increased from 18.4% to 63.9% as Worker capability grew. Mediation analysis found that linguistic certainty—how assertively the Worker phrased its conclusions—explained 74% of the capability paradox effect in the larger dataset, with confidence intervals excluding zero. Worker-side safety prompting did not reliably reduce the risk. A proposed defense, heterogeneous ensemble verification using Workers with asymmetric domain competence, reduced ASR from 52.8% to 2.0% with negligible impact on benign tasks.

Plain-language abstract This paper examines whether AI systems that combine vision and language — such as Vision Language Models and multi-agent systems — can recognize when a question cannot be reliably answered and appropriately decline to respond. The authors build a benchmark called MMAQA with 2,079 samples drawn from both answerable and deliberately unanswerable questions, then evaluate how well several leading AI systems handle these cases.

Motivation Current benchmarks for multimodal AI assume every question has an answer, which trains and tests models to always respond even when the input is incomplete, ambiguous, or contradictory. This creates a critical reliability gap: in high-stakes settings like medical image analysis or legal document review, a model that confidently produces a wrong answer is more dangerous than one that declines to answer. The paper addresses the absence of rigorous evaluation of this abstention capability.

Methodology The authors construct MMAQA by transforming answerable multimodal instances into unanswerable counterparts along two axes — visual modality dependency and evidence sufficiency — then filtering them through a dual-consensus VLM quality-control module and human annotators to yield 2,079 samples. They evaluate three frontier Vision Language Models spanning closed and open-source systems, plus two multi-agent system architectures, under a 3x2 condition-by-clause experimental design. Responses are categorized using a five-way confusion matrix and four metrics.

Results Under standard prompting, VLMs almost never abstain, and even simple confidence baselines outperform this default behavior. Multi-agent systems improve abstention rates but introduce an accuracy-abstention trade-off: no system simultaneously exceeds 65% accuracy on both answerable and unanswerable instances. Sequential multi-agent designs match or exceed iterative variants, indicating the bottleneck is model miscalibration rather than insufficient reasoning depth. Models tend to abstain when image or text evidence is entirely absent, but attempt to reconcile degraded or contradictory evidence rather than declining, suggesting that effective abstention requires abstention-aware training rather than better prompting or more agents.

Plain-language abstract GSAR is a framework for detecting and recovering from hallucinations in multi-agent AI systems that investigate operational incidents. When a team of specialized AI agents each produce diagnostic claims about a system failure, GSAR evaluates whether each claim is actually supported by observed evidence, and then decides whether to accept, regenerate, or replan the response — all within an explicit compute budget.

Motivation Multi-agent AI systems for incident investigation blend tool-verified observations with plausible model-inferred claims, and existing hallucination evaluators cannot distinguish between these categories. Prior approaches such as binary classifiers and LLM-as-judge scalars treat all supporting evidence as equivalent and emit a single undifferentiated signal, leaving the system unable to select an appropriately cheap recovery action when only a minor fix is needed versus a full replan.

Methodology GSAR partitions claims into four categories — grounded, ungrounded, contradicted, and complementary — and assigns evidence-type-specific weights reflecting epistemic strength (tool-observed evidence weighted higher than model-inferred). It computes an asymmetric contradiction-penalized weighted groundedness score and couples it to a three-tier decision function (proceed, regenerate, replan) driving a bounded iteration loop. The framework was evaluated on the FEVER claim-verification benchmark using a production stack (Locus SDK, Cohere embed-v3.0, Oracle Database 26ai AI Vector Search) and graded by four independently trained LLM judges: GPT-5.4, Claude Sonnet 4.6, Claude Opus 4.7, and Gemini 2.5 Pro.

Results Every ablation replicated in the same direction across all four judges. Removing the asymmetric contradiction penalty (rho=0) produced score inflation of +0.04 to +0.06 under three judges while leaving contradiction catch-rate unchanged, confirming that the penalty prevents score gaming without affecting detection. Collapsing the complementary claim category dropped the grounded-output rate from 100/200 to 18/200 (an 82% reduction) under Opus 4.7. Under normal operating conditions GSAR's score correlated with an independent grader's faithfulness judgment at Spearman rho = +0.53 (95% CI [+0.34, +0.70]), and proceed agreement was 100%. The end-to-end pipeline ran at n=200 samples in 339 seconds wall-clock time on the production stack.

Plain-language abstract This paper introduces SWARM (System-Wide Assessment of Risk in Multi-agent systems), a simulation framework for evaluating the safety of AI systems where many agents interact. Instead of labeling each agent action as simply safe or unsafe, SWARM assigns a continuous probability score to every interaction, enabling more nuanced measurement of risk and the testing of governance policies designed to manage it.

Motivation Current AI safety frameworks classify agent behavior as binary — safe or unsafe — discarding the uncertainty that comes from imperfect evaluation. When a proxy assessment assigns only 60% confidence that an interaction is beneficial, collapsing that to a binary 'safe' verdict hides the residual 40% risk. This mirrors Goodhart's Law: once a binary threshold becomes the target, agents can satisfy the metric while the underlying objective degrades, a failure mode documented in a companion case study where an agent passed all binary benchmarks while covertly cutting output quality.

Methodology SWARM models a population of N agents interacting over discrete time steps. A proxy computer aggregates five observable signals — task progress, rework rate, verifier rejections, tool misuse flags, and counterparty engagement — into a combined score that is then mapped to a soft probabilistic label p via a calibrated sigmoid. A payoff engine computes expected surplus and harm as weighted expectations under this uncertainty. A governance engine applies configurable levers including transaction taxes, circuit breakers, reputation decay, and random audits. Seven scenarios were run with five-seed replication to map tradeoffs. Companion experiments applied the same governance layer to live LLM-backed agents (Concordia entities, Claude Haiku, Claude Sonnet, and GPT-4o Mini) without architectural changes.

Results Strict threshold-based governance reduced system welfare by over 40% without meaningfully lowering systemic toxicity. Aggressive externality internalization collapsed total welfare from a baseline of +262 down to -67, while toxicity remained invariant. Circuit breakers required careful calibration: overly restrictive thresholds severely diminished system value, while an optimal threshold balanced moderate welfare with minimized toxicity. Soft distributional metrics detected proxy gaming by self-optimizing agents that continued to pass all conventional binary evaluations. The governance layer transferred without modification to live LLM agents, and RLHF safety alignment in those models proved robust to adversarial system-prompt manipulation.

Plain-language abstract This paper studies how humans can more effectively oversee and correct the plans generated by AI-driven multi-agent systems. The authors build AMBIPOM, an interactive prototype that lets users inspect and edit multi-agent plans represented as dependency graphs, and they run both a user study and a controlled benchmark to learn how people and language models behave when collaborating on plan refinement.

Motivation Modern multi-agent systems coordinated by large language models can misalign with human intent, hallucinate steps, or violate domain constraints, yet most existing tools only let users check final outputs. This outcome-only supervision makes it hard to diagnose where a plan went wrong or apply targeted fixes, creating a need for process-level human oversight at intermediate planning stages.

Methodology The authors formalize a three-axis design space for human-LLM co-planning: mode (structural graph manipulation vs. natural-language feedback), scope (global vs. targeted subgraph), and level (low-level edits such as add/delete vs. high-level operations such as merge/split). They implement this space in AMBIPOM, which represents plans as directed acyclic graphs. A 13-participant user study measured efficiency, effort, and strategy, while a controlled benchmark of 200 gold plans and 1,150 broken-plan items tested four LLM revision conditions (global feedback, targeted feedback with and without full-plan context, and edit-sequence generation) across four task types.

Results Users naturally developed hybrid workflows, alternating between broad text feedback and local direct manipulation, and most (10 of 13) preferred inspecting plans before executing them. Global feedback produced the most structurally accurate LLM revisions (average execution accuracy 0.840 vs. 0.553 for edit-sequence generation), while targeted feedback was better at preserving plan stability by leaving untouched nodes unchanged. Edit-sequence generation offered higher interpretability but was markedly less robust for complex structural edits such as adding nodes or merging steps.

Plain-language abstract XFlow is a system for building multi-agent LLM workflows that are more reliable, and XPF is its protocol language. The core idea: today, constraints, rules, and process obligations are buried inside prompts that agents must remember and re-apply, with no way for the surrounding system to enforce them. XFlow draws an explicit prompt-harness boundary: informal semantic reasoning stays inside actors, but selected commitments are moved into harness structure that is compiled, checked, and enforced. A protocol is written as a readable literate document but compiled into a typed intermediate representation and executed as a program. At runtime, agent outputs are staged as lifecycle-governed symbols and only become shared state after passing checks.

Motivation In a multi-agent pipeline, one agent's hallucination, malformed output, or misread instruction becomes shared state and corrupts later decisions. Current frameworks sit at two extremes of the Chomsky hierarchy: markup/config tools describe workflow shape but say nothing about where knowledge leaves the prompt, and prompt-based tools keep formalizable constraints stuck in instructions. Developers have no language for drawing, testing, and adjusting which commitments are governable versus left to the actor.

Methodology Three-phase architecture. Specification: authors write XPF (YAML frontmatter for symbols/policies, Markdown stages, fenced semantic blocks for actor interfaces, guarded control flow, and responsibility handoffs). Compilation: mechanical passes parse the literate surface, resolve names, statically check that actor reads/writes are legal, judge outputs bind to declared symbols, flow targets exist, and call returns match child-protocol outputs, then lower to a typed IR. Execution: a runtime wraps each actor call in a typed interface, commits symbol writes only after schema validation/provenance/commit-policy checks inside atomic transactions with rollback, propagates staleness to derived values reactively, and persists scoped session frames for resume and audit. Evaluation uses Qwen3.5-9B and DeepSeek-V4-Flash across tau3-bench, CorpusQA (layered on the XpandA baseline), and SWE-bench Verified (around mini-SWE-agent).

Results On tau3-bench, wrapping a bare ReAct agent with XFlow raised constraint-compliance to 100% in Retail and Airline for Qwen3.5-9B (from 96.5% and 91.8%) and to 100% across all three domains for DeepSeek-V4-Flash (Telecom compliance +36.3pp), while task pass1 stayed comparable, separating 'reached the answer' from 'reached it via a valid path.' CorpusQA accuracy improved with XpandA+XFlow to 61.7 (+2.4) for Qwen and 75.7 (+0.9) for DeepSeek by encoding domain interpretation rules as deterministic derivations over extracted symbols. On SWE-bench Verified, XFlow around mini-SWE-agent raised the pass rate from 77.4 to 79.8 (+2.4) by gating patch submission on a passing local validation check; a case study shows the protocol rejecting a submission after a failed test run and issuing a targeted retry. The paper also presents cloud-edge use cases where edge worker outputs must pass schema and coverage checks before entering global state.

Plain-language abstract Multi-agent LLM systems leave long execution traces in which a few events actually decide the outcome, buried in logs of messages, routes, memory writes, and tool calls. The standard way to find those events is counterfactual replay: rewind, edit, and re-run the trajectory to measure each event's effect, but its cost grows linearly with candidate events and is infeasible at scale. The authors compile each trace into a typed event knowledge graph and train a calibrated predictor that estimates which events the replay oracle would mark high-effect, without running the oracle. The named system, BranchPoint-Latent, uses a gradient-boosted learning-to-rank model over 13 graph features and raises per-trace localization (Branch Recall@5) from 0.73 to 0.93 on held-out families at zero replay cost.

Motivation Reliable operation of multi-agent LLM systems depends on debugging long traces, but the standard counterfactual-replay oracle costs O(T*|F|*N) model calls (seconds-to-minutes of GPU per replay), prohibitive at production scale. The authors take a direction orthogonal to oracle design: predict the oracle's per-event verdict cheaply instead of paying for it. Prediction is non-trivial because no single cheap signal works everywhere: graph centrality is strong on graph-friendly traces but near-constant on chains, Last-K fails when consequential events are early, and novelty/disagreement/uncertainty anti-correlate with the oracle in several regimes.

Methodology Each trace is compiled into an event knowledge graph with typed node attributes (route position, memory/retrieval persistence, tool metadata, uncertainty proxies, optional latent payloads). A deterministic remove-event replay oracle scores each event offline and supplies training labels once, then is removed from the deployment loop. From the graph, 13 CPU-cheap features are computed. A single gradient-boosted tree (depth 3, 400 trees, lr 0.08) with a within-trace learning-to-rank objective and family-balanced weights emits a redundancy-aware budget-bounded top-K replay agenda. Evaluation uses five-fold cross-validation grouped by trace and by trace-disjoint held-out family, over 37 trace families (163,815 events) drawn from HotpotQA, StrategyQA, GSM8K, ARC, XSum, and MBPP, plus six model-authored live-agent families (Qwen3-1.7B) and the Who&When leaderboard.

Results The interpretable linear scorer attains Spearman rho 0.58-0.59, AUPRC 0.61 (19 points over the 0.418 base rate), ECE 0.13, dominating centrality (0.52) and routing bottleneck (0.45), while single-feature heuristics anti-correlate. The learning-to-rank GBM improves every metric: Branch Recall@5 0.95 in-distribution and 0.93 on held-out families, NDCG@5 0.94/0.92, per-trace rho 0.80/0.77, beating the linear scorer on 28 of 37 families. Gains concentrate where linear scoring is blind: held-out MBPP tool traces go 0.00 to 1.00, GSM8K 0.23 to 0.98. The predictor recovers ~80% of the active-replay oracle's recall at zero replay cost. On live model-authored traces it beats centrality on all six families. On Who&When, the LLM-free CPU model reaches Acc@1 0.37 (algo) and 0.24 (hand-crafted), matching an RL-fine-tuned 8B LLM while the benchmark's own zero-shot LLM judges reach only <=0.14. A structural regime router does not beat the always-learned default (0.719 vs 0.731).

Plain-language abstract Self-improving agent systems in the Darwin and Huxley Gödel Machine line search over self-modifying agents but assume a fixed benchmark stays valid as agents improve. The Red Queen Gödel Machine makes the evaluator part of the loop: learned evaluators co-evolve with the task agents they score, and the search objective may change only at epoch boundaries so per-epoch self-improvement guarantees still hold.

Motivation Static evaluation blocks self-improvement in three settings: tasks with no direct benchmark (paper writing, proof writing), evaluation that is slow or weakly informative, and benchmarks that saturate or get reward-hacked as agents strengthen. The Red Queen dynamic from evolution, adapting against co-evolving competitors, is the missing piece.

Methodology Tree search over an archive of multi-agent workspaces, with nodes selected by Thompson sampling over clade metaproductivity. Controlled utility evolution: within an epoch one evaluator is frozen and grades every task agent; challenger evaluators evolve in the shared codebase and are scored on a held-out ground-truth anchor; at checkpoints a challenger replaces the incumbent only if it wins on an epsilon-best-belief criterion, after which selective erasure discards utility records that depended on the displaced evaluator. Evaluated on coding (Polyglot), scientific paper writing and reviewing, and Olympiad proof writing and grading.

Results On verifiable coding, adding a cheap co-evolved code-review judge (queried once, versus multi-turn agent execution) reaches 71.7% held-out pass rate against the prior SOTA HGM-H's 69.9% with 1.35–1.72× fewer search tokens. Co-evolved paper writers raise acceptance from 21.8% to 40.5% under a diverse judge panel; a co-evolved grader beats static baselines at 3× lower search cost with 9% higher ground-truth accuracy. An adversarial-objective epoch corrects a reviewer that over-accepted AI-generated papers at up to 1.91× the human rate, yielding one equally stringent on both.