Nested enterprise schemas push NL-to-SQL to 91.7%, while agent-construction benchmarks stall near 25%

91.7% answer correctness on 900 execution-verified enterprise queries, 54.6 points clear of the next-best system. That is the headline number from a DevRev team’s NL-to-SQL paper posted today, and the interesting part is what the gap is measuring. Their new benchmark targets the schema shape academic suites skip: nested types, link graphs between entities, semi-structured columns, the stuff a real ticketing-and-CRM warehouse accumulates. They pair it with a Semantic Depth Score, a schema-agnostic rubric for how much analytical reasoning a question demands, so a query set can be characterized by depth rather than by table count. The system itself is a single-generation agentic pipeline whose schema-selection, metadata-retrieval, and error-repair stages are each designed around that nested regime, and the same architecture stays competitive on Spider 2.0 Snowflake at one generation per query. For the semantic-governance charter this is the right kind of evidence: the accuracy jump comes from grounding the agent in the schema’s structure and metadata, not from more sampling, which is the claim that “semantic context changes agent correctness” needs to be tested against. What the abstract does not report is the wrong-join and wrong-column breakdown, so read the full paper for whether the 54.6 points come from schema linking or from repair.

The evals story in the last few days is about how far agents sit below expert ceilings once the task is realistic. τ^τ-bench, from Quan Shi, Keshav Dhandhania, Karthik Narasimhan, and Victor Barres, makes agent construction the task: a developer agent gets a business’s actual records, a client who holds the requirements, a production API, an inherited codebase, and a serving-cost budget, and has to deliver a working customer-service agent that is then scored against held-out simulated users. Across 53 tasks in four domains the best configuration, Claude Opus 5 under Claude Code, passes 23.9% of evaluation simulations against an expert-authored reference at 82.2%. The failure modes read like a code review of a junior contractor: shallow queries against the records instead of reading them, near-zero communication with the client, and no experimentation with architecture or spend, shipping the first design that runs. That last point is the one to bring to your own harness. Cost-and-architecture search is a step models skip unless forced.

Harbor Adapters and Harbor-Index, a very large author list anchored by Ludwig Schmidt, Alex Shaw, and Mike Merrill, is the infrastructure paper the field kept asking for. They ported more than 80 agentic benchmarks behind one adapter interface, validated the ports with parity experiments, and ran 8 models across 54 benchmarks each under Terminus-2 and a native harness. The distilled artifact is Harbor-Index: 82 tasks from 29 benchmarks, filtered for difficulty and audited by humans and models, sized to be affordable to run. No model-harness pair clears 30%; GPT-5.5 with Codex tops it at 28.0%. Read with τ^τ-bench, the two papers set a consistent ceiling for September 2026: roughly a quarter of hard, realistic agentic tasks, regardless of who is measuring.

On the retrieval side, Embedding Surgery from Maddalena Amendola, Antonio Mallia, and Raffaele Perego attacks the static-index problem directly. Dense retrievers compute document embeddings offline, so they cannot absorb editorial feedback or a shift in intent without a rebuild. The method applies minimal, localized updates to selected document embeddings, framed as a convex optimization that enforces ranking constraints while bounding how far each vector moves. On TREC DL-Hard with editorial feedback that yields up to a 60.64% relative gain in nDCG@10, corrections propagate to semantically related queries, and the updates go into an ANN index by in-place overwrite, no reconstruction. It composes with query-side methods like CoRocchio and holds up better under noisy feedback. If your agent’s retrieval lane has a “the ranking is wrong for this query and we know it” problem, this is a cheaper answer than retraining the encoder.

Two production write-ups from r/LLMDevs carry the reliability thread. A post-mortem on in-memory agent state graphs describes running agents around the clock on LangGraph and CrewAI-style runtimes and hitting three failure classes: context windows that accumulate noise and drive cost up, one unhandled exception in a coroutine corrupting the whole runtime graph, and any OOM or segfault erasing every in-flight trajectory. Their replacement, open-sourced as samantha-core, moves durable knowledge into Markdown on disk, routes inter-agent messages through atomic inbox directories with explicit acks, replaces unbounded coordinator loops with short-lived scoped executors tracked by file plans, and cascades across model tiers on rate limits. It is, in effect, a durable-execution engine rebuilt from filesystem primitives, and the design decisions map one-to-one onto what Temporal-class engines provide.

The second is a confession: 883 commits and eight months on a coding-agent harness where the model was a replaceable worker and the system owned planning, state, permissions, verification, and evidence, abandoned because the scope kept growing (code graphs, replayable runs, model routing, dashboards, computer use) until it was a headless experiment nobody used. The archived repo is offered as parts, and the list of salvageable ideas is the same list the post-mortem above arrived at from the other direction: state and verification outside the model, replayable runs, orchestrator-worker separation, context budgeting and handoffs, permission gates, and checking agent claims against evidence the agent did not generate. Two independent practitioners converging on that list in the same week is a stronger signal than either post alone.

Zhongjie Wang and Mingyi Liu put a theoretical frame around the same convergence. Their argument is that agents make execution elastic while problem framing, semantic commitment, verification, integration, and residual-risk acceptance stay bounded by human cognition and organizational authority. They propose Trustworthy Change as the engineering object that moves from intent through delegated execution to acceptance, a Responsibility Topology that classifies organizations by how many independent authorities must accept residual risk, and the Human-Agent Cell as the unit that produces candidates and evidence but holds no acceptance authority. It is explicitly hypotheses, not results, and the authors say so. The useful contribution is naming the axis: as execution scales and authority does not, the coordination cost lands on shared engineering facts and invalidation, which is where the harness builders above were bleeding.

What to watch: whether the DevRev benchmark publishes per-error-class rates so the schema-grounding claim can be separated from repair, and whether anyone runs τ^τ-bench with the cost-and-architecture search step forced rather than optional. The ceiling numbers are now consistent enough across suites that the next interesting result is one that moves them.