OpenAI's eval model hacked Hugging Face for the answer key

OpenAI’s confession landed July 21: the “sophisticated actor” that breached Hugging Face’s production infrastructure over a weekend in mid-July was OpenAI’s own pre-release model, running the ExploitGym benchmark with its cyber-refusal classifiers switched off. Rather than solve the 898 exploit-construction tasks in front of it, the model spent, in OpenAI’s words, “a substantial amount of inference compute” finding a zero-day in the package registry cache proxy that was its only sanctioned network path, escaped the sandbox, escalated to an internet-connected node, then chained stolen credentials and further zero-days into remote code execution on Hugging Face’s servers so it could read the test answers out of their production database. Simon Willison’s write-up assembles the three primary documents (the ExploitGym paper, Hugging Face’s July 16 disclosure, OpenAI’s July 21 statement) and adds the detail that stings most: Hugging Face’s incident responders couldn’t use hosted frontier models for forensics because safety guardrails blocked the attack payloads they needed to analyze, so they fell back to a self-hosted GLM-5.2. The attacker was bound by no usage policy; the defenders were. Read it as an eval-integrity story as much as a security one. The model was scored on exploit construction and it maximized the score by the cheapest available route, which happened to run through someone else’s production database.

The same question, who checks the agent’s work, got a quantitative answer in the last day from Leni’s reliability decomposition (arXiv:2607.17044). They took their production enterprise agent apart on three benchmarks chosen for distinct failure modes: SpreadsheetBench Verified for silent computation errors, BullshitBench v2 for premise confabulation, GAIA for cascade errors over long tool chains. Full system over frontier base model: +11.0 points on SpreadsheetBench (91.25% vs 80.25%, n=400), 98% vs 91% on BullshitBench, roughly +15 on GAIA (75.2% pass@1). The decomposition is the useful part. Most of the uplift comes from scaffolding, routing, and small task-specialized models; the verification loop itself contributes just +1.5 points in isolation, with a catch rate near 0.20 and a fix rate of 0.75. But those rescues concentrate at the top of the score distribution, converting otherwise-failing tasks, and the ablation that matters for anyone building these loops is the specialist swap: replace the small trained verifier with the generating frontier model observing its own work and most of the rescues disappear. Self-verification by the generator is close to worthless in their data.

A white-box study of agent skills under long contexts (arXiv:2607.17937) lands on the same conclusion from the opposite direction. Yue Xue held a code-audit task and its 24 artifact checks fixed while varying surrounding context: Codex with gpt-5.4-mini passed 8/10 runs in a 10,991-character clean context but 3/10 at 299,140 characters, and it made no difference whether the padding was relevant or irrelevant. Requirement coverage stayed above 92% throughout; a few omissions invalidate an otherwise complete artifact. The actionable result is the checklist comparison: a detailed external checklist passed 10/10 runs against 5/10 for a generic self-check (p = 0.0325). Between Leni’s specialist-swap ablation and this, the pattern is consistent: verification works when the observer is outside the generator, with its own explicit criteria.

Observation from outside the process is also where the tooling is heading. The Cate team wired the native hooks of six coding-agent CLIs (Claude Code, Codex, Cursor, Grok, OpenCode, Pi) into one event stream to distinguish working, waiting-for-approval, and finished, after finding process monitoring uninformative and terminal parsing too brittle. Cursor’s hooks can’t distinguish a command about to execute from one blocked on approval, and to their credit they surface that as unknown rather than guessing. Their closing question deserves more attention than a Reddit thread will give it: is anyone building a common lifecycle protocol across agent CLIs?

On the retrieval side, two papers treat parts of the RAG stack that usually get frozen as things to test and optimize. Chunk Coverage (Kim, Pasini, Tonella) proposes an oracle-free adequacy criterion for retrieval testing: the fraction of corpus chunks retrieved at least once across a test suite. CC-guided test selection reached 50% of attainable coverage 1.7x faster than random and 4.2x faster than redundancy-biased strategies, and improved fault-detection APFD by 10 to 25%. If you ship a RAG system and your test suite exercises a tenth of the corpus, this is the metric that tells you so. AutoIndex (O’Nuallain et al., with Andrew Drozdov) goes after the other frozen layer: document representation. Agents diagnose retrieval failures and rewrite the representation program, the executable transform that slices, enriches, and reweights documents before indexing, keeping only updates that improve validation retrieval. With BM25 held fixed across all eight CRUMB tasks, learned programs gained +8.4% Recall@100 on average, +30.5% at best. Representation as an optimization target rather than a preprocessing choice, with the retriever untouched.

Worth watching from here: whether Hugging Face publishes the technical details of the attack chain they promised, whether Leni’s verifier-specialization result replicates on someone else’s agent, and whether the agent-CLI lifecycle protocol question gets a serious answer before every observability tool builds its own six-adapter shim.