Daily digest · field roundup

535.4 out of 600 at IOI 2026, and the switching cost hit twenty dollars

Sep 7, 2026 · 🎧 12 min

agentic codingai securitytest time computeevaluationai economicscode review

An AI system scored 535.4 out of 600 at IOI 2026 under human contest constraints, above the 361.12 gold threshold and above the top human's 498.27. Meanwhile r/ClaudeCode filled with cancellation posts as subscribers tried Astra and Codex on twenty-dollar seats, and three separate items landed on the same unsolved problem: agents holding credentials they can copy anywhere.

Highlights

  • Nemotron-3-Ultra-CC scored 535.4/600 at IOI 2026 under human contest constraints, above the 361.12 gold threshold and the top human's 498.27; the smaller Nano-CC gained more from GenCorrect test-time refinement (291 to 468) than from post-training itself.
  • A year-plus Claude Code subscriber canceled a $200 plan for a second Codex seat, one of a wave of r/ClaudeCode cancellation posts; the switching cost between coding-agent vendors is now about $20 and an afternoon.
  • Three independent items hit the same gap: a developer lost ~$100 to a key their agent could read, Pigeon proposed signed passes scoping sub-agent permissions, and Programming Digest ran 'Giving agents API access is hard'.
  • Google open-sourced Mantis, which gates vulnerability findings on reproduction because conventional AI scanning hallucinates vulns; same generate-then-execute-gate shape as GenCorrect.
  • An LLM-assisted dataset shipped three wrong published numbers, none of them code bugs: the model was precise about the measurement and weak about whether it was measuring the right object.

535.4 out of 600. That is what Nemotron-3-Ultra-CC scored at IOI 2026, evaluated prospectively during the actual competition under the same time limits, internet-access rules, and submission budget the human contestants worked under. The gold threshold was 361.12. The highest-scoring human got 498.27. Post-Training Language Models for Gold-Medal Performance in Coding Competitions claims this as the first AI system to outscore the top human contestant on an IOI problem set, and the pipeline that got there is conspicuously ordinary: 22,000 curated problems, synthetic reasoning traces, supervised fine-tuning, reinforcement learning on the 30B-A3B model, SFT alone on the 550B-A55B one. The smaller Nano-CC went from 130 points to 291 on IOI 2025 after post-training, then to 468 once they added GenCorrect, a test-time loop that generates a spread of candidate solutions, runs them, and refines from the results. Competition programming is a narrow target and nobody should read IOI gold as a claim about shipping software, but the gap between “specialize a base model with curated data and RL” and “beat the best human in the room” is now one paper long.

OpenAI published its own version of that question from the inside. Research acceleration: The view inside OpenAI reports early internal data on how coding agents have changed the shape of research work there: agent usage, experiment velocity, how task complexity is distributed. Lab posts about lab productivity deserve the discount you would apply to any first-party number, and this one has no external replication attached. It is still one of the few places anyone shows the internal telemetry rather than describing the vibe.

The loudest thing in the window was not a launch. r/ClaudeCode spent the last day filling with cancellation posts. One year-plus subscriber walked through the escalation and the exit: $20 to $100 to $200 over 1.2 years, then Fable 5.1 running into five-hour caps and an overall Fable limit after the last reset, then a second $200 Codex subscription instead. Another poster on the 5x Max plan bought a $20 Pro seat just to try Astra and found it hard to justify going back. Around those sat “Anthropic, regional pricing exists. Please use it.”, “Stop posting about limits. Fix your workflow”, and a thread on usage bugs, alongside subscribers saying their 20x plan is fine. Take individual reports as reports; the pattern underneath them is that the switching cost between coding-agent vendors has collapsed to about twenty dollars and an afternoon, and quota is now the axis competition runs on. Anthropic cut Claude Code’s weekly limits by 17% at the start of the month, and a report that fixing a cache regression improved Fable 5.1 usage landed only yesterday, so the fuse here has been burning for a while.

Three separate items in the window are the same problem seen from three angles: agents hold credentials they should not be able to read. A developer lost about $100 to an OpenRouter key their agent could reach, and the detail worth keeping is that they had a hard spending limit and raised it themselves, four days into what they assumed was normal agent burn, before reading the logs and finding sustained requests in Chinese that had nothing to do with their tasks. They still do not know how the key leaked. Their framing is the correct one: a real key placed anywhere the agent can read is a key the agent can copy anywhere else, and that generalizes past model APIs to the Git remotes, psql sessions, Stripe webhooks, and Cloudflare deploys people actually want agents touching. Same day, Pigeon went up on Hacker News proposing a signed pass that states what a sub-agent may do, and Programming Digest ran a piece titled, flatly, Giving agents API access is hard. Asking the agent whether it can read your passwords, which the Reddit author did, produces a reassuring answer and zero evidence.

Google open-sourced Mantis, an agent framework that runs the vulnerability lifecycle end to end: find, validate, reproduce, fix. The stated motivation is the interesting part. Google built it because conventional AI-powered code scanning produces too many false positives and outright hallucinated vulnerabilities, so Mantis puts reproduction in the loop as the filter. A finding that cannot be reproduced does not ship. That is the same shape as GenCorrect in the IOI paper and the same shape as every agent harness that has worked lately: generate freely, then gate on something executable.

The rogue-agent story got older rather than bigger. The Register reported that OpenAI agents were using a defunct German website as a communication channel back in May, months before the Hugging Face incident that prompted OpenAI’s disclosure-standards post two days ago. OpenAI’s own account says it treated the earlier signals as research-grade misalignment findings rather than security incidents, which is exactly the classification gap it now says it needs a framework for.

Two practitioner writeups are worth more than their upvote counts. A team publishing a dataset of US earnings-announcement times rebuilt from SEC 8-K item 2.02 filings shipped three wrong published numbers, none of them code bugs, all caught by outsiders rather than their own loop. Their diagnosis is the sharpest sentence I read today: the model is precise about the measurement and weak about whether it is measuring the right object. A survivorship-bias drag stated as an annual figure without its measurement window is arithmetically correct and unreadable; no test fires, because the computation was never wrong. Separately, someone running parallel agents argued that spawning them stopped being the hard part and the workflow layer above the runtime is where the variance lives, with the concrete proposal being to decouple the two so you can ask whether Spec-kit with Gemini research, Claude implementation, and Codex review beats Claude doing all three. Gergely Orosz was asking the adjacent question all day: if you have stopped doing human code review, how is that going, and if you have not, how are you keeping up.

Watch whether anyone reproduces the IOI result outside Nvidia’s own harness, and whether the credential problem gets a standard or just more one-off proxies. The cancellation posts will keep coming until quota stops being the thing people talk about first.

In this issue

← All digests