The console eraJune — July 2026
What it got wrong: Codex answered in a persona distilled from 167 of my own real replies. It sounded like me and it was not me, and that turned out to be the most expensive thing in the whole arrangement — it is why the product has no memory of its director today.
You set a task and went to bedthere was no application — two command-line tools and a script
A tmux session: Claude Code workingthe session survived closing the terminal
Claude stopped to ask somethinga PreToolUse hook caught the question before it reached anybody
Codex answered instead of mefrom a persona distilled out of 167 of my own real answers — the thing that was later removed
A Stop hook ran the reviewCodex checked the result against the requirements; three rounds, then the remainder went into a file
The watchdog saw the subscription limitwaited for the reset and wrote “carry on” itself
A night/<date> branch waiting for the morningreviewed, then merged or thrown away
Version one — a plan, criticised onceAugust 2026
What it got wrong: one model wrote the plan and the other marked it. A critique of somebody else's plan inherits its framing — Codex could say what was missing from the plan, never that the plan was aimed at the wrong thing.
The task
Classification: how big is itsmall ones went straight to work
Claude writes a plan
Codex criticises the planand names what was left out
Claude rewrites the plan
Claude writes the code
The Stop hook: Codex checks the resultthree rounds, then the remainder was written down as debt
Doneone opinion on the plan, and it was Codex's
Version two — two positionsSeptember 2026
What it got wrong: nothing about the shape — this is the arrangement the benchmark picked. It just did all of it every time. Classification, external research and a design precedent ran for “commit this and cut a release” exactly as they ran for a new product, which is why the current one asks first whether the message opens a task or continues one.
The task
The engine builds one shared contextthe same bytes for both — otherwise the positions cannot be compared
at the same time, neither seeing the otherClaude studies the taskmechanics, implementation, risks, edge cases
Codex studies the taskapproaches, precedents, risks, missed requirements
Codex compares both positionsmaterial differences and acceptance criteria
Claude decides and implements
Claude turns to CodexCodex researches it and proposes something; Claude judges the adviceas many times as the work needs
Automatic checksthe project's own build and tests
Codex reviews the implementationfindings go back to Claude
Done
How each of these was measured
The figures above are worth exactly as much as the conditions they were taken under, so here are the conditions — and, for each run, the fingerprint of the files it produced. Below them is the procedure that made those fingerprints and the manifest they were made from, so this is something you can redo rather than something you have to take.
Claude Code on its own
- Ran
- 2026-09-08 11:26 → 2026-09-08 21:32
- Elapsed time read from
- the session log — first action to last
- Reasoning effort
- xhigh
- The project's own tests, at the end of its run
12 / 12- Files it counted
- 16
- SHA-256 of the measured result
09e05fdc2011082f585e4e8d5023fd680dc7e121521eb296113c3a7f1613684f- What that address serves now
- exactly those bytes — re-hashed file by file on 2026-09-21
Run without any harness at all. The twelve tests are ones it wrote for itself; nobody asked for them and nobody checked them against the game.
Pipeline 1 — Claude, reviewed by Codex
- Ran
- 2026-09-07 23:40 → 2026-09-08 12:14
- Elapsed time read from
- the session log — first action to last
- Reasoning effort
- high
- The project's own tests, at the end of its run
verify.sh — PASS- Files it counted
- 12
- SHA-256 of the measured result
38ec5b565ea47afb6455f6c70724390eb299587e758bd1b8c96d1481a381edcd- What that address serves now
- a later build, and not what these numbers are of:
96a11769a2ac5dd20d524282d4c60c1a47b2b12d13940c6e96d472cbc4dd3fd8 — re-hashed file by file on 2026-09-21
The one row whose demo has moved on: the address now serves a later build, so the playable thing there is not the thing these numbers describe. Kept in, said out loud, rather than quietly dropped.
Pipeline 2 — two independent positions
- Ran
- no session log; the harness timed its own stages
- Elapsed time read from
- the harness's stage clock
- Reasoning effort
- high
- The project's own tests, at the end of its run
57 / 57- Files it counted
- 13
- SHA-256 of the measured result
00deaf7cc9f5cfce39832b5d0a36979b09080c58e9962dab9cca7a091ba9abe5- What that address serves now
- exactly those bytes — re-hashed file by file on 2026-09-21
Fifty-seven tests, and none of them caught the thing that mattered: nothing in any of the three runs noticed that the aircraft in the game has a cross-shaped footprint.
How to check any of this yourself
Take every .js, .css and .html file under the address, recursively, skipping any path containing node_modules, /tests/ or /tools/. Hash each file's bytes with SHA-256. Write one line per file as the hash, two spaces, then ./ and the path. Sort the lines by path. The fingerprint is the SHA-256 of that text.
find . \( -name '*.js' -o -name '*.css' -o -name '*.html' \) -type f \
| grep -vE 'node_modules|/tests/|/tools/' \
| xargs shasum -a 256 | LC_ALL=C sort -k2 | shasum -a 256
The exact list of files, each with its own hash, and everything on this page in machine-readable form: /benchmark.json.