Judge field trials
These trials ask one question: which of the facts loop-claude's judge reports could code read instead?
The judge is the headless claude that rules on each worker run; its reply named the items the chunk took, whether a release was ready to cut, and any escalation the worker asked for.
They ran on 2026-11-11, over the 709 cases archived by then. docs/decisions.md → routing: loop-claude reads the items a chunk takes records what was decided from them.
Terms
- Case: an outcomes file in
~/.claude/loop-claude-transcripts/, its first verdict, and the transcript the judge read for it. - Final message: the last text the worker wrote after the last prompt loop-claude or the human gave it.
Results
| Field | What code tried | Result |
|---|---|---|
| Escalation | "higher rung", "rung up" or "one rung above" in the final message | Code is wrong in each of the 39 disagreements: 37 where only code finds a request, 2 where only the judge does. Code matched a worker asking for the human's yes at the top rung and a review report quoting "rerun one rung up", and missed requests made before the final message. The judge keeps the field. |
| Release ready to cut | The plan as the default branch and tags stood at the verdict, read by release_plan.release_standing |
Of 240 interrupts, 31 ask about a cut, and code reads ReadyToCut for 3. Two misses are code's: in two projects a release review had merged without the Release review of <release> commit that code looks for. The rest asked under older rules or after the cut. The judge keeps the field, and a todo.md item files the fix. |
| Items the chunk takes | An item stated with its L and E + I anywhere in the worker's text | 413 of 709 cases state one, 77 of them more than one, and workers also state items they file, finish or only mention. Code reads the items from a line that holds only them, which workers write only since loop-claude asks for it, so the archive can't measure it. |
Running
Each script reads the archive on this machine through cases.py and prints its counts last; run it from the checkout's root, as in python3 judge-field-trials/escalations.py.
The lines above the counts quote transcripts and name projects, so they stay on this machine.
escalations.pyprints each escalation disagreement with the words around it.release_ready.pyprints each interrupt's question beside whether code reads the release ready to cut.items.pyprints a few cases stating more than one item.