Files
..

Judge field trials

These trials ask one question: which of the facts loop-claude's judge reports could code read instead? The judge is the headless claude that rules on each worker run; its reply named the items the chunk took, whether a release was ready to cut, and any escalation the worker asked for. They ran on 2026-11-11, over the 709 cases archived by then. docs/decisions.md → routing: loop-claude reads the items a chunk takes records what was decided from them.

Terms

  • Case: an outcomes file in ~/.claude/loop-claude-transcripts/, its first verdict, and the transcript the judge read for it.
  • Final message: the last text the worker wrote after the last prompt loop-claude or the human gave it.

Results

Field What code tried Result
Escalation "higher rung", "rung up" or "one rung above" in the final message Code is wrong in each of the 39 disagreements: 37 where only code finds a request, 2 where only the judge does. Code matched a worker asking for the human's yes at the top rung and a review report quoting "rerun one rung up", and missed requests made before the final message. The judge keeps the field.
Release ready to cut The plan as the default branch and tags stood at the verdict, read by release_plan.release_standing Of 240 interrupts, 31 ask about a cut, and code reads ReadyToCut for 3. Two misses are code's: in two projects a release review had merged without the Release review of <release> commit that code looks for. The rest asked under older rules or after the cut. The judge keeps the field, and a todo.md item files the fix.
Items the chunk takes An item stated with its L and E + I anywhere in the worker's text 413 of 709 cases state one, 77 of them more than one, and workers also state items they file, finish or only mention. Code reads the items from a line that holds only them, which workers write only since loop-claude asks for it, so the archive can't measure it.

Running

Each script reads the archive on this machine through cases.py and prints its counts last; run it from the checkout's root, as in python3 judge-field-trials/escalations.py. The lines above the counts quote transcripts and name projects, so they stay on this machine.

  • escalations.py prints each escalation disagreement with the words around it.
  • release_ready.py prints each interrupt's question beside whether code reads the release ready to cut.
  • items.py prints a few cases stating more than one item.