This commit was merged in pull request #478.
This commit is contained in:
@@ -6,12 +6,13 @@
|
||||
|
||||
`Bar = 0.1`
|
||||
|
||||
`Next ID = 303`
|
||||
`Next ID = 304`
|
||||
|
||||
## Items
|
||||
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 303 | 0.8.0 | | defect | **Archive each judge run's prompt and json reply beside the verdict it gave, so judge cost, replays and backups cover the judge as they cover the worker.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3, 3 by 2 | 4.8 |
|
||||
| 297 | 0.8.0 | | | **Have loop-claude compute the judge's taken-in items, release-ready and escalation fields from `chunk-preflight`, git and the worker's structured lines, and leave the judge only the verdict kind.** | 3 | 2 | 3 | 6 | 4 | 1.4 by 2, code health by 2 | 3.7 |
|
||||
| 302 | 0.8.0 | | | **Call a scoring run for an item whose tagline makes the comprehension floor its target, and not for one that only names the floor, such as in a quoted decision title.** | 2 | 2 | 3 | 5 | 3 | code health by 2 | 3.6 |
|
||||
| 293 | 0.8.0 | | | **Run mutation tests on a chunk's changed lines in the worker's pre-review gate, through one command that picks the tool by language, and hand stability-reviewer the surviving mutants.** | 3 | 2 | 3 | 6 | 5 | 1.1 by 5, 1.4 by 4, 1.5 by 2 | 2.8 |
|
||||
@@ -58,6 +59,12 @@
|
||||
|
||||
## Details
|
||||
|
||||
### 303. Archive each judge run's prompt and json reply beside the verdict it gave, so judge cost, replays and backups cover the judge as they cover the worker.
|
||||
|
||||
Today no judge transcript reaches loop-claude's archive. `bin/claude_code/reply.py` runs every headless reply, the judge's included, with `--no-session-persistence`, so its runs stay out of the session picker; Claude Code then writes no session file for the archive to copy. No decision chose to leave the judge out: `loop-claude: the archive keeps the raw transcript` was written for the worker's `.jsonl`. Goal 3 asks for the transcripts of unattended runs to be backed up, and the judge is one. The gap also leaves the judge's spend measurable only from the `Judge cost:` lines, which older verdicts lack ($136 recorded over 2026-09-30 to 2026-11-11), and leaves `judge-replay` rebuilding each prompt from today's records: of the 39 verdicts it replayed on 2026-10-15, 8 may rest on records that had changed since.
|
||||
|
||||
The judge runs with no tools and one turn (`judge_args` in `bin/loop_claude/cycle/judge.py`), so its whole transcript is the prompt it read and the json reply. Recommended: keep `--no-session-persistence`, and have the archive write the prompt and the reply beside the outcomes entry of the verdict, interrupt or rejected reply they produced, deduplicated by content hash as the worker's copies are. A failed write stops the loop, as that decision says. Persisting the session and copying its file would also work, but fills the picker and needs a cleanup. The prompt repeats the worker's transcript, which the archive already holds: measure the size per cycle first, and if it outgrows "small enough to copy whole", keep the prompt without the transcript part and name the worker copy and entry range it held. Then let `judge-replay` read the archived prompt where one exists, and price the judge from the archived reply's usage. Update that decision entry to say the judge's runs are archived too.
|
||||
|
||||
### 297. Have loop-claude compute the judge's taken-in items, release-ready and escalation fields from `chunk-preflight`, git and the worker's structured lines, and leave the judge only the verdict kind.
|
||||
|
||||
`bin/loop_claude/cycle/judge.py` already checks quotes in code (`quotes_checked`, `grounded`, `re_ask_checked`). The `Reply` fields `items_taken_in`, `release_ready` and `escalation` state facts the plan and git already hold, so code can derive them with certainty, and the judge's prompt shrinks to what needs judgement. Before switching, recompute each field for the 668 archived verdicts offline and count where code and judge disagree; read each disagreement and keep the judge for a field where code is wrong. Seats found `quotes_checked` hard to read (PR #397's scoring run: the misquote rule is split across `judge.py` and `escalation_mark.py`); make it easier to read where the change touches it.
|
||||
|
||||
Reference in New Issue
Block a user