Archive each judge run's prompt and reply beside its verdict, and replay judge-replay on the archived prompt #568

Merged
lilleman merged 9 commits from item-303 into main 2026-11-17 01:37:43 +01:00

9 Commits

Author SHA1 Message Date
lilleman f2a4347d78 Finish item 303 in the plan
CI / tests (pull_request) Successful in 2m54s
2026-11-17 01:31:56 +01:00
lilleman afd7abaa5e Merge remote-tracking branch 'origin/main' into item-303
CI / tests (pull_request) Successful in 2m53s
2026-11-17 01:30:19 +01:00
lilleman e1e9d03df1 Answer product-owner round 1: 0 🔴, 2 🟡 (0 misread); the report names the prompt each missed verdict was replayed on, the help says where records change the kind, and the decision drops pricing from usage 2026-11-17 01:30:15 +01:00
lilleman b067d8cda8 Answer stability round 2: 0 🔴, 1 🟡 (0 misread); the run-pairing test holds each prompt with its reply again, and the task heading test now kills its mutant 2026-11-17 01:30:15 +01:00
lilleman 38544dd2d3 Kill the task heading mutant of the archived prompt's split 2026-11-17 01:25:07 +01:00
lilleman 2f0708f390 Answer prose round 1: 0 🔴, 5 🟡 (0 misread); the decision entry and the help text reworded with the measurement redone, the title names the judge runs, and the hash comments say prompt and reply 2026-11-17 01:15:35 +01:00
lilleman 2600fbb254 Answer stability round 1: 1 🔴, 5 🟡 (0 misread); imports sorted, the replay cuts the task and the rules from its grounds and fails on a missing heading, outcome parameters keyword-only, the mtime order dropped 2026-11-17 01:13:39 +01:00
lilleman b5afb4c36a Kill the mutants of the judge run archive that its tests missed 2026-11-17 00:51:39 +01:00
lilleman e1d2eae7f4 Archive each judge run's prompt and json reply beside the verdict it gave, and replay from the archived prompt
CI / tests (pull_request) Failing after 2m45s
2026-11-16 23:42:36 +01:00