Answer architecture round 1: 0 🔴, 4 🟡 (0 misread); item 330 replays only the judge, on the Fable 5.2 rungs, and the four decisions' rewording is filed as question item 331
CI / tests (pull_request) Successful in 2m22s

This commit was merged in pull request #493.
This commit is contained in:
2026-11-12 00:24:40 +01:00
parent 93c81c48ab
commit 05806b517d
+19 -5
View File
@@ -6,13 +6,14 @@
`Bar = 0.1`
`Next ID = 331`
`Next ID = 332`
## Items
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 330 | 0.8.0 | | | **Replay the judge and every review pass on the Fable 5.2 ladder, and record what each replay shows.** | 2 | 2 | 2 | 4 | 3 | 1.2 by 4, 1.1 by 3 | 2.9 |
| 331 | 0.8.0 | | question | **Reword four decisions to expire when the rungs their replays tested change, not whenever a model joins the ladder.** | 1 | 1 | 1 | 1 | 2 | 1.4 by 2 | 0.9 |
| 330 | 0.8.0 | | | **Replay the judge on each Fable 5.2 rung an Opus 5.5 project's E + I climbs it to, and keep it off any rung that misses an interrupt.** | 3 | 3 | 2 | 6 | 3 | 1.2 by 4, 1.1 by 3 | 2.9 |
| 320 | 0.8.0 | | | **Bench a worker below the project's rung with the advisor tool against the same worker alone and the project's rung, and file the switch if it wins.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3, 1.1 by 2 | 2.0 |
| 315 | 0.8.0 | | | **Start each reviewer under `agents/` without `CLAUDE.md`, and hand it the vocabulary, the prose rules and the project's decision titles.** | 3 | 2 | 3 | 6 | 4 | 1.4 by 6 | 2.0 |
| 312 | 0.8.0 | | | **Research the tooling agents read, search and run commands with, and file an item for each tool that cuts tokens a replay shows keep quality.** | 1 | 2 | 1 | 2 | 3 | 1.4 by 3, 1.5 by 2 | 2.0 |
@@ -73,11 +74,24 @@
## Details
### 330. Replay the judge and every review pass on the Fable 5.2 ladder, and record what each replay shows.
### 331. Reword four decisions to expire when the rungs their replays tested change, not whenever a model joins the ladder.
Item 292 put Fable 5.2 on the routing ladder. Two decisions expire when a new model joins it: `routing: on a model judge-replay covered, the judge runs …` ("replay then") and `routing: a review pass runs below the project's rung only on a rung that a replay …` ("a replay of every pass"). On an Opus 5.5 project the judge now climbs onto Fable 5.2 at low, medium and high at E + I 5, 6 and 8, rungs `judge-replay` never covered.
The architect's 🔀 answer on PR #493 (item 292): four decisions expire "when a new model joins the ladder". Fable 5.2 joined above every project model's rung, and none of these replays tested a rung above the project's, so the event fired with nothing new to test. Every later ladder rebuild would fire all four again.
Run `bin/judge-replay claude-opus-5-5` on the new ladder, and `bin/review-replay` for each pass the 2026-10-15 to 2026-10-18 runs covered. Record each result in its decision, and renew or change each "Valid" line. Where a judge rung on Fable 5.2 misses an interrupt the project's rung returns, keep the judge off it until a decision says otherwise.
- `review: each pass keeps a reviewer of its own`: the merge trials ran on Opus 5.5 at medium. Reword the "Valid" line to name that rung.
- `review: the principles audit runs only in the release review…`: its trial ran on the project's rung. Reword the line to name that rung.
- `routing: a review pass runs below the project's rung only on a rung that a replay…`: `review-replay` tests only rungs below the project's rung. Have the line expire when those rungs change, the event item 196 names ("until a cheaper model joins the ladder").
- `routing: on a model judge-replay covered, the judge runs…`: the replay tested the project's rung and below, while the climb above rests on the ladder's order alone. Have the line name both, and point the climb at item 330.
Recommendation: reword all four as above. They are decisions the repo held before item 292, so the rewording is lilleman's call.
### 330. Replay the judge on each Fable 5.2 rung an Opus 5.5 project's E + I climbs it to, and keep it off any rung that misses an interrupt.
Item 292 put Fable 5.2 on the routing ladder. On an Opus 5.5 project the judge now climbs with its chunk's E + I onto Fable 5.2 at low, medium and high, at E + I 5, 6 and 8. Fable 5.2 has never returned a judge verdict, and the 2026-10-15 replay showed that another model can miss an interrupt in kind, not only in strength. `routing: on a model judge-replay covered, the judge runs …` rests on that replay.
`bin/judge-replay` replays only the project's rung and the rungs below it (`down_from` in `judge_replay/main.py`), so as it stands it never runs a Fable 5.2 rung. Give it the rungs the judge climbs to as well, run `bin/judge-replay claude-opus-5-5`, and record the result in that decision. Where a Fable 5.2 rung misses an interrupt the reference judge returns, keep the judge off it until a decision says otherwise.
The review passes need no replay: `review-replay` tests only rungs below the project's rung, which item 292 left as they were.
### 320. Bench a worker below the project's rung with the advisor tool against the same worker alone and the project's rung, and file the switch if it wins.