Start the stability, prose, product-owner and architecture reviewers without CLAUDE.md

This commit is contained in:
2026-11-12 14:30:24 +01:00
parent 545a57f01c
commit d81da4d413
7 changed files with 7 additions and 14 deletions
+1
View File
@@ -2,6 +2,7 @@
## Unreleased
- Five of the six reviewers start without `CLAUDE.md`: principles-auditor, stability-reviewer, prose-reviewer, product-owner-reviewer and systems-architect. `review-replay --omit-claude-md` replays each round three times unchanged and once without it, so a pass is held to the noise of an unchanged reviewer.
- larv-review skips stability round 2 where round 1 reported a 🔴 and every 🔴 and 🟡 asked only that a named mutant be killed or a named failing test pass, once the tests and the mutation command run clean on the fixes.
- `review-replay` now qualifies a rung whose share of a pass's archived findings is no more than 10 points below the mean of two replays on the project's rung, over at least 8 rounds, where it misses no 🔴 that both found. It used to demand every finding the project's rung found again, which the project's rung itself failed. It tries the project's model at each lower effort first, then the ladder below. Its run on the prose pass of Opus 5.5 projects qualified no rung: Opus 5.5 at low effort found 49% against a bar of 66%.
- loop-claude records beside each verdict, interrupt and quoted answer the claude-config release and commit that ran it, the tokens the worker, each subagent and the judge spent, by model and kind, with the list prices in force, and any typed answer of yours the judge quotes as reversing a choice the worker took on its own recommendation; `score-outcomes` counts those reversals per finished chunk.
+1
View File
@@ -2,6 +2,7 @@
name: "product-owner-reviewer"
description: "Use this agent to review a change to an external surface (API, UI, CLI, SDK, webhook) from an end customer's view: ease of use, edge cases and persona fit."
tools: Read, Grep, Glob, Write
omitClaudeMd: true
---
You are a seasoned Product Owner with deep expertise in user experience, API design, and customer empathy. You have shipped consumer and B2B products across web, mobile, REST APIs, and developer-facing CLIs. Your superpower is putting yourself in the end customer's shoes and surfacing the friction, confusion, and edge cases that engineers tend to miss.
+1
View File
@@ -2,6 +2,7 @@
name: prose-reviewer
description: "Use this agent for larv-review's prose pass."
tools: Read, Grep, Glob, Write
omitClaudeMd: true
---
You read the prose as someone who did not write it: you see only what it says, never what the author meant.
+1
View File
@@ -3,6 +3,7 @@ name: "stability-reviewer"
description: "Use this agent to review changed code for stability: bugs, error handling, edge cases and the project's technical principles, after a feature, refactor or fix."
color: orange
tools: Bash, Read, Grep, Glob, Write
omitClaudeMd: true
---
You are an elite stability-focused code reviewer with deep expertise in defensive programming, fault-tolerant system design, and production incident prevention. You have decades of experience identifying subtle bugs, race conditions, edge cases, and architectural weaknesses that lead to outages and unexpected behavior. You think like a chaos engineer — always asking "what could go wrong?" — and like a maintainer of ten years, always asking "what will this cost to live with?" There are no bugs, and no security holes, in code that doesn't exist.
+1
View File
@@ -2,6 +2,7 @@
name: systems-architect
description: "Use this agent to design a system or review architecture: patterns, component boundaries, fitness for purpose and long-term maintainability, not individual PR lines."
tools: Read, Grep, Glob, WebFetch, WebSearch, Write
omitClaudeMd: true
---
You are an elite Systems Architect with decades of experience designing software systems that endure for as long as their horizon demands. Your philosophy is uncompromising on fundamentals: clarity, simplicity, and deliberate structural choices trump short-term convenience every time. You design systems people will read, understand, and extend for their whole recorded lifetime.
+2 -2
View File
@@ -553,7 +553,7 @@ On 2026-11-12, first gate (two unchanged replays, no control):
| prose | 68 | 34, 36 | 24 | 39 | 1 |
| overrides | no archived round | | | | |
Only principles-auditor met the gate, so only it has `omitClaudeMd`. Its evidence is thin: two rounds of each replay failed, and two findings were found both times. Without `CLAUDE.md` each replayed reviewer found more findings than it found both times with it, and three of the five found more than in either replay with it. So the misses may be the reviewer's own noise; todo item 340 asks lilleman whether a gate with a control may decide the other five.
Only principles-auditor met the first gate. Its evidence is thin: two rounds of each replay failed, and two findings were found both times. Without `CLAUDE.md` each replayed reviewer found more findings than it found both times with it, and three of the five found more than in either replay with it. So the misses may be the reviewer's own noise, which the second gate measures.
On 2026-11-12, second gate, with the control (three unchanged replays, then one without), on claude-opus-5-5 at medium effort. Rounds a replay failed on are left out, so the counts differ from the first table's:
@@ -568,7 +568,7 @@ On 2026-11-12, second gate, with the control (three unchanged replays, then one
Every pass passes the second gate. The unchanged replays alone missed up to 8 findings the other two found, so the first gate's one miss in stability, prose and product-owner was noise. The second gate still rests on 6 to 9 rounds a pass, and one run each.
Whether stability-reviewer, and the other four, start without `CLAUDE.md` waits on lilleman's call through todo item 340; only principles-auditor has `omitClaudeMd` now.
2026-11-12, lilleman: every pass the second gate passed starts without `CLAUDE.md`, so principles-auditor, stability-reviewer, prose-reviewer, product-owner-reviewer and systems-architect have `omitClaudeMd`. override-reviewer keeps it: no archived round of its pass exists to replay. Rerun the gate for a pass whose reviewer starts to miss findings; each pass rests on 6 to 9 rounds and one run.
Valid while Claude Code honours `omitClaudeMd` for a subagent and passes `agent_type` to a `SubagentStart` hook.
-12
View File
@@ -12,7 +12,6 @@
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 340 | 0.9.0 | | owner | **Let the stability reviewer start without `CLAUDE.md`, as principles-auditor does, once a replay with a control has been run on every pass and reported to lilleman.** | 2 | 2 | 3 | 5 | 4 | 1.4 by 5 | 1.7 |
| 319 | 0.9.0 | | | **Start a fresh session with a brief of the chunk when a rung step changes the worker's model, in place of resuming the old session.** | 3 | 2 | 3 | 6 | 4 | 1.1 by 2, 1.4 by 2 | 1.4 |
| 309 | 0.9.0 | | | **Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.** | 2 | 2 | 2 | 4 | 2 | 1.4 by 3 | 1.3 |
| 299 | 0.9.0 | | | **Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3 | 1.2 |
@@ -68,17 +67,6 @@
## Details
### 340. Let the stability reviewer start without `CLAUDE.md`, as principles-auditor does, once a replay with a control has been run on every pass and reported to lilleman.
Item 315's gate kept `omitClaudeMd` only where a replay without `CLAUDE.md` found every finding that two unchanged replays both found. Read so, five passes failed it on 2026-11-12: stability missed 1 of 11, prose 1 of 24, product-owner 1 of 4, architecture 4 of 27, and overrides had no archived round to replay. The decision `agents/: a reviewer starts without CLAUDE.md only where a replay of its pass missed none of the findings two unchanged replays both found` holds the figures. The unchanged reviewer finds only about 70% of its findings again, so that gate can't tell a loss from the reviewer's own noise.
lilleman's call, 2026-11-12: principles-auditor and stability-reviewer start without `CLAUDE.md`; the other reviewers keep it. The call is provisional until the data below is in.
1. Give `review-replay --omit-claude-md` a control. It replays each round three times unchanged and once without `CLAUDE.md`, and prints for each unchanged replay how many findings it missed that the other two found. Each run costs four replays of 10 rounds on the project's model.
2. Run it for every pass that has archived rounds: principles, stability, prose, product-owner and architecture.
3. Report each pass's figures to lilleman, saying whether they change the picture: a pass passes where its count without `CLAUDE.md` is no higher than the highest of its three unchanged counts. The picture changes where stability or principles fails, or another pass passes.
4. Where the picture holds, add `omitClaudeMd: true` to stability-reviewer, and record the figures and the call in the decision above. Where it changes, ask lilleman before changing any agent; lilleman makes the call again from the figures. The hook and the header already serve any agent that has the key.
### 319. Start a fresh session with a brief of the chunk when a rung step changes the worker's model, in place of resuming the old session.
On SWE-bench Verified with Haiku 4.5 and Opus 4.7, handing the strong model the weak model's transcript solved 69.2% at $1.61 a task; starting the strong model fresh solved 79.2% at $0.90 (arXiv 2608.24358, The Handoff Tax). A model switch also rereads the whole context uncached (Claude Code prompt caching docs). A worker that steps from Sonnet 5.5 to Opus 5.5 by `--resume` is the raw handoff; an effort-only step within one model keeps the cache and isn't. The escalated subagent, which starts fresh with the trigger's evidence, already follows the pattern that won.