Rerun the pass that raised a 🟡 when its fix changes what a test asserts or whether it runs
CI / tests (pull_request) Successful in 3m2s

This commit is contained in:
2026-11-18 09:49:26 +01:00
parent 43dfd375b8
commit 6c52766a27
2 changed files with 6 additions and 3 deletions
+3 -1
View File
@@ -314,7 +314,7 @@ Valid while no published standard sets either number, the reading-cost window st
2026-10-08, lilleman. Serves goal 1.4: run beside the full scan, `/security-review` started seconds apart from it each round and returned a near-identical report. Valid while the full scan covers what `/security-review` reports.
## review: a fix for a 🔴 reruns the pass that found it, unless that pass is at its cap, and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟡, a 🟢, a ❓ or a prose finding reruns none; the stability pass doesn't rerun on a round's fixes where every 🔴 and 🟡 of that round asked only that a named mutant be killed or a named failing test pass, and the tests and the mutation command run clean on the fixes
## review: a fix for a 🔴 reruns the pass that found it, unless that pass is at its cap, and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟢, a ❓ or a prose finding reruns none, and one for a 🟡 reruns only the pass that raised it, and only when it changes what a test asserts or whether it runs; the stability pass doesn't rerun on a round's fixes where every 🔴 and 🟡 of that round asked only that a named mutant be killed or a named failing test pass, and the tests and the mutation command run clean on the fixes
2026-10-08, lilleman. Serves goal 1.4. An audit of the four costliest archived chunks found nearly every finding that mattered in a pass's first round; when every fix, a nit's included, re-entered review, later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes change only what a user wouldn't notice.
@@ -324,6 +324,8 @@ Valid while no published standard sets either number, the reading-cost window st
2026-11-12, lilleman through todo item 308. Serves goal 1.4. Since the worker runs the mutation command before each stability round, a finding that asks only that a named mutant be killed or a named failing test pass has an answer a run can check. The archive from 2026-09-30 to 2026-11-12 holds 326 stability reports of round 2 or later, from 13 projects, and 81 of those reports open with BLOCK (`stability-skip-count/`). For each of the 326, the 🔴 and 🟡 lines of the report before it were matched on mutant and test-failure wording. Every such line matched in 5 of those earlier reports. Read by hand, 1 of the 5 asked only that a failing test pass, and the round 2 after each of the 5 approved. So none of the 81 BLOCKs fell in a round this skips. The skip needs every 🟡 to be of that kind too: in one chunk on 2026-11-12, round 1's only 🔴 was a red suite, and round 2 blocked on a 🟡 whose answer was wrong, which no test or mutant showed. A fix may not edit the named failing test, since a test made to pass that way doesn't show anything. Fixes from such a round still rerun every other pass whose row they meet. Valid while code that such a skipped round's fixes wrote is rarely what a later chunk's 🔴 findings fix.
2026-11-18, lilleman through todo item 399. Serves goals 1.1 and 1.4. The 2026-10-29 entry's validity condition broke once: in a client project's pull request, stability round 1 raised a 🟡 that an end-to-end payroll test passes on a used database, and proposed asserting the created row counts. The answer commit added that assertion behind a `test.skip` that runs when every count is zero. A broken payroll also creates zero rows, so a regression became a skip; the gate and the mutation run don't catch it, since a skipped test counts as passing, and only the forge's auto-reviewer did. So a 🟡 fix that changes what a test asserts or whether it runs, which is a change to what review trusts, reruns the pass that raised it, as a 🔴 fix does. Every other 🟡 fix keeps the 2026-10-29 rule, since the 10-29 audit found such reruns mostly re-checking wording. The rerun reviews only the answers, and the cap of two rounds still ends the pass. Valid while 🟡 fixes that change tests are a small share of 🟡 fixes, so the extra round stays cheap; check that again after the next ten chunks that use it.
## review: a comment is prose in any file; a branch changing only Markdown no agent follows runs only the prose pass and, for README or docs an outside reader lands on, the product-owner pass; plan rows a chunk files from a finding, re-scores or moves get no review
2026-10-29, lilleman. Serves goals 1.4 and 1.5. One project's chunk on 2026-10-29 changed a README paragraph, `todo.md` and three lines of `gate.yml`, and the principles, security and stability passes all ran on it, each twice: 6 of the chunk's 10 rounds. A change to config is one that changes what runs, so a comment in it fires only the prose pass, as a comment in code already did. A branch that changes only Markdown no agent follows changes no code, config or tests, so no pass that reads those runs on it. A plan row a chunk files from a finding, re-scores or moves is read by the chunk that ships it, so reviewing it at filing reviewed it twice: that chunk spent a prose round of 4 minutes on its plan change. Markdown an agent follows, such as a skill, is config, since a change there changes what agents do. Valid while defects in prose-only chunks stay rare in later chunks' fixes.
+3 -2
View File
@@ -87,18 +87,19 @@ Give a reviewer without Bash `git ls-files`'s output in its prompt, since some h
- **Done.** A pass is done when its latest round leaves no 🔴 or 🟡 standing: none was reported, or each was shown factually wrong, declined, filed, or named under the bar per "Findings", or settled in the PR summary per "Comprehension check" → "Places", or, for the prose pass and a pass at its cap, fixed. A stability pass whose round's fixes skip the next round per "Re-entry" → "Mechanical answers" is done once that rule's run is clean. Resolve that round's 🟢s and ❓s per "Findings" too.
- **Cap.** A pass takes at most two rounds per chunk, and the release review's architecture and principles passes five; don't ask the user for more. Answer its last round's findings per "Findings": those fixes merge once the gate passes, unreviewed by that pass. rule-hook's `round-cap` refuses a round commit that would take a pass past its cap, and no override waives it. A review the auto-reviewer posts after its second isn't a round: fix none of it and rerun no pass for it, but file a security finding it raises per `~/.claude/CLAUDE.md` → "Working loop", and any other 🔴 in `todo.md`.
**Re-entry.** A fix is a change no pass has seen. A fix for a 🟡, a 🟢, a ❓ or a prose finding reruns no pass, whatever it changes. A round's fixes for 🔴 findings re-enter together: run every pass that a row below names for any of them, a done pass included, in the waves of "Waves", then §4; "Mechanical answers" below names the one exception. A pass at its cap doesn't run again ("Round cap" → "Cap").
**Re-entry.** A fix is a change no pass has seen. A fix for a 🟢, a ❓ or a prose finding reruns no pass, whatever it changes. A fix for a 🟡 reruns none either, except one that changes what a test asserts or whether it runs, which meets the second row only. A round's fixes for 🔴 findings, and its 🟡 fixes that meet that row, re-enter together: run every pass that a row below names for any of them, a done pass included, in the waves of "Waves", then §4; "Mechanical answers" below names the one exception. A pass at its cap doesn't run again ("Round cap" → "Cap").
| The fix | Reruns |
|---------|--------|
| answers a 🔴 of a pass other than prose | that pass |
| answers a 🟡 of a pass other than prose and changes what a test asserts or whether it runs: an assertion added, removed or loosened, an expected value changed, or a test skipped, deleted, or put behind a condition or a catch | that pass |
| carries an architectural decision, in the release review | Architecture |
| changes code, config or tests, in the release review | Principles |
| opens an untrusted-input boundary or touches authn/authz, secrets or crypto | Security, unless the full scan reruns on it |
| made rule-hook log an override | Overrides |
| changes behaviour: what the code or its tests do | Stability |
A fix meets every row it fits. Prose has no row, so a comment or doc edit inside a fix doesn't rerun any pass. Example: a fix for a stability 🔴 that changes code and a comment reruns Principles and Stability.
A fix for a 🔴 meets every row it fits; a fix for a 🟡 meets only the second. Prose has no row, so a comment or doc edit inside a fix doesn't rerun any pass. Example: a fix for a stability 🔴 that changes code and a comment reruns Principles and Stability.
**Mechanical answers.** A stability round's fixes may skip the next stability round. That holds where the round reported a 🔴, and each of its 🔴 and 🟡 findings asked only that a named mutant be killed or a named failing test pass. Run §2's tests and mutation command on the fixes, and fix again per §2 until they leave no failing test and no surviving mutant that changes behaviour. Each named failing test must pass as the round found it, not edited or deleted. Then the stability pass doesn't run again, and the round's answer commit title ends `; tests and mutants clean at <commit>`. The fixes still rerun each other pass whose row they meet. Where the run can't be made clean, or a fix edits a named failing test, the stability pass runs again as the table says.