Answer prose round 1: the CHANGELOG states one finding per test, and the README gives mutmut's real launcher limit

This commit is contained in:
2026-10-13 11:09:46 +02:00
parent 2114998673
commit ac4a40eb02
5 changed files with 9 additions and 9 deletions
+1 -1
View File
@@ -2,7 +2,7 @@
## Unreleased
- The stability reviewer mutation-tests the lines each change adds or modifies: it makes small changes to that code, such as flipping a comparison, and reports each one that changes behaviour and that no test notices as a 🟡 naming the test to add or strengthen. It uses the project's documented mutation command, or else mutmut, Stryker, cargo-mutants or Infection. The technical principles replace "Test coverage should not decline" with that rule, and say what a valuable test is: it checks behaviour through the unit's interface, and fails when that behaviour breaks. README → "Tools" gives the mutmut command for `bin/`. Expect longer stability passes and more 🟡s about tests. In a project whose README documents no mutation command, the reviewer picks and installs a tool on each run; document one there to make the runs consistent.
- The stability reviewer mutation-tests the lines each change adds or modifies: it makes small changes to that code, such as flipping a comparison. For each test to add or strengthen, it reports one 🟡 naming the changes that test would catch. It uses the project's documented mutation command, or else cargo-mutants, Infection, mutmut or Stryker. The technical principles replace "Test coverage should not decline" with "Every mutant of a changed line fails a test, unless the mutant leaves behaviour unchanged", and say what a valuable test is: it checks behaviour through the unit's interface, and fails when that behaviour breaks. README → "Tools" gives the mutmut command for `bin/`. Expect longer stability passes and more 🟡s about tests. In a project whose README documents no mutation command, the reviewer picks and installs a tool on each run; document one there to make the runs consistent.
- Each new `loop-claude` worker session's prompt now quotes the README's Goals and Audience. A line above them says that the chunk's `todo.md` items, or else the task, set its scope, and that the goals only choose how to deliver it. A session outside the loop reads Goals and Audience before starting a chunk, as `CLAUDE.md` now says.
- A session no longer reads the technical principles before writing code. It writes the simplest change that solves its item, test first wherever a test can check the change, and reads a principle only when the item changes it, a finding names it, or a decision or `todo.md` item it writes cites it. larv-review's reviewers hold the change to the principles, and a finding against one names it. Expect more findings from the prose and stability reviewers, since sessions no longer load the rules for comments and READMEs.
- The rules for comments and READMEs, "Quality" and the examples behind "Prose" moved to `skills/larv-review/standards/`, which the prose and stability reviewers and `loop-claude`'s judge read. `CLAUDE.md` keeps a short core of "Prose"; sorting where order carries no meaning lives only in the technical principles.
+2 -2
View File
@@ -77,7 +77,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`
Mutation-test a changed module, here `larv_review/rounds.py`, in a scratch copy of `bin/` (`cp -r bin "$scratch"`), since mutmut writes files beside its config. Name the module and its tests in a `setup.cfg` there, and leave out each test class that starts a tool's launcher through `subprocess`: the launcher's Python has no mutmut.
To mutation-test a changed module, copy `bin/` to a scratch directory (`cp -r bin "$scratch"`): mutmut writes files beside its config. In the copy, name the module and its tests in a `setup.cfg`, as below for `larv_review/rounds.py`. Leave out each test class that starts a tool's launcher through `subprocess`: a mutated module fails to load in a process mutmut did not start.
```ini
[mutmut]
@@ -88,7 +88,7 @@ pytest_add_cli_args=-k
not LarvReviewTool
```
Then, in the copy: `uvx --with pytest 'mutmut>=3,<4' run && uvx --with pytest 'mutmut>=3,<4' results`, and `uvx --with pytest 'mutmut>=3,<4' show <mutant>` for a mutant's line. The config keys are mutmut 3's.
Then, in the copy: `uvx --with pytest 'mutmut>=3,<4' run && uvx --with pytest 'mutmut>=3,<4' results`, and `uvx --with pytest 'mutmut>=3,<4' show <mutant>` to see a mutant's change. The config keys are mutmut 3's.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them, and `rule_hook` imports `rank_todo.py`'s check. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
+2 -2
View File
@@ -13,7 +13,7 @@ You are an elite stability-focused code reviewer with deep expertise in defensiv
2. **Read every file under `~/.claude/principles/`**, the shared principles, and `~/.claude/skills/larv-review/standards/quality.md`. These are mandatory org guidelines; all code MUST comply with each that applies. A file's `Applies to` line says what it covers; one without the line covers every change. Re-read each time — they change. Flag violations prominently.
3. **Read README.md, AGENTS.md and `docs/decisions.md`** in the current project for project-specific conventions, build instructions, tooling, and the accepted trade-offs a reviewer should honor rather than re-flag. Note the documented test, lint, and static-analysis commands — you will run them.
4. **Verify, don't just read.** Run the test suite and static analysis/linters through the project's container tooling, unless a decision puts them on the host; tear down test containers afterwards. A full-suite run catches broken callers that diff-reading misses. When a suspected bug is cheap to confirm, reproduce it — a scratch script, a targeted test run — before reporting it. Put scratch work in a directory you create for this run, and delete it when done; delete nothing else, since a directory the prompt names may hold other agents' files. Leave the working tree exactly as you found it. Anything you could not run goes in Coverage & Verification, never silently skipped.
5. **Mutation-test the changed lines.** Run the project's documented mutation command, or else a widely deployed mutation tool for the language: mutmut for Python, Stryker for JavaScript and TypeScript, cargo-mutants for Rust, Infection for PHP. Limit it to the lines the change adds or modifies, through the tool's own option where it has one (cargo-mutants `--in-diff <diff file>`, Infection `--git-diff-lines`, Stryker `--mutate <file>:<lines>`); otherwise mutate only the changed files and keep only the mutants on changed lines. Run it on a scratch copy, since the tools write files, through the project's tooling as in step 4. Report its wall time and mutant count in Coverage & Verification, or why nothing was mutated. Report one 🟡 per test to add or strengthen, naming the surviving mutants it would catch. Drop a mutant that leaves behaviour unchanged, and one that changes only message or log text no caller or user depends on.
5. **Mutation-test the changed lines.** Run the project's documented mutation command, or else a widely deployed mutation tool for the language: cargo-mutants for Rust, Infection for PHP, mutmut for Python, Stryker for JavaScript and TypeScript. Limit it to the lines the change adds or modifies, through the tool's own option where it has one (cargo-mutants `--in-diff <diff file>`, Infection `--git-diff-lines`, Stryker `--mutate <file>:<lines>`); otherwise mutate only the changed files and keep only the mutants on changed lines. Run it through the project's tooling, as in step 4, on a scratch copy: the tools write files. Report its wall time and mutant count in Coverage & Verification, or why nothing was mutated. Ignore a surviving mutant that leaves behaviour unchanged, or changes only message or log text no caller or user depends on. Report one 🟡 per test to add or strengthen, naming the surviving mutants it would catch.
6. **Two passes minimum, then hunt absences.**
- *Breadth:* every hunk in every changed file, no skimming. Track which files you have covered; batch large diffs, never truncate the review.
- *Depth:* trace the riskiest data paths end-to-end (input → logic → persistence/response), the interactions between changed pieces, and between the change and config (timeout hierarchies, retries × idempotency, queue worker counts, env parity).
@@ -120,7 +120,7 @@ Every PR gets at least this sweep.
### 11. Tests
- Does the PR ship tests for the new behavior? (principle: most code has automated tests, and tests come first.)
- Do they exercise the risky and error paths, not just the happy path?
- Does a mutant of a changed line survive (First Steps, step 5)? (principle: every mutant of a changed line that changes behaviour fails a test.)
- Does a mutant of a changed line survive (First Steps, step 5)? (principle: every mutant of a changed line fails a test, unless the mutant leaves behaviour unchanged.)
- Was a test deleted while the behaviour it pinned remains? Flag it; restore the test.
- Were tests and implementation changed together in a way that could mask a regression — e.g. an assertion weakened to make a test pass? Flag it; these should be separate changes.
- Would each test fail if the behavior regressed? Watch for assertion-free tests, tautological asserts, and mocks so broad the test exercises the mock instead of the code.
+3 -3
View File
@@ -167,10 +167,10 @@ Valid while review rounds per pass do not rise. A principle found late, such as
## review: the stability pass mutation-tests the lines a chunk changes, in place of measuring test coverage
2026-10-13, lilleman through todo item 108, implemented by claude. Serves goals 1.1 and 1.4. Coverage counts the lines a test runs, so a test without a meaningful assertion still raises it. A surviving mutant shows a test that would let behaviour break. Every mutant of an untested changed line survives, so mutation testing of the changed lines also shows what a coverage drop on them would. A deleted test is in the diff, which the stability reviewer reads. Measuring coverage beside it would add cost without raising quality, so todo item 69 was dropped and `principles/technical-principles.md` replaced "Test coverage should not decline" with the mutant rule.
2026-10-13, lilleman through todo item 108, implemented by claude. Serves goals 1.1 and 1.4. Coverage counts the lines a test runs, so a test without a meaningful assertion still raises it. A surviving mutant shows a test that would let behaviour break. Every mutant of an untested changed line survives, so mutation testing of the changed lines also shows what a coverage drop on them would. Coverage would also catch a deleted test; the stability reviewer sees one in the diff without it. Measuring coverage beside it would add cost without raising quality, so todo item 69 was dropped and `principles/technical-principles.md` replaced "Test coverage should not decline" with the mutant rule.
The reviewer limits each run to the changed lines, so its cost grows with the chunk. The tools are those the language's ecosystem most uses, each with its own way to limit a run to a diff: cargo-mutants `--in-diff` (https://mutants.rs/in-diff.html), Infection `--git-diff-lines` (https://infection.github.io/guide/command-line-options.html), Stryker's mutation ranges (https://stryker-mutator.io/docs/stryker-js/configuration/). mutmut has no such option, so the reviewer mutates the changed files and keeps the mutants on changed lines. On 2026-10-13 mutmut 3 ran on this repo's `larv_review/rounds.py` in under two seconds, as README → "Tools" describes.
The reviewer limits each run to the changed lines, so its cost grows with the chunk. The tools are widely deployed, each with its own way to limit a run to a diff: cargo-mutants `--in-diff` (https://mutants.rs/in-diff.html), Infection `--git-diff-lines` (https://infection.github.io/guide/command-line-options.html), Stryker's mutation ranges (https://stryker-mutator.io/docs/stryker-js/configuration/). mutmut has no such option, so the reviewer mutates the changed files and keeps the mutants on changed lines. On 2026-10-13 mutmut 3 ran on this repo's `larv_review/rounds.py` in under two seconds, set up as README → "Tools" says.
The run has no time cap. Limiting it to the changed lines already bounds it by the chunk's size, and no published default would set a cap. Each run's wall time in the reviewer's report shows when the premise below fails.
The run has no time cap. Limiting it to the changed lines already bounds it by the chunk's size, and no published standard gives a cap to copy. Each run's wall time in the reviewer's report shows when the premise below fails.
Valid while a mutation run on a chunk's changed lines stays cheap beside the review itself.
+1 -1
View File
@@ -44,7 +44,7 @@
- Write tests first for the behaviour we want, then implement until they pass.
- Changing tests and changing other code are two separate processes — be very careful about doing both at once.
- Use properly randomised UUID v7s — never `aaaa-0000`-style placeholders.
* Every mutant of a changed line that changes behaviour fails a test. A mutant is a small change to the code, such as a flipped comparison; one no test notices marks behaviour the tests would let break. The stability pass checks this with a mutation run on the chunk's changed lines.
* Every mutant of a changed line fails a test, unless the mutant leaves behaviour unchanged. A mutant is a small change to the code, such as a flipped comparison. One that no test notices marks behaviour the tests would let break. The stability pass checks this with a mutation run on the chunk's changed lines.
* Idempotent data writing.
- Typical example: `PUT /resource/{id}` instead of `POST /resource` + Idempotent-Id header.
* Handle errors explicitly, as values — avoid the exception path.