Merge pull request 'Decide the ratchet by a paired ruling, and check the floor on each release's last item' (#228) from item-75 into main

This commit was merged in pull request #228.
This commit is contained in:
2026-10-08 21:06:59 +02:00
10 changed files with 207 additions and 44 deletions
+3 -2
View File
@@ -12,7 +12,8 @@ Decisions live in `docs/decisions.md`:
- loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
- review: a pass gets at most three rounds per chunk; instead of a fourth, the run asks the human
- review: a fix for a 🔴 or 🟡 reruns the pass that found it and each pass whose trigger names fixes like it; a fix for a 🟢, a ❓ or a prose finding reruns none
- review: a project decision may suspend the scoring run for the plan items it names
- review: a paired ruling against the last measured commit decides the ratchet; the four-seat scoring run checks the floor when a release's items run out or an item names the floor
- review: a project decision may suspend the comprehension check for the plan items it names
- review: a pass skips only on a precedent: a decision, or a plan item crediting the pass, that the repo held before the chunk began
- loop-claude: the archive keeps the raw transcript, and a failed archive stops the loop
- loop-claude: a `claude` that cannot start is a failed worker run, retried
@@ -33,7 +34,7 @@ Decisions live in `docs/decisions.md`:
## Scoring run
The project values for `skills/comprehension-panel/scoring-run.md`:
The project values for `skills/comprehension-panel/seat-prompts.md`:
- `{language}`: Python
- `{kind}`: a Claude Code configuration with an agent loop and review skills
+5 -5
View File
@@ -2,20 +2,20 @@
## Unreleased
- larv-review blocks a merge when simulated readers rule that the PR makes the project read worse than the last measured commit: the default branch's newest merged change that added a scoring run to `docs/comprehension-history.md`, or whose last ruling there is better. A ruling of same leaves that commit in place, so a worse ruling may name places earlier chunks wrote. In each architecture pass Mid A and the inherited architect vote better, same or worse; when they disagree, the maintainability senior and Junior A vote too. A ruling needs a lead of two or more votes, else it is same, and a vote that names no place counts as same. A worse ruling never ships: after three rounds larv-review asks you for another round, a no leaves the PR unmerged, and so does an unattended run. The four-seat scoring run checks the comprehension floor only when a chunk ships or removes a release's last plan item, or ships an item naming the floor; between those runs the floor blocks nothing. A score under the floor blocks the merge until fixed or until you decide to ship; a decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
- larv-review reruns fewer passes after a fix. A fix reruns the pass whose 🔴 or 🟡 it answers, plus stability when the fix changes behaviour, security when it opens an untrusted-input boundary or touches authn/authz, secrets or crypto, and architecture when it carries an architectural decision. A rerun reviews only what changed since that pass last reviewed. A fix for a 🟢, a ❓ or any prose finding reruns no pass, and prose takes one round.
- larv-review runs one security pass on each change: the full scan alone when it runs, else `/security-review`.
- larv-review runs the new `bin/reading-cost`, which measures each chunk `loop-claude` finished from its archive: the new input tokens the worker and its subagents took in before the worker's first commit, and the files git tracks that they read most. It appends each figure the project's `docs/reading-cost.md` lacks. Once 10 chunks are measured, and the last 5 chunks' median is above the 5 before them, larv-review files one plan item naming the files read most, or refreshes it. The chunk that ships it runs `reading-cost --reset`, so the trend counts only chunks that finished after it.
- larv-review appends every scoring run to the project's `docs/comprehension-history.md`: the commit the seats read, each seat's five scores and the seats' mean, and the places each seat named. The ratchet compares a run with the history's last run on the default branch. The `comprehension floor:` entry in `docs/decisions.md` keeps only the decisions to ship under the floor; a project's next scoring run moves the last scores the entry still holds into the history. The seats do not read the history.
- larv-review appends every paired ruling and scoring run to the project's `docs/comprehension-history.md`: the commit the seats read, a ruling's base, each seat's vote or scores, and the places each seat named. The PR summary gives each ruling with its base and votes. The `comprehension floor:` entry in `docs/decisions.md` keeps only the decisions to ship under the floor; a project's next scoring run moves the last scores the entry still holds into the history. The seats do not read the history.
- The README's goals put quality with the fewest interrupts first, as goal 1; sharing the setup, backups and leaking nothing follow as goals 2–4.
- larv-review reruns the comprehension scoring run before merge when any commit since the last run changes more than `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`, so the recorded scores are the merged commit's. A score under the floor, or below the default branch's recorded scores, blocks the merge and is never filed in `todo.md`. A fix lifts either; your decision to ship under the floor lifts only a score under the floor. A rerun counts toward the three-round cap only when it reports a blocking score that no decision of yours covers.
- larv-review skips the comprehension scoring run for a chunk whose plan items a project decision names. The decision also names the later item whose scoring run ends the suspension. Neither the comprehension floor nor the ratchet can block a skipped chunk, its summary cites the decision, and a chunk shipping its release's last item is always measured.
- larv-review reruns the comprehension check before merge when any commit since its last run changes more than `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`, so the recorded result is the merged commit's. A result that blocks the merge is never filed in `todo.md`. A rerun counts toward the three-round cap only when it reports a block that no decision of yours covers.
- larv-review skips the comprehension check for a chunk whose plan items a project decision names. The decision also names the later item whose check ends the suspension. A skipped chunk cannot be blocked for reading worse or for the floor, and its summary cites the decision. A chunk that ships or removes a release's last plan item, or ships an item naming the floor, still gets the scoring run.
- README → Install and Update give the commands that check the latest tag out into `~/.claude`: on a new machine, whether or not Claude Code has already created the directory, and on one already on an older tag.
- A session commits changes Claude Code made to `settings.json` only once their diff holds no credential, and asks you to move any credential out of the file first.
- A technical principle, "Sane defaults, where a default serves better than an explicit ask". A default suits a value most callers want, does no harm when wrong, and can be seen and changed. It lives at the program's edge, such as a CLI; a library still asks for data, the clock and randomness.
- larv-review's principles audit reads only the files under `~/.claude/principles/` that apply to the project, launching a file's auditor when unsure. A file's `Applies to` line says what it covers; one without the line covers every project. The summary names each file left out. After its first round, the architecture pass reruns the scoring run and only the reviewers whose 🔴 or 🟡 the fixes answer, unless the fixes carry an architectural decision.
- larv-review's principles audit reads only the files under `~/.claude/principles/` that apply to the project, launching a file's auditor when unsure. A file's `Applies to` line says what it covers; one without the line covers every project. The summary names each file left out. After its first round, the architecture pass reruns only the reviewers whose 🔴 or 🟡 the fixes answer, unless the fixes carry an architectural decision.
- When `loop-claude`'s judge rules a chunk unfinished, the resumed worker is shown the passage the judge quoted as the work remaining. A loop started before this change exits 1 at its next reload; rerun it.
- larv-review's architecture pass and full security scan get short names, `Architecture` and `Full scan`. Each round's commit is titled with its pass's name in lower case: `Answer full scan round 1: …`.
- larv-review's comprehension floor now says, without another file open, what the scoring run measures. A run scoring below the last one blocks the merge until fixed. After three rounds larv-review asks you; an unattended run leaves the PR unmerged with the question. A PR under the floor that scores no lower than the last run merges once you decide to ship. A decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
- `loop-claude`'s judge reads your `~/.claude/CLAUDE.md` beside the project's `AGENTS.md` and `docs/decisions.md`. It no longer interrupts for a choice that file settles, and it answers the worker's questions from it.
- `loop-claude`'s judge holds a chunk open until each answer you typed in it that a later chunk could need is recorded. An answer counts as recorded once the README's goals or audience, `AGENTS.md`, `docs/decisions.md` or `~/.claude/CLAUDE.md` hold it. It also counts once the transcript shows the worker writing it into the README's goals or audience, `AGENTS.md`, `docs/decisions.md`, `todo.md` or a code comment. The worker writing to `~/.claude/CLAUDE.md` does not count, since new rules there are yours. The resumed worker is told this rule.
- `loop-claude` archives each answer the judge quotes, with the question it answers and its UTC time, in `<sha256>.outcomes.md`.
+2 -2
View File
@@ -34,10 +34,10 @@ How work proceeds in any repo, unless the repo's `AGENTS.md` says otherwise. The
## Quality
Quality is proven by review. Every change is reviewed as a diff. The whole project is measured when a change carries an architectural decision, and before each release, because a project can drift from the principles one clean diff at a time. A project decision may suspend only the comprehension panel's scoring run (`larv-review` §3 → "Architecture" → "Suspended"). `larv-review` §3 says which review runs when. Quality has three parts. When two pull against each other, the earlier one wins.
Quality is proven by review. Every change is reviewed as a diff. The whole project is measured when a change carries an architectural decision, and before each release, because a project can drift from the principles one clean diff at a time. `larv-review` §3 says which review runs when. Quality has three parts. When two pull against each other, the earlier one wins.
1. **Correct.** The work does what the goals and the plan say, and tests prove it.
2. **True to the technical principles** (`~/.claude/principles/technical-principles.md`). The `comprehension-panel` skill scores KISS; a score under the comprehension floor (`larv-review` §3 → "Architecture") blocks a merge. **Comprehension never falls:** a chunk starts from the last panel recorded on the default branch, and when its own panel scores any dimension or the overall lower, even within the panel's noise, improve what the readers named and score again. An equal score passes. After three such rounds, stop and ask the human.
2. **True to the technical principles** (`~/.claude/principles/technical-principles.md`). The `comprehension-panel` skill measures KISS in `larv-review`'s comprehension check (§3 → "Architecture"), which a project decision may suspend, unlike any other review. **Comprehension never falls:** when readers comparing the chunk with the last commit the panel scored or ruled better rule that it reads worse, improve what they named and rule again. After three such rounds, stop and ask the human. At a release's end, or for an item naming it, a score under the comprehension floor blocks a merge.
3. **Plainly written**, per "Prose".
## Prose
+106
View File
@@ -262,3 +262,109 @@ Inherited architect, would restructure first:
- `bin/loop_claude/cycle/judge_reply.py:12` `JUDGE_RULES`, beside `judge.py:53` `judge_prompt`
- `bin/loop_claude/project/checkout.py:7` `checkout_root` and `bin/reading_cost/main.py:113` `located`, and `checkout.py:32` `read_utf8`
- `bin/failure.py`, `bin/file_writes.py` and `bin/markdown_sections.py`, into one package
## 2026-10-08T18:37:21Z, PR #228 at 2682668, against 8c9bf09
Ruling: worse
| Seat | Vote |
|---|---|
| Mid A | worse |
| Inherited architect | worse |
Mid A, decided by:
- `skills/larv-review/SKILL.md:91` "Paired ruling": the base, its fallbacks, a two-stage vote and a tie rule in one paragraph
- `skills/comprehension-panel/scoring-run.md:1` and `:67` "Paired ruling": one file name over two kinds of prompt
- `skills/comprehension-panel/SKILL.md:102` "The paired ruling": the procedure spread over three files
- `skills/larv-review/SKILL.md:89` "Architecture": the comprehension check, paired ruling, scoring run, ratchet and floor to hold at once
- `skills/larv-review/SKILL.md:105` "History": two record shapes in one bullet
- `skills/larv-review/SKILL.md:66` "Re-entry", better: "Rerun" names its mechanism
Inherited architect, decided by:
- `skills/larv-review/SKILL.md:89` "Architecture": two mechanisms, each with its own trigger and seats, and a three-way rule for the base
- `skills/larv-review/SKILL.md:105` "History": two entry shapes, and which commit of a paired entry is the next base
- `CHANGELOG.md:5` against `:11`, `:16`, `:19`: two accounts of what blocks a merge
- `CLAUDE.md:37` and `:40` "Quality": "comprehension check" undefined, and one sentence packing the floor and the ruling
- `skills/comprehension-panel/scoring-run.md:1` "Paired ruling": the name no longer says what is inside
- `AGENTS.md:15`: the new decision title packs both mechanisms and the vote rule
- `skills/larv-review/SKILL.md:35` and `:64` "Re-entry", better: a rerun traces from the table alone
## 2026-10-08T18:46:12Z, PR #228 at fbe753e, against c7333c0
Ruling: same
| Seat | Vote |
|---|---|
| Mid A | worse |
| Inherited architect | better |
| Maintainability senior | same |
| Junior A | worse |
Mid A, decided by:
- `skills/larv-review/SKILL.md:89` "Architecture": two kinds of run under one umbrella, each with its own trigger, and a vote rule to work through by example
- `skills/larv-review/SKILL.md:93` "Base": the comparison point found by how earlier chunks appended to the history
- `skills/larv-review/SKILL.md:35` the Architecture row: a forward reference to the scoring run's trigger
- `CLAUDE.md:37` and `:40` "Quality": one concept under several names, and "the last measured commit" undefined
- `skills/comprehension-panel/SKILL.md:102` "The paired ruling": who votes is left to the caller
- `skills/comprehension-panel/seat-prompts.md:1`, better: the name tells the truth, and `{forbidden}` serves both briefs
- `skills/larv-review/SKILL.md:64` "Re-entry", better: the rerun rule is explicit
Inherited architect, decided by:
- `skills/larv-review/SKILL.md:64` "Re-entry", better: why a pass ran a fourth time is answered in one paragraph
- `skills/larv-review/SKILL.md:33` the pass table, better: each pass's rule for fixes sits in its own Trigger cell
- `skills/larv-review/SKILL.md:37` and `:123` "Security", better: when each security pass runs
- `skills/larv-review/SKILL.md:93` "Base": base selection depends at a distance on the wording of a line in the history format
- `skills/larv-review/SKILL.md:35` the Architecture row: a pointer 60 lines down
- `skills/larv-review/SKILL.md:95` "Ruling": the vote margin worked out by hand
- `CLAUDE.md:37` and `AGENTS.md:16`: one concept, three names
Maintainability senior, decided by:
- `skills/larv-review/SKILL.md:89` "Paired ruling": the base defined by an action, and the vote arithmetic
- `skills/larv-review/SKILL.md:35` and `:97`: a forward reference, though one definition
- `skills/larv-review/SKILL.md:64` "Re-entry", better: deterministic reruns
- `skills/larv-review/SKILL.md:59` "Later rounds", better: every rerun reviews only its diff
- `skills/comprehension-panel/seat-prompts.md:3`, better: the briefs cannot drift apart
- `AGENTS.md:35` "Scoring run": the heading no longer matches its use
- `CLAUDE.md:40`: "last measured commit" defined only in larv-review
Junior A, decided by:
- `skills/larv-review/SKILL.md:91` "Paired ruling": the base, the escalation and the vote rule in three rereads
- `skills/larv-review/SKILL.md:35` the Architecture row: a forward reference to a two-condition rule
- `skills/larv-review/SKILL.md:35`, `:51`, `:89`, `CLAUDE.md:37`: four names for the comprehension machinery
- `skills/larv-review/SKILL.md:36` the Full scan and Security rows: read together now
- `skills/comprehension-panel/seat-prompts.md:8`: placeholders nesting placeholders
## 2026-10-08T18:57:27Z, PR #228 at 50249b1, against 12d6d75
Ruling: worse
| Seat | Vote |
|---|---|
| Mid A | worse |
| Inherited architect | worse |
Mid A, decided by:
- `skills/larv-review/SKILL.md:93` "Base": a `git log` command, a two-way condition, a forward reference to the history's format and two fallbacks
- `skills/larv-review/SKILL.md:89` "Architecture": two kinds of run, four sub-bullets and a vote rule; the section's fan-out roughly doubled
- `skills/larv-review/SKILL.md:35`, `:98` and `:120`: one trigger written out three times
- `CLAUDE.md:40` "Quality": "the last commit the panel scored or ruled better" reads two ways
- `AGENTS.md:35` "Scoring run": the heading no longer names what it feeds
- `CHANGELOG.md:5`: ten sentences in one entry
- `skills/larv-review/SKILL.md:66` "Re-entry", `:37` and `:124` "Security", `skills/comprehension-panel/seat-prompts.md:7`, better
Inherited architect, decided by:
- `skills/larv-review/SKILL.md:93` "Base": finding the base at 3am takes git archaeology across merge commits
- `CHANGELOG.md:5` against `skills/larv-review/SKILL.md:93` and `:107`: only the changelog says a ruling of same leaves the base, so a chunk may be told to fix places it never touched
- `skills/larv-review/SKILL.md:95` "Seats" and `:96` "Ruling": the vote table built in the head
- `CLAUDE.md:40` "Quality": a folded noun phrase in the charter
- `skills/larv-review/SKILL.md:35`, `:98` and `:120`: the trigger three times, and seven concepts where there were three
- `AGENTS.md:35` "Scoring run": the heading not renamed with the file
- `skills/larv-review/SKILL.md:64` "Re-entry" and `:40` the Prose row, `:37` and `:124` "Security", better
+9 -4
View File
@@ -48,8 +48,9 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
- 2026-10-03, PR #202: 7.5, 7.0, 7.0, 6.75, 7.0.
- 2026-10-03, PR #208: 7.5, 7.0, 7.13, 6.63, 6.88.
- 2026-10-04, PR #215: 7.38, 6.63, 7.0, 6.88, 6.88.
- 2026-10-08, PR #224: 7.00, 6.50, 7.00, 6.50, 6.75. lilleman shipped it below the ratchet as well: its 20 lines of skill prose left the code as it was, so the fall is the panel's noise between runs, which `todo.md` item 75 addresses.
- 2026-10-08, PR #224: 7.00, 6.50, 7.00, 6.50, 6.75. lilleman shipped it below the ratchet as well: its 20 lines of skill prose left the code as it was, so the fall is the panel's noise between runs, which the paired ruling cancels.
- 2026-10-08, PR #225: 7.25, 6.75, 7.00, 6.88, 6.88. Above PR #224's run on every score but Shape, which held.
- 2026-10-08, PR #228: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet: the seats blamed the new ratchet's own rules, which `todo.md` item 85 moves into a tested tool.
## loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
@@ -63,9 +64,13 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
2026-10-08, lilleman, replacing the 2026-09-30 rule that every fix, a nit's included, re-enters review. Serves goal 1.4. An audit of the four costliest archived chunks (2026-10-08) found nearly every finding that mattered in a pass's first round; later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes keep changing nothing a user would notice.
## review: a project decision may suspend the scoring run for the plan items it names
## review: a paired ruling against the last measured commit decides the ratchet; the four-seat scoring run checks the floor when a release's items run out or an item names the floor
2026-10-04, lilleman. Serves goal 1.4. In a restructure, each step's scoring run measures code the next step deletes. It costs a run and, below the floor, blocks the step, yet says nothing about what the restructure buys. The scoring run after the last step measures that. Valid while such a decision names its items and the plan item that ends it.
2026-10-08, lilleman. Serves goals 1.1 and 1.4. Two independent four-seat runs moved up to 0.4 on near-identical code, so a fall within that noise forced rounds that changed nothing (PR #224). A seat reading both versions judges the difference on the same material, so the noise the versions share cancels. No conventional significance level lets a sign test settle with four votes, so the human chose a lead of two votes over every other ruling, same included. Two seats that agree settle it, so a PR that leaves the reading as it was costs two seats. A third seat alone can never settle it, so the third and fourth launch together. The base is the default branch's newest merged change that adds a scoring run to the history, or whose last added paired ruling is better. The human chose the last measured commit on the architect's finding that with a merge base as the base, chunks without an architecture pass, and a suspended restructure's steps, could each lower comprehension unseen. A ruling of same leaves the base where it was, or a run of small falls each ruled same would add up unseen. A scoring run moves it too: otherwise a project never ruled better would compare against its first ruling forever, and a release's floor check is where the human sees the absolute level. Reading the default branch one merged change at a time keeps the rule working for merge commits and squash merges alike: a branch's own early ruling of better never counts, only the merged change's last ruling. The scoring run also runs on a chunk shipping an item that names the floor, so the chunk that lifts a project to the floor can prove it. A chunk that removes a release's last item without shipping it runs it too, so no release ends unmeasured. Valid while seats reading both versions agree more often than two absolute runs do.
## review: a project decision may suspend the comprehension check for the plan items it names
2026-10-04, lilleman. Serves goal 1.4. In a restructure, each step's scoring run measures code the next step deletes. It costs a run and, below the floor, blocks the step, yet says nothing about what the restructure buys. The scoring run after the last step measures that. 2026-10-08: the paired ruling replaced the scoring run in most chunks, and the suspension covers it too; the ruling after the last step compares with the commit measured before the first step. Valid while such a decision names its items and the plan item that ends it.
## review: a pass skips only on a precedent: a decision, or a plan item crediting the pass, that the repo held before the chunk began
@@ -115,7 +120,7 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
## reading cost: larv-review measures each finished chunk from loop-claude's archive and appends the figure to `docs/reading-cost.md` in the next chunk's PR
2026-10-08, lilleman. Serves goal 1.1: comprehension measured from real work, at no model cost (goal 1.4). loop-claude archives from the main checkout, outside any PR, so a tracked file it wrote there would dirty that checkout. So `bin/reading-cost` measures each finished chunk `docs/reading-cost.md` lacks from the archive, in the next chunk's worktree. A PR that never merges then loses no figure, and loop-claude needs no change. The figures keep a file of their own, so `docs/comprehension-history.md` keeps one writer, the scoring run. Serves goal 4 by naming only files git tracks, and token counts. Rejected: loop-claude opening a PR of its own, one more PR per chunk and a tie to the forge; the worker measuring its own transcript before merging, which misses the chunk's tail. The figure counts new input tokens, uncached and cache-written, because workers read mostly through Bash and subagents, where parsed paths alone miss what they read. Valid while loop-claude archives every finished chunk with its subagents' transcripts.
2026-10-08, lilleman. Serves goal 1.1: comprehension measured from real work, at no model cost (goal 1.4). loop-claude archives from the main checkout, outside any PR, so a tracked file it wrote there would dirty that checkout. So `bin/reading-cost` measures each finished chunk `docs/reading-cost.md` lacks from the archive, in the next chunk's worktree. A PR that never merges then loses no figure, and loop-claude needs no change. The figures keep a file of their own, so `docs/comprehension-history.md` keeps one writer, the comprehension check. Serves goal 4 by naming only files git tracks, and token counts. Rejected: loop-claude opening a PR of its own, one more PR per chunk and a tie to the forge; the worker measuring its own transcript before merging, which misses the chunk's tail. The figure counts new input tokens, uncached and cache-written, because workers read mostly through Bash and subagents, where parsed paths alone miss what they read. Valid while loop-claude archives every finished chunk with its subagents' transcripts.
## reading cost: a rise is the last 5 chunks' median above the 5 before them, and files or refreshes one plan item
+10 -2
View File
@@ -95,9 +95,17 @@ the end. Tell everyone not to manufacture confusion.
A scoring run needs four seats — Junior A, Mid A, the maintainability senior and the inherited
architect — plus the calibration seat ("The score") when more than one project is in play. Build
the four seats' prompts from `scoring-run.md`, filled in from the project's `AGENTS.md` → "Scoring
the four seats' prompts from `seat-prompts.md`, filled in from the project's `AGENTS.md` → "Scoring
run". Change neither between runs, or the scores are not comparable. A project without that
section gets it at its first scoring run.
section gets it at its first scoring run or paired ruling.
## The paired ruling
A paired ruling asks whether a change makes a project read better, the same or worse. Each seat
reads both versions, so what the two versions share cannot move the ruling. Check out the older
version as a detached worktree in your scratchpad, and remove it afterwards. Build each seat's
prompt from `seat-prompts.md` → "Paired ruling"; the seats and the rule for adding one are the
caller's.
## What the seven report
@@ -1,10 +1,12 @@
# The scoring run's seat prompts
# The comprehension seats' prompts
Each seat's prompt is the four parts below, in order: the opening, the seat's background, the shared brief, and the report for its kind of seat. Copy the text verbatim and replace each `{placeholder}`:
A scoring-run seat's prompt is the opening, the seat's background, the shared brief, and the report for its kind of seat, in that order. Copy the text verbatim and replace each `{placeholder}`:
- `{checkout}` and `{branch}`: the worktree the seats read, and its branch.
- `{main}`: the project's main checkout.
- `{files}`: every tracked non-test file the shared brief does not forbid, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, such as `bin/loop_claude/**/*.py`.
- `{forbidden}`: `todo.md, docs/decisions.md, docs/comprehension-history.md or docs/reading-cost.md (the plan, the decision log, past panels' records and past chunks' reading cost)`.
- `{git}`: `git -C {checkout} ls-files`, and for a paired seat also `git -C {base checkout} ls-files`, each in backticks.
- `{files}`: every tracked non-test file but those `{forbidden}` names, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, such as `bin/loop_claude/**/*.py`.
- `{preloaded}`: the files the harness pre-loads into a seat, found per `SKILL.md` → "What makes the result trustworthy".
- `{anchors}` and `{dimensions}`: the six anchor bullets and the four dimension bullets in `SKILL.md` → "The score", verbatim, without bold and each on one line.
- `{language}`, `{kind}`, `{domain}`, `{domain docs}` and `{3am question}`: the project's `AGENTS.md` → "Scoring run".
@@ -12,7 +14,7 @@ Each seat's prompt is the four parts below, in order: the opening, the seat's ba
## Opening
```
You are one seat of a comprehension panel. Tools: Read, Grep, Glob, and Bash only for `git -C {checkout} ls-files` and `ls`. Never write, edit, run, build or test anything.
You are one seat of a comprehension panel. Tools: Read, Grep, Glob, and Bash only for {git} and `ls`. Never write, edit, run, build or test anything.
```
## Backgrounds
@@ -27,7 +29,7 @@ You are one seat of a comprehension panel. Tools: Read, Grep, Glob, and Bash onl
```
Target: the project checked out at {checkout} (branch `{branch}`). Read ONLY there — never {main}, which is a different version.
Depth 1, the whole project: read README.md, AGENTS.md and CLAUDE.md (the charter), then every non-test tracked file: {files}. Do NOT open todo.md, docs/decisions.md, docs/comprehension-history.md or docs/reading-cost.md (the plan, the decision log, past panels' scores and past chunks' reading cost), nor any review output. You may open a test file when the code does not answer a question; say when you did.
Depth 1, the whole project: read README.md, AGENTS.md and CLAUDE.md (the charter), then every non-test tracked file: {files}. Do NOT open {forbidden}, nor any review output. You may open a test file when the code does not answer a question; say when you did.
Your harness pre-loads these files into your context before your first tool call, from a DIFFERENT version (main): {preloaded}. Treat the worktree's own copies as the ones under review. A pre-loaded AGENTS.md may list decision titles; do not use them as evidence.
@@ -63,3 +65,30 @@ Enforce this order:
3. Read enough implementation to check the map (depth 1: every non-test file is in scope, but your job is shape).
Report: the verbatim map; where it was wrong and how costly each correction was; level-by-level fan-out with the worst level named; names that mislead, concepts with two names, one name over two concepts, areas you could not name; what you would restructure, ranked, with files and costs; what the structure gets right. The 3am question: {3am question} — walking in cold, how long to the right file and unit? Where does it rot first, and where would the next two features land? Close with Navigation, Locality, Shape, Self-sufficiency and overall, each with one sentence naming its anchor and the concrete thing holding it there.
```
## Paired ruling
A paired seat's prompt is the opening, the seat's background, and the paired brief below. Its placeholders besides those above:
- `{base}` and `{head}`: the commits of the older and the newer version; `{base checkout}`: the worktree holding `{base}`.
- `{changed}`: the output of `git -C {checkout} diff --name-status {base} {head}`, less the files `{forbidden}` names.
```
Target: two versions of one project. The older, commit {base}, is checked out at {base checkout}; the newer, commit {head}, at {checkout}. Read ONLY there — never {main}, which is a different version.
Depth 1, the whole project in both versions: read README.md, AGENTS.md and CLAUDE.md (the charter), then every non-test tracked file at {checkout}: {files}. These files differ between the versions; read each at {base checkout} too, where it exists there:
{changed}
Do NOT open {forbidden}, nor any review output. You may open a test file when the code does not answer a question; say when you did.
Your harness pre-loads these files into your context before your first tool call, from main, which may match either version or neither: {preloaded}. Treat the two worktrees' own copies as the ones under review. A pre-loaded AGENTS.md may list decision titles; do not use them as evidence.
No running anything, no builds, no tests, no edits of any kind.
Rule whether the project as a whole reads better, the same or worse at {head} than at {base}, judged on these four dimensions:
{dimensions}
Same means a reader of either version would find it as easy to understand and change. Do not manufacture a difference.
Report, in this shape:
1. Your ruling: better, same or worse.
2. The places that decided it, most decisive first. Each: path:line and the unit's name, in the version the place is in, and one or two sentences on what reads easier or harder there than in the other version. A better or worse ruling with no named place is discarded.
```
+25 -16
View File
@@ -32,7 +32,7 @@ A pass is one kind of review, a row of the table, named by its first column. Its
| Pass | Reviewer | Trigger |
|------|----------|----------------------------------------|
| Architecture | systems-architect, then principles-auditor and the scoring run | ships its release's last item, or carries an architectural decision: a new, restructured or moved component, module, service or boundary; a new dependency or first use of an infrastructure pattern (queue, cron, cache, websocket, …); a new or changed external contract (routes, request/response shapes, events, webhooks, published file formats, CLI flags and exit codes); a schema change beyond an additive column; a fix, only where it carries an architectural decision |
| Architecture | systems-architect, then principles-auditor and the comprehension check | ships or removes the last plan item of a release that has work landed towards it, or ships an item naming the comprehension floor; or carries an architectural decision: a new, restructured or moved component, module, service or boundary; a new dependency or first use of an infrastructure pattern (queue, cron, cache, websocket, …); a new or changed external contract (routes, request/response shapes, events, webhooks, published file formats, CLI flags and exit codes); a schema change beyond an additive column; a fix, only where it carries an architectural decision |
| Full scan | the `claude-security` plugin at high effort: `/claude-security scan this branch's changes at high` | touches authn/authz, secrets or crypto, opens an untrusted-input boundary, or adds an external surface |
| Security | `/security-review` | changes code or config, unless the full scan runs on the same change; a fix, only where it opens an untrusted-input boundary or touches authn/authz, secrets or crypto, and the full scan does not rerun on it |
| Stability | stability-reviewer | changes code, config or tests; a fix, only where it changes behaviour: what the code or its tests do |
@@ -48,15 +48,15 @@ Give a reviewer without Bash the branch diff and `git ls-files`'s output in its
- **Skip.** A precedent is a `docs/decisions.md` entry, or a plan item crediting the pass, that the repo held before the chunk began. Where one holds the decision this change executes, skip the pass and cite that entry or item.
- **Departure.** A change that departs from the precedent is unreviewed, however thorough the precedent is.
- **Ask.** Fired and not skipped → ask the user whether to run it, with the evidence and your recommendation; no user reachable → run it. For the full scan, give the evidence and your recommendation, then launch it: the plugin asks the user to confirm its cost, and that prompt is the ask.
- **Audit.** A precedent or the user's no skips only the architect, never the principles audit: no precedent checked the code this chunk writes. The scoring run skips only as "Architecture" → "Suspended" says.
- **Audit.** A precedent or the user's no skips only the architect, never the principles audit: no precedent checked the code this chunk writes. The comprehension check skips only as "Architecture" → "Suspended" says.
**Round cap.**
- **Round.** A round is one launch of a pass's reviewer: the first, each re-entry, and the rerun after the architect returns Stage 0 questions ("Architecture" below). A rescore is a round only as "Architecture" → "Fresh" says.
- **Architecture round.** The architecture pass's first round is the architect (unless skipped), the principles audit and the scoring run, which "Architecture" → "Suspended" may skip. A later round relaunches each reviewer whose 🔴 or 🟡 the previous round's fixes answer, the scoring run only as "Architecture" → "Fresh" says. Where those fixes carry an architectural decision, the architect and the principles audit run again.
- **Architecture round.** The architecture pass's first round is the architect (unless skipped), the principles audit and the comprehension check, which "Architecture" → "Suspended" may skip. A later round relaunches each reviewer whose 🔴 or 🟡 the previous round's fixes answer, the comprehension check only as "Architecture" → "Fresh" says. Where those fixes carry an architectural decision, the architect and the principles audit run again.
- **Auto-reviewer.** The external auto-reviewer (§4) is a pass of its own, each review it posts a round.
- **Count.** Close each round with one commit that holds the round's fixes and is titled `Answer <pass> round <n>: …`, `<pass>` being the pass's name in lower case, such as `full scan`. A round that changed nothing gets an empty commit (`git commit --allow-empty`). Push the commits when §4 runs next, where there is a remote. A pass's count is the highest `<n>` among titles holding `Answer <pass> round <n>:` in `git log origin/<default branch>..HEAD` after a `git fetch`, or in `git log <default branch>..HEAD` with no remote. The auto-reviewer's count is the number of its reviews on the PR, less its reviews of a `Rescore:` commit.
- **Later rounds.** A rerun pass reviews only what changed since it last reviewed: `git diff <the commit it last reviewed, or else the commit before this round's fixes> HEAD`, since the fixes may be unpushed. The scoring run's seats still read the whole project. From round 2, prompt an agent under `~/.claude/agents/` with its own last round's findings verbatim too.
- **Later rounds.** A rerun pass reviews only what changed since it last reviewed: `git diff <the commit it last reviewed, or else the commit before this round's fixes> HEAD`, since the fixes may be unpushed. The comprehension check's seats still read the whole project. From round 2, prompt an agent under `~/.claude/agents/` with its own last round's findings verbatim too.
- **Done.** A pass is done when its latest round leaves no 🔴 or 🟡 standing: none was reported, or each was shown factually wrong, declined, filed, or named under the bar per "Findings", or, for the prose pass, fixed. Resolve that round's 🟢s and ❓s per "Findings" too.
- **Cap.** A pass takes at most three rounds per chunk. Instead of a fourth, list what is open and ask the user. For the auto-reviewer, ask before the push that would summon its fourth, and hold the fixes unpushed until the answer.
- **Answer.** A yes grants that pass one more round, after which the ask repeats; a no leaves the PR unmerged.
@@ -86,29 +86,38 @@ Launch systems-architect: review the system as a whole entered through this chan
Then the principles audit: one principles-auditor for the README's goals, and one per file in `~/.claude/principles/` whose `Applies to` line the project meets, judged from the README, `git ls-files` and the branch diff. A file without the line applies to every project. Unsure → launch it.
Then the comprehension floor, which the scoring run measures. Four simulated readers — Junior A, Mid A, the maintainability senior and the inherited architect — each read the whole project, not the diff, and score it from 1 to 10 overall and on four dimensions: Navigation, Locality, Shape and Self-sufficiency. A seat's overall may not exceed its lowest dimension by more than 1. Each score is the mean of the four seats, to two decimals. Launch the seats per `~/.claude/skills/comprehension-panel/SKILL.md` → "The scoring run".
Then the comprehension check: simulated readers, launched per `~/.claude/skills/comprehension-panel/SKILL.md`, each read the whole project, not the diff. It has two kinds of run.
Two files of the project keep the floor's records:
**Paired ruling**, in every architecture pass. Each seat votes whether the project reads better, the same or worse at this branch's head than at the base.
- **History.** `docs/comprehension-history.md` keeps every scoring run, appended and never edited. A run is a `## <UTC time, ISO 8601>, <PR or branch> at <commit>` heading naming when the seats finished and the commit they read; a table of each seat's five scores, with a last row holding the four seats' mean of each; and under each seat, labelled `<seat>, hardest first:` or, for the architect, `<seat>, would restructure first:`, its places, each as `path:line` and the unit's name. Appending a run re-enters no pass. Resolve a merge conflict in the file by putting the default branch's runs first.
- **Entry.** `docs/decisions.md` keeps one entry whose title opens `comprehension floor:`. It holds each decision to ship under the floor, with its scores.
- **Base.** The last measured commit: the newest commit on `origin/<default branch>`, after a `git fetch`, whose change to the history (below) adds a scoring run, or whose last added paired ruling is better. Read each merged change whole, a merge commit as one change: `git log --first-parent --diff-merges=first-parent -p origin/<default branch> -- docs/comprehension-history.md`; with no remote, use `<default branch>`. No such commit → the base the first paired ruling in that history names; no ruling either → the merge base of the head and the default branch.
- **Head.** Before each ruling, merge `origin/<default branch>` into this branch unless the head already holds it, so the seats see only this branch's change on top of the base.
- **Seats.** Mid A and the inherited architect. When their votes differ, the maintainability senior and Junior A vote too, launched together.
- **Ruling.** The vote that leads each other vote by two or more; without one, same. A better or worse vote that names no place counts as same.
A `comprehension floor:` entry written before the history may still hold the last scoring run's five scores. While the default branch's history holds no run, the ratchet compares with those scores. The next scoring run moves them, and only them, into the history as its first run: a heading of the date or time the entry gives and whatever it names the run by, its commit included where named, and a table with the mean alone. The rest of the entry stays.
**Scoring run**, when the chunk ships or removes the last plan item of a release that has work landed towards it, or ships an item naming the comprehension floor. Junior A, Mid A, the maintainability senior and the inherited architect each score the project at the head from 1 to 10 overall and on four dimensions: Navigation, Locality, Shape and Self-sufficiency. A seat's overall may not exceed its lowest dimension by more than 1. Each score is the mean of the four seats, to two decimals.
The chunk does not merge while either holds:
- **Floor.** Any dimension or the overall is below 7.0.
- **Ratchet.** Any score is below the mean of the history's last run as it stands on the default branch, even within the panel's noise. An equal score passes, and so does every run while the default branch holds neither a run in the history nor such scores in the entry.
- **Ratchet.** The paired ruling is worse.
- **Floor.** The scoring run puts any dimension or the overall below 7.0.
Each score under the floor or the ratchet is a 🔴 of the scoring run. It is never filed, and stands until a later scoring run clears it or, for a score under the floor, a decision to ship answers it.
Each is a 🔴 of the comprehension check. It is never filed, and stands until a later run clears it or, for the floor, a decision to ship answers it.
Fix the places the seats name under the lowest dimension and under each dimension that fell. When only the floor holds, the user may decide to ship instead. A decision that lets every chunk ship under the floor, held in the project's `docs/decisions.md` before the chunk began, counts as theirs while it stands. A decision to ship listed in the `comprehension floor:` entry covers only the chunk it names. No user reachable → fix. A fall never ships: after three rounds, "Round cap" → "Cap" asks the user for another round.
Fix a worse ruling at the places the seats voting worse name, and the floor at the places the seats name under the lowest dimension. Under the floor alone, the user may decide to ship instead. A decision that lets every chunk ship under the floor, held in the project's `docs/decisions.md` before the chunk began, counts as theirs while it stands. A decision to ship listed in the `comprehension floor:` entry covers only the chunk it names. No user reachable → fix. A worse ruling never ships: after three rounds, "Round cap" → "Cap" asks the user for another round.
Put each run's five scores in the summary, and append the run to the history on this branch. List in the entry each decision to ship under the floor, and file its places in `todo.md` per `~/.claude/CLAUDE.md` → "Todo items". A project without the entry gets it at its first scoring run, titled `comprehension floor: every dimension and the overall at 7.0 or above` and applying KISS (`~/.claude/principles/technical-principles.md`).
Two files of the project keep the records:
**Fresh.** Where the chunk ran a scoring run, the architecture pass is done only once its last scoring run measured the commit to merge. That run may predate commits that change only `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`. Any other commit after it, a prose fix included, because the seats read the docs, calls for a rescore: the scoring run again, once every other pass is done, the auto-reviewer included. A rescore reporting a 🔴 that no decision to ship answers is an architecture round. Any other rescore is not a round: append only its run to the history, in a commit titled `Rescore: …`. A `Rescore:` commit re-enters no pass: push it and wait for the forge's checks (§4 step 3), not for an auto-review.
- **History.** `docs/comprehension-history.md` keeps every paired ruling and scoring run, appended and never edited. Each opens with a `## <UTC time, ISO 8601>, <PR or branch> at <commit>` heading naming when the seats finished and the commit they read; a paired ruling's heading ends `, against <base>`. A paired ruling holds a `Ruling: <better, same or worse>` line, a table of each seat's vote, and under each seat, labelled `<seat>, decided by:`, its places. A scoring run holds a table of each seat's five scores, with a last row holding the four seats' mean of each, and under each seat, labelled `<seat>, hardest first:` or, for the architect, `<seat>, would restructure first:`, its places. A place is `path:line` and the unit's name. Appending re-enters no pass. Resolve a merge conflict in the file by putting the default branch's entries first.
- **Entry.** `docs/decisions.md` keeps one entry whose title opens `comprehension floor:`. It holds each decision to ship under the floor, with its scores.
**Suspended.** A decision in the project's `docs/decisions.md`, held before the chunk began, may name plan items to ship without a scoring run. A named item counts as `todo.md` worded it before the chunk began. The decision must name the later plan item whose chunk ends the suspension with a scoring run. A chunk may ship named items only with each other. Such a chunk skips the scoring run, so neither the floor nor the ratchet can block it, and its summary cites the decision. A chunk shipping its release's last item runs the scoring run, even when every item it ships is named.
A `comprehension floor:` entry written before the history may still hold a scoring run's five scores. The next scoring run moves them, and only them, into the history as its first run: a heading of the date or time the entry gives and whatever it names the run by, its commit included where named, and a table with the mean alone. The rest of the entry stays.
Put each ruling with its base and votes, and each run's five scores, in the summary, and append them to the history on this branch. List in the entry each decision to ship under the floor, and file its places in `todo.md` per `~/.claude/CLAUDE.md` → "Todo items". A project without the entry gets it at its first scoring run, titled `comprehension floor: every dimension and the overall at 7.0 or above` and applying KISS (`~/.claude/principles/technical-principles.md`).
**Fresh.** Where the chunk ran a comprehension check, the architecture pass is done only once its last paired ruling, and its last scoring run where it ran one, read the commit to merge. They may predate commits that change only `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`. Any other commit after them, a prose fix included, because the seats read the docs, calls for a rescore: the comprehension check again, once every other pass is done, the auto-reviewer included. A rescore reporting a 🔴 that no decision to ship answers is an architecture round. Any other rescore is not a round: append only its results to the history, in a commit titled `Rescore: …`. A `Rescore:` commit re-enters no pass: push it and wait for the forge's checks (§4 step 3), not for an auto-review.
**Suspended.** A decision in the project's `docs/decisions.md`, held before the chunk began, may name plan items to ship without a comprehension check. A named item counts as `todo.md` worded it before the chunk began. The decision must name the later plan item whose chunk ends the suspension with a comprehension check. A chunk may ship named items only with each other. Such a chunk skips the comprehension check, so neither the ratchet nor the floor can block it, and its summary cites the decision. A chunk that meets the scoring run's trigger runs it, even when every item it ships is named.
### Security
+12 -7
View File
@@ -6,7 +6,7 @@
`Bar = 9`
`Next ID = 84`
`Next ID = 86`
| Goal | W |
|---|---|
@@ -24,10 +24,10 @@
| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |
|---|---|---|---|---|---|---|---|---|---|
| 75 | 0.1.0 | decision | **Decide the ratchet by a paired run: Mid A and the inherited architect read the whole project at the default branch and at the PR head, and rule whether it reads better, the same or worse once the PR lands; add seats only until the votes settle.** | 5 | 5 | 10 | 9 | 1.1, 1.4 | 31.6 |
| 73 | 0.1.0 | | **Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries.** | 3 | 4 | 10 | 8 | 1.4 | 31.4 |
| 77 | 0.1.0 | | **Measure each project with cheap static metrics — cognitive complexity, how far a change spreads, state shared or mutated across methods — and keep only those the panel's history shows tracking comprehension.** | 3 | 7 | 9 | 8 | 1.1, 1.4 | 29.1 |
| 78 | 0.1.0 | | **Generate comprehension questions with deterministic scripts — from symbols, the call graph and the tests, each answer checkable by a tool — and bench the scores an agent gets on them against the panel, reading cost and static metrics.** | 3 | 7 | 9 | 8 | 1.1, 1.4 | 29.1 |
| 85 | 0.1.0 | decision | **Move larv-review's mechanical rules — the paired ruling's base and vote count, each pass's round count, the scoring run's trigger and the comprehension history's format — into one tested tool under `bin/`, so the skill says only when to run it and what each answer means.** | 4 | 7 | 8 | 9 | 1.1 | 28.1 |
| 1 | 0.1.0 | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 3 | 5 | 8 | 9 | 3, 4 | 27.0 |
| 62 | 0.1.0 | defect | **Lift Locality, Self-sufficiency and the overall to the comprehension floor, at the places the seats name.** | 4 | 6 | 6 | 8 | 1.1 | 22.3 |
| 21 | 0.1.0 | | **Give `settings.json`'s model one writer, and sort its keys.** | 3 | 3 | 6 | 7 | 1.3 | 21.8 |
@@ -37,6 +37,7 @@
| 80 | 0.1.0 | | **State in README → Install that every machine taking this checkout must be a sandbox, as `settings.json`'s auto mode assumes.** | 1 | 1 | 5 | 5 | 4 | 16.8 |
| 83 | 0.1.0 | | **Group `bin/` by tool, so each tool's modules and tests sit together.** | 3 | 4 | 4 | 6 | 1.1 | 15.9 |
| 5 | 0.1.0 | decision | **Key a project on its checkout's full path, and refuse to start while another session uses that key.** | 5 | 5 | 5 | 6 | 1.1 | 15.6 |
| 84 | 0.1.0 | | **Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.** | 2 | 2 | 4 | 5 | 1.4 | 15.1 |
| 67 | 0.1.0 | principle | **Point stability-reviewer at the technical principles in place of its own restatement of them.** | 2 | 3 | 4 | 5 | 1.3 | 14.9 |
| 66 | 0.1.0 | principle | **Hold systems-architect's method thresholds only while the comprehension panel finds the flagged code hard.** | 3 | 3 | 4 | 5 | 1.1 | 14.2 |
| 69 | 0.1.0 | principle | **Measure test coverage, so a chunk that lowers it is seen.** | 3 | 4 | 4 | 5 | 1.1 | 13.9 |
@@ -53,10 +54,6 @@
## Details
### 75. Decide the ratchet by a paired run: Mid A and the inherited architect read the whole project at the default branch and at the PR head, and rule whether it reads better, the same or worse once the PR lands; add seats only until the votes settle.
The human ranks review cost first, and wants the project as a whole measured, since comprehension drifts across chunks that each look fine alone. Today the ratchet compares two independent absolute runs of four seats, whose scores move up to about 0.4 between runs on near-identical code, so a fall within that noise forces rounds that move nothing. A seat comparing the two commits judges the difference on the same material, so the noise the commits share cancels. Start with two seats; add one at a time until a sign test settles better or worse, and call it the same if four seats leave it unsettled. The four-seat scoring run still checks the floor, once, on each release's last item. Append each paired ruling to `docs/comprehension-history.md`, with each seat's vote and the places it named.
### 73. Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries.
The human ranks review cost first. Across 54 archived chunks the worker's own thread was 54% of spend, mostly cache reads: its context averaged about 350k tokens a turn and peaked near 700k, filled by reviewer reports, each of which arrived twice (the spawn result and the task notification).
@@ -78,6 +75,10 @@ The human wants these explored and benched against the panel, with a global push
A hand-written question bank goes stale as the project grows and leaves everything it does not ask about unmeasured. A script that samples symbols at random each run ("what calls X?", "where is Y validated?", "which test fails if Z changes?") covers the whole project over time, and the call graph, grep and the tests hold the answers, so no judge is needed. Each answer names the code it rests on, so a question whose code changed is regenerated. Keep the method only where the bench shows it tracks the panel at a lower cost.
### 85. Move larv-review's mechanical rules — the paired ruling's base and vote count, each pass's round count, the scoring run's trigger and the comprehension history's format — into one tested tool under `bin/`, so the skill says only when to run it and what each answer means.
2026-10-08, lilleman: next after the items that lower token use (73, 77, 78). Every comprehension run since PR #224 named `skills/larv-review/SKILL.md` §3 "Architecture" the hardest place, and PR #228's paired ruling ruled worse on rules a program should hold: the base's `git log` search and fallbacks, the vote arithmetic, a trigger written out three times, and the round count PR #225's seats said only prose enforces. PR #228's reviewers found about twenty edge cases in that text, among them squash against merge commits, entry order, a vote with no place, and a trigger judged before items are removed; tests can hold each, prose could not. Keep the skill's judgement calls in prose, and give the tool the history's format, so the base search does not hang on the wording of a line.
### 1. Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.
It lives under `~/.claude`, which `.gitignore` keeps from the remote, so a lost disk loses every unattended run's record. Back it up only to a destination private to me and encrypted at rest: the worker runs with permissions bypassed, so the archived tool results can hold credentials it read.
@@ -124,6 +125,10 @@ The key names the project's session, archive folder and models entry. `project_k
`bin/reading-cost` keys the archive by the main checkout, the parent of git's common dir, since it runs in a chunk's worktree; `loop-claude` keys it by `--show-toplevel`, so a loop started inside a worktree archives where `reading-cost` never looks. Derive both keys one way.
### 84. Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.
Raised by the principles audit of PR #228: the `review: a paired ruling …` entry in `docs/decisions.md` is valid only while that holds, and `docs/comprehension-history.md` keeps every vote, yet nothing reads the agreement rate. Scoring runs on near-identical code are rare now, so compare with those already in the history, such as PR #224's and PR #225's, or make a second run for the check.
### 67. Point stability-reviewer at the technical principles in place of its own restatement of them.
DRY, in `principles/technical-principles.md`: `agents/stability-reviewer.md` restates principles in its own words, and the copy has drifted: it asks for "properly randomised UUIDs" where the principle requires UUID v7.
@@ -138,7 +143,7 @@ KISS, in `principles/technical-principles.md`, which keeps a limit only while re
### 29. Make the contributor a standing member of every project's audience, unless its `## Audience` says maintainability does not matter.
State that in `CLAUDE.md`'s goals-and-audience rule, and have larv-review's comprehension floor name the contributor as the reader it serves. The floor then gates every chunk while the contributor stands, and a project that drops them skips it.
State that in `CLAUDE.md`'s goals-and-audience rule, and have larv-review's comprehension floor name the contributor as the reader it serves. The floor's block then applies to a project whose audience includes the contributor, and a project that drops them skips it.
### 68. Test `workflow-bench/compare.py`'s blinding and `bench.py`'s run plan.
+1 -1
View File
@@ -16,7 +16,7 @@ Report: write it to `{report}`.
- Launch every agent in the foreground (`run_in_background: false`): background agents die when this headless turn ends.
- No remote and no forge: skip the PR, CI, the external auto-reviewer and the merge; review locally.
- Do not run the repo's test suites. A targeted probe is fine, in the repo's own container tooling.
- Skip the comprehension floor's scoring run; say so in the report.
- Skip the comprehension check; say so in the report.
- No human is reachable. Where the process names a no-human branch, take it; where it names none, take your recommendation and list the question in the report.
- Commit each round's fixes on `chunk`, messages prefixed `{case} - review: `. Touch nothing outside this repository except the report.