52 KiB
todo
Scoring
Score = 100 * V / (2**S + 8)
Bar = 0.1
Next ID = 168
Items
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 166 | 0.2.0 | question | Decide whether docs/decisions.md names rule-hook's refusal of a review launched below its rung, in routing: and in rule-hook:. |
1 | 2 | 1 | 2 | 1 | 1.2 by 3 | 4.4 | |
| 152 | 0.2.0 | decision | Anchor each score level in rules/todo-items.md with words, and each project's with two or three items it shipped, and have a scorer compare a new item against them. |
3 | 2 | 3 | 6 | 4 | 1.1 by 7 | 6.4 | |
| 50 | 0.2.0 | principle | Grant the bench's runs only the tools a review needs, in place of skipping every permission. | 2 | 2 | 1 | 3 | 2 | 1.1 by 3 | 5.5 | |
| 65 | 0.2.0 | defect | Archive each judge reply the loop rejects. | 2 | 1 | 3 | 4 | 4 | 3 by 3 | 5.5 | |
| 153 | 0.2.0 | decision | Route the judge and the reviewers on the highest E + I in their chunk, an item the worker takes in included. | 3 | 1 | 3 | 5 | 4 | 1.2 by 5 | 3.0 | |
| 161 | 0.2.0 | Have loop-claude run a preflight script before each fresh worker and put its findings in the worker's prompt, so the worker no longer spends tool calls finding the chunk's mechanical facts. | 4 | 2 | 3 | 7 | 6 | 1.1 by 4, 1.4 by 6, 1.5 by 6 | 2.9 | ||
| 84 | 0.2.0 | Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes. | 2 | 2 | 2 | 4 | 4 | 1.4 by 5 | 2.3 | ||
| 73 | 0.2.0 | Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries. | 3 | 2 | 3 | 6 | 5 | 1.4 by 4 | 1.1 | ||
| 159 | 0.2.0 | decision | Run each review pass on the lowest rung that a replay of its archived rounds shows finding every 🔴 and 🟡 the project's rung finds, even below the worker's rung. | 4 | 2 | 3 | 7 | 6 | 1.4 by 7 | 1.1 | |
| 80 | 0.3.0 | defect | State in README → Install that every machine taking this checkout must be a sandbox, as settings.json's auto mode assumes. |
1 | 2 | 1 | 2 | 1 | 2 by 4 | 17.5 | |
| 21 | 0.3.0 | defect | Give settings.json's model one writer, and sort its keys. |
3 | 3 | 3 | 7 | 3 | 2 by 5 | 13.7 | |
| 29 | 0.3.0 | Make the contributor a standing member of every project's audience, unless its ## Audience says maintainability does not matter. |
2 | 2 | 3 | 5 | 2 | 1.1 by 5, 1.4 by 5 | 13.7 | ||
| 119 | 0.3.0 | Retitle the comprehension-floor decision so it says a chunk lifting a project towards the floor works first on its lowest-scoring dimension, and that the floor blocks nothing. | 1 | 2 | 1 | 2 | 1 | 1.1 by 4, 1.4 by 4 | 13.1 | ||
| 127 | 0.3.0 | Define seat, place, paired ruling, scoring run and the judge's records in CLAUDE.md → "Vocabulary". |
1 | 2 | 3 | 4 | 1 | 1.1 by 4, 1.4 by 4 | 13.1 | ||
| 129 | 0.3.0 | Explain bin/rule_hook/rules.py's OVERRIDE pattern beside it: why the reason is lazy, what the lookahead splits and why --> is stripped. |
1 | 2 | 1 | 2 | 1 | 1.1 by 4, 1.4 by 4 | 13.1 | ||
| 114 | 0.3.0 | decision | Run git through git_repo.py in rule_hook, metric_bench, reading_cost, stability_replay and loop_claude, keeping rule-hook's guards as an option, so one module owns how a tool runs git. |
3 | 1 | 3 | 5 | 4 | 1.1 by 9, 1.4 by 9 | 12.3 | |
| 120 | 0.3.0 | Rewrite larv-review §3's "Round cap" and "Re-entry" as one table that says, for each kind of fix, which passes rerun and what a round reviews. | 2 | 2 | 3 | 5 | 4 | 1.1 by 6, 1.2 by 6, 1.4 by 6 | 11.9 | ||
| 49 | 0.3.0 | principle | Check that AGENTS.md lists exactly the titles of docs/decisions.md. |
1 | 1 | 1 | 1 | 2 | 1.1 by 6 | 10.9 | |
| 117 | 0.3.0 | Give todo_table.items one way to say a plan has no items table, so no caller tells it from an empty plan by truthiness. |
1 | 1 | 3 | 3 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 126 | 0.3.0 | State when work has landed towards a release once, in bin/larv_review/plan.py, and have larv-review's help point to it. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 131 | 0.3.0 | Describe the transcript archive's layout in bin/transcript_archive/, which owns it, and have loop-claude's help point there. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 133 | 0.3.0 | Show, beside bin/rule_hook/override_log.py's this_chunks, one example per case it decides: an amended or rebased commit, a renamed branch, and a commit on the default branch. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 134 | 0.3.0 | Name in bin/shell_command.py's segments the shell forms it does not parse, so a caller knows what it misses. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 154 | 0.3.0 | defect | Have bin/shell_command.py's segments keep a command's words together across a redirect, so git > out commit reads as one git commit. |
1 | 1 | 3 | 3 | 2 | 1.1 by 6 | 10.9 | |
| 157 | 0.3.0 | Split docs/decisions.md → todo.md: one table of items …'s 2026-10-02 paragraph into one paragraph per choice. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 106 | 0.3.0 | Make larv-review's "Do not over-read a clean result" cite a rule that says Renovate or Dependabot owns dependency CVEs, or drop the citation. | 1 | 2 | 1 | 2 | 1 | 1.1 by 5 | 10.9 | ||
| 167 | 0.3.0 | Refuse a /security-review or /claude-security Skill call below the project's rung, as rule-hook refuses such an Agent call. |
3 | 2 | 3 | 6 | 2 | 1.1 by 4 | 7.3 | ||
| 82 | 0.3.0 | principle | Sort AGENTS.md's decision titles and docs/decisions.md's entries by title. |
2 | 2 | 1 | 3 | 4 | 1.1 by 4, 1.4 by 4 | 5.5 | |
| 162 | 0.3.0 | owner | Script each step that 0.2.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release. | 2 | 2 | 1 | 3 | 3 | 1.4 by 4, 1.5 by 4 | 4.9 | |
| 165 | 0.3.0 | Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant. | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, 1.4 by 3 | 3.0 | ||
| 122 | 0.4.0 | Assemble the judge's prompt in bin/loop_claude/cycle/judge.py, importing only wording the worker also sees, and call the worker "the worker" in it. |
2 | 2 | 3 | 5 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 | ||
| 158 | 0.4.0 | decision | Measure the fixed cost of a chunk from loop-claude's archive, in tokens and hours against its items' S, and set F in Score = 100 * V / (2**S + F) to it. |
2 | 2 | 1 | 3 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 | |
| 116 | 0.4.0 | Report L by the share of chunks with a 🔴, once 10 chunks that each shipped one item record finding counts and most record no 🔴. | 1 | 1 | 2 | 2 | 2 | 1.1 by 5 | 9.1 | ||
| 121 | 0.4.0 | Check that bin/round_titles.py's PASSES lists exactly the passes of larv-review §3's table. |
1 | 1 | 1 | 1 | 2 | 1.1 by 5 | 9.1 | ||
| 112 | 0.4.0 | Count an item as shipped only where its chunk built it, so an item a chunk dropped from the plan gets no outcome. | 3 | 2 | 2 | 5 | 3 | 1.1 by 6 | 8.2 | ||
| 130 | 0.4.0 | Move the vote counting out of bin/larv_review/ruling.py's votes, so reading a votes file decides nothing. |
2 | 1 | 2 | 3 | 3 | 1.1 by 4, 1.4 by 4 | 8.2 | ||
| 125 | 0.4.0 | decision | Give "record", "outcome" and "places" one meaning each across bin/, and rename bin/larv_review/git.py by what it holds. |
3 | 1 | 3 | 5 | 5 | 1.1 by 9, 1.4 by 9 | 7.4 | |
| 81 | 0.4.0 | principle | Have workflow-bench/bench.py return each run's failure as a value, so one failed clone does not hide the later runs' results. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4 | 7.3 | |
| 102 | 0.4.0 | Give loop-claude's retry delay an environment override, so the black-box test stops replacing time.sleep through sitecustomize. |
1 | 1 | 2 | 2 | 2 | 1.1 by 4 | 7.3 | ||
| 66 | 0.4.0 | principle | Hold systems-architect's method thresholds only while the comprehension panel finds the flagged code hard. | 2 | 2 | 3 | 5 | 3 | 1.1 by 5 | 6.8 | |
| 101 | 0.4.0 | principle | Have larv_review/record.py's read_seats take the time, the commit check and the history as inputs, so it parses and decides without effects. |
2 | 1 | 2 | 3 | 3 | 1.1 by 5 | 6.8 | |
| 111 | 0.4.0 | Measure the chunks of a project that merges by squash or fast-forward, which score-outcomes skips today. |
3 | 2 | 2 | 5 | 3 | 1.1 by 5 | 6.8 | ||
| 113 | 0.4.0 | Compute an item's S from the code it names, where it names any, and show it beside the finished items score-outcomes lists. |
3 | 2 | 2 | 5 | 4 | 1.1 by 5, 1.4 by 5 | 6.8 | ||
| 137 | 0.4.0 | Archive a chunk's route as a record loop-claude reads back field by field, in place of the Worker: prose ROUTED parses, and name stepped and the judge's bundled by what they do. |
4 | 2 | 3 | 7 | 4 | 1.1 by 4, 1.4 by 4 | 5.5 | ||
| 110 | 0.4.0 | Count, for each finished chunk, the later changes that fixed code it wrote, and show them beside its guessed E. | 3 | 2 | 2 | 5 | 5 | 1.1 by 6, 1.4 by 6 | 4.9 | ||
| 139 | 0.4.0 | Split bin/routing_figures/measure.py by concern, and keep the figures' measured cycle and each raised call as fields loop-claude reads back, not as the prose it renders. |
3 | 1 | 2 | 4 | 5 | 1.1 by 5, 1.4 by 5 | 4.1 | ||
| 149 | 0.4.0 | decision | Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release. | 4 | 2 | 3 | 7 | 5 | 1.1 by 7 | 3.8 | |
| 12 | 0.5.0 | Run the test suite in CI on every PR, or record in docs/decisions.md why the local run is enough. |
3 | 2 | 1 | 4 | 4 | 1.1 by 6 | 5.5 | ||
| 105 | 0.5.0 | decision | Compare review rounds per pass and reading cost for the five chunks before and after the worker: decision moved the principles out of the worker's context, and file an item if either rose. |
1 | 2 | 1 | 2 | 2 | 1.4 by 6 | 5.5 | |
| 135 | 0.5.0 | Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop. | 2 | 2 | 3 | 5 | 3 | 1.1 by 4 | 5.5 | ||
| 68 | 0.5.0 | principle | Test workflow-bench/compare.py's blinding and bench.py's run plan. |
2 | 1 | 1 | 2 | 4 | 1.1 by 5 | 4.6 | |
| 4 | 0.5.0 | Name each chunk's archive folder by the chunk's start time, so <key>/ lists chunks in the order they ran. |
3 | 2 | 3 | 6 | 3 | 1.1 by 3 | 4.1 | ||
| 59 | 0.5.0 | Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end. | 2 | 1 | 3 | 4 | 3 | 1.4 by 6 | 4.1 | ||
| 123 | 0.5.0 | Have bin/comprehension_history.py parse each history entry once into seats, votes and places, so latest_places, worse_places and worse_votes neither depend on table order nor split a label to find a seat. |
2 | 1 | 2 | 3 | 5 | 1.1 by 5, 1.4 by 5 | 4.1 | ||
| 132 | 0.5.0 | Lift each dimension still under the comprehension floor after release 0.2.0's scoring run, at the places that run names. | 3 | 2 | 2 | 5 | 5 | 1.1 by 5, 1.4 by 5 | 4.1 | ||
| 124 | 0.5.0 | decision | Group the shared modules at bin/'s root into packages named by purpose, so the root holds only launchers and stand-alone tools. |
4 | 1 | 3 | 6 | 6 | 1.1 by 9, 1.4 by 9 | 4.1 | |
| 156 | 0.5.0 | decision | Drop rank_todo.py's ranking of an old-shape table once no project under ~/code holds one. |
2 | 1 | 3 | 4 | 4 | 1.1 by 4 | 3.6 | |
| 155 | 0.5.0 | Lower the reading cost of the files a chunk reads most before its first commit. | 3 | 2 | 2 | 5 | 5 | 1.4 by 6 | 1.6 | ||
| 1 | 0.6.0 | question | Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume. | 1 | 2 | 1 | 2 | 1 | 0.0 | ||
| 6 | 0.6.0 | decision | Move what loop-claude writes out of ~/.claude.json, and ask the human about each write that must stay. |
3 | 3 | 3 | 7 | 3 | 1.1 by 5 | 6.8 | |
| 5 | 0.6.0 | decision | Key a project on its checkout's full path, and refuse to start while another session uses that key. | 4 | 2 | 3 | 7 | 5 | 1.1 by 6 | 3.3 | |
| 77 | 0.6.0 | Teach bin/metric-bench Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel. |
4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 3.2 | ||
| 51 | 0.6.0 | principle | Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints. | 2 | 1 | 3 | 4 | 5 | 1.1 by 5 | 2.7 | |
| 151 | 0.6.0 | decision | Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes. | 3 | 2 | 2 | 5 | 5 | 1.2 by 7 | 2.6 | |
| 52 | 0.6.0 | principle | Record why this checkout lives in Claude Code's own directory, or move it out and link it in. | 4 | 3 | 4 | 9 | 6 | 1.1 by 4 | 1.2 |
Details
166. Decide whether docs/decisions.md names rule-hook's refusal of a review launched below its rung, in routing: and in rule-hook:.
Item 163 (PR #295) has rule-hook refuse a review launch below the rung loop-claude gives its worker. The routing decision still says loop-claude's prompt has the worker launch reviewers, comprehension seats and the security pass on the project's rung, and does not name the hook. Recommendation: amend that paragraph to say rule-hook refuses a launch below the rung too. The decision was held before the chunk began, so the human decides. The rule-hook: entry's title says a marker overrides one rule, and reviewer-rung takes none. The amendment need not give the reason no marker waives the rule: the comment above reviewer-rung in bin/rule_hook/rules.py records it.
152. Anchor each score level in rules/todo-items.md with words, and each project's with two or three items it shipped, and have a scorer compare a new item against them.
LLM ratings drift to the middle of an unanchored scale, and worked examples cut estimation error. A project's anchors are shipped items, cited by ID and commit, chosen from what score-outcomes measured for them.
50. Grant the bench's runs only the tools a review needs, in place of skipping every permission.
"Always grant explicit access to data", in principles/technical-principles.md: workflow-bench/bench.py starts claude with --dangerously-skip-permissions, and no rule confines it to a sandbox, so a run on the host reaches the host's credentials and other checkouts.
65. Archive each judge reply the loop rejects.
A defect: goal 3, keeping unattended runs' transcripts backed up, does not hold while a rejected judge reply reaches only stderr. Raised by the principles audit of PR #215: the judge runs with --no-session-persistence, so the archive cannot answer the 3am question of why the loop rejected a verdict.
153. Route the judge and the reviewers on the highest E + I in their chunk, an item the worker takes in included.
The judge reports the L of an item the worker takes in, and routing.stepped lifts the worker to the rung it picks, but the judge and the reviewers route on the routed item's E + I alone, which the archived route line holds. So when the worker takes in an item whose defects would escape further or break more, the judge and the reviewers stay on a lower rung than it needs: a missed interrupt lets the worker settle a decision that is lilleman's (goal 1.2), and a weaker review lets a defect through (goal 1.1). The judge must report the highest E + I taken in too, and the route must carry it, archived so a rerun keeps it; loop-claude then hands the reviewers' rung it picks to the worker's prompt and environment.
161. Have loop-claude run a preflight script before each fresh worker and put its findings in the worker's prompt, so the worker no longer spends tool calls finding the chunk's mechanical facts.
In item 140's chunk, the worker spent 5 min 46 s of 30 min, 16 of its 68 messages and 226k of its 491k fresh tokens before its first edit, through 12 Bash and 5 Read calls. About half of those found facts a program computes the same way in every chunk:
- the open PR and worktree for the chunk, the last tag and how many changes are unreleased;
- the next release, the first plan item that does not wait, its L, its E + I and its detail, after
rank_todo.py --check; - the items that may bundle with it under
CLAUDE.md→ "Working loop", "Work one chunk at a time, and make it as large as one purpose allows"; - what
larv-review needs-scoring-run,reading-costandlarv-review placesprint; - a worktree off a freshly fetched
origin/main.
The preflight also lists the files and components the item's detail names, for the worker to judge the item's size from.
Compute these in one tool and give loop-claude's prompt its output as facts, so the worker reads them rather than finding them. The reading the chunk needs, such as the decision an item changes, stays the worker's. After its first read of the code, the worker counts the item's size again: the lines, files and components it will change. Where that count would change the item's L, a subagent on the project's rung re-scores the item. A re-score whose L picks a rung above the worker's is escalation trigger 1. Measure the next few chunks' time and fresh tokens before the first edit against item 140's figures. The comprehension seats found the prompt assembly in bin/loop_claude/cycle/prompts.py hard: put the preflight's facts in a function of their own there.
84. Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.
Raised by the principles audit of PR #228: the review: the comprehension check records … entry in docs/decisions.md is valid only while that holds, and docs/comprehension-history.md keeps every vote, yet nothing reads the agreement rate. Scoring runs on near-identical code are rare now, so compare with those already in the history, such as PR #224's and PR #225's, or make a second run for the check.
73. Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries.
2026-10-08, lilleman: rescored from A10/G8 to A4/G4 on the measurement below, so other items that lower token use go first.
Measured over the 68 transcripts in loop-claude's archive on 2026-10-08, matching each Agent call to its tool result and to any <task-notification> naming its tool-use id. No report reached the worker twice: 813 arrived once as the hand-back, and 363 once as a notification after Claude Code ran the agent in the background. Reports took a median 19k tokens per chunk, 8% of the worker's peak context (24% at most). Lines carrying a severity mark are 9–24% of a reviewer's report, and the worker would still open the file for each 🔴 or 🟡 it fixes. A third of the report volume is the comprehension seats', whose votes and places the worker copies into the history. File reads fill more of the worker's context: sed -n output is 30% of its Bash results and grep output 11%, which reading-cost targets.
The reports would live in the session's scratchpad, such as <scratchpad>/larv-review/prose-round-1.md, so no project's checkout sees them. A later round gets the path of that pass's earlier report in place of the findings pasted in. How a report reaches the file:
- Write access. Grant each reviewer Write. No reviewer under
agents/holds it today, an agent's tool list cannot confine Write to one folder, and~/.claude/CLAUDE.md→ "Working with the codebase" denies edits to a reviewer that only reads. Check whether a PreToolUse hook can tell a subagent's call from the worker's and deny a Write outside the scratchpad. - A wrapper. A general-purpose subagent launches the reviewer, saves the report and returns only the summary lines. Claude Code's documentation says a subagent cannot start another; drop this route unless a test shows one can.
This risks goal 1.1: a worker fixing from a one-line summary can miss context it needs.
159. Run each review pass on the lowest rung that a replay of its archived rounds shows finding every 🔴 and 🟡 the project's rung finds, even below the worker's rung.
lilleman asked, on 2026-10-16, for the prose pass to run on the worker's rung or lower, and the same day, answering PR #296, for every pass to drop below the project's rung only where a replay shows it may. Today bin/loop_claude/cycle/prompts.py's review_launches keeps every reviewer at or above the project's rung whatever the item's E + I, so a docs-only chunk such as item 140's, routed to Haiku 5.5, runs its reviewers on Opus 5.5.
Whether a pass runs cheaper without lowering quality (goal 1.4 against 1.1) is measured, not assumed, as docs/decisions.md → routing: the stability reviewer stays on the project's rung… measures the stability reviewer: replay a pass's archived rounds on the code each reviewed, first on the project's rung, then on each rung below it, and take the lowest rung that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again. A missed prose finding breaks no behaviour and a later change fixes it cheaply, so replay the prose pass first. Generalise bin/stability-replay to take the pass, in place of a replay tool per pass. Record each run and the rung it chose in a routing: decision, valid until a new model joins the ladder or the pass's prompt changes. Give each pass a row of its own in review_launches' table of review kinds, rather than a branch.
80. State in README → Install that every machine taking this checkout must be a sandbox, as settings.json's auto mode assumes.
A defect: goal 2, keeping the setup shareable between computers, does not hold while a new machine is never told it must be a sandbox. Raised by the principles audit of PR #224. The settings.json decision is valid while every machine taking this checkout is a sandbox, but Install asks for a sandbox only for loop-claude.
21. Give settings.json's model one writer, and sort its keys.
A defect: goal 2, keeping the setup shareable between computers, does not hold while one machine's model choice overwrites every other's. Claude Code rewrites "model" in this tracked file whenever a model is picked, so one /model dirties the checkout and overrides the synced choice.
Every write Claude Code makes to the file rewrites "model", claude plugin install included: Claude Code 2.1.288 turned "opus" into "opus[1m]". Each write also puts the keys back in Claude Code's own order, so a sorted file stays sorted only until the next write.
29. Make the contributor a standing member of every project's audience, unless its ## Audience says maintainability does not matter.
State that in ~/.claude/rules/goals-and-audience.md → "Goals and Audience", and have larv-review's comprehension check name the contributor as the reader it serves. The check then runs in a project whose audience includes the contributor, and a project that drops them skips it.
119. Retitle the comprehension-floor decision so it says a chunk lifting a project towards the floor works first on its lowest-scoring dimension, and that the floor blocks nothing.
Today's title, "a project lifting itself to the comprehension floor starts at the lowest dimension", reads as either where the project scores or where work begins. Beside the title saying the check blocks no merge, a reader cannot tell whether Floor: means stop; a seat named AGENTS.md:5 among the places it found hard. "First" means within such a chunk, not across a release: since the decision's 2026-10-16 line, items that lift the lowest dimension compete with the others like any item.
127. Define seat, place, paired ruling, scoring run and the judge's records in CLAUDE.md → "Vocabulary".
Two seats of PR #257's scoring run assembled these terms from four files: bin/larv_review/main.py's help, skills/larv-review/SKILL.md:99, bin/comprehension_history.py:82 and skills/comprehension-panel/seat-prompts.md. "Records" is defined only in a comment at bin/loop_claude/project/records.py:6.
129. Explain bin/rule_hook/rules.py's OVERRIDE pattern beside it: why the reason is lazy, what the lookahead splits and why --> is stripped.
The maintainability senior of PR #257's scoring run named bin/rule_hook/rules.py:49 OVERRIDE: nothing in the code explains the pattern. Make OVERRIDE easier to read.
114. Run git through git_repo.py in rule_hook, metric_bench, reading_cost, stability_replay and loop_claude, keeping rule-hook's guards as an option, so one module owns how a tool runs git.
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per docs/decisions.md → larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension.
The comprehension seats on PR #254 found five git runners with different contracts: git_repo.git, rule_hook/git.py's guarded one, metric_bench/main.py's, reading_cost/main.py's located and loop_claude/effects.py's. PR #276 added a sixth, stability_replay/main.py's git. The inherited architect of PR #257's scoring run named bin/git_repo.py:8 git again. Make git_repo.git easier to read.
120. Rewrite larv-review §3's "Round cap" and "Re-entry" as one table that says, for each kind of fix, which passes rerun and what a round reviews.
Junior A, Mid A and the maintainability senior of PR #257's scoring run named skills/larv-review/SKILL.md:63 "Round cap" hardest, two of them with :74 "Re-entry"; the inherited architect named :71 "Cap". A reader holds the trigger table's "a fix, only where …" clauses, the cap, later rounds' scope and the prose, 🟢 and ❓ exceptions at once. Junior A could not say what a fix to a stability 🟡 that also touches a comment reruns. Make those sections easier to read.
49. Check that AGENTS.md lists exactly the titles of docs/decisions.md.
DRY, in principles/technical-principles.md: the list is copied by hand, and sessions work from the copy.
117. Give todo_table.items one way to say a plan has no items table, so no caller tells it from an empty plan by truthiness.
The comprehension seats on PR #254 found items returning a list, None or a Failure, and larv_review/plan.py plan_reasons reading None through items(before) or [], which works only because a Failure is truthy.
126. State when work has landed towards a release once, in bin/larv_review/plan.py, and have larv-review's help point to it.
Two seats of PR #257's scoring run named bin/larv_review/plan.py:9 scoring_run_due. It reads the default branch's plan and the chunk's together, and the landed rule is written twice: in its docstring and in bin/larv_review/main.py's help. Make scoring_run_due easier to read.
131. Describe the transcript archive's layout in bin/transcript_archive/, which owns it, and have loop-claude's help point there.
Two seats of PR #257's scoring run named bin/transcript_archive/write.py:13 archive. Two invariants are explained only in loop-claude's help, bin/loop_claude/loop/main.py: folders are named by a hash of the transcript's first timestamped line, and a copy is replaced only by one grown from it. Make archive easier to read.
133. Show, beside bin/rule_hook/override_log.py's this_chunks, one example per case it decides: an amended or rebased commit, a renamed branch, and a commit on the default branch.
Junior A, Mid A and the maintainability senior of PR #257's scoring run named bin/rule_hook/override_log.py:36 this_chunks: which overrides belong to the chunk rests on reflog behaviour a reader must trust or work out. Make this_chunks easier to read.
134. Name in bin/shell_command.py's segments the shell forms it does not parse, so a caller knows what it misses.
Junior A and Mid A of PR #257's scoring run named bin/shell_command.py:31 segments: a hand-rolled tokenizer with dense regexes and a second lexing path for unbalanced quotes, which the rule hook and reading-cost both depend on. Its comments state each quirk but not its limits. Make segments easier to read.
154. Have bin/shell_command.py's segments keep a command's words together across a redirect, so git > out commit reads as one git commit.
segments('echo hi > f arg') returns arg as a command of its own, and segments('git > /tmp/x commit -m hi') returns git and commit -m hi apart, though the shell runs one git commit. rule-hook's commit rules read the commands segments finds, so such a commit escapes commit-one-line, no-ai-attribution, round-title-counts and todo-rank-check. The comprehension seats' scoring run in PR #280 found it.
157. Split docs/decisions.md → todo.md: one table of items …'s 2026-10-02 paragraph into one paragraph per choice.
Raised by PR #283's prose pass. The paragraph runs about 330 words and holds six choices: the bar, the purge, re-raising findings, the gate check, the principle exemption and the 2026-10-08 weights note. A reader after one of them reads all six.
106. Make larv-review's "Do not over-read a clean result" cite a rule that says Renovate or Dependabot owns dependency CVEs, or drop the citation.
Found by the goals audit of PR #245. skills/larv-review/SKILL.md §3 → "Security" says Renovate or Dependabot owns dependency CVEs per CLAUDE.md → "Working with the codebase", which says nothing about either.
167. Refuse a /security-review or /claude-security Skill call below the project's rung, as rule-hook refuses such an Agent call.
rule-hook checks Agent calls (item 163) but no Skill calls. A worker that invokes /security-review in its own context, instead of in a subagent, runs the pass on its own rung, and nothing blocks it. Check Skill calls too: breaches_of in bin/rule_hook/main.py takes a Skill call whose skill is security-review or claude-security as a review launch, and settings.json's PreToolUse matcher gains Skill. Found by the systems-architect's review of PR #295.
82. Sort AGENTS.md's decision titles and docs/decisions.md's entries by title.
"Deterministic order", in principles/technical-principles.md, raised by the principles audit of PR #224: their order carries no meaning, so two chunks adding a decision conflict at the end, and a reader cannot find a title by scanning.
162. Script each step that 0.2.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.
lilleman asked, on 2026-10-16, for a standing audit each release, so the loop keeps moving work a program does as well from the model to scripts (goals 1.4 and 1.5). Item 161 scripts the steps found before the first edit in item 140's chunk. This audit reads the archived transcripts of the chunks that shipped release 0.2.0 for every step repeated by hand across chunks whose answer a program computes the same way, before, during and after review: a lookup, a check, a prompt assembled for a reviewer, a cleanup. File one item per such step, with the time and tokens it cost, into the next release with items left. Leave a step that needs judgement with the model.
The chunk that ships this item files a copy of it in the release after the one it audits, rewording "0.2.0" to the release that copy audits, and marks the copy owner, so the audit recurs each release until lilleman drops it.
165. Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.
A test audit across lilleman's projects on 2026-10-16 found 10 of 33 sampled tests in this repo pinning wording that is not the contract. Examples of these and other weak tests: bin/loop_claude/test_loop_claude.py:165, :271 and most asserts in :664 to :687 pin whole sentences of the question, archive or prompt text, up to about 750 characters, and :864 asserts help-text wording. :906 and :936 parse the source with ast and list each Stop(...) and Failure(...) call by function name. bin/model_routing/test_ladder.py:16 asserts len(LADDER) == 11. Rewrite each to the technical principles' test bullet: assert the facts the text must carry, such as an item ID, a rung or a model argument, and delete a test no real bug could fail. Before and after rewriting a test, run the mutation tool on the code it covers: every mutant the old test killed, the new one kills.
122. Assemble the judge's prompt in bin/loop_claude/cycle/judge.py, importing only wording the worker also sees, and call the worker "the worker" in it.
All four seats of PR #257's scoring run named bin/loop_claude/cycle/judge.py:16 JUDGE_RULES. The prompt is spliced from constants in cycle/prompts.py (ANSWER_DESTINATIONS, ANSWERS_RECORDED, either) and project/records.py (HUMAN_RULE_NAMES), so a reader rebuilds it from three files. It says "the agent" where the code and CLAUDE.md → "Vocabulary" say "worker". Wording the worker also sees, such as ANSWER_DESTINATIONS in prompts.resumed, may stay in cycle/prompts.py. Make JUDGE_RULES easier to read.
158. Measure the fixed cost of a chunk from loop-claude's archive, in tokens and hours against its items' S, and set F in Score = 100 * V / (2**S + F) to it.
F is 8 until then, about one S 3 change, lilleman's call in PR #283. A chunk costs more than its size: the worker reads about 130k tokens before its first commit whatever the item, and every chunk takes its review passes. Fit cost against 2**S from the finished chunks score-outcomes measures, in the same units 2**S counts, and say in docs/decisions.md → todo.md: … how far the fit holds. A chunk bundling tiny items shares F between them.
116. Report L by the share of chunks with a 🔴, once 10 chunks that each shipped one item record finding counts and most record no 🔴.
score-outcomes reports the median 🔴 count at each guessed R, and how well L orders chunks by 🔴 findings; the part of R a 🔴 tests is L. Where most chunks have no 🔴, every median reads 0 and the rank correlation reads none, which tells a scorer nothing. No round title held counts before PR #254, so how often a chunk has a 🔴 is not yet known.
121. Check that bin/round_titles.py's PASSES lists exactly the passes of larv-review §3's table.
The inherited architect of PR #257's scoring run named bin/round_titles.py:5 PASSES: it repeats skills/larv-review/SKILL.md:33 with no check tying them. It leaves out the auto-reviewer, whose rounds SKILL.md:68 counts by hand. A test under bin/repo_checks/ can hold the two lists together. Make PASSES easier to read.
112. Count an item as shipped only where its chunk built it, so an item a chunk dropped from the plan gets no outcome.
score-outcomes counts every item a merge removed from todo.md as shipped. A merge that changed only the plan ships nothing, but one that built an item and also dropped another gives the dropped item the built one's outcome, as PR #253 did with item 69. Mark a dropped item in the change that drops it, such as by the commit or merge title, and leave it out.
130. Move the vote counting out of bin/larv_review/ruling.py's votes, so reading a votes file decides nothing.
Mid A of PR #257's scoring run named bin/larv_review/ruling.py:8 votes. It calls counted while validating, to reject the maintainability senior's and Junior A's lines when Mid A and the inherited architect already agree overall: a ruling hidden in a parser. The counting rule is spread over seats.py, ruling.py, comprehension_history.place_dimensions and the help. Make votes easier to read.
125. Give "record", "outcome" and "places" one meaning each across bin/, and rename bin/larv_review/git.py by what it holds.
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per docs/decisions.md → larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension.
The inherited architect of PR #257's scoring run named bin/reading_cost/measure.py:16 Reading and bin/larv_review/git.py:6 history_changes, and found each word used for several concepts. "Record": the history writer larv_review/record.py, the judge's loop_claude/project/records.py, and reading_cost/figures.py's RECORD. "Outcome": transcript_archive/outcomes.py, score_outcomes/outcomes.py Outcome and rule_hook/rules.py Outcome. "Places": comprehension_history.Place and Reading.places. larv_review/git.py holds history logic, not a git runner. Make Reading and history_changes easier to read.
81. Have workflow-bench/bench.py return each run's failure as a value, so one failed clone does not hide the later runs' results.
"Errors as values", in principles/technical-principles.md, raised by the principles audit of PR #224: bench.py raises KeyError and CalledProcessError, and pool.map re-raises the first while the other threads run on unreported.
102. Give loop-claude's retry delay an environment override, so the black-box test stops replacing time.sleep through sitecustomize.
Found by the principles audit of PR #242. bin/loop_claude/test_black_box.py:113-114 swaps the global time.sleep, the shape "Functional core, imperative shell" bans. An override of the delay lets the test run the code as production runs it.
66. Hold systems-architect's method thresholds only while the comprehension panel finds the flagged code hard.
KISS, in principles/technical-principles.md, which keeps a limit only while readers find the flagged code hard. agents/systems-architect.md makes cyclomatic 7 and 24 lines gate rules from the first commit, a 🔴 unless a reason is stated.
101. Have larv_review/record.py's read_seats take the time, the commit check and the history as inputs, so it parses and decides without effects.
Found by the principles audit of PR #242, against "Functional core, imperative shell" (principles/technical-principles.md). bin/larv_review/record.py:67 reads time.time(), then checks git (:70, commit_exists) and the history file (:75) inside the function that parses the seats file and decides. A test cannot fix the heading's time, and every test of read_seats touches the real checkout.
111. Measure the chunks of a project that merges by squash or fast-forward, which score-outcomes skips today.
score-outcomes reads a chunk from a merge commit: its branch's round titles, and its change against the first parent. Two of the projects under ~/code merge without merge commits, so their scorers see no finished item.
113. Compute an item's S from the code it names, where it names any, and show it beside the finished items score-outcomes lists.
score-outcomes grounds L, E, I and S in finished chunks only. Where an item names files or units, their size bounds what the change can touch.
137. Archive a chunk's route as a record loop-claude reads back field by field, in place of the Worker: prose ROUTED parses, and name stepped and the judge's bundled by what they do.
PR #262's paired ruling found the project reads worse: a rerun resumes on its rung only by parsing Rung.__str__'s and Route.__str__'s wording back with bin/loop_claude/routing.py's ROUTED, a string contract over four files, where a reword silently routes afresh. Write the route as a structured line, such as JSON after Worker:, that transcript_archive reads into fields. stepped both raises to a bundled item's rung and counts towards a one-rung step; Reply.bundled names neither L nor highest. Make bin/loop_claude/loop/main.py's run_loop easier to read where it threads the route.
110. Count, for each finished chunk, the later changes that fixed code it wrote, and show them beside its guessed E.
score-outcomes measures risk only by the 🔴 findings review caught. A serious bug review missed shows later, as a change that fixes code the chunk wrote: the defect that escaped, which E guesses.
139. Split bin/routing_figures/measure.py by concern, and keep the figures' measured cycle and each raised call as fields loop-claude reads back, not as the prose it renders.
The paired ruling on PR #265 ruled worse. Make these places easier to read: measure.py measured and render (token attribution, subagent rungs, round counting, calls and rendering in one module; measured, Measured and measure_archive side by side); calls with archive.py raised and last_measured, where a call's identity is its sentence and the measured cycle is read back from rendered markdown; and Reports.searches, which holds characters, not searches.
149. Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.
A release's capacity is a sum of its items' cost, 2**S. Each project states its own; the human sets the first value per project. Nothing is deleted by score.
12. Run the test suite in CI on every PR, or record in docs/decisions.md why the local run is enough.
The repo has no CI, so larv-review's gate is the local run, and larv-review's Gate skips even that when the last stability round left the suite green: an unattended chunk can merge on the worker's own word.
When CI runs a rule check, such as one of bin/rule-hook's, let <!-- rule-override <rule>: <reason> --> in the PR body override it.
105. Compare review rounds per pass and reading cost for the five chunks before and after the worker: decision moved the principles out of the worker's context, and file an item if either rose.
The worker: decision is valid while review rounds per pass do not rise. Count the rounds from each PR's Answer <pass> round <n>: commits, and take reading cost from docs/reading-cost.md. Run it once five chunks have merged after the one that recorded the decision.
135. Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop.
The maintainability senior and the inherited architect of PR #257's scoring run named bin/loop_claude/loop/reload.py:38 fingerprint and :50 exec_if_edited: the fingerprint covers every bin/**/*.py, so editing statusline.py or rank_todo.py re-execs a running loop. Make fingerprint easier to read.
68. Test workflow-bench/compare.py's blinding and bench.py's run plan.
"Most, if not all, code should have automated tests", in principles/technical-principles.md: compare.blind() keeps the comparison blind, and a version name it misses invalidates the bench silently.
4. Name each chunk's archive folder by the chunk's start time, so <key>/ lists chunks in the order they ran.
Today the folder is the SHA-256 of the transcript's first timestamped line, which tells the human nothing; that line carries the start time. Drop the fallback to the first line, which every chunk of a project shares, and give the tests' fake transcripts ISO timestamps. bin/reading-cost takes a chunk's id from the folder's first 12 characters and skips a chunk whose id docs/reading-cost.md holds, so give it a way to match a figure written under the old name.
59. Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.
Raised by the product-owner review of PR #211. The resumed worker gets the judge's quote of the work remaining, but a worker that still thinks the chunk done can answer each cycle the same way, and each cycle costs a worker and a judge run (goal 1.4).
123. Have bin/comprehension_history.py parse each history entry once into seats, votes and places, so latest_places, worse_places and worse_votes neither depend on table order nor split a label to find a seat.
Three seats of PR #257's scoring run named bin/comprehension_history.py:106 latest_places; the logic it rests on is :114 worse_places and :126 worse_votes. They regex the markdown the module writes. worse_votes keeps a seat's first row through setdefault, because the votes table comes first. worse_places recovers the seat with label.split(",")[0]. Entry still holds scoring seats' places for rulings recorded before PR #255. Make these functions easier to read.
132. Lift each dimension still under the comprehension floor after release 0.2.0's scoring run, at the places that run names.
Replaces item 62, which release 0.3.0's items split. PR #257's scoring run put Locality and Self-sufficiency at 6.63, Shape at 6.25 and the overall at 6.50, against the floor of 7.0 in docs/decisions.md → comprehension floor: …. Split this into items, one per place, once 0.3.0's scoring run has named them.
124. Group the shared modules at bin/'s root into packages named by purpose, so the root holds only launchers and stand-alone tools.
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per docs/decisions.md → larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension.
PR #257's scoring run put Shape at 6.25, the lowest dimension. bin/ holds about 30 entries: 6 launchers, 6 tool packages, 3 shared packages and 13 root modules. Two root modules are tools (rank_todo.py, statusline.py) and 11 are shared libraries, and nothing in the layout tells them apart; only README's import-root paragraph maps them. Candidate packages: git running and parsing, the todo table with rank_todo.py's formula language and goal weighing, and the comprehension history with round titles. rank_todo.py keeps its path, which every project's rules name. The .gitignore allowlist and README follow the move.
156. Drop rank_todo.py's ranking of an old-shape table once no project under ~/code holds one.
While rank_todo.py ranks two shapes, projects follow two sets of scoring rules, against goal 1.3. Once every project has reshaped, drop OLD_SHAPE's weighing from rank_todo.py, RESHAPE, and the old shape's lines in rank_todo.py --help and rules/todo-items.md. Keep todo_table reading old tables: score_outcomes reads every past merge's todo.md. PR #283's paired ruling found the old-shape path made rank_todo.py's score, Scoring.weights, unknown_columns and todo_table.py's Moves harder to read; leave each easier, with one shape and moves that are always moves, and the Moves cell's rules in one place.
155. Lower the reading cost of the files a chunk reads most before its first commit.
bin/reading-cost on 2026-10-16: the last 5 chunks' median is 189873 new input tokens before their first commit, the 5 before them 90662. Read most over the last 5, by opens and searches: todo.md 56, docs/decisions.md 19, CHANGELOG.md 13, README.md 13, bin/test_rank_todo.py 10, AGENTS.md 8, skills/larv-review/SKILL.md 7, agents/stability-reviewer.md 6, bin/loop_claude/test_loop_claude.py 6, bin/rank_todo.py 6. The chunk that ships this item runs bin/reading-cost --reset.
1. Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.
It lives under ~/.claude, which .gitignore keeps from the remote, so a lost disk loses every unattended run's record. Back it up only to a destination private to me and encrypted at rest: the worker runs with permissions bypassed, so the archived tool results can hold credentials it read.
2026-10-02, the human: ai-sandbox is the VM loop-claude will run on, hosted on the server bender. bender-infra, the infrastructure-as-code project that manages bender, will back up ai-sandbox's data volume once ai-sandbox is rebuilt. The item waits for the human to say the rebuild has landed; do not remind them. Before writing the README line, check that ~/.claude lives on that volume and that the backup is private and encrypted at rest.
The README line covers the bench's transcripts in workflow-bench/logs/ too (goal 3).
6. Move what loop-claude writes out of ~/.claude.json, and ask the human about each write that must stay.
Claude Code owns that file and writes it. trust reads it, changes it and writes it back, so a write that lands in between is lost.
5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
The key names the project's session, archive folder and models entry. project_key drops the leading /, so ~/code/a and /a share all three, and one loop can resume or delete the other's session.
bin/reading-cost keys the archive by the main checkout, the parent of git's common dir, since it runs in a chunk's worktree; loop-claude keys it by --show-toplevel, so a loop started inside a worktree archives where reading-cost never looks. Derive both keys one way.
77. Teach bin/metric-bench Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.
2026-10-08, lilleman: no metric enters review until a rerun on more data shows it tracking the panel.
Before the rerun, set the AUC and top-k lift over chance that count as tracking, from a published threshold, so the rerun ends in a decision. The rerun needs places from a project in Go or TypeScript, the languages most other projects use, so the bench first measures those languages, run through each project's own tooling. Function and file length stay warnings to look into, never limits: a plain long function can be simple, and a hard one may need five-line pieces.
On 2026-10-08 only this repo's history named places: five scoring runs, all that day. The medians:
| Metric | AUC | Top k |
|---|---|---|
| Cognitive complexity | 0.66 | 0.26 |
| Length | 0.74 | 0.18 |
| Shared state | constant | constant |
| Change spread | 0.67 | 0.18 |
AUC is 0.5 for a coin toss, and top k scores 0.10 by chance. Shared state cannot be judged here: every unit in bin/ scores 0 on it. Every run's first place was a section of skills/larv-review/SKILL.md, which no code metric sees.
On 2026-10-09 a question bench asked a headless claude, allowed only Read, Grep and Glob, three questions a script answers from the syntax tree: which units call a unit, which units it calls, and which test files refer to it. It asked them about every unit the history names whose name is unique, plus as many units the history does not name. Five of the history's six scoring runs named such a unit. The tool is in commit 083cc78; it was deleted after the run.
Claude's misses ranked the named units at a coin toss. Top k scores 0.50 by chance on these units:
| Metric | AUC | Top k |
|---|---|---|
| Question misses | 0.49 | 0.50 |
| Cognitive complexity | 0.66 | 0.59 |
| Length | 0.75 | 0.69 |
| Change spread | 0.57 | 0.54 |
Claude's answers overlapped the truth by a median Jaccard index of 0.76. A run took in 62,643 new input tokens and cost $0.70; it did not record which model answered. Grep answers these questions whatever the code's shape, so no question enters review; a new question bench would need questions that take understanding the code, not finding it.
51. Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints.
"Functional core, imperative shell", in principles/technical-principles.md: archive, trust, retried, the worker, interrupt and model_choice print to stderr themselves, and the tests swap the global stream.
151. Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes.
Each case is a question the project settled and the answer it chose. A fresh-context agent given only the README's premises answers each case; a case it answers differently says the goals no longer decide it. Written guidance can fail to change behaviour, which is why the goals get a test.
52. Record why this checkout lives in Claude Code's own directory, or move it out and link it in.
"One owner per value" and "Prefer the vendor-neutral option", in principles/technical-principles.md: git, Claude Code and loop-claude all write ~/.claude, kept apart only by the .gitignore allowlist, and the repo's records live in a vendor directory.