55 KiB
todo
Scoring
Score = 100 * V / (2**S + 8)
Bar = 0.1
Next ID = 176
Items
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 84 | 0.2.0 | Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes. | 2 | 2 | 2 | 4 | 4 | 1.4 by 5 | 2.3 | ||
| 168 | 0.2.0 | Have the human answer an interrupt on loop-claude's terminal and resume the worker headless with the answer, opening the interactive session only when they ask for it. | 4 | 2 | 3 | 7 | 5 | 1.5 by 8 | 1.8 | ||
| 73 | 0.2.0 | Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries. | 3 | 2 | 3 | 6 | 5 | 1.4 by 4 | 1.1 | ||
| 159 | 0.2.0 | decision | Run each review pass on the lowest rung that a replay of its archived rounds shows finding every 🔴 and 🟡 the project's rung finds, even below the worker's rung. | 4 | 2 | 3 | 7 | 6 | 1.4 by 7 | 1.1 | |
| 80 | 0.3.0 | defect | State in README → Install that every machine taking this checkout must be a sandbox, as settings.json's auto mode assumes. |
1 | 2 | 1 | 2 | 1 | 2 by 4 | 17.5 | |
| 21 | 0.3.0 | defect | Give settings.json's model one writer, and sort its keys. |
3 | 3 | 3 | 7 | 3 | 2 by 5 | 13.7 | |
| 29 | 0.3.0 | Make the contributor a standing member of every project's audience, unless its ## Audience says maintainability does not matter. |
2 | 2 | 3 | 5 | 2 | 1.1 by 5, 1.4 by 5 | 13.7 | ||
| 119 | 0.3.0 | Retitle the comprehension-floor decision so it says a chunk lifting a project towards the floor works first on its lowest-scoring dimension, and that the floor blocks nothing. | 1 | 2 | 1 | 2 | 1 | 1.1 by 4, 1.4 by 4 | 13.1 | ||
| 127 | 0.3.0 | Define seat, place, paired ruling, scoring run and the judge's records in CLAUDE.md → "Vocabulary". |
1 | 2 | 3 | 4 | 1 | 1.1 by 4, 1.4 by 4 | 13.1 | ||
| 129 | 0.3.0 | Explain bin/rule_hook/rules.py's OVERRIDE pattern beside it: why the reason is lazy, what the lookahead splits and why --> is stripped. |
1 | 2 | 1 | 2 | 1 | 1.1 by 4, 1.4 by 4 | 13.1 | ||
| 114 | 0.3.0 | decision | Run git through git_repo.py in rule_hook, metric_bench, reading_cost, stability_replay and loop_claude, keeping rule-hook's guards as an option, so one module owns how a tool runs git. |
3 | 1 | 3 | 5 | 4 | 1.1 by 9, 1.4 by 9 | 12.3 | |
| 120 | 0.3.0 | Rewrite larv-review §3's "Round cap" and "Re-entry" as one table that says, for each kind of fix, which passes rerun and what a round reviews. | 2 | 2 | 3 | 5 | 4 | 1.1 by 6, 1.2 by 6, 1.4 by 6 | 11.9 | ||
| 49 | 0.3.0 | principle | Check that AGENTS.md lists exactly the titles of docs/decisions.md. |
1 | 1 | 1 | 1 | 2 | 1.1 by 6 | 10.9 | |
| 117 | 0.3.0 | Give todo_table.items one way to say a plan has no items table, so no caller tells it from an empty plan by truthiness. |
1 | 1 | 3 | 3 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 126 | 0.3.0 | State when work has landed towards a release once, in bin/larv_review/plan.py, and have larv-review's help point to it. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 131 | 0.3.0 | Describe the transcript archive's layout in bin/transcript_archive/, which owns it, and have loop-claude's help point there. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 133 | 0.3.0 | Show, beside bin/rule_hook/override_log.py's this_chunks, one example per case it decides: an amended or rebased commit, a renamed branch, and a commit on the default branch. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 134 | 0.3.0 | Name in bin/shell_command.py's segments the shell forms it does not parse, so a caller knows what it misses. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 154 | 0.3.0 | defect | Have bin/shell_command.py's segments keep a command's words together across a redirect, so git > out commit reads as one git commit. |
1 | 1 | 3 | 3 | 2 | 1.1 by 6 | 10.9 | |
| 157 | 0.3.0 | Split docs/decisions.md → todo.md: one table of items …'s 2026-10-02 paragraph into one paragraph per choice. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4, 1.4 by 4 | 10.9 | ||
| 106 | 0.3.0 | Make larv-review's "Do not over-read a clean result" cite a rule that says Renovate or Dependabot owns dependency CVEs, or drop the citation. | 1 | 2 | 1 | 2 | 1 | 1.1 by 5 | 10.9 | ||
| 167 | 0.3.0 | Refuse a /security-review or /claude-security Skill call below the project's rung, as rule-hook refuses such an Agent call. |
3 | 2 | 3 | 6 | 2 | 1.1 by 4 | 7.3 | ||
| 172 | 0.3.0 | Name the project's rung project across bin/loop_claude, in place of anchor, as bin/model_routing/ladder.py names it. |
2 | 1 | 3 | 4 | 3 | 1.1 by 3, 1.4 by 3 | 6.2 | ||
| 82 | 0.3.0 | principle | Sort AGENTS.md's decision titles and docs/decisions.md's entries by title. |
2 | 2 | 1 | 3 | 4 | 1.1 by 4, 1.4 by 4 | 5.5 | |
| 173 | 0.3.0 | Have rule-hook's todo-rank-check run a worktree's own bin/rank_todo.py when the worktree is of this repo, so a branch that changes todo.md's shape commits without an override. |
2 | 1 | 3 | 4 | 2 | 1.1 by 2, 1.4 by 2 | 5.5 | ||
| 162 | 0.3.0 | owner | Script each step that 0.2.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release. | 2 | 2 | 1 | 3 | 3 | 1.4 by 4, 1.5 by 4 | 4.9 | |
| 175 | 0.3.0 | Compare the first five chunks loop-claude starts with chunk-preflight against the five before them, by time and fresh tokens before the first edit and by review rounds and unfinished verdicts, and file an item where the preflight did not pay. |
1 | 2 | 1 | 2 | 2 | 1.4 by 3, 1.5 by 3 | 4.9 | ||
| 171 | 0.3.0 | decision | Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised. | 3 | 2 | 2 | 5 | 4 | 1.2 by 3, 1.4 by 3 | 3.2 | |
| 165 | 0.3.0 | Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant. | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, 1.4 by 3 | 3.0 | ||
| 169 | 0.3.0 | Have rule-hook's todo-rank-check catch a todo.md that the same Bash call edits and then commits. |
3 | 1 | 3 | 5 | 3 | 1.1 by 2 | 2.7 | ||
| 174 | 0.3.0 | In a chunk no item routed, ask the worker to name each todo.md item it takes, with its L and E + I, so loop-claude can route on an item it takes in. | 1 | 2 | 3 | 4 | 2 | 1.1 by 1 | 1.8 | ||
| 122 | 0.4.0 | Assemble the judge's prompt in bin/loop_claude/cycle/judge.py, importing only wording the worker also sees, and call the worker "the worker" in it. |
2 | 2 | 3 | 5 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 | ||
| 158 | 0.4.0 | decision | Measure the fixed cost of a chunk from loop-claude's archive, in tokens and hours against its items' S, and set F in Score = 100 * V / (2**S + F) to it. |
2 | 2 | 1 | 3 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 | |
| 170 | 0.4.0 | decision | Re-tune WORKER_STEPS and REVIEW_STEPS once 10 chunks have shipped under them, from how well L, E and I order their review rounds and 🔴 findings. |
2 | 2 | 3 | 5 | 2 | 1.1 by 3, 1.4 by 5 | 10.0 | |
| 116 | 0.4.0 | Report L by the share of chunks with a 🔴, once 10 chunks that each shipped one item record finding counts and most record no 🔴. | 1 | 1 | 2 | 2 | 2 | 1.1 by 5 | 9.1 | ||
| 121 | 0.4.0 | Check that bin/round_titles.py's PASSES lists exactly the passes of larv-review §3's table. |
1 | 1 | 1 | 1 | 2 | 1.1 by 5 | 9.1 | ||
| 112 | 0.4.0 | Count an item as shipped only where its chunk built it, so an item a chunk dropped from the plan gets no outcome. | 3 | 2 | 2 | 5 | 3 | 1.1 by 6 | 8.2 | ||
| 130 | 0.4.0 | Move the vote counting out of bin/larv_review/ruling.py's votes, so reading a votes file decides nothing. |
2 | 1 | 2 | 3 | 3 | 1.1 by 4, 1.4 by 4 | 8.2 | ||
| 125 | 0.4.0 | decision | Give "record", "outcome" and "places" one meaning each across bin/, and rename bin/larv_review/git.py by what it holds. |
3 | 1 | 3 | 5 | 5 | 1.1 by 9, 1.4 by 9 | 7.4 | |
| 81 | 0.4.0 | principle | Have workflow-bench/bench.py return each run's failure as a value, so one failed clone does not hide the later runs' results. |
1 | 2 | 1 | 2 | 2 | 1.1 by 4 | 7.3 | |
| 102 | 0.4.0 | Give loop-claude's retry delay an environment override, so the black-box test stops replacing time.sleep through sitecustomize. |
1 | 1 | 2 | 2 | 2 | 1.1 by 4 | 7.3 | ||
| 66 | 0.4.0 | principle | Hold systems-architect's method thresholds only while the comprehension panel finds the flagged code hard. | 2 | 2 | 3 | 5 | 3 | 1.1 by 5 | 6.8 | |
| 101 | 0.4.0 | principle | Have larv_review/record.py's read_seats take the time, the commit check and the history as inputs, so it parses and decides without effects. |
2 | 1 | 2 | 3 | 3 | 1.1 by 5 | 6.8 | |
| 111 | 0.4.0 | Measure the chunks of a project that merges by squash or fast-forward, which score-outcomes skips today. |
3 | 2 | 2 | 5 | 3 | 1.1 by 5 | 6.8 | ||
| 113 | 0.4.0 | Compute an item's S from the code it names, where it names any, and show it beside the finished items score-outcomes lists. |
3 | 2 | 2 | 5 | 4 | 1.1 by 5, 1.4 by 5 | 6.8 | ||
| 137 | 0.4.0 | Archive a chunk's route as a record loop-claude reads back field by field, in place of the Worker: prose ROUTED parses, and name stepped and the judge's bundled by what they do. |
4 | 2 | 3 | 7 | 4 | 1.1 by 4, 1.4 by 4 | 5.5 | ||
| 110 | 0.4.0 | Count, for each finished chunk, the later changes that fixed code it wrote, and show them beside its guessed E. | 3 | 2 | 2 | 5 | 5 | 1.1 by 6, 1.4 by 6 | 4.9 | ||
| 139 | 0.4.0 | Split bin/routing_figures/measure.py by concern, and keep the figures' measured cycle and each raised call as fields loop-claude reads back, not as the prose it renders. |
3 | 1 | 2 | 4 | 5 | 1.1 by 5, 1.4 by 5 | 4.1 | ||
| 149 | 0.4.0 | decision | Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release. | 4 | 2 | 3 | 7 | 5 | 1.1 by 7 | 3.8 | |
| 12 | 0.5.0 | Run the test suite in CI on every PR, or record in docs/decisions.md why the local run is enough. |
3 | 2 | 1 | 4 | 4 | 1.1 by 6 | 5.5 | ||
| 105 | 0.5.0 | decision | Compare review rounds per pass and reading cost for the five chunks before and after the worker: decision moved the principles out of the worker's context, and file an item if either rose. |
1 | 2 | 1 | 2 | 2 | 1.4 by 6 | 5.5 | |
| 135 | 0.5.0 | Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop. | 2 | 2 | 3 | 5 | 3 | 1.1 by 4 | 5.5 | ||
| 68 | 0.5.0 | principle | Test workflow-bench/compare.py's blinding and bench.py's run plan. |
2 | 1 | 1 | 2 | 4 | 1.1 by 5 | 4.6 | |
| 4 | 0.5.0 | Name each chunk's archive folder by the chunk's start time, so <key>/ lists chunks in the order they ran. |
3 | 2 | 3 | 6 | 3 | 1.1 by 3 | 4.1 | ||
| 59 | 0.5.0 | Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end. | 2 | 1 | 3 | 4 | 3 | 1.4 by 6 | 4.1 | ||
| 123 | 0.5.0 | Have bin/comprehension_history.py parse each history entry once into seats, votes and places, so latest_places, worse_places and worse_votes neither depend on table order nor split a label to find a seat. |
2 | 1 | 2 | 3 | 5 | 1.1 by 5, 1.4 by 5 | 4.1 | ||
| 132 | 0.5.0 | Lift each dimension still under the comprehension floor after release 0.2.0's scoring run, at the places that run names. | 3 | 2 | 2 | 5 | 5 | 1.1 by 5, 1.4 by 5 | 4.1 | ||
| 124 | 0.5.0 | decision | Group the shared modules at bin/'s root into packages named by purpose, so the root holds only launchers and stand-alone tools. |
4 | 1 | 3 | 6 | 6 | 1.1 by 9, 1.4 by 9 | 4.1 | |
| 156 | 0.5.0 | decision | Drop rank_todo.py's ranking of an old-shape table once no project under ~/code holds one. |
2 | 1 | 3 | 4 | 4 | 1.1 by 4 | 3.6 | |
| 155 | 0.5.0 | Lower the reading cost of the files a chunk reads most before its first commit. | 3 | 2 | 2 | 5 | 5 | 1.4 by 6 | 1.6 | ||
| 1 | 0.6.0 | question | Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume. | 1 | 2 | 1 | 2 | 1 | 0.0 | ||
| 6 | 0.6.0 | decision | Move what loop-claude writes out of ~/.claude.json, and ask the human about each write that must stay. |
3 | 3 | 3 | 7 | 3 | 1.1 by 5 | 6.8 | |
| 5 | 0.6.0 | decision | Key a project on its checkout's full path, and refuse to start while another session uses that key. | 4 | 2 | 3 | 7 | 5 | 1.1 by 6 | 3.3 | |
| 77 | 0.6.0 | Teach bin/metric-bench Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel. |
4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 3.2 | ||
| 51 | 0.6.0 | principle | Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints. | 2 | 1 | 3 | 4 | 5 | 1.1 by 5 | 2.7 | |
| 151 | 0.6.0 | decision | Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes. | 3 | 2 | 2 | 5 | 5 | 1.2 by 7 | 2.6 | |
| 52 | 0.6.0 | principle | Record why this checkout lives in Claude Code's own directory, or move it out and link it in. | 4 | 3 | 4 | 9 | 6 | 1.1 by 4 | 1.2 |
Details
84. Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.
Raised by the principles audit of PR #228: the review: the comprehension check records … entry in docs/decisions.md is valid only while that holds, and docs/comprehension-history.md keeps every vote, yet nothing reads the agreement rate. Scoring runs on near-identical code are rare now, so compare with those already in the history, such as PR #224's and PR #225's, or make a second run for the check.
168. Have the human answer an interrupt on loop-claude's terminal and resume the worker headless with the answer, opening the interactive session only when they ask for it.
Today an interrupt runs claude --resume on the terminal and waits for it to exit (bin/loop_claude/cycle/interrupt.py). Once the human answers, the worker carries on in that interactive session, so the loop waits until the human exits it, often for the rest of the chunk. Instead, the loop prints the question and reads the answer on its own terminal. The next cycle resumes the worker headless with a prompt like prompts.answered's, saying the human gave the answer. A word typed in place of an answer, such as /talk, opens today's interactive session, for when the human wants to read the diff or discuss first. An empty answer stops the loop, as an interrupt with nothing typed does today. A release ready to cut still opens the interactive session. Bench the word, and the line that asks for the answer, as CLAUDE.md says.
The judge must treat an answer typed at the loop's terminal as the human's. bin/claude_code/transcript.py's render labels every -p prompt [loop], so an answer passed on headless would read as one quoted from the records. The judge would then never hold the chunk open until that answer is recorded. The change touches two places the comprehension seats found hard: bin/loop_claude/cycle/judge.py's run_judge and bin/loop_claude/loop/main.py's help. Leave each easier to read.
73. Have each reviewer write its full report to a file and return the worker only its findings' one-line summaries.
2026-10-08, lilleman: rescored from A10/G8 to A4/G4 on the measurement below, so other items that lower token use go first.
Measured over the 68 transcripts in loop-claude's archive on 2026-10-08, matching each Agent call to its tool result and to any <task-notification> naming its tool-use id. No report reached the worker twice: 813 arrived once as the hand-back, and 363 once as a notification after Claude Code ran the agent in the background. Reports took a median 19k tokens per chunk, 8% of the worker's peak context (24% at most). Lines carrying a severity mark are 9–24% of a reviewer's report, and the worker would still open the file for each 🔴 or 🟡 it fixes. A third of the report volume is the comprehension seats', whose votes and places the worker copies into the history. File reads fill more of the worker's context: sed -n output is 30% of its Bash results and grep output 11%, which reading-cost targets.
The reports would live in the session's scratchpad, such as <scratchpad>/larv-review/prose-round-1.md, so no project's checkout sees them. A later round gets the path of that pass's earlier report in place of the findings pasted in. How a report reaches the file:
- Write access. Grant each reviewer Write. No reviewer under
agents/holds it today, an agent's tool list cannot confine Write to one folder, and~/.claude/CLAUDE.md→ "Working with the codebase" denies edits to a reviewer that only reads. Check whether a PreToolUse hook can tell a subagent's call from the worker's and deny a Write outside the scratchpad. - A wrapper. A general-purpose subagent launches the reviewer, saves the report and returns only the summary lines. Claude Code's documentation says a subagent cannot start another; drop this route unless a test shows one can.
This risks goal 1.1: a worker fixing from a one-line summary can miss context it needs.
159. Run each review pass on the lowest rung that a replay of its archived rounds shows finding every 🔴 and 🟡 the project's rung finds, even below the worker's rung.
lilleman asked, on 2026-10-16, for the prose pass to run on the worker's rung or lower, and the same day, answering PR #296, for every pass to drop below the project's rung only where a replay shows it may. Today bin/loop_claude/cycle/prompts.py's review_launches keeps every reviewer at or above the project's rung whatever the item's E + I, so a docs-only chunk such as item 140's, routed to Haiku 5.5, runs its reviewers on Opus 5.5.
Whether a pass runs cheaper without lowering quality (goal 1.4 against 1.1) is measured, as docs/decisions.md → routing: the stability reviewer runs no lower than the project's rung… measures the stability reviewer: replay a pass's archived rounds on the code each reviewed, first on the project's rung, then on each rung below it, and take the lowest rung that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again. A missed prose finding breaks no behaviour and a later change fixes it cheaply, so replay the prose pass first. Generalise bin/stability-replay to take the pass, in place of a replay tool per pass. Record each run and the rung it chose in a routing: decision, valid until a new model joins the ladder or the pass's prompt changes. Give each pass a row of its own in REVIEWS, review_launches' table of review kinds in bin/loop_claude/cycle/prompts.py, rather than a branch.
80. State in README → Install that every machine taking this checkout must be a sandbox, as settings.json's auto mode assumes.
A defect: goal 2, keeping the setup shareable between computers, does not hold while a new machine is never told it must be a sandbox. Raised by the principles audit of PR #224. The settings.json decision is valid while every machine taking this checkout is a sandbox, but Install asks for a sandbox only for loop-claude.
21. Give settings.json's model one writer, and sort its keys.
A defect: goal 2, keeping the setup shareable between computers, does not hold while one machine's model choice overwrites every other's. Claude Code rewrites "model" in this tracked file whenever a model is picked, so one /model dirties the checkout and overrides the synced choice.
Every write Claude Code makes to the file rewrites "model", claude plugin install included: Claude Code 2.1.288 turned "opus" into "opus[1m]". Each write also puts the keys back in Claude Code's own order, so a sorted file stays sorted only until the next write.
29. Make the contributor a standing member of every project's audience, unless its ## Audience says maintainability does not matter.
State that in ~/.claude/rules/goals-and-audience.md → "Goals and Audience", and have larv-review's comprehension check name the contributor as the reader it serves. The check then runs in a project whose audience includes the contributor, and a project that drops them skips it.
119. Retitle the comprehension-floor decision so it says a chunk lifting a project towards the floor works first on its lowest-scoring dimension, and that the floor blocks nothing.
Today's title, "a project lifting itself to the comprehension floor starts at the lowest dimension", reads as either where the project scores or where work begins. Beside the title saying the check blocks no merge, a reader cannot tell whether Floor: means stop; a seat named AGENTS.md:5 among the places it found hard. "First" means within such a chunk, not across a release: since the decision's 2026-10-16 line, items that lift the lowest dimension compete with the others like any item.
127. Define seat, place, paired ruling, scoring run and the judge's records in CLAUDE.md → "Vocabulary".
Two seats of PR #257's scoring run assembled these terms from four files: bin/larv_review/main.py's help, skills/larv-review/SKILL.md:99, bin/comprehension_history.py:82 and skills/comprehension-panel/seat-prompts.md. "Records" is defined only in a comment at bin/loop_claude/project/records.py:6.
129. Explain bin/rule_hook/rules.py's OVERRIDE pattern beside it: why the reason is lazy, what the lookahead splits and why --> is stripped.
The maintainability senior of PR #257's scoring run named bin/rule_hook/rules.py:49 OVERRIDE: nothing in the code explains the pattern. Make OVERRIDE easier to read.
114. Run git through git_repo.py in rule_hook, metric_bench, reading_cost, stability_replay and loop_claude, keeping rule-hook's guards as an option, so one module owns how a tool runs git.
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per docs/decisions.md → larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension.
The comprehension seats on PR #254 found five git runners with different contracts: git_repo.git, rule_hook/git.py's guarded one, metric_bench/main.py's, reading_cost/main.py's located and loop_claude/effects.py's. PR #276 added a sixth, stability_replay/main.py's git. The inherited architect of PR #257's scoring run named bin/git_repo.py:8 git again. Make git_repo.git easier to read.
120. Rewrite larv-review §3's "Round cap" and "Re-entry" as one table that says, for each kind of fix, which passes rerun and what a round reviews.
Junior A, Mid A and the maintainability senior of PR #257's scoring run named skills/larv-review/SKILL.md:63 "Round cap" hardest, two of them with :74 "Re-entry"; the inherited architect named :71 "Cap". A reader holds the trigger table's "a fix, only where …" clauses, the cap, later rounds' scope and the prose, 🟢 and ❓ exceptions at once. Junior A could not say what a fix to a stability 🟡 that also touches a comment reruns. Make those sections easier to read.
49. Check that AGENTS.md lists exactly the titles of docs/decisions.md.
DRY, in principles/technical-principles.md: the list is copied by hand, and sessions work from the copy.
117. Give todo_table.items one way to say a plan has no items table, so no caller tells it from an empty plan by truthiness.
The comprehension seats on PR #254 found items returning a list, None or a Failure, and larv_review/plan.py plan_reasons reading None through items(before) or [], which works only because a Failure is truthy.
126. State when work has landed towards a release once, in bin/larv_review/plan.py, and have larv-review's help point to it.
Two seats of PR #257's scoring run named bin/larv_review/plan.py:9 scoring_run_due. It reads the default branch's plan and the chunk's together, and the landed rule is written twice: in its docstring and in bin/larv_review/main.py's help. Make scoring_run_due easier to read.
131. Describe the transcript archive's layout in bin/transcript_archive/, which owns it, and have loop-claude's help point there.
Two seats of PR #257's scoring run named bin/transcript_archive/write.py:13 archive. Two invariants are explained only in loop-claude's help, bin/loop_claude/loop/main.py: folders are named by a hash of the transcript's first timestamped line, and a copy is replaced only by one grown from it. Make archive easier to read.
133. Show, beside bin/rule_hook/override_log.py's this_chunks, one example per case it decides: an amended or rebased commit, a renamed branch, and a commit on the default branch.
Junior A, Mid A and the maintainability senior of PR #257's scoring run named bin/rule_hook/override_log.py:36 this_chunks: which overrides belong to the chunk rests on reflog behaviour a reader must trust or work out. Make this_chunks easier to read.
134. Name in bin/shell_command.py's segments the shell forms it does not parse, so a caller knows what it misses.
Junior A and Mid A of PR #257's scoring run named bin/shell_command.py:31 segments: a hand-rolled tokenizer with dense regexes and a second lexing path for unbalanced quotes, which the rule hook and reading-cost both depend on. Its comments state each quirk but not its limits. Make segments easier to read.
154. Have bin/shell_command.py's segments keep a command's words together across a redirect, so git > out commit reads as one git commit.
segments('echo hi > f arg') returns arg as a command of its own, and segments('git > /tmp/x commit -m hi') returns git and commit -m hi apart, though the shell runs one git commit. rule-hook's commit rules read the commands segments finds, so such a commit escapes commit-one-line, no-ai-attribution, round-title-counts and todo-rank-check. The comprehension seats' scoring run in PR #280 found it.
157. Split docs/decisions.md → todo.md: one table of items …'s 2026-10-02 paragraph into one paragraph per choice.
Raised by PR #283's prose pass. The paragraph runs about 330 words and holds six choices: the bar, the purge, re-raising findings, the gate check, the principle exemption and the 2026-10-08 weights note. A reader after one of them reads all six.
106. Make larv-review's "Do not over-read a clean result" cite a rule that says Renovate or Dependabot owns dependency CVEs, or drop the citation.
Found by the goals audit of PR #245. skills/larv-review/SKILL.md §3 → "Security" says Renovate or Dependabot owns dependency CVEs per CLAUDE.md → "Working with the codebase", which says nothing about either.
167. Refuse a /security-review or /claude-security Skill call below the project's rung, as rule-hook refuses such an Agent call.
rule-hook checks Agent calls (item 163) but no Skill calls. A worker that invokes /security-review in its own context, instead of in a subagent, runs the pass on its own rung, and nothing blocks it. Check Skill calls too: breaches_of in bin/rule_hook/main.py takes a Skill call whose skill is security-review or claude-security as a review launch, and settings.json's PreToolUse matcher gains Skill. Found by the systems-architect's review of PR #295.
172. Name the project's rung project across bin/loop_claude, in place of anchor, as bin/model_routing/ladder.py names it.
The architecture pass of PR #296 found one rung under two names: bin/model_routing/ladder.py calls it project, while bin/loop_claude/routing.py, cycle/prompts.py, cycle/run.py's Cycle.anchor and loop/main.py call it anchor, and gloss it as "anchor, the project's rung" in docstring after docstring. A reader works out that the two are one rung each time. Make bin/loop_claude/routing.py's first_route easier to read.
82. Sort AGENTS.md's decision titles and docs/decisions.md's entries by title.
"Deterministic order", in principles/technical-principles.md, raised by the principles audit of PR #224: their order carries no meaning, so two chunks adding a decision conflict at the end, and a reader cannot find a title by scanning.
173. Have rule-hook's todo-rank-check run a worktree's own bin/rank_todo.py when the worktree is of this repo, so a branch that changes todo.md's shape commits without an override.
Raised by the overrides pass of PR #296. The hook imports the main checkout's rank_todo.py, so while a branch of this repo changes the table's shape, every commit touching todo.md needs a todo-rank-check override: 33 log lines across four branches, the score-outcomes, plan-136, plan-148 and item-146 ones. The hook then checks nothing, and the rule rests on the worker chaining the branch's own check. Run another repo's bin/rank_todo.py never: that would run its code before every commit.
162. Script each step that 0.2.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.
lilleman asked, on 2026-10-16, for a standing audit each release, so the loop keeps moving work a program does as well from the model to scripts (goals 1.4 and 1.5). Item 161 scripts the steps found before the first edit in item 140's chunk. This audit reads the archived transcripts of the chunks that shipped release 0.2.0 for every step repeated by hand across chunks whose answer a program computes the same way, before, during and after review: a lookup, a check, a prompt assembled for a reviewer, a cleanup. File one item per such step, with the time and tokens it cost, into the next release with items left. Leave a step that needs judgement with the model.
The chunk that ships this item files a copy of it in the release after the one it audits, rewording "0.2.0" to the release that copy audits, and marks the copy owner, so the audit recurs each release until lilleman drops it.
175. Compare the first five chunks loop-claude starts with chunk-preflight against the five before them, by time and fresh tokens before the first edit and by review rounds and unfinished verdicts, and file an item where the preflight did not pay.
Item 161 put chunk-preflight's facts into a fresh worker's prompt, so the worker no longer finds them itself. In item 140's chunk, before it, the worker spent 5 min 46 s of 30 min, 16 of its 68 messages and 226k of its 491k fresh tokens before its first edit, through 12 Bash and 5 Read calls. Measure those figures from loop-claude's archive, as medians over the first five chunks that started with the preflight and the five before them: one chunk is too noisy a baseline, as docs/decisions.md → reading cost: a rise is … records. Measure goal 1.4's counter-signal beside them, review rounds and unfinished verdicts per chunk, since a worker that reads less may start sooner and need more rounds. Where time or tokens did not fall, or rounds or unfinished verdicts rose, file an item naming the facts the workers still found by hand, or what the preflight got wrong.
171. Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.
Raised by the goals audit of PR #296. bin/routing_figures/measure.py groups a chunk's review rounds by the rung its worker started on, and calls for a routing change when a lower rung took more of them. Since PR #296 the reviewers may run above the worker's rung, on the one their item's E + I picks, and no figure groups rounds by it, though the archived worker line names the E + I it follows from. A cheap worker on an item of high E + I gets stronger reviewers, who may raise more rounds, and a call could then blame the worker's rung for them: an interrupt of lilleman that settles nothing (goal 1.2).
165. Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.
A test audit across lilleman's projects on 2026-10-16 found 10 of 33 sampled tests in this repo pinning wording that is not the contract. Examples of these and other weak tests: bin/loop_claude/test_loop_claude.py:165, :271 and most asserts in :664 to :687 pin whole sentences of the question, archive or prompt text, up to about 750 characters, and :864 asserts help-text wording. :906 and :936 parse the source with ast and list each Stop(...) and Failure(...) call by function name. bin/model_routing/test_ladder.py:16 asserts len(LADDER) == 11. Rewrite each to the technical principles' test bullet: assert the facts the text must carry, such as an item ID, a rung or a model argument, and delete a test no real bug could fail. Before and after rewriting a test, run the mutation tool on the code it covers: every mutant the old test killed, the new one kills.
169. Have rule-hook's todo-rank-check catch a todo.md that the same Bash call edits and then commits.
The hook judges a call before it runs, and changed in bin/rule_hook/todo_check.py compares todo.md with the index and HEAD as they stand then. A call that edits todo.md and commits it, such as a heredoc script followed by git commit, passes unchecked: PR #296's commit 149108e did, and the override marker it carried was never logged. larv-review's gate still runs rank_todo.py --check, so an unranked plan stops at merge rather than at commit. A git pre-commit hook would see the staged file.
174. In a chunk no item routed, ask the worker to name each todo.md item it takes, with its L and E + I, so loop-claude can route on an item it takes in.
The principles pass of PR #301 found that only routed_by in bin/loop_claude/cycle/prompts.py asks the worker to name each item it takes with its L and E + I, and fresh uses it only where an item routed the chunk. In an unrouted chunk the judge has no levels to report, so neither the taken-in L nor the taken-in E + I lifts the worker, the reviewers or the judge, and the first prompt names no reviewers' rung for a higher E + I.
122. Assemble the judge's prompt in bin/loop_claude/cycle/judge.py, importing only wording the worker also sees, and call the worker "the worker" in it.
All four seats of PR #257's scoring run named bin/loop_claude/cycle/judge.py:16 JUDGE_RULES. The prompt is spliced from constants in cycle/prompts.py (ANSWER_DESTINATIONS, ANSWERS_RECORDED, either) and project/records.py (HUMAN_RULE_NAMES), so a reader rebuilds it from three files. It says "the agent" where the code and CLAUDE.md → "Vocabulary" say "worker". Wording the worker also sees, such as ANSWER_DESTINATIONS in prompts.resumed, may stay in cycle/prompts.py. Make JUDGE_RULES easier to read.
158. Measure the fixed cost of a chunk from loop-claude's archive, in tokens and hours against its items' S, and set F in Score = 100 * V / (2**S + F) to it.
F is 8 until then, about one S 3 change, lilleman's call in PR #283. A chunk costs more than its size: the worker reads about 130k tokens before its first commit whatever the item, and every chunk takes its review passes. Fit cost against 2**S from the finished chunks score-outcomes measures, in the same units 2**S counts, and say in docs/decisions.md → todo.md: … how far the fit holds. A chunk bundling tiny items shares F between them.
170. Re-tune WORKER_STEPS and REVIEW_STEPS once 10 chunks have shipped under them, from how well L, E and I order their review rounds and 🔴 findings.
docs/decisions.md → routing: one ladder… calls the steps lilleman approved on 2026-10-16 a first guess. Once 10 chunks that each shipped one item scored with L, E and I have merged, read score-outcomes' rank correlations for L, E and I and the routed-chunks table of routing-figures.md, and propose new steps to lilleman, whose call they are.
116. Report L by the share of chunks with a 🔴, once 10 chunks that each shipped one item record finding counts and most record no 🔴.
score-outcomes reports the median 🔴 count at each guessed R, and how well L orders chunks by 🔴 findings; the part of R a 🔴 tests is L. Where most chunks have no 🔴, every median reads 0 and the rank correlation rests on the few that have one, which tells a scorer little. No round title held counts before PR #254, so how often a chunk has a 🔴 is not yet known.
121. Check that bin/round_titles.py's PASSES lists exactly the passes of larv-review §3's table.
The inherited architect of PR #257's scoring run named bin/round_titles.py:5 PASSES: it repeats skills/larv-review/SKILL.md:33 with no check tying them. It leaves out the auto-reviewer, whose rounds SKILL.md:68 counts by hand. A test under bin/repo_checks/ can hold the two lists together. Make PASSES easier to read.
112. Count an item as shipped only where its chunk built it, so an item a chunk dropped from the plan gets no outcome.
score-outcomes counts every item a merge removed from todo.md as shipped. A merge that changed only the plan ships nothing, but one that built an item and also dropped another gives the dropped item the built one's outcome, as PR #253 did with item 69. Mark a dropped item in the change that drops it, such as by the commit or merge title, and leave it out.
130. Move the vote counting out of bin/larv_review/ruling.py's votes, so reading a votes file decides nothing.
Mid A of PR #257's scoring run named bin/larv_review/ruling.py:8 votes. It calls counted while validating, to reject the maintainability senior's and Junior A's lines when Mid A and the inherited architect already agree overall: a ruling hidden in a parser. The counting rule is spread over seats.py, ruling.py, comprehension_history.place_dimensions and the help. Make votes easier to read.
125. Give "record", "outcome" and "places" one meaning each across bin/, and rename bin/larv_review/git.py by what it holds.
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per docs/decisions.md → larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension.
The inherited architect of PR #257's scoring run named bin/reading_cost/measure.py:16 Reading and bin/larv_review/git.py:6 history_changes, and found each word used for several concepts. "Record": the history writer larv_review/record.py, the judge's loop_claude/project/records.py, and reading_cost/figures.py's RECORD. "Outcome": transcript_archive/outcomes.py, score_outcomes/outcomes.py Outcome and rule_hook/rules.py Outcome. "Places": comprehension_history.Place and Reading.places. larv_review/git.py holds history logic, not a git runner. Make Reading and history_changes easier to read.
81. Have workflow-bench/bench.py return each run's failure as a value, so one failed clone does not hide the later runs' results.
"Errors as values", in principles/technical-principles.md, raised by the principles audit of PR #224: bench.py raises KeyError and CalledProcessError, and pool.map re-raises the first while the other threads run on unreported.
102. Give loop-claude's retry delay an environment override, so the black-box test stops replacing time.sleep through sitecustomize.
Found by the principles audit of PR #242. bin/loop_claude/test_black_box.py:113-114 swaps the global time.sleep, the shape "Functional core, imperative shell" bans. An override of the delay lets the test run the code as production runs it.
66. Hold systems-architect's method thresholds only while the comprehension panel finds the flagged code hard.
KISS, in principles/technical-principles.md, which keeps a limit only while readers find the flagged code hard. agents/systems-architect.md makes cyclomatic 7 and 24 lines gate rules from the first commit, a 🔴 unless a reason is stated.
101. Have larv_review/record.py's read_seats take the time, the commit check and the history as inputs, so it parses and decides without effects.
Found by the principles audit of PR #242, against "Functional core, imperative shell" (principles/technical-principles.md). bin/larv_review/record.py:67 reads time.time(), then checks git (:70, commit_exists) and the history file (:75) inside the function that parses the seats file and decides. A test cannot fix the heading's time, and every test of read_seats touches the real checkout.
111. Measure the chunks of a project that merges by squash or fast-forward, which score-outcomes skips today.
score-outcomes reads a chunk from a merge commit: its branch's round titles, and its change against the first parent. Two of the projects under ~/code merge without merge commits, so their scorers see no finished item.
113. Compute an item's S from the code it names, where it names any, and show it beside the finished items score-outcomes lists.
score-outcomes grounds L, E, I and S in finished chunks only. Where an item names files or units, their size bounds what the change can touch.
137. Archive a chunk's route as a record loop-claude reads back field by field, in place of the Worker: prose ROUTED parses, and name stepped and the judge's bundled by what they do.
PR #262's paired ruling found the project reads worse: a rerun resumes on its rung only by parsing Rung.__str__'s and Route.__str__'s wording back with bin/loop_claude/routing.py's ROUTED, a string contract over four files, where a reword silently routes afresh. Write the route as a structured line, such as JSON after Worker:, that transcript_archive reads into fields. stepped both raises to a bundled item's rung and counts towards a one-rung step; Reply.bundled names neither L nor highest. Make bin/loop_claude/loop/main.py's run_loop easier to read where it threads the route.
110. Count, for each finished chunk, the later changes that fixed code it wrote, and show them beside its guessed E.
score-outcomes measures risk only by the 🔴 findings review caught. A serious bug review missed shows later, as a change that fixes code the chunk wrote: the defect that escaped, which E guesses.
139. Split bin/routing_figures/measure.py by concern, and keep the figures' measured cycle and each raised call as fields loop-claude reads back, not as the prose it renders.
The paired ruling on PR #265 ruled worse. Make these places easier to read: measure.py measured and render (token attribution, subagent rungs, round counting, calls and rendering in one module; measured, Measured and measure_archive side by side); calls with archive.py raised and last_measured, where a call's identity is its sentence and the measured cycle is read back from rendered markdown; and Reports.searches, which holds characters, not searches.
149. Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.
A release's capacity is a sum of its items' cost, as the formula counts it. Each project states its own; the human sets the first value per project. Nothing is deleted by score.
12. Run the test suite in CI on every PR, or record in docs/decisions.md why the local run is enough.
The repo has no CI, so larv-review's gate is the local run, and larv-review's Gate skips even that when the last stability round left the suite green: an unattended chunk can merge on the worker's own word.
When CI runs a rule check, such as one of bin/rule-hook's, let <!-- rule-override <rule>: <reason> --> in the PR body override it.
105. Compare review rounds per pass and reading cost for the five chunks before and after the worker: decision moved the principles out of the worker's context, and file an item if either rose.
The worker: decision is valid while review rounds per pass do not rise. Count the rounds from each PR's Answer <pass> round <n>: commits, and take reading cost from docs/reading-cost.md. Run it once five chunks have merged after the one that recorded the decision.
135. Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop.
The maintainability senior and the inherited architect of PR #257's scoring run named bin/loop_claude/loop/reload.py:38 fingerprint and :50 exec_if_edited: the fingerprint covers every bin/**/*.py, so editing statusline.py or rank_todo.py re-execs a running loop. Make fingerprint easier to read.
68. Test workflow-bench/compare.py's blinding and bench.py's run plan.
"Most, if not all, code should have automated tests", in principles/technical-principles.md: compare.blind() keeps the comparison blind, and a version name it misses invalidates the bench silently.
The tests also cover workflow-bench/bench.py's permission_flags, with a probe of a headless claude given its flags. It is denied a read outside the granted trees, a bare git status, and wrapper calls with --output, --no-index, a glued -aF/abs, --fil=/abs or a path outside the clone. A cd <elsewhere> && python3 <wrapper> … acts on the clone only. It is allowed the wrapper's log, diff, show, status, ls-files, add and commit -m. workflow-bench/test_review_git.py tests the wrapper's refusals directly.
4. Name each chunk's archive folder by the chunk's start time, so <key>/ lists chunks in the order they ran.
Today the folder is the SHA-256 of the transcript's first timestamped line, which tells the human nothing; that line carries the start time. Drop the fallback to the first line, which every chunk of a project shares, and give the tests' fake transcripts ISO timestamps. bin/reading-cost takes a chunk's id from the folder's first 12 characters and skips a chunk whose id docs/reading-cost.md holds, so give it a way to match a figure written under the old name.
59. Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.
Raised by the product-owner review of PR #211. The resumed worker gets the judge's quote of the work remaining, but a worker that still thinks the chunk done can answer each cycle the same way, and each cycle costs a worker and a judge run (goal 1.4).
123. Have bin/comprehension_history.py parse each history entry once into seats, votes and places, so latest_places, worse_places and worse_votes neither depend on table order nor split a label to find a seat.
Three seats of PR #257's scoring run named bin/comprehension_history.py:106 latest_places; the logic it rests on is :114 worse_places and :126 worse_votes. They regex the markdown the module writes. worse_votes keeps a seat's first row through setdefault, because the votes table comes first. worse_places recovers the seat with label.split(",")[0]. Entry still holds scoring seats' places for rulings recorded before PR #255. Make these functions easier to read.
132. Lift each dimension still under the comprehension floor after release 0.2.0's scoring run, at the places that run names.
Replaces item 62, which release 0.3.0's items split. PR #257's scoring run put Locality and Self-sufficiency at 6.63, Shape at 6.25 and the overall at 6.50, against the floor of 7.0 in docs/decisions.md → comprehension floor: …. Split this into items, one per place, once 0.3.0's scoring run has named them.
124. Group the shared modules at bin/'s root into packages named by purpose, so the root holds only launchers and stand-alone tools.
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per docs/decisions.md → larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension.
PR #257's scoring run put Shape at 6.25, the lowest dimension. bin/ holds about 30 entries: 6 launchers, 6 tool packages, 3 shared packages and 13 root modules. Two root modules are tools (rank_todo.py, statusline.py) and 11 are shared libraries, and nothing in the layout tells them apart; only README's import-root paragraph maps them. Candidate packages: git running and parsing, the todo table with rank_todo.py's formula language and goal weighing, and the comprehension history with round titles. rank_todo.py keeps its path, which every project's rules name. The .gitignore allowlist and README follow the move.
156. Drop rank_todo.py's ranking of an old-shape table once no project under ~/code holds one.
While rank_todo.py ranks two shapes, projects follow two sets of scoring rules, against goal 1.3. Once every project has reshaped, drop OLD_SHAPE's weighing from rank_todo.py, RESHAPE, and the old shape's lines in rank_todo.py --help and rules/todo-items.md. Keep todo_table reading old tables: score_outcomes reads every past merge's todo.md. PR #283's paired ruling found the old-shape path made rank_todo.py's score, Scoring.weights, unknown_columns and todo_table.py's Moves harder to read; leave each easier, with one shape and moves that are always moves, and the Moves cell's rules in one place.
155. Lower the reading cost of the files a chunk reads most before its first commit.
bin/reading-cost on 2026-10-16: the median of the last 5 chunks is 189,487 new input tokens before their first commit, up from 146,584 for the 5 before them. Read most over the last 5, by opens and searches: todo.md 51, README.md 12, CHANGELOG.md 11, bin/loop_claude/test_loop_claude.py 11, AGENTS.md 7, bin/todo_table.py 7, docs/decisions.md 7, agents/stability-reviewer.md 6, bin/loop_claude/cycle/judge.py 5, bin/rule_hook/test_rule_hook.py 5. The chunk that ships this item runs bin/reading-cost --reset.
1. Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.
It lives under ~/.claude, which .gitignore keeps from the remote, so a lost disk loses every unattended run's record. Back it up only to a destination private to me and encrypted at rest: the worker runs with permissions bypassed, so the archived tool results can hold credentials it read.
2026-10-02, the human: ai-sandbox is the VM loop-claude will run on, hosted on the server bender. bender-infra, the infrastructure-as-code project that manages bender, will back up ai-sandbox's data volume once ai-sandbox is rebuilt. The item waits for the human to say the rebuild has landed; do not remind them. Before writing the README line, check that ~/.claude lives on that volume and that the backup is private and encrypted at rest.
The README line covers the bench's transcripts in workflow-bench/logs/ too (goal 3).
6. Move what loop-claude writes out of ~/.claude.json, and ask the human about each write that must stay.
Claude Code owns that file and writes it. trust reads it, changes it and writes it back, so a write that lands in between is lost.
5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
The key names the project's session, archive folder and models entry. project_key drops the leading /, so ~/code/a and /a share all three, and one loop can resume or delete the other's session.
bin/reading-cost keys the archive by the main checkout, the parent of git's common dir, since it runs in a chunk's worktree; loop-claude keys it by --show-toplevel, so a loop started inside a worktree archives where reading-cost never looks. Derive both keys one way.
77. Teach bin/metric-bench Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.
2026-10-08, lilleman: no metric enters review until a rerun on more data shows it tracking the panel.
Before the rerun, set the AUC and top-k lift over chance that count as tracking, from a published threshold, so the rerun ends in a decision. The rerun needs places from a project in Go or TypeScript, the languages most other projects use, so the bench first measures those languages, run through each project's own tooling. Function and file length stay warnings to look into, never limits: a plain long function can be simple, and a hard one may need five-line pieces.
On 2026-10-08 only this repo's history named places: five scoring runs, all that day. The medians:
| Metric | AUC | Top k |
|---|---|---|
| Cognitive complexity | 0.66 | 0.26 |
| Length | 0.74 | 0.18 |
| Shared state | constant | constant |
| Change spread | 0.67 | 0.18 |
AUC is 0.5 for a coin toss, and top k scores 0.10 by chance. Shared state cannot be judged here: every unit in bin/ scores 0 on it. Every run's first place was a section of skills/larv-review/SKILL.md, which no code metric sees.
On 2026-10-09 a question bench asked a headless claude, allowed only Read, Grep and Glob, three questions a script answers from the syntax tree: which units call a unit, which units it calls, and which test files refer to it. It asked them about every unit the history names whose name is unique, plus as many units the history does not name. Five of the history's six scoring runs named such a unit. The tool is in commit 083cc78; it was deleted after the run.
Claude's misses ranked the named units at a coin toss. Top k scores 0.50 by chance on these units:
| Metric | AUC | Top k |
|---|---|---|
| Question misses | 0.49 | 0.50 |
| Cognitive complexity | 0.66 | 0.59 |
| Length | 0.75 | 0.69 |
| Change spread | 0.57 | 0.54 |
Claude's answers overlapped the truth by a median Jaccard index of 0.76. A run took in 62,643 new input tokens and cost $0.70; it did not record which model answered. Grep answers these questions whatever the code's shape, so no question enters review; a new question bench would need questions that take understanding the code, not finding it.
51. Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints.
"Functional core, imperative shell", in principles/technical-principles.md: archive, trust, retried, the worker, interrupt and model_choice print to stderr themselves, and the tests swap the global stream.
151. Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes.
Each case is a question the project settled and the answer it chose. A fresh-context agent given only the README's premises answers each case; a case it answers differently says the goals no longer decide it. Written guidance can fail to change behaviour, which is why the goals get a test.
52. Record why this checkout lives in Claude Code's own directory, or move it out and link it in.
"One owner per value" and "Prefer the vendor-neutral option", in principles/technical-principles.md: git, Claude Code and loop-claude all write ~/.claude, kept apart only by the .gitignore allowlist, and the repo's records live in a vendor directory.