Files
claude-config/todo.md
T

47 KiB

todo

Scoring

Score = 100 * V / (2**S + 32)

Bar = 0.1

Next ID = 300

Items

ID Release Blocked by Exempt Item L E I R S Moves Score
296 0.8.0 Run the scoring run that releases 0.5.0 to 0.7.0 skipped, and make a due scoring run fail a check the worker cannot dismiss. 2 2 3 5 3 code health by 4 7.2
294 0.8.0 Check the files a commit stages against rule-hook's file rules, so a file a Bash command wrote is held to them as an Edit or Write is. 2 2 3 5 4 1.3 by 5 5.2
297 0.8.0 Have loop-claude compute the judge's taken-in items, release-ready and escalation fields from chunk-preflight, git and the worker's structured lines, and leave the judge only the verdict kind. 3 2 3 6 4 1.4 by 2, code health by 2 3.7
293 0.8.0 Run mutation tests on a chunk's changed lines in the worker's pre-review gate, through one command that picks the tool by language, and hand stability-reviewer the surviving mutants. 4 2 3 7 5 1.1 by 5, 1.4 by 4, 1.5 by 2 2.8
292 0.8.0 owner Rebuild the routing ladder for Fable 5.2, placing each of its efforts by the Artificial Analysis Intelligence Index beside Opus 5.5's and Sonnet 5.5's, and replace Fable 5.1 wherever it is named. 3 2 3 6 4 1.1 by 4, 1.4 by 3 2.4
295 0.8.0 Have chunk-pr merge wait for the forge's required checks itself, and give workers one command that waits for a background log or report file. 2 2 3 5 3 1.4 by 2, 1.5 by 2 1.6
299 0.8.0 Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt. 2 2 2 4 3 1.4 by 3 1.2
298 0.8.0 297 Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where judge-replay changes no verdict kind. 3 3 3 7 4 1.4 by 2, 1.5 by 1 1.0
287 0.9.0 Move the forge helpers Forge, forge_of, runner, open_pull, merged_pull, pull_at and gitea into one module that chunk_preflight, chunk_pr, deploy_release and larv_review all import from, so no tool imports another tool's internals for them. 3 1 3 5 5 code health by 3 3.4
284 0.9.0 owner Script each step that 0.7.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release. 1 2 1 2 2 1.4 by 3, 1.5 by 3 2.6
288 0.9.0 Point the two reading-cost decisions at code health and goal 1.4's counter-signal, in place of the goal 1.1 they cite. 1 2 1 2 1 code health by 1, 1.4 by 1 2.6
270 0.9.0 Move each check a review pass repeats across chunks, other than the mutation tests bin/mutate-changes runs, into the worker's pre-review gate, starting with the passes most often taken to a second round. 2 2 3 5 3 1.1 by 3, 1.4 by 3 2.5
77 0.9.0 Teach bin/metric-bench Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel. 4 2 1 5 6 1.1 by 7, 1.4 by 7 2.4
273 0.9.0 decision Re-tune the E + I steps of REVIEW_STEPS for 6 to 8 once 10 finished chunks have scored an E + I of 6 or more, from how well E + I orders their review rounds and 🔴 findings. 1 2 2 3 2 1.1 by 2, 1.4 by 3 2.3
268 0.9.0 End a run whose release is ready to cut with a notice on loop-claude's terminal, in place of an interrupt that waits for an answer. 2 2 3 5 3 1 by 5 2.2
171 0.9.0 decision Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised. 3 2 2 5 4 1.2 by 3, 1.4 by 3 2.0
264 0.9.0 Report the comprehension seats and the escalated subagents as subagent types of their own in routing-figures.md, with their dollars. 2 2 1 3 3 1.4 by 5 2.0
149 0.9.0 decision Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release. 4 2 3 7 5 1.1 by 7 1.8
165 0.9.0 Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant. 3 2 1 4 5 1.1 by 4, 1.4 by 3 1.8
151 0.9.0 decision Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes. 3 2 2 5 5 1.2 by 7 1.8
68 0.9.0 principle Test workflow-bench/compare.py's blinding and bench.py's run plan. 2 1 1 2 4 1.1 by 5 1.8
102 0.9.0 Give loop-claude's retry delay, and the file it reads the machine's memory from, environment overrides, so the black-box test stops replacing time.sleep and machine_load.memory.MEMINFO through sitecustomize. 1 1 2 2 3 1.1 by 4 1.7
135 0.9.0 Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop. 2 2 3 5 3 1.1 by 4 1.7
5 0.9.0 decision Key a project on its checkout's full path, and refuse to start while another session uses that key. 4 2 3 7 5 1.1 by 6 1.6
291 0.9.0 owner Let principles/technical-principles.md → "Sane defaults" allow a library whose README goals ask that it work with no config to default the clock and randomness in its public constructor, where a documented option overrides each. 1 2 3 4 1 1.2 by 3 1.4
156 0.9.0 decision Drop rank_todo.py's ranking of an old-shape table once no project under ~/code holds one. 2 1 3 4 4 1.1 by 4 1.4
196 0.9.0 decision Replay the architecture, principles, overrides and product-owner passes with review-replay on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it. 2 2 3 5 2 1.4 by 3 1.3
260 0.9.0 decision Compare the first 20 chunks whose worker got the reading rule with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and drop the rule where any rose. 1 2 1 2 2 1.4 by 3 1.3
51 0.9.0 principle Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints. 2 1 3 4 5 1.1 by 5 1.3
201 0.9.0 owner Save the human's tmux setups and restore them after a reboot, so work is up and running again quickly. 3 2 1 4 4 1.5 by 4 1.3
267 0.9.0 Keep waiting when an interrupt gets an empty answer, at the terminal or in the claude opened for it, in place of stopping the loop. 2 1 3 4 2 1.5 by 3 1.3
4 0.9.0 Name each chunk's archive folder by the chunk's start time, so <key>/ lists chunks in the order they ran. 3 2 3 6 3 1.1 by 3 1.3
187 0.9.0 Count the tokens a worker spends reading reviewers' reports, so goal 1.4's signal shows whether report files lowered what a review costs the worker's context. 2 2 1 3 3 1.4 by 3 1.2
266 0.9.0 Move the chunks archived under the key home/lilleman/.claude to the key claude-config, so the routing figures and score-outcomes count this repo's chunks as one project. 3 2 2 5 3 1.4 by 3 1.2
181 0.9.0 principle Have the test of a quoted answer's step-up assert the stepped rung's facts as literals, not through prompts.rungs_held. 1 1 1 1 1 1.1 by 2 1.0
192 0.9.0 principle Sort .gitignore's allowlist block, from !.claude/ to !workflow-bench/test_review_git.py, by code point, so rule-hook's ignorefile-sorted checks later edits to it again. 1 1 1 1 1 1.1 by 2 1.0
193 0.9.0 Narrow readme_goals's mark to Mark by a typed lookup in place of typing.cast, so a mark added to the regex but not the lookup fails at run time, and one added to the lookup but not Mark fails mypy. 1 1 3 3 1 1.1 by 2 1.0
194 0.9.0 Make test_waits_out_a_usage_limit_which_ends_a_run_of_failures expect a usage-limit wait independent of the wall-clock time that passes during the run. 1 1 1 1 1 1.1 by 2 1.0
202 0.9.0 Add CLAUDE.md → "Prose"'s rule "Write negations as people say them", with its examples, to skills/larv-review/standards/prose.md, so the prose reviewer holds prose to it. 1 2 2 3 1 1.1 by 2 1.0
265 0.9.0 Archive each wait that costs a chunk wall time, for a usage limit, low memory, a failed worker start or a heavy slot, with its start and end stamps, so goal 1.5's hours split waiting from work. 3 2 2 5 4 1.5 by 3 1.0
169 0.9.0 Have rule-hook's todo-rank-check catch a todo.md that the same Bash call edits and then commits. 3 1 3 5 3 1.1 by 2 0.8
174 0.9.0 In a chunk no item routed, ask the worker to name each todo.md item it takes, with its L and E + I, so loop-claude can route on an item it takes in. 1 2 3 4 1 1.1 by 1 0.5
197 0.9.0 Let the human edit what they type at loop-claude's terminal questions with the arrow keys, so Left moves the cursor in place of printing ^[[D. 2 2 3 5 2 1.5 by 1 0.4
1 0.10.0 question Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume. 1 2 1 2 1 0.0

Details

296. Run the scoring run that releases 0.5.0 to 0.7.0 skipped, and make a due scoring run fail a check the worker cannot dismiss.

The last scoring run in docs/comprehension-history.md is from 2026-10-29: overall 6.00, Locality 5.63, under the 7.0 floor. v0.5.0, v0.6.0 and v0.7.0 were cut without one, and larv-review places still prints that run's places, 254 commits stale. Workers twice dismissed needs-scoring-run's due line as preflight reading a stale branch's plan; find why before choosing the check. Without a run, the floor item the scoring run files under 7.0 never fires.

294. Check the files a commit stages against rule-hook's file rules, so a file a Bash command wrote is held to them as an Edit or Write is.

From 2026-11-08 to 2026-11-10, workers in other projects wrote 1,266 files through Bash (a python heredoc, sed -i, cat >) against 260 through Edit and Write, and rule-hook --help says it doesn't check a file a Bash command writes. So action-pin, image-pin, ignorefile-sorted and the attribution check miss most edits. rule-hook already checks at git commit for todo-rank-check, so checking staged files there follows a pattern the repo has. Ship item 192 in the same chunk: its unsorted .gitignore block is one symptom of this gap.

297. Have loop-claude compute the judge's taken-in items, release-ready and escalation fields from chunk-preflight, git and the worker's structured lines, and leave the judge only the verdict kind.

bin/loop_claude/cycle/judge.py already checks quotes in code (quotes_checked, grounded, re_ask_checked). The Reply fields items_taken_in, release_ready and escalation state facts the plan and git already hold, so code can derive them with certainty, and the judge's prompt shrinks to what needs judgement. Before switching, recompute each field for the 668 archived verdicts offline and count where code and judge disagree; read each disagreement and keep the judge for a field where code is wrong. Seats found quotes_checked hard to read (PR #397's scoring run: the misquote rule is split across judge.py and escalation_mark.py); make it easier to read where the change touches it.

293. Run mutation tests on a chunk's changed lines in the worker's pre-review gate, through one command that picks the tool by language, and hand stability-reviewer the surviving mutants.

Of stability-reviewer's 428 round-1 findings from 2026-11-02 to 2026-11-10, 223 (52%) were test gaps, such as a surviving mutant or an unpinned boundary, and 31 of its 50 round 2s held one. 133 of its 144 reports ran mutation tests by hand: today the reviewer picks and runs mutmut, Stryker, cargo-mutants or Infection itself, per agents/stability-reviewer.md step 5. A command run once before review, on the lines the branch changed, finds the same survivors deterministically, so the worker kills them before review and the reviewer judges only what remains. This takes over the stability half of item 270. Ship it alone: the nearest finished items, 227, 210, 180 and 73, changed 261 to 402 lines and took 3 to 8 rounds past the first.

L is 4, re-scored on 2026-11-11 after the first read: the change is S 6 or more, spans several components (a new bin/ tool, skills/larv-review/SKILL.md, agents/stability-reviewer.md, bin/rule_hook/heavy_command.py) and adds a new tool. mutmut 3 doesn't print the source line a mutant changes. In its mutants/ copy, each mutant x_<function>__mutmut_<n> keeps the original's lines, tabs included, so the first line other than the def where it differs from x_<function>__mutmut_orig sits at the same offset from the source function's def; on 2026-11-11 that mapped records/markdown_text.py's four survivors to their lines.

292. Rebuild the routing ladder for Fable 5.2, placing each of its efforts by the Artificial Analysis Intelligence Index beside Opus 5.5's and Sonnet 5.5's, and replace Fable 5.1 wherever it is named.

Fable 5.2 (claude-fable-5-2) has been released. The ladder in bin/model_routing/ladder.py tops out at Rung("claude-fable-5-1", "max"), and the decision routing: one ladder of (model, effort) rungs holds "while no newer model joins the catalog", so it is due for a rebuild.

Read the index again (https://artificialanalysis.ai/leaderboards/models), at each effort it scores, with its cost per task. Rebuild the ladder by the decision's cut: a rung stays only where no cheaper rung scores as high. Fable 5.2 may earn more than one rung, and a cheaper effort of it may push an Opus 5.5 rung off. Where the order changes which rung a project's model starts on, or how far the L and E + I steps reach, say so in the decision. The rungs are the human's choice (CLAUDE.md → "Working loop", "Ask, don't guess"), so propose the new ladder with the index's figures and get the human's approval before changing code.

Then replace claude-fable-5-1 where it is named: ladder.py, bin/routing_figures/prices.py (Fable 5.2's prices), settings.json → modelSettings, the tests in bin/model_routing/, bin/loop_claude/, bin/judge_replay/ and bin/rule_hook/, AGENTS.md, docs/decisions.md and presentation/index.html. Keep Fable 5.1 in prices.py while archived chunks that ran on it are still priced from there.

295. Have chunk-pr merge wait for the forge's required checks itself, and give workers one command that waits for a background log or report file.

In 0.7.0's 35 chunks, workers made 65 tea api calls in 26 chunks, mostly polling commit status and trying merge methods in turn, and ran 25 until … sleep loops that took 1,744 seconds. chunk-pr merge takes --gate-passed on the worker's word; waiting for the required checks itself makes that flag a fact, not a claim. This is the kind of step item 284 files, and can count as its first.

299. Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt.

The judge's verdict kind is the one judge decision that item 297 leaves an LLM. No single rule found so far separates interrupts: a "?" in the worker's final message appears in 48 interrupts and misses 66. A smaller LLM already failed the replay (docs/decisions.md → routing: on a model judge-replay covered, …: Sonnet returned another kind on 3 interrupts, Haiku missed 7). Untested are combined code rules, a decision table, and a small classifier trained and run on this machine on the archived verdicts (372 finished, 234 interrupt, 42 unfinished, 20 quoted). Run each offline, nothing sent to an outside service (goal 4), and count disagreements by kind; the bar is zero missed interrupts. A hosted decision model is out of scope, since it would send transcripts to an outside service.

298. Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where judge-replay changes no verdict kind.

153 Judge cost: entries total $109.76, a median of about $0.76 a verdict at every rung, and the cost follows the size of the records, not the transcript: about 76k characters in log and 188k in claude-config. In 224 of 251 recent verdicts (89%) the evidence sits in the worker's final message. A missed interrupt is the costly error, so the bar is zero changed interrupts and zero new rejections across the replay. It waits for item 297, which moves the judge's factual fields into code, since that shrinks the prompt first and changes what this cut measures. The judge is about 4.5% of dollars, so the saving is small; each cycle also gets faster.

287. Move the forge helpers Forge, forge_of, runner, open_pull, merged_pull, pull_at and gitea into one module that chunk_preflight, chunk_pr, deploy_release and larv_review all import from, so no tool imports another tool's internals for them.

Today the helpers live in two tools. Forge is defined in bin/chunk_preflight/gathered.py and imported by bin/chunk_preflight/choices.py, bin/chunk_preflight/facts.py, bin/chunk_preflight/main.py and bin/chunk_pr/main.py, and again through choices by bin/deploy_release/forge.py. forge_of lives in choices.py, which bin/chunk_pr/main.py, bin/deploy_release/cut.py and bin/larv_review/reviewer_header.py import. open_pull, runner and gitea live in bin/deploy_release/forge.py, which bin/chunk_pr/main.py and bin/larv_review/reviewer_header.py import. So deploy_release imports chunk_preflight, and chunk_pr and larv_review import both.

Both comprehension seats of PR #453's paired ruling (2026-11-09) named this web as a place that cost them time, and neither saw a reason for it: a second forge would need edits in chunk_preflight, deploy_release, chunk_pr and larv_review.

Put the seven helpers in a folder named by what it holds, such as bin/forge/, and import them from there. Keep each helper's behaviour and its tests. chunk_preflight/choices.py keeps pulls_command, which is preflight's own. Check AGENTS.md and docs/decisions.md for a mention of where a helper lives, and update it.

284. Script each step that 0.7.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.

lilleman asked, on 2026-10-16, for a standing audit each release, so the loop keeps moving work a program does as well from the model to scripts (goals 1.4 and 1.5). This audit reads the archived transcripts of the chunks that shipped release 0.7.0 for every step that chunks repeated by hand, before, during or after review. It keeps the steps whose answer a program computes the same way each time, such as a lookup, a check, a prompt assembled for a reviewer, a cleanup. File one item per such step, with the time and tokens it cost, into the next release with items left. Leave a step that needs judgement with the model. Run item 283's tool first, once it has shipped.

The chunk that ships this item files a copy of this audit in the release after this item's own, to audit this item's release, rewording "0.7.0" to the release that copy audits, and marks the copy owner, so the audit recurs each release until lilleman drops it.

288. Point the two reading-cost decisions at code health and goal 1.4's counter-signal, in place of the goal 1.1 they cite.

Both docs/decisions.md entries under reading cost: a rise is … and reading cost: larv-review measures each finished chunk … open "Serves goal 1.1: comprehension …". No goal says that any more: code health (CLAUDE.md → "Shared goal") holds comprehension, and since PR #455 a rising reading cost is goal 1.4's first counter-signal. Raised by the stability review of PR #455 as a 🟢 outside that change.

270. Move each check a review pass repeats across chunks, other than the mutation tests bin/mutate-changes runs, into the worker's pre-review gate, starting with the passes most often taken to a second round.

The review folders under ~/.local/state/larv-review/, review replays left out, hold a stability round 2 beside 137 of 211 round-1 reports (65%), product-owner 73 of 173 (42%) and prose 8 of 277 (3%), on 2026-11-08. The architecture and principles passes run up to five rounds in a release review, so they don't compare. From 2026-11-02 to 2026-11-10, 223 of stability's 428 round-1 findings (52%) were test gaps, and 31 of its 50 round 2s held one; bin/mutate-changes now runs the mutation tests that find most of them before review. Check which kinds of finding the other round 2s repeat, such as an unhandled error or a product-owner wording fix. Move only a check a program or the worker can run before review; leave judgement with the reviewer.

77. Teach bin/metric-bench Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.

2026-10-08, lilleman: no metric enters review until a rerun on more data shows it tracking the panel.

Before the rerun, set the AUC and top-k lift over chance that count as tracking, from a published threshold, so the rerun ends in a decision. The rerun needs places from a project in Go or TypeScript, the languages most other projects use, so the bench first measures those languages, run through each project's own tooling. Function and file length stay warnings to look into, never limits: a plain long function can be simple, and a hard one may need five-line pieces.

On 2026-10-08 only this repo's history named places: five scoring runs, all that day. The medians:

Metric AUC Top k
Cognitive complexity 0.66 0.26
Length 0.74 0.18
Shared state constant constant
Change spread 0.67 0.18

AUC is 0.5 for a coin toss, and top k scores 0.10 by chance. Shared state cannot be judged here: every unit in bin/ scores 0 on it. Every run's first place was a section of skills/larv-review/SKILL.md, which no code metric sees.

On 2026-10-09 a question bench asked a headless claude, allowed only Read, Grep and Glob, three questions a script answers from the syntax tree: which units call a unit, which units it calls, and which test files refer to it. It asked them about every unit the history names whose name is unique, plus as many units the history does not name. Five of the history's six scoring runs named such a unit. The tool is in commit 083cc78; it was deleted after the run.

Claude's misses ranked the named units at a coin toss. Top k scores 0.50 by chance on these units:

Metric AUC Top k
Question misses 0.49 0.50
Cognitive complexity 0.66 0.59
Length 0.75 0.69
Change spread 0.57 0.54

Claude's answers overlapped the truth by a median Jaccard index of 0.76. A run took in 62,643 new input tokens and cost $0.70; it did not record which model answered. Grep answers these questions whatever the code's shape, so no question enters review; a new question bench would need questions that take understanding the code, not finding it.

273. Re-tune the E + I steps of REVIEW_STEPS for 6 to 8 once 10 finished chunks have scored an E + I of 6 or more, from how well E + I orders their review rounds and 🔴 findings.

docs/decisions.md → routing: one ladder… records why lilleman kept the steps on 2026-11-08. Of the 36 chunks it measured, 1 scored an E + I of 6 or more and none 7 or more, so the steps for 6 to 8 are untested. Start once score-outcomes shows 10 finished chunks with an E + I of 6 or more. Then read its rank correlation for E + I, and the routed-chunks table of routing-figures.md in loop-claude's archive ($LOOP_CLAUDE_TRANSCRIPTS, default ~/.claude/loop-claude-transcripts). Propose new steps to lilleman, whose call they are.

268. End a run whose release is ready to cut with a notice on loop-claude's terminal, in place of an interrupt that waits for an answer.

On 2026-11-08 the human answered notice only: the loop prints that the release is ready to cut and ends; no interrupt, and nothing waits in loop-inbox.

Releases are the human's, and CLAUDE.md → "Working loop" already ends the loop once a release is ready to cut, so the ask settles nothing. 15 interrupts in the archive ask whether the human will cut a release, and about 6 of the 45 interrupts from 2026-11-02 to 2026-11-10 said a release was ready to cut; log's 2.5.0 was asked twice within an hour on 2026-11-10. On 2026-11-08 one of the 4 questions waiting in ~/.local/state/loop-claude/ is such an ask, plainpages-plugin-fastjira's 0.3.0 since 2026-11-03. Today a release ready to cut opens claude's session at once (bin/loop_claude/cycle/run.py ask), and the loop stops if the human leaves it with nothing typed. The change also touches what an interrupt covers in CLAUDE.md → "Vocabulary". Where the change touches ask, make it easier to read, as the seats named it.

171. Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.

Raised by the goals audit of PR #296. bin/routing_figures/calls.py groups a chunk's review rounds by the rung its worker started on, and calls for a routing change when a lower rung took more of them. Since PR #296 the reviewers may run above the worker's rung, on the one their item's E + I picks, and no figure groups rounds by it, though the archived worker line names the E + I it follows from. A cheap worker on an item of high E + I gets stronger reviewers, who may raise more rounds, and a call could then blame the worker's rung for them: an interrupt of lilleman that settles nothing (goal 1.2).

264. Report the comprehension seats and the escalated subagents as subagent types of their own in routing-figures.md, with their dollars.

Seats launch as general-purpose, so they hide inside that type: $301.24 of the $962.17 the figures of 2026-11-04 count, from 221 subagents. By 2026-11-10 general-purpose was the second-largest spend, $585 of about $2,385, and by transcript size about two thirds of it were seats (291 launches), then 177 security passes. Escalations are counted by trigger only, never in dollars. bin/routing_figures/chunk.py groups subagents by their subagent_type; a seat is told apart by its prompt, and an escalated subagent by the escalation marker its prompt opens with.

149. Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.

A release's capacity is a sum of its items' cost, as the formula counts it. Each project states its own; the human sets the first value per project. Nothing is deleted by score.

165. Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.

A test audit across lilleman's projects on 2026-10-16 found 10 of 33 sampled tests in this repo pinning wording that is not the contract. Examples of these and other weak tests: many tests in bin/loop_claude/test_loop_claude.py pin whole sentences of the question, archive, prompt or help text, up to about 750 characters. test_every_exit_1_stop_has_its_clause_in_help and test_every_failed_judge_run_has_its_clause_in_help parse the source with ast and list each Stop(...) and Failure(...) call by function name. bin/model_routing/test_ladder.py asserts len(LADDER) == 6. Rewrite each to the technical principles' test bullet: assert the facts the text must carry, such as an item ID, a rung or a model argument, and delete a test no real bug could fail. Before and after rewriting a test, run the mutation tool on the code it covers: every mutant the old test killed, the new one kills.

151. Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes.

Each case is a question the project settled and the answer it chose. A fresh-context agent given only the README's premises answers each case; a case it answers differently says the goals no longer decide it. Written guidance can fail to change behaviour, which is why the goals get a test.

68. Test workflow-bench/compare.py's blinding and bench.py's run plan.

"Most, if not all, code should have automated tests", in principles/technical-principles.md: compare.blind() keeps the comparison blind, and a version name it misses invalidates the bench silently.

The tests also cover workflow-bench/bench.py's permission_flags, with a probe of a headless claude given its flags. It is denied a read outside the granted trees, a bare git status, and wrapper calls with --output, --no-index, a glued -aF/abs, --fil=/abs or a path outside the clone. A cd <elsewhere> && python3 <wrapper> … acts on the clone only. It is allowed the wrapper's log, diff, show, status, ls-files, add and commit -m. workflow-bench/test_review_git.py tests the wrapper's refusals directly.

102. Give loop-claude's retry delay, and the file it reads the machine's memory from, environment overrides, so the black-box test stops replacing time.sleep and machine_load.memory.MEMINFO through sitecustomize.

Found by the principles audit of PR #242. The SITECUSTOMIZE block in bin/loop_claude/test_black_box.py swaps the global time.sleep, the shape "Functional core, imperative shell" bans, and the audit of PR #376 found the same block swapping machine_load.memory.MEMINFO, the constant in bin/machine_load/memory.py. An override of each lets the test run the code as production runs it.

135. Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop.

The maintainability senior and the inherited architect of PR #257's scoring run named fingerprint and exec_if_edited in bin/loop_claude/loop/reload.py: the fingerprint covers every bin/**/*.py, so editing statusline.py or rank_todo.py re-execs a running loop. Make fingerprint easier to read.

5. Key a project on its checkout's full path, and refuse to start while another session uses that key.

The key names the project's session, archive folder and models entry. project_key, in bin/transcript_archive/project_key.py, strips ~/code/ and then any leading /, so ~/code/a and /a share all three, and one loop can resume or delete the other's session.

bin/reading-cost keys the archive by the main checkout, the parent of git's common dir, since it runs in a chunk's worktree; loop-claude keys it by --show-toplevel, so a loop started inside a worktree archives where reading-cost never looks. Derive both keys one way.

291. Let principles/technical-principles.md → "Sane defaults" allow a library whose README goals ask that it work with no config to default the clock and randomness in its public constructor, where a documented option overrides each.

"Sane defaults" has a library ask for every input that can change between runs, the clock and randomness included, and leaves defaulting them to the code that wires a program together. A logging library's principles audit found that its constructor defaults both, as its first goal, useful with no config, asks. Its decision log records the defaults as a departure from the principle, and names this item as what ends it. Until then, each principles audit there may raise the defaults again.

2026-11-10, the human: keep the library's defaults, and give the principle an exception for this case, worded about as the tagline says. The exception has to square with "Functional core, imperative shell", which the same bullet cites and no other principle overrides.

156. Drop rank_todo.py's ranking of an old-shape table once no project under ~/code holds one.

While rank_todo.py ranks two shapes, projects follow two sets of scoring rules, against goal 1.3. Once every project has reshaped, drop OLD_SHAPE's weighing from bin/release_plan/ranking.py, RESHAPE, and the old shape's lines in rank_todo.py --help and rules/todo-items.md. Keep todo_table reading old tables: score_outcomes reads every past merge's todo.md. PR #283's paired ruling found the old-shape path made ranking.py's score, Scoring.weights, unknown_columns and todo_table.py's Moves harder to read; leave each easier, with one shape and moves that are always moves, and the Moves cell's rules in one place. Release 0.3.0's seats named ranking.py's score, weigh and rank and todo_table.py's OLD_SHAPE for the same fork: "every rule exists twice for the old table shape" (Mid A). Make weigh and rank easier to read too.

196. Replay the architecture, principles, overrides and product-owner passes with review-replay on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.

Item 159's chunk, PR #322, replayed only the prose and stability passes. Sonnet 5.5 at high missed 44 to 55% of what the project's rung found again in each, so the agent judged a replay of the other four unlikely to qualify a rung. The product-owner review of PR #322 asked that the plan keep them, so the ranking decides when they run. Each run costs hours of reviewer replays (goal 1.4), so the score keeps it low until a cheaper model joins the ladder. docs/decisions.md → routing: a review pass runs below the project's rung only on a rung that a replay of its rounds shows finding every 🔴 and 🟡 the project's rung finds… records the runs so far. The release review of 0.6.0 deleted the code that lowered a pass, since no rung had qualified; the decision says where it goes back.

260. Compare the first 20 chunks whose worker got the reading rule with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and drop the rule where any rose.

Start once 20 chunks have run under the rule. The rule is READING_RULE in bin/loop_claude/cycle/prompts.py, which every fresh worker prompt holds; git log --first-parent -S READING_RULE on the default branch finds the merge that added it. A chunk got it when its first worker started after that merge. Measure as docs/decisions.md → loop-claude: the worker's claude compacts its context before it fills a 200k window… measured the window: rounds past each pass's first and the 🔴, 🟡 and misread counts from the round commit titles, unfinished verdicts from the outcomes files, hours from the worker's first message to its last, and dollars from the chunk's cost-state entry. Compare means, since the decision weighs the rule by all spend.

51. Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints.

"Functional core, imperative shell", in principles/technical-principles.md: say and input_or in bin/loop_effects/effects.py take the effects but print to stderr or stdout themselves, and retried and the rest print through say. Two bare print(file=sys.stderr) calls remain outside them, in bin/loop_claude/cycle/interrupt.py and bin/loop_claude/project/model_choice.py. The tests swap the global stream.

201. Save the human's tmux setups and restore them after a reboot, so work is up and running again quickly.

The human asked on 2026-10-21 for tmux support, above all for saving tmux setups. Don't build anything until the human has said exactly what they need. Ask them at least:

  • What a setup holds, such as sessions, windows, panes, each pane's working directory and the command running in it, like one loop-claude per project.
  • When a setup is saved: on demand or on its own.
  • How a setup comes back after a reboot.
  • Whether the code that saves and restores setups must work on every machine that takes this checkout (goal 2), which decides whether that code lives in this repo.
  • What else tmux support should cover.

Weigh the widely used tmux-resurrect and tmux-continuum plugins against a script of our own. Then re-score this item from the answers and that choice, and split it into items where it is more than one chunk.

267. Keep waiting when an interrupt gets an empty answer, at the terminal or in the claude opened for it, in place of stopping the loop.

11 runs in the archive ended with "Stopped: the human is needed to answer the question above, but the interrupt ended with nothing typed", each leaving its chunk open until the human restarted the loop. That stop comes from human_typed in bin/loop_claude/cycle/interrupt.py, after the claude opened for /talk or for a release ready to cut ends with no human turn. An empty line at the terminal stops the loop the same way, in first_answer. Item 268 removes one of the cases that opens claude.

4. Name each chunk's archive folder by the chunk's start time, so <key>/ lists chunks in the order they ran.

Today the folder is the SHA-256 of the transcript's first timestamped line, which tells the human nothing; that line carries the start time. Drop the fallback to the first line, which every chunk of a project shares, and give the tests' fake transcripts ISO timestamps. bin/reading-cost takes a chunk's id from the folder's first 12 characters and skips a chunk whose id docs/reading-cost.md holds, so give it a way to match a figure written under the old name.

187. Count the tokens a worker spends reading reviewers' reports, so goal 1.4's signal shows whether report files lowered what a review costs the worker's context.

Before PR #310, reports reached the worker as the Agent call's result, a median 19k tokens per chunk (2026-10-08, 68 archived transcripts). Now a reviewer returns its verdict and one line per finding, and the worker reads a 🔴 or 🟡 finding's entry from the report file. reading-cost counts only tracked files (bin/reading_cost/measure.py), and routing-figures counts only the Agent call's result (bin/routing_figures/chunk.py), so neither sees the report reads. Found by the architecture review of PR #310.

266. Move the chunks archived under the key home/lilleman/.claude to the key claude-config, so the routing figures and score-outcomes count this repo's chunks as one project.

The archive holds 277 chunk folders under home/lilleman/.claude, from 2026-09-24 on, and 39 under claude-config (2026-11-10). The routing figures of 2026-11-04 list no project home/lilleman/.claude. They find a chunk's later fixes on the default branch of the checkout its key names (bin/routing_figures/archive.py measure_project), and /home/lilleman/.claude is a release checkout on a detached HEAD. Item 5 fixes how keys are derived, not the chunks already archived. ~/.claude/loop-claude-models.json still holds a home/lilleman/.claude entry; it is machine state outside the repo, so removing it is a one-off step on each machine, not part of the change.

181. Have the test of a quoted answer's step-up assert the stepped rung's facts as literals, not through prompts.rungs_held.

bin/loop_claude/test_loop_claude.py's test_a_quoted_answer_that_cites_an_escalation_trigger_steps_the_worker_up_before_it_resumes_and_archives_the_step builds its expected prompt from prompts.rungs_held, the code under test, so a bug there passes it. The technical principles' test bullet bans that shape. Assert the new rung and one review launch as literals, as the typed answer's step-up test does. Raised by the principles audit of PR #308.

192. Sort .gitignore's allowlist block, from !.claude/ to !workflow-bench/test_review_git.py, by code point, so rule-hook's ignorefile-sorted checks later edits to it again.

Found by review of PR #317. The block, .gitignore:17-142 on 2026-11-10, is out of order in several places: !bin/chunk-preflight sits below !bin/chunk_pr/, and !bin/loop-inbox below !bin/loop_claude/, though - sorts before _. LC_ALL=C sort -c on the block names the first one. ignore_breaches in bin/rule_hook/rules.py skips a block that was already unsorted before an edit, so while this one stays unsorted no later edit to it is checked. The unsorted block breaches the technical principle "Deterministic order wherever order carries no meaning".

193. Narrow readme_goals's mark to Mark by a typed lookup in place of typing.cast, so a mark added to the regex but not the lookup fails at run time, and one added to the lookup but not Mark fails mypy.

Found by review of PR #317. bin/release_plan/ranking.py's readme_goals writes typing.cast(Mark, mark or None), which asserts the type goal_weights branches on without checking it: a mark added to the regex reaches goal_weights as a value Mark lacks. mypy does not read the regex, so no lookup lets it catch that. Replace the cast with a lookup typed dict[str, Mark], from each mark the regex captures, and "" for none, to its Mark value. A mark added to the regex but not the lookup then raises KeyError at run time, and a value added to the lookup but not Mark fails mypy.

194. Make test_waits_out_a_usage_limit_which_ends_a_run_of_failures expect a usage-limit wait independent of the wall-clock time that passes during the run.

Found by review of PR #317. The black-box test, bin/loop_claude/test_black_box.py:609, not the unit test of the same name in test_loop_claude.py, which runs on a fake clock, sets resets_at from the test's time.time() and expects the wait, rounded to tens, to be 3720. When more than about 5 s pass between that read and the loop's, the wait rounds to 3710 and the test fails with no bug. Item 102 covers only replacing the time.sleep swap; this item changes the expected value, for example by asserting the wait against resets_at + 120 minus the time the loop read, or a bound on it.

202. Add CLAUDE.md → "Prose"'s rule "Write negations as people say them", with its examples, to skills/larv-review/standards/prose.md, so the prose reviewer holds prose to it.

Found by the prose review of PR #329. prose.md opens by calling itself the full rules that CLAUDE.md → "Prose" keeps a core of, yet it lacks that rule, so a prose reviewer whose bar is the standards files never sees it.

265. Archive each wait that costs a chunk wall time, for a usage limit, low memory, a failed worker start or a heavy slot, with its start and end stamps, so goal 1.5's hours split waiting from work.

No record holds these waits today. An outcomes file holds verdicts, stops, step-ups and the judge's cost (bin/transcript_archive/outcomes.py), and ~/.local/state/run-heavy/ holds only the slots' lock files. loop-claude waits out a usage limit and low memory in bin/loop_claude/cycle/retry.py; run-heavy waits for a slot in bin/run_heavy/. A heavy slot's wait happens inside a worker's Bash call, so loop-claude doesn't see it; run-heavy would write it where the archive can find it.

169. Have rule-hook's todo-rank-check catch a todo.md that the same Bash call edits and then commits.

The hook judges a call before it runs, and changed in bin/rule_hook/todo_check.py compares todo.md with the index and HEAD as they stand then. A call that edits todo.md and commits it, such as a heredoc script followed by git commit, passes unchecked: PR #296's commit 149108e did, and the override marker it carried was never logged. larv-review's gate still runs rank_todo.py --check, so an unranked plan stops at merge rather than at commit. A git pre-commit hook would see the staged file.

174. In a chunk no item routed, ask the worker to name each todo.md item it takes, with its L and E + I, so loop-claude can route on an item it takes in.

The principles pass of PR #301 found that only routed_by in bin/loop_claude/cycle/prompts.py asks the worker to name each item it takes with its L and E + I, and fresh uses it only where an item routed the chunk. In an unrouted chunk the judge has no levels to report, so neither the taken-in L nor the taken-in E + I lifts the worker or the judge.

197. Let the human edit what they type at loop-claude's terminal questions with the arrow keys, so Left moves the cursor in place of printing ^[[D.

bin/loop_effects/effects.py hands loop-claude Python's builtin input, and nothing under bin/ imports readline, so the terminal gets no line editing: fixing a typo means erasing back to it. loop-claude asks two questions at its terminal. effects.input reads the pick of the project's model, in project/model_choice.py's ask_for_model. effects.input_or reads the interrupt's answer, in cycle/interrupt.py's first_answer, straight from stdin, since it also watches loop-inbox. Importing readline in effects.py gives input line editing, at two costs; input_or would also need to read through input while it watches. It moves the prompt from stderr to stdout, while the question above it stays on stderr. test_black_box.py waits for the prompt on stderr. readline's bracketed paste takes a pasted block whole, so an answer can hold several lines. prompts.typed_answer ends the answer at its first blank line, so the judge would read the rest of such a paste as loop-claude's words. The product-owner review of PR #308 raised the interrupt's case. lilleman, 2026-10-18: "Not super-important, but a bit irritating."

1. Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.

It lives under ~/.claude, which .gitignore keeps from the remote, so a lost disk loses every unattended run's record. Back it up only to a destination private to me and encrypted at rest: the worker runs with permissions bypassed, so the archived tool results can hold credentials it read.

2026-10-02, the human: ai-sandbox is the VM loop-claude will run on, hosted on the server bender. bender-infra, the infrastructure-as-code project that manages bender, will back up ai-sandbox's data volume once ai-sandbox is rebuilt. The item waits for the human to say the rebuild has landed; do not remind them. Before writing the README line, check that ~/.claude lives on that volume and that the backup is private and encrypted at rest.

The README line covers the bench's transcripts in workflow-bench/logs/ too (goal 3).