Answer prose round 1: 0 🔴, 14 🟡; keep each routing reason in one entry, retitle the stability reviewer's decision, and say what each help line and comment means
This commit is contained in:
@@ -46,7 +46,7 @@ Decisions live in `docs/decisions.md`:
|
||||
- routing: a worker hands a task one rung up on each escalation trigger, up to two rungs above the rung its item picks unless the human agrees, and opens the subagent's prompt with the trigger so the routing figures count it
|
||||
- routing: a worker whose run ends asking for a higher rung, citing an escalation trigger the judge finds in the transcript, resumes one rung up next cycle, up to two rungs above the rung its item picks
|
||||
- routing: on a model judge-replay covered, the judge runs on the rung its chunk's item's E + I picks for a review, never below the project's rung
|
||||
- routing: the stability reviewer stays on the project's rung while the worker runs below it, until `stability-replay` shows a lower rung finding every 🔴 and 🟡 the project's rung finds
|
||||
- routing: the stability reviewer runs no lower than the project's rung while the worker runs below it, until `stability-replay` shows a lower rung finding every 🔴 and 🟡 the project's rung finds
|
||||
|
||||
## Working rules
|
||||
|
||||
|
||||
+1
-1
@@ -10,7 +10,7 @@
|
||||
- An item that restores a maintain goal the project breaks today is exempt as `defect`, and its Moves may name that maintain goal, weighed as the heaviest achieve goal. A defect ranks by its score like any item. Only two kinds rank ahead of the rest of their release: questions first, since an answer may change other items, then security items.
|
||||
- The stability reviewer applies `principles/technical-principles.md` as written, in place of its own copies of it. Its test-data check now asks for UUID v7, where it asked for randomised UUIDs.
|
||||
- larv-review's principles audit is a pass of its own, and runs on every chunk that changes code, config or tests, not only on one with an architectural decision. A chunk of prose or planning alone skips it.
|
||||
- `rule-hook` refuses an Agent call that launches a reviewer, a comprehension seat or the security pass on a rung below the one `loop-claude` names for it, and takes no override: pass the model and effort the refusal names. A seat or the security pass is held to the project's rung, and a reviewer to the reviewers' rung above.
|
||||
- `rule-hook` refuses an Agent call that launches a reviewer, a comprehension seat or the security pass on a rung below the one `loop-claude` names for it, and takes no override: pass the model and effort the refusal names.
|
||||
|
||||
## 0.1.0
|
||||
|
||||
|
||||
@@ -89,7 +89,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `rank_todo.py [--check] [todo.md]` scores and ranks a project's `todo.md`; `rank_todo.py --help` states the format.
|
||||
- `reading-cost [--reset] [record]` appends a figure for each chunk `loop-claude` finished to a project's `docs/reading-cost.md`, and says whether the figures rise; `reading-cost --help` states the measure.
|
||||
- `rule-hook [overrides]` blocks a tool call that breaks a rule a program can check, unless the call overrides the rule; `settings.json` → `hooks` runs it before each Bash, Edit, MultiEdit, Write and Agent call, and `rule-hook --help` states the rules and the override.
|
||||
- `score-outcomes [L E I S]` lists the finished items whose guessed scores sit nearest the given ones, with what their chunks measured, the median outcome at each guessed R and S, and how well L, E and I each order the chunks; `score-outcomes --help` states the measures.
|
||||
- `score-outcomes [L E I S]` lists the finished items whose guessed scores sit nearest the given ones, with what their chunks measured, the median outcome at each guessed R and S, and how well L, E and I each order the chunks by review rounds and 🔴 findings; `score-outcomes --help` states the measures.
|
||||
- `stability-replay <model>` replays archived stability reviews of the projects run on the model, on the code each reviewed, and prints the lowest rung below the project's that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again; `stability-replay --help` states the replay. Run it only in a sandbox, like loop-claude: each replayed reviewer has Bash and skips every permission prompt, and an archived prompt may name paths on the host.
|
||||
- `statusline.py` renders Claude Code's status line; `settings.json` → `statusLine` runs it.
|
||||
- `subagent-rung [--plan] <model> <L>` prints the Agent tool's model and effort for a subagent's task, picked by its L from the project's model, and with `--plan` never below the project's rung; `subagent-rung --help` states the rule.
|
||||
|
||||
@@ -24,7 +24,7 @@ def routed_by(route: Route, anchor: Rung | None) -> str:
|
||||
|
||||
|
||||
def review_launches(route: Route, anchor: Rung | None) -> str:
|
||||
"""Which reviews to launch on a rung other than the worker's, and on which; below the project's rung, anchor, also where to launch a subagent that plans. Empty off the ladder, and where every review runs on the worker's rung.
|
||||
"""Which reviews to launch on a rung other than the worker's, and on which; below the project's rung, anchor, also where to launch a subagent that files or scores todo.md items. Empty off the ladder, and where every review runs on the worker's rung.
|
||||
|
||||
For a worker on Sonnet 5.5 at high, on an Opus 5.5 project, routed by item 3 of E + I 5, it opens: ` Launch each reviewer with the Agent tool set to model opus, effort high, the rung item 3's E + I of 5 picks. Launch each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.`
|
||||
"""
|
||||
|
||||
@@ -46,13 +46,15 @@ interrupt run on one of the ladder's rungs: a model at an effort. As a chunk sta
|
||||
item in the next release that does not wait routes it: an item waits while it is marked question,
|
||||
or while an item that blocks it is still in the table. The item's L picks the worker's rung,
|
||||
stepping from the model's cheapest rung by WORKER_STEPS in bin/model_routing/ladder.py, below it
|
||||
for a low L; without such an item, or with no L, the chunk runs on the model's cheapest rung. The
|
||||
for a low L; without such an item, or with its L, E or I unscored, the chunk runs on the model's
|
||||
cheapest rung. The
|
||||
item's E + I picks the reviewers' rung, stepping up from the cheapest rung by REVIEW_STEPS there,
|
||||
never below the worker's rung. The worker's first prompt names the rung and the item, and asks the worker to
|
||||
name each item the chunk takes, with its L and E + I. The prompt has the worker launch each review
|
||||
that runs on a rung other than its own on that rung: the reviewers on theirs, and each
|
||||
comprehension seat and the security pass on the cheapest rung. Below the cheapest rung, it has the
|
||||
worker launch each subagent that plans on the rung subagent-rung --plan prints. The prompt after
|
||||
name each item the chunk takes, with its L and E + I. The prompt names the rung of each review that
|
||||
runs on a rung other than the worker's: the reviewers' rung for the reviewers, and the cheapest rung
|
||||
for each comprehension seat and the security pass. When the worker runs below the cheapest rung,
|
||||
the prompt has it launch each subagent that files or scores todo.md items on the rung
|
||||
subagent-rung --plan prints. The prompt after
|
||||
each unfinished verdict says this again, since a step-up may have moved the worker. A reviewer or
|
||||
seat that rules the work below standard may ask to rerun one rung up; each prompt names that rung.
|
||||
loop-claude hands the worker's claude its own rung, the project's and the reviewers', in
|
||||
@@ -68,8 +70,8 @@ the worker is below that limit. Each finished or unfinished verdict, stop and st
|
||||
archive names the worker's rung, with the item, L and E + I that routed the chunk; a rerun resumes
|
||||
an open chunk on the latest rung named there, and counts its unfinished verdicts afresh. A rung no
|
||||
longer on the ladder, or below the one its item's L picks, is dropped, and the chunk is routed
|
||||
afresh from todo.md, as is one an archived line names by R + C, which loop-claude routed on before
|
||||
L. A todo.md or an archive that cannot be read or parsed is said on stderr, and the chunk is routed
|
||||
afresh from todo.md. So is a chunk whose archived line names an R + C, as loop-claude wrote before
|
||||
it routed on L. A todo.md or an archive that cannot be read or parsed is said on stderr, and the chunk is routed
|
||||
without it.
|
||||
|
||||
The judge of a model listed in JUDGE_REPLAYED (bin/loop_claude/routing.py) runs on the rung the
|
||||
|
||||
@@ -58,7 +58,7 @@ def decoded_as(value: object, kind: typing.Any) -> typing.Any:
|
||||
if not isinstance(value, dict) or not required <= value.keys() <= shape.keys():
|
||||
return REJECTED
|
||||
fields = {name: decoded_as(field, shape[name]) for name, field in value.items()}
|
||||
# A dataclass that checks its fields, as Route does, names the check fits.
|
||||
# A dataclass that checks its fields, as Route does, does so in a static method named fits.
|
||||
if any(field in (REJECTED, RETIRED) for field in fields.values()) or not getattr(kind, "fits", lambda _: True)(fields):
|
||||
return REJECTED
|
||||
return kind(**fields)
|
||||
|
||||
@@ -34,5 +34,5 @@ def quoted(passage: str) -> str:
|
||||
|
||||
|
||||
def listed(names: typing.Sequence[str], conjunction: str) -> str:
|
||||
"""The names as `a, b and c`, with conjunction, such as and or or, before the last."""
|
||||
"""The names as `a, b and c`, with conjunction, such as `and` or `or`, before the last."""
|
||||
return names[0] if len(names) == 1 else f"{', '.join(names[:-1])} {conjunction} {names[-1]}"
|
||||
|
||||
@@ -61,7 +61,7 @@ LADDER = (
|
||||
)
|
||||
# The rungs a task's L steps its worker from the project's rung.
|
||||
WORKER_STEPS = {1: -2, 2: -1, 3: 0, 4: 1}
|
||||
# The rungs a task's E + I steps a review up from the project's rung; one under 5 steps none.
|
||||
# The rungs a task's E + I steps a review up from the project's rung; an E + I under 5 steps none.
|
||||
REVIEW_STEPS = {5: 1, 6: 2, 7: 3, 8: 4}
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ from model_routing.ladder import Rung
|
||||
|
||||
# How a worker opens the prompt of a subagent it escalates a task to, `Escalation trigger 2: …`: docs/decisions.md → `routing: a worker hands a task one rung up`.
|
||||
ESCALATION_MARKER = "Escalation trigger"
|
||||
# An archived worker line: the rung, then the item and the levels that routed it, if any. resume reads only LEVELS_ROUTED's levels, so a line an older loop-claude wrote, naming an R + C, routes its chunk afresh; the figures still read its rung.
|
||||
# An archived worker line: the rung, then the item and the levels that routed it, if any. route_from reads only LEVELS_ROUTED's levels, so a line an older loop-claude wrote, naming an R + C, routes its chunk afresh; archived_worker still reads its rung for the routing figures.
|
||||
ROUTED = re.compile(r"(?P<rung>.+?)(?:, routed by item (?P<item>[0-9]+)'s (?P<levels>.+))?")
|
||||
LEVELS_ROUTED = re.compile(r"L of (?P<likelihood>[0-9]+) and E \+ I of (?P<escape_and_impact>[0-9]+)")
|
||||
|
||||
@@ -32,7 +32,7 @@ def levels_fit(item: int, likelihood: int, escape_and_impact: int) -> bool:
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Route:
|
||||
"""A chunk's worker rung, and the plan item that picked it, 0 for none, with its L, which picked the worker's rung, and its E + I, which picks its reviewers'. A route holds only levels that levels_fit, so no rung is picked from a level off its scale.
|
||||
"""A chunk's worker rung and the plan item that picked it, 0 for none. The item's L picked the worker's rung; its E + I picks the reviewers'. A route holds only levels that levels_fit, so no rung is picked from a level off its scale.
|
||||
|
||||
unfinished counts the unfinished verdicts towards the next one-rung step; a reload hands it over with the rung.
|
||||
"""
|
||||
@@ -45,7 +45,7 @@ class Route:
|
||||
unfinished: int = 0
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# A guard against a programming error, which nothing catches: route_from and the reload decoder check fits before they build a route.
|
||||
# Raises on a programming error only, so nothing catches it: route_from and the reload decoder check levels_fit before they build a route.
|
||||
if not levels_fit(self.item, self.likelihood, self.escape_and_impact):
|
||||
raise ValueError(f"a route by item {self.item} cannot hold an L of {self.likelihood} and an E + I of {self.escape_and_impact}")
|
||||
|
||||
|
||||
+4
-4
@@ -43,7 +43,8 @@ A valid todo.md; ## Details and each item's detail are optional:
|
||||
goal.
|
||||
- The bar is judged on the Score as shown, to one decimal.
|
||||
- A new item takes Next ID, and Next ID then goes up by one; every ID stays below it.
|
||||
- An item's L, E and I are levels, from 1 to 4, 3 and 5, that ~/.claude/rules/todo-items.md anchors.
|
||||
- An item's L, E and I are levels that ~/.claude/rules/todo-items.md anchors: L from 1 to 4, E from 1
|
||||
to 3 and I from 1 to 5.
|
||||
rank_todo.py writes its R, L + E + I - 2, at least 7 when I is 5 and at least 5 when I is 4. Its S
|
||||
is a whole number from 1 to 10.
|
||||
- An item's Moves cell is empty, or names each goal it moves and how far, a whole number from 1 to
|
||||
@@ -58,9 +59,8 @@ A valid todo.md; ## Details and each item's detail are optional:
|
||||
loop. An ID at or above Next ID fails; one below it that the table no longer holds was finished
|
||||
or deleted, and is dropped.
|
||||
- ## Items holds the table and nothing else: an order goes in the scores or the Blocked by column.
|
||||
- The table holds the columns above. A table without Blocked by or R gains it. One without L, E and
|
||||
I fails, and so does one with a column this version does not know, since writing it would drop
|
||||
that column.
|
||||
- The table holds the columns above. One missing L, E or I fails, and so does one with a column this
|
||||
version does not know, since writing it would drop that column.
|
||||
- A table in the old shape, with C, A, G and Goals columns in place of L, E, I and Moves, still
|
||||
ranks, by a formula over R, S, A, G and W and a | Goal | W | table that weighs every README goal, W
|
||||
being the heaviest cited goal's weight; ranking it says to reshape it.
|
||||
|
||||
@@ -41,7 +41,7 @@ RULES: dict[RuleName, Rule] = {
|
||||
),
|
||||
# A rung is not a judgment call, and the refusal names the arguments to pass, so no marker waives it.
|
||||
"reviewer-rung": Rule(
|
||||
"a reviewer, a comprehension seat or the security pass's subagent runs on the rung loop-claude names for it: for a seat or the security pass the project's rung, and for a reviewer the highest of the project's rung, the worker's and the one its item's E + I picks, so a worker below the project's rung cannot hand a review to its own cheaper model",
|
||||
"a reviewer, a comprehension seat or the security pass's subagent runs on the rung loop-claude names for it: for a seat or the security pass the project's rung, and for a reviewer the highest of the project's rung, the worker's and the one the chunk's item's E + I picks, so a worker below the project's rung cannot hand a review to its own cheaper model",
|
||||
"an Agent call, in a claude loop-claude started, whose subagent type is an agent under agents/, or whose prompt opens with seat-prompts.md's opening sentence or the one larv-review's Security section sets",
|
||||
overridable=False,
|
||||
),
|
||||
|
||||
@@ -8,9 +8,9 @@ reads origin as last fetched and says so on stderr.
|
||||
|
||||
Git history is the record. A chunk is a merge commit on the default branch's first-parent line
|
||||
whose change removed items from todo.md, and changed more than todo.md and the records below; it
|
||||
shipped those items. Each item's guess is its scores in the todo.md the merge's first parent held:
|
||||
L, E, I, R and S, or, before L, E and I replaced C, its R and S, and its C where a table with A, G
|
||||
and Goals scored one. The chunk measures:
|
||||
shipped those items. Each item's guess is the scores it held in the todo.md of the merge's first
|
||||
parent: L, E, I, R and S. An item scored before L, E and I replaced C holds R and S, and C too where
|
||||
its table also had A, G and Goals. The chunk measures:
|
||||
|
||||
- lines changed: lines added plus lines deleted against the first parent, leaving out todo.md and
|
||||
the records tools append to, docs/comprehension-history.md and docs/reading-cost.md;
|
||||
@@ -23,7 +23,7 @@ and Goals scored one. The chunk measures:
|
||||
|
||||
A project that merges without merge commits, by fast-forward or squash, has no chunk to measure.
|
||||
|
||||
Given L, E, I and S, each on the scale ~/.claude/rules/todo-items.md anchors, it works out R from
|
||||
Given L, E and I, on the levels ~/.claude/rules/todo-items.md anchors, and S, it works out R from
|
||||
L, E and I, as rank_todo.py does, and lists the 5 finished items whose guesses sit closest, closest
|
||||
first: the least mean squared difference over the scores both hold, so R and S alone for an item
|
||||
scored before L, E and I, the newest first among equals. Each line gives what the item's chunk
|
||||
@@ -32,7 +32,7 @@ measured, which items shipped with it, and its tagline.
|
||||
Then the median of the outcome R and S each predict at each value they were guessed at: R against
|
||||
🔴 findings, S against lines changed. And how well L, E and I each order chunks by review rounds
|
||||
past each pass's first and by 🔴 findings: Spearman's rank correlation, from -1 to +1, where +1
|
||||
orders them exactly; none where every chunk held one value. It counts only chunks that shipped one
|
||||
orders them exactly; none where every chunk held the same level, or the same outcome. It counts only chunks that shipped one
|
||||
item, since a chunk measures all its items at once, and reports a score only once 10 such chunks
|
||||
hold it and its outcomes.
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ class Outcome:
|
||||
Measure = tuple[str, typing.Callable[[Outcome], int | None]]
|
||||
# The scores whose median outcome the report gives at each guessed value.
|
||||
MEDIANS: dict[str, Measure] = {"R": ("🔴 findings", lambda outcome: outcome.findings[0] if outcome.findings else None), "S": ("lines changed", lambda outcome: outcome.lines)}
|
||||
# The levels R is built from, for each of which the report gives how well it orders chunks by both outcomes.
|
||||
# The levels R is built from; for each, the report gives how well it orders chunks by review rounds and by 🔴 findings.
|
||||
ORDERED = ("L", "E", "I")
|
||||
# The scores render shows, in this order; C only on an item scored before L, E and I replaced it.
|
||||
SHOWN = ("L", "E", "I", "R", "S", "C")
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Replay archived stability rounds on the rungs below the project's, to find which rungs may run the stability reviewer: docs/decisions.md → `routing: the stability reviewer stays on the project's rung`.
|
||||
"""Replay archived stability rounds on the rungs below the project's, to find which rungs may run the stability reviewer: docs/decisions.md → `routing: the stability reviewer runs no lower than the project's rung`.
|
||||
|
||||
usage: stability-replay <model>
|
||||
|
||||
@@ -20,7 +20,7 @@ that misses one. For each replay, claude on the project's rung matches the repor
|
||||
on record, and counts a finding only where the report raises it as 🔴 or 🟡. The tool prints each
|
||||
finding a rung missed and the lowest rung that, like each rung above it, missed none. That rung is
|
||||
the evidence for running the stability reviewer on the worker's rung whenever the worker runs at
|
||||
or above it; docs/decisions.md → `routing: the stability reviewer stays on the project's rung`
|
||||
or above it; docs/decisions.md → `routing: the stability reviewer runs no lower than the project's rung`
|
||||
records the run.
|
||||
|
||||
Like the loop, it tries claude three times running before it counts a reply as failed. A failed
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Which archived stability rounds to replay, the code each reviewed, and the report: docs/decisions.md → `routing: the stability reviewer stays on the project's rung`."""
|
||||
"""Which archived stability rounds to replay, the code each reviewed, and the report: docs/decisions.md → `routing: the stability reviewer runs no lower than the project's rung`."""
|
||||
import dataclasses
|
||||
import datetime
|
||||
import pathlib
|
||||
|
||||
+2
-2
@@ -25,7 +25,7 @@ class Shape:
|
||||
"""A shape of the items table: its columns, and how its goals cell reads and writes."""
|
||||
# The score columns, in the order the table holds them.
|
||||
score_columns: tuple[str, ...]
|
||||
# The columns a reader does without: a missing BLOCKED_BY reads as blocked by nothing, a missing score as no score, and a missing Score as no score yet.
|
||||
# The columns a reader does without: a missing BLOCKED_BY reads as blocked by nothing, a missing score column as that score unset, and a missing Score column as not yet ranked.
|
||||
optional: tuple[str, ...]
|
||||
# The columns rank_todo.py needs to rank a row, so no chunk starts on an item whose risk is unscored.
|
||||
risk_columns: tuple[str, ...]
|
||||
@@ -74,7 +74,7 @@ def write_cited(moves: Moves) -> str:
|
||||
return ", ".join(goal for goal, _ in moves)
|
||||
|
||||
|
||||
# L, E and I are optional, since score_outcomes reads tables from before they replaced C out of history; R too, since rank_todo.py writes it from them.
|
||||
# L, E and I are optional, since score_outcomes reads, out of git history, tables from before they replaced C; R too, since rank_todo.py writes it from them.
|
||||
SHAPE = Shape(("L", "E", "I", "R", "S"), (BLOCKED_BY, "L", "E", "I", "R", "Score"), ("L", "E", "I"), ("R",), MOVES, parse_moves, write_moves)
|
||||
# The shape from before MOVES replaced A, G and Goals, and L, E and I replaced C, which older todo.md files hold; one from before C joined lacks it.
|
||||
OLD_SHAPE = Shape(("R", "S", "C", "A", "G"), (BLOCKED_BY, "C", "Score"), (), (), "Goals", parse_cited, write_cited)
|
||||
|
||||
+12
-10
@@ -62,7 +62,7 @@
|
||||
|
||||
2026-10-15, lilleman asked for it through todo item 144, implemented by claude. Serves goal 1.1: a cheap reviewer that finds the work below standard can have a more capable one check that ruling, and work below standard gets a more capable rung to fix it.
|
||||
|
||||
The work is below standard where systems-architect returns BLOCK, a reviewer reports a finding against larv-review's `standards/quality.md`, or `record-ruling` counts a comprehension seat's vote in a paired ruling as worse. That reviewer or seat may ask, and the worker then relaunches it on the rung loop-claude's prompt names, one above the rung it ran on. `bin/loop_claude/cycle/prompts.py`'s `reruns` names it: the seats run on the project's rung, and the reviewers on the rung `review_rungs` in `bin/model_routing/ladder.py` gives them. Off the ladder, or for a reviewer or seat already on the top rung, the prompt names none, and its report stands.
|
||||
The work is below standard where systems-architect returns BLOCK, a reviewer reports a finding against larv-review's `standards/quality.md`, or `record-ruling` counts a comprehension seat's vote in a paired ruling as worse. That reviewer or seat may ask, and the worker then relaunches it on the rung loop-claude's prompt names, one above the rung it ran on. `bin/loop_claude/cycle/prompts.py`'s `reruns` names it: the seats run on the project's rung, and the reviewers on the highest of the worker's rung and the one their item's E + I picks. Off the ladder, or for a reviewer or seat already on the top rung, the prompt names none, and its report stands.
|
||||
|
||||
A reviewer's relaunch gets the report that asked and confirms or dismisses each of its findings with a reason, so no finding goes unanswered; a seat's relaunch reads afresh, since a seat reports no findings. The relaunch's report stands in place of the one that asked, in the same round, so it adds no round, and a relaunch cannot ask again. So the round cap still bounds a pass, and one round launches a reviewer at most twice. The ruling that stands is the fourth escalation trigger in `CLAUDE.md`. lilleman ruled that no ruling raises the judge's rung, which `routing: on a model judge-replay covered` sets.
|
||||
|
||||
@@ -130,9 +130,9 @@ Valid while the ladder orders its rungs by competence.
|
||||
|
||||
2026-10-02, lilleman. Serves goal 1: findings were filed faster than chunks closed them, so the plan grew without telling what mattered. Each item is scored as `~/.claude/rules/todo-items.md` says; this repo's formula counts value highest and size least. An item under the bar is not kept; this repo's bar started at 10 and was set to 9 the same day, purging the 17 items under it. An Exempt cell keeps a row past the bar where a rule needs it kept. lilleman retired the two decisions whose plans the purge removed: the stats line in the transcript archive, and three benches before a workflow change merges. A finding under the bar is recorded nowhere, so a later review may raise it again, and the run rescores it. That costs one scoring; a ledger of dropped findings would regrow the list the bar caps. larv-review's "Round cap" → "Done" counts a finding named under the bar as settled, so one raised again holds no pass open. `rank_todo.py --check` is part of larv-review's gate, since only a check keeps the stored Score from going stale. The format is mandatory in every project, as the premises are (goal 1.3); the formula and bar are each project's own. A script ranks the table so a re-rank after each addition or edit is deterministic and cheap. A 🟡 against a technical principle is exempt as `principle`, because its audience value scores low and the bar would drop every one. A 🟢 against a principle is scored like any other finding. 2026-10-08, lilleman: the goal weights in `todo.md` depart from an even spread by position, which would weigh goal 4, leaking nothing, at 0.11. Valid while one human sets the bar and reads the ranking, and while the items marked `principle` stay few enough to rank.
|
||||
|
||||
2026-10-15, lilleman: the scores get a researched method, which `todo.md` items 149, 151 and 152 build, after PR #283 shipped the ranking and PR #296 the risk scores; the evidence is in `docs/scoring-and-goals-research.md`. R becomes L + E + I − 2: the chance the change holds a defect, set first by its size; the chance the defect escapes tests, review and monitoring; and the worst impact dimension, each about ×10 per level. I5, irreversible, holds R at 7 or more, and I4 at 5 or more, since a severe enough consequence outranks how unlikely it is. C goes from every table and tool: what it scored beyond size, spread and novelty, which L already holds, is complexity, which adds almost nothing once size is accounted for. A task's R alone then picks its rung. Items rank by V / C, value per cost, the cost being 2^S, and V sums each cited achieve goal's sum-reciprocal weight times how far the item moves it. A goes, and the Audience says whose value the goals serve. R stays out of the ranking and sets how carefully an item is done; an item that lowers risk scores it as value. Each release fills by priority up to a capacity it states, in place of the bar. Each README goal is `maintain`, which vetoes an item that breaks it, or `achieve`, with a signal, a counter-signal and a one-line reason, as this README's Goals now mark them; each project keeps settled decisions to test its goals against. Each part of this method holds once the item that builds it ships. 2026-10-16, lilleman: this replaces "A task's R alone then picks its rung". A task's worker routes on its L, and its reviewers on its E + I (goals 1.1 and 1.4). How much a change does, and how new that is, make a worker err. How far a defect escapes, and what it breaks, set how hard review must look. I's reach counts every project and session that runs the change, so a change to a tool every session runs scores I at level 3, every consumer affected.
|
||||
2026-10-15, lilleman: the scores get a researched method, which `todo.md` items 149, 151 and 152 build, after PR #283 shipped the ranking and PR #296 the risk scores; the evidence is in `docs/scoring-and-goals-research.md`. R becomes L + E + I − 2: the chance the change holds a defect, set first by its size; the chance the defect escapes tests, review and monitoring; and the worst impact dimension, each about ×10 per level. I5, irreversible, holds R at 7 or more, and I4 at 5 or more, since a severe enough consequence outranks how unlikely it is. C goes from every table and tool: what it scored beyond size, spread and novelty, which L already holds, is complexity, which adds almost nothing once size is accounted for. Items rank by value per cost, V / 2^S, and V sums each cited achieve goal's sum-reciprocal weight times how far the item moves it. A goes, and the Audience says whose value the goals serve. R stays out of the ranking and sets how carefully an item is done; an item that lowers risk scores it as value. Each release fills by priority up to a capacity it states, in place of the bar. Each README goal is `maintain`, which vetoes an item that breaks it, or `achieve`, with a signal, a counter-signal and a one-line reason, as this README's Goals now mark them; each project keeps settled decisions to test its goals against. Each part of this method holds once the item that builds it ships. 2026-10-16, lilleman: a task's L and E + I pick its rungs in place of R, as `routing: one ladder` records.
|
||||
|
||||
2026-10-16, PR #283: items rank by `Score = 100 * V / 2**S`. 2**S is the cost; the 100 lets a score read to one decimal. V sums, over the achieve goals an item's Moves cell names, each goal's weight times how far the item moves it, 1 to 10. The goal ranked k of n achieve goals in reading order weighs (1/k) / (1 + 1/2 + … + 1/n), the sum-reciprocal weight. A Moves column replaces A, G and Goals, and `rank_todo.py` derives the weights from README.md, so the goal weights table and the 2026-10-08 weights are gone. An item that cited only maintain goals now names the achieve goal it moves, or, as a defect restoring one, that maintain goal, as the entry below says. A panel of three readers seated as lilleman picked the spelling: two put a Moves column holding `1.1 by 8, 1.2 by 5` first, and one put `1.1: 8` first; two chose `2**S`, since a Python reader takes `^` for XOR; all three chose one decimal of `100 * V / 2**S` over three significant digits or a log. The bar moved to 0.1, lilleman's call answering the review of PR #283: on one-decimal scores it drops an item scoring under 0.05, in practice one that moves no achieve goal, and item 149 revisits it. A table in the old shape still ranks by the formula and weights it states, and ranking it says to reshape it, so no other project's commits stop; reshaping it is that project's only work until done, as `CLAUDE.md` → "Working loop" says. PR #296 dropped the C column, and put L, E and I right before R, which `rank_todo.py` writes from them: of three readers seated as lilleman, two ranked that order first, since a reader meets each level before the R it builds and R sits beside S, and the third ranked it second, behind S before the levels. The old shape's ranking is valid until `todo.md` item 156.
|
||||
2026-10-16, PR #283: items rank by `Score = 100 * V / 2**S`. 2**S is the cost; the 100 lets a score read to one decimal. V sums, over the achieve goals an item's Moves cell names, each goal's weight times how far the item moves it, 1 to 10. The goal ranked k of n achieve goals in reading order weighs (1/k) / (1 + 1/2 + … + 1/n), the sum-reciprocal weight. A Moves column replaces A, G and Goals, and `rank_todo.py` derives the weights from README.md, so the goal weights table and the 2026-10-08 weights are gone. An item that cited only maintain goals now names the achieve goal it moves, or, as a defect restoring one, that maintain goal, as the entry below says. A panel of three readers seated as lilleman picked the spelling: two put a Moves column holding `1.1 by 8, 1.2 by 5` first, and one put `1.1: 8` first; two chose `2**S`, since a Python reader takes `^` for XOR; all three chose one decimal of `100 * V / 2**S` over three significant digits or a log. The bar moved to 0.1, lilleman's call answering the review of PR #283: on one-decimal scores it drops an item scoring under 0.05, in practice one that moves no achieve goal, and item 149 revisits it. A table in the old shape still ranks by the formula and weights it states, and ranking it says to reshape it, so no other project's commits stop; reshaping it is that project's only work until done, as `CLAUDE.md` → "Working loop" says. PR #296 dropped the C column, and put L, E and I right before R, which `rank_todo.py` writes from them: of three readers seated as lilleman, two ranked that order first, since a reader meets each level before the R it builds and R sits beside S, and the third ranked it second, after an order with S before the levels. The old shape's ranking is valid until `todo.md` item 156.
|
||||
|
||||
2026-10-16, lilleman, answering the review of PR #283. An item that restores a maintain goal the project breaks today is a defect: exempt as `defect`. Its Moves name that maintain goal and how much of it is broken, and the goal weighs as the heaviest achieve goal, since a broken maintain goal outranks any achieve goal. Every defect ranks by its score, so a trivial one waits behind valuable work. Only questions and then security findings rank first in their release: a question's answer may change other items, and a question waits for the human anyway, so ranking it first delays no chunk. An item names the most specific goal it moves, and a parent only for a move none of its sub-goals covers, so a sub-goal's move does not count twice; `rank_todo.py` refuses a goal named beside its own sub-goal. Each project reshapes its own `todo.md`; `CLAUDE.md` → "Working loop" makes the reshape the only work until done, as it makes writing missing premises. An item that makes this project easier to read moves goal 1.4 as well as 1.1, since reading cost is measured in tokens per chunk; a goal of its own for readability is left for later. Cost gains a fixed part, F in `Score = 100 * V / (2**S + F)`: every chunk carries a cost its size does not shrink, the reading before its first commit, about 130k tokens, and its review passes, so `2**S` alone overrated tiny items. F is 8, about one S 3 change, until `todo.md` item 158 measures it.
|
||||
|
||||
@@ -172,7 +172,7 @@ Valid while no published standard sets either number, the reading-cost window st
|
||||
|
||||
## rule-hook: a PreToolUse hook enforces each rule a program can check, a `# rule-override <rule>: <reason>` marker overrides one, and larv-review's overrides pass rules on each override before merge
|
||||
|
||||
2026-10-12, lilleman. Serves goals 1.2 and 1.3. Anthropic's best-practices page calls `CLAUDE.md` advisory and hooks deterministic, so a rule that must hold every time moves into `bin/rule-hook`, and `CLAUDE.md` keeps it only where the hook cannot check it. A Claude Code hook ties the check to one vendor ("Prefer the vendor-neutral option", `principles/technical-principles.md`), which is the target itself here: this repo configures Claude Code. What the tie buys: the hook blocks before a write lands, Edit and Write calls included, in every project with no per-repo install; a git `commit-msg` hook sees only commits, and CI sees them only after the push. The hook checks each write, so the agent fixes the line it just wrote; checking the staged diff at commit time would also cover files a Bash command writes, at the cost of later feedback. The marker's spelling was picked by a panel of three fresh readers, two to one. Each override is logged in the checkout's git directory, which is never pushed (goal 4). The plan had loop-claude's judge to rule on overrides inside a loop; lilleman approved larv-review's overrides pass in every run, because it rules before merge and an outrageous override still reaches the human through its ask. 2026-10-16, lilleman, answering the question plan item 166 asked: `reviewer-rung` takes no marker, since a rung is no judgment call and the refusal names the arguments to pass, as the comment above it in `bin/rule_hook/rules.py` says; `routing: a chunk's worker runs on the rung its item's L picks` names the rungs it holds a launch to. Valid while a PreToolUse hook that exits 2 blocks the call and hands its stderr to the model, and while an agent commits through Claude Code's Bash tool.
|
||||
2026-10-12, lilleman. Serves goals 1.2 and 1.3. Anthropic's best-practices page calls `CLAUDE.md` advisory and hooks deterministic, so a rule that must hold every time moves into `bin/rule-hook`, and `CLAUDE.md` keeps it only where the hook cannot check it. A Claude Code hook ties the check to one vendor ("Prefer the vendor-neutral option", `principles/technical-principles.md`), which is the target itself here: this repo configures Claude Code. What the tie buys: the hook blocks before a write lands, Edit and Write calls included, in every project with no per-repo install; a git `commit-msg` hook sees only commits, and CI sees them only after the push. The hook checks each write, so the agent fixes the line it just wrote; checking the staged diff at commit time would also cover files a Bash command writes, at the cost of later feedback. The marker's spelling was picked by a panel of three fresh readers, two to one. Each override is logged in the checkout's git directory, which is never pushed (goal 4). The plan had loop-claude's judge to rule on overrides inside a loop; lilleman approved larv-review's overrides pass in every run, because it rules before merge and an outrageous override still reaches the human through its ask. 2026-10-16, lilleman, answering plan item 166: `reviewer-rung` is the one rule no marker overrides; the comment above it in `bin/rule_hook/rules.py` says why. Valid while a PreToolUse hook that exits 2 blocks the call and hands its stderr to the model, and while an agent commits through Claude Code's Bash tool.
|
||||
|
||||
## rules/: `CLAUDE.md` holds the worker's rules that no file path can scope, and each topic's rules for the worker live in a `rules/` file that Claude Code loads by its `paths:` frontmatter
|
||||
|
||||
@@ -232,7 +232,7 @@ Valid while a mutation run on a chunk's changed lines stays cheap beside the rev
|
||||
|
||||
The rungs, least competent first: Haiku 5.5 at low, medium, high, xhigh and max; Sonnet 5.5 at high; Opus 5.5 at medium, high, xhigh and max; Fable 5.1 at max. The order and the cut come from the Artificial Analysis Intelligence Index (https://artificialanalysis.ai/leaderboards/models, read 2026-10-14), which scores each model at each effort and gives a cost per task. A rung stays only where no cheaper rung scores as high, so a step up always buys more score (goal 1.4). That cut drops Sonnet 5.5 at low, medium, xhigh and max, Opus 5.5 at low, and every Fable 5.1 rung, and every tie in the index falls among the dropped rungs. lilleman put Fable 5.1 at max back on top: the index scores it under Opus 5.5 at high, but Anthropic's model catalog names Fable the most capable model, and for the hardest tasks lilleman wants the highest quality at any price.
|
||||
|
||||
A project's rung is its model's cheapest rung, which for Opus 5.5 is its default effort, medium. A model off the ladder routes nothing: claude runs it at the effort its settings give. A task's L steps its worker from the project's rung: 1 two rungs down, 2 one down, 3 none, 4 one up. Its E + I steps its reviewers up from the project's rung: 4 or less none, 5 one up, 6 two up, 7 three up, 8 four up. A step past either end stops there. A worker errs more the more it changes and the newer that is, which L scores; review must look harder the further a defect would escape and the more it would break, which E + I scores (goals 1.1 and 1.4). Size sets L, so a large change runs its worker higher. The steps spread the values this repo's plan held when rescored on 2026-10-16, L 1 to 4 and E + I 2 to 7, so a worker on most items runs at or below the project's rung, and a review climbs only where a defect would escape far or break much. Until then, from 2026-10-14, one band of R + C picked the worker's rung. lilleman approved these steps on 2026-10-16 as a first guess: `score-outcomes` reports how well L, E and I each order finished chunks by review rounds and 🔴 findings, and `todo.md` item 170 re-tunes the steps once 10 chunks have shipped under them. A change to the ladder or the steps is lilleman's.
|
||||
A project's rung is its model's cheapest rung, which for Opus 5.5 is its default effort, medium. A model off the ladder routes nothing: claude runs it at the effort its settings give. A task's L steps its worker from the project's rung: 1 two rungs down, 2 one down, 3 none, 4 one up. Its E + I steps its reviewers up from the project's rung: 4 or less none, 5 one up, 6 two up, 7 three up, 8 four up. A step past either end stops there. A worker errs more the more it changes and the newer that is, which L scores; review must look harder the further a defect would escape and the more it would break, which E + I scores (goals 1.1 and 1.4). Size raises L, so a large change runs its worker higher. The steps spread the values this repo's plan held when rescored on 2026-10-16, L 1 to 4 and E + I 2 to 7, so a worker on most items runs at or below the project's rung, and a review climbs only where a defect would escape far or break much. lilleman approved these steps on 2026-10-16 as a first guess: `score-outcomes` reports how well L, E and I each order finished chunks by review rounds and 🔴 findings, and `todo.md` item 170 re-tunes the steps once 10 chunks have shipped under them. A change to the ladder or the steps is lilleman's.
|
||||
|
||||
Valid while the index ranks these models, and while no newer model joins the catalog.
|
||||
|
||||
@@ -240,9 +240,11 @@ Valid while the index ranks these models, and while no newer model joins the cat
|
||||
|
||||
2026-10-14, lilleman. Serves goals 1.1 and 1.4.
|
||||
|
||||
Only chunks run below the project's rung can prove that a cheaper worker holds quality (goal 1.1). So on 2026-10-14 lilleman chose a trial on real chunks: a replay of past chunks would spend on work that never ships (goal 1.4). Review holds quality meanwhile. While the worker runs below the project's rung, loop-claude's prompt has it launch each reviewer, each comprehension seat and the security pass's subagent on the project's rung, through the Agent tool's model and effort. The worker runs a full scan only from such a subagent, because the `claude-security` plugin's researchers and verifiers inherit the model that starts them. A subagent that cannot run the scan leaves it unavailable, so larv-review runs `/security-review` in its place. lilleman chose the project's rung, medium, knowing `settings.json` runs Opus 5.5 at xhigh, so a routed worker runs below the effort an interactive session takes.
|
||||
Only chunks run below the project's rung can prove that a cheaper worker holds quality (goal 1.1). So on 2026-10-14 lilleman chose a trial on real chunks: a replay of past chunks would spend on work that never ships (goal 1.4). Review holds quality meanwhile. While the worker runs below the project's rung, loop-claude's prompt has it launch each comprehension seat and the security pass's subagent on the project's rung, and each reviewer there too unless its item's E + I picks a higher rung, through the Agent tool's model and effort. The worker runs a full scan only from such a subagent, because the `claude-security` plugin's researchers and verifiers inherit the model that starts them. A subagent that cannot run the scan leaves it unavailable, so larv-review runs `/security-review` in its place. lilleman chose the project's rung, medium, knowing `settings.json` runs Opus 5.5 at xhigh, so a routed worker runs below the effort an interactive session takes.
|
||||
|
||||
While the worker runs above the project's rung, loop-claude's prompt has it launch the comprehension seats and the security pass's subagent on the project's rung too. So the seats score every chunk on one rung, and the security pass spends no more than the project chose. The prompt after each unfinished verdict repeats this, since a step-up may have taken the worker above the project's rung. The other reviewers inherit the worker's higher rung. On 2026-10-15 lilleman dropped routing them by R + C instead: a worker climbs because its chunk proved harder than its score said, which is when a cheaper reviewer is the bigger risk. On 2026-10-16 lilleman routed them on their item's E + I as well, which may lift them above both rungs: how far a defect would escape, and what it would break, set how hard review must look (goal 1.1). loop-claude names that rung in the worker's prompt, and hands it to the worker's claude beside the project's and the worker's, so rule-hook holds each reviewer launch to it. A low E + I never steps a reviewer below the project's rung: on 2026-10-16 lilleman chose to lower a pass only where a replay shows a lower rung finding every 🔴 and 🟡 the project's rung finds, which `todo.md` item 159 measures for each pass. A reviewer or seat that rules the work below standard may rerun one rung up: `review: a reviewer or comprehension seat that rules the work below standard`.
|
||||
While the worker runs above the project's rung, loop-claude's prompt has it launch the comprehension seats and the security pass's subagent on the project's rung too. So the seats score every chunk on one rung, and the security pass spends no more than the project chose. The prompt after each unfinished verdict repeats this, since a step-up may have taken the worker above the project's rung. The other reviewers never run below the worker's rung: a worker climbs because its chunk proved harder than its score said, which is when a cheaper reviewer is the bigger risk. A reviewer or seat that rules the work below standard may rerun one rung up: `review: a reviewer or comprehension seat that rules the work below standard`.
|
||||
|
||||
On 2026-10-16 lilleman routed the reviewers on their item's E + I as well, on either side of the project's rung, which may lift them above both the worker's rung and the project's: how far a defect would escape, and what it would break, set how hard review must look (goal 1.1). loop-claude names that rung in the worker's prompt, and hands it to the worker's claude beside the project's and the worker's, so rule-hook holds each reviewer launch to it. A low E + I never steps a reviewer below the project's rung: lilleman chose to lower a pass only where a replay shows a lower rung finding every 🔴 and 🟡 the project's rung finds, which `todo.md` item 159 measures for each pass.
|
||||
|
||||
Valid until the trial on real chunks shows whether a worker below the project's rung holds quality.
|
||||
|
||||
@@ -260,7 +262,7 @@ Valid while a model switch inside a session discards its prompt cache.
|
||||
|
||||
The cost of a cheaper worker shows as extra review rounds and unfinished verdicts. Every 20 cycles across the archive, loop-claude measures them in code, with no model run (goal 1.4), and interrupts lilleman with the calls not raised before, when chunks routed onto a rung took more of either than chunks routed onto the next rung up, each rung with 5 chunks or more. Five keeps one odd chunk from calling for a change (goal 1.2). It also calls when the reports of 5 or more subagents of one type were longer than the tool results they read, so a delegated search cost the worker more context than it saved. `bin/routing_figures/` measures and words the calls. A loop on another project cannot commit here, so it writes the figures and the calls raised at the archive's root, and this repo's next chunk files the plan item, as `AGENTS.md` says. Off a terminal, a call is said on stderr and the loop goes on: it is about this setup, not the running project's chunk. A reset line in the calls file restarts the figures after a routing change, so they show whether the change worked.
|
||||
|
||||
Since 2026-10-16 the reviewers run on the rung their item's E + I picks, apart from the worker's, so a lower worker rung's chunks may take more rounds from stronger reviewers, and a call could blame the worker's rung for them; `todo.md` item 171 groups the rounds by the reviewers' rung too. Valid while loop-claude's archive holds every finished chunk's review rounds and unfinished verdicts.
|
||||
Since 2026-10-16 the reviewers may run above the worker's rung, on the one their item's E + I picks, so chunks whose worker ran on a lower rung may take more rounds from stronger reviewers, and a call could blame the worker's rung for them; `todo.md` item 171 groups the rounds by the reviewers' rung too. Valid while loop-claude's archive holds every finished chunk's review rounds and unfinished verdicts.
|
||||
|
||||
## routing: a worker delegates a task only where it needs just the task's conclusion, and each subagent runs on the rung its task's L picks from the project's rung, never below the project's rung when it files or scores todo.md items
|
||||
|
||||
@@ -300,11 +302,11 @@ On a model listed in `JUDGE_REPLAYED` in `bin/loop_claude/routing.py`, the judge
|
||||
|
||||
On 2026-10-15, `bin/judge-replay claude-opus-5-5` replayed the latest 15 archived verdicts of each kind from the Opus 5.5 projects, 39 in all. The reference judge, Opus 5.5 at the xhigh effort `settings.json` gave every judge before, reproduced 31 of them. The other 8 may rest on records or judge rules that have changed since; either way no rung is held to them. On the 31, Haiku 5.5 missed 7 at xhigh and 8 at max. That run tried each judge once. 4 of Haiku's misses, 1 at xhigh and 3 at max, misquoted the transcript, which the loop and `judge-replay` now retry; without them it missed 6 and 5. Sonnet 5.5 at high returned another kind on 3 interrupts, and Opus 5.5 at medium missed none. A missed interrupt lets the worker settle a decision that is lilleman's (goal 1.2), so the judge never runs below the project's rung.
|
||||
|
||||
Only Opus 5.5 was replayed, so the judge of a project on another model keeps the effort its settings give until `judge-replay` covers that model (goal 1.1). Every routed chunk on record had an item of R + C 9, so the replay could not show that the project's rung suffices for harder items, and the judge climbs with the routed item. Since 2026-10-16, lilleman's call, it climbs with the item's E + I, as a review does: the judge checks the worker's decisions as the reviewers check its code, and a decision it lets through reaches lilleman as a defect review lets through reaches the users (goal 1.2). It runs above the project's rung only on an E + I of 5 or more (goal 1.4). An item the worker takes in beyond the routed one lifts the worker's rung but not the judge's, until `todo.md` item 153 lifts the judge too.
|
||||
Only Opus 5.5 was replayed, so the judge of a project on another model keeps the effort its settings give until `judge-replay` covers that model (goal 1.1). Every routed chunk on record had an item of R + C 9, so the replay could not show that the project's rung suffices for harder items, and the judge climbs with the routed item. Since 2026-10-16, lilleman's call, it climbs with the item's E + I, as a review does: the judge checks the worker's decisions as the reviewers check its code, and a decision the judge lets through reaches lilleman, as a defect that review lets through reaches the users (goal 1.2). It runs above the project's rung only on an E + I of 5 or more (goal 1.4). An item the worker takes in beyond the routed one lifts the worker's rung but not the judge's, until `todo.md` item 153 lifts the judge too.
|
||||
|
||||
Valid while the archived verdicts show what a judge should return, and until a new model joins the ladder: replay then.
|
||||
|
||||
## routing: the stability reviewer stays on the project's rung while the worker runs below it, until `stability-replay` shows a lower rung finding every 🔴 and 🟡 the project's rung finds
|
||||
## routing: the stability reviewer runs no lower than the project's rung while the worker runs below it, until `stability-replay` shows a lower rung finding every 🔴 and 🟡 the project's rung finds
|
||||
|
||||
2026-10-15, lilleman proposed running the stability reviewer on the worker's rung; the replay that gates it is the agent's proposal. Serves goals 1.1 and 1.4: the stability pass is the only one that mutation-tests the changed lines, so a defect it misses may merge, and a cheaper reviewer saves money only where it misses nothing.
|
||||
|
||||
|
||||
+3
-3
@@ -9,7 +9,7 @@ paths:
|
||||
- **Order items only by score and the Blocked by column.** An item's Blocked by cell lists the IDs of the items that must be done before it starts, such as `12, 14`; `rank_todo.py --help` says how it ranks them. No text in `todo.md` outside the table orders items: where an item must go first, score it so, and say why in its detail.
|
||||
- **Plan only on the project's rung or above.** loop-claude's prompt tells you when you run below it. Below it, never score or re-score an item, move it between releases, split a milestone into items, delete an item under the bar, or rewrite a filed item's tagline or detail. Draft each new item, and each change to a filed item, instead. Before your next commit, launch one subagent to apply the drafts, score them and set their release. Launch it on the rung `CLAUDE.md`'s delegation rule gives a subagent that files or scores items. Keep the scores and releases it writes. For a security finding, launch it at once. Any rung may remove a finished item's row and detail, and score a delegated task's L. A subagent launched for another task leaves `todo.md` alone.
|
||||
- **An item's release allows the bump that semver read strictly asks for.** An item that changes what exists needs a major, or before 1.0.0 a minor. One that only adds needs a minor, and a fix a patch.
|
||||
- **Score an item when it is filed or edited.** S, the size of the change, is a whole number from 1 to 10. Score L, E and I by the levels below, each about ten times the one beneath it. `rank_todo.py` writes R, the risk that the change creates serious bugs or issues: L + E + I − 2, at least 7 when I is 5 and at least 5 when I is 4, since a severe enough consequence outranks how unlikely it is. A task's L picks its worker's rung, and its E + I its reviewers' rung.
|
||||
- **Score an item when it is filed or edited.** S, the size of the change, is a whole number from 1 to 10. Score L, E and I by the levels below, each level about ten times as likely, or as damaging, as the one below it. `rank_todo.py` writes R, the risk that the change creates serious bugs or issues: L + E + I − 2, at least 7 when I is 5 and at least 5 when I is 4, since a severe enough consequence outranks how unlikely it is. A task's L picks its worker's rung, and its E + I its reviewers' rung.
|
||||
|
||||
| Level | L — chance the change holds a defect | E — chance the defect escapes | I — damage, worst of availability, data, secrets, reach and reversibility |
|
||||
|---|---|---|---|
|
||||
@@ -19,10 +19,10 @@ paths:
|
||||
| 4 | two or more of level 3's, or a rewrite of code with known defects | — | everything down, a backup restore, or a secret or PII exposed |
|
||||
| 5 | — | — | irreversible: data lost for good, or a one-way door |
|
||||
|
||||
- **I's reach counts every project and session that runs the change,** not the people in the README's audience. A change to a hook or a tool that every session runs is I level 3, every consumer affected, even where one person uses it.
|
||||
- **I's reach counts every project and session that runs the change.** A change to a hook or a tool that every session runs is I level 3, every consumer affected, even where one person uses it.
|
||||
- **Name in Moves each achieve goal in `README.md` → `## Goals` the item moves, and how far,** from 1 to 10, judged by its value to the audience in `## Audience`: `1.1 by 8, 1.2 by 5`. Name the most specific goal it moves, and a parent goal only for a move none of its sub-goals covers, never beside its own sub-goal. Leave out every maintain goal, which vetoes, except on an item marked `defect` that restores it: name that goal, and how much of it is broken. `rank_todo.py --help` says how it weighs each goal.
|
||||
- **A table with C, A, G and Goals in place of L, E, I and Moves is in the old shape.** Reshaping it is the project's only work until done, as `CLAUDE.md` → "Working loop" says, and `rank_todo.py` says how. Until then, A is the item's value to the audience and G its value to the goals it cites.
|
||||
- **Check L, E, I and S against finished chunks.** Guess them, then run `~/.claude/bin/score-outcomes <L> <E> <I> <S>` in the project. It lists the finished items guessed nearest, what their chunks measured, the median outcome at each guessed R and S, and how well L, E and I each order the chunks. Set each score to the value guessed for the finished items whose outcome matches what you expect of this one. Keep your guess when no listed item resembles the one you are scoring.
|
||||
- **Check L, E, I and S against finished chunks.** Guess them, then run `~/.claude/bin/score-outcomes <L> <E> <I> <S>` in the project. It lists the finished items guessed nearest, what their chunks measured, the median outcome at each guessed R and S, and how well L, E and I each order the chunks by review rounds and 🔴 findings. Set each score to the value guessed for the finished items whose outcome matches what you expect of this one. Keep your guess when no listed item resembles the one you are scoring.
|
||||
- **Read the places the comprehension seats found hard before filing an item.** `~/.claude/bin/larv-review places`, run in the project, prints them. Where the item's change touches one, its detail says to make that place easier to read.
|
||||
- **An item scoring under the bar is deleted, with every item that needs it, and a finding under the bar is not filed.** The chunk's summary names each, with its score.
|
||||
- **The Exempt cell keeps an item whatever its score:** `security` for a security finding, `defect` for a 🔴 finding or an item that restores a maintain goal the project breaks today, `question` for a question only the human can answer, which ranks first in its release, `decision` for an item `docs/decisions.md` cites, `owner` for an item the human told you to keep, and, where no other reason fits, `principle` for a 🟡 finding against a file under `~/.claude/principles/`, whose item's detail names that principle. Every item `docs/decisions.md` cites carries one of these reasons.
|
||||
|
||||
@@ -46,7 +46,7 @@
|
||||
| 169 | 0.3.0 | | | **Have rule-hook's `todo-rank-check` catch a `todo.md` that the same Bash call edits and then commits.** | 3 | 1 | 3 | 5 | 3 | 1.1 by 2 | 2.7 |
|
||||
| 122 | 0.4.0 | | | **Assemble the judge's prompt in `bin/loop_claude/cycle/judge.py`, importing only wording the worker also sees, and call the worker "the worker" in it.** | 2 | 2 | 3 | 5 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 |
|
||||
| 158 | 0.4.0 | | decision | **Measure the fixed cost of a chunk from loop-claude's archive, in tokens and hours against its items' S, and set F in `Score = 100 * V / (2**S + F)` to it.** | 2 | 2 | 1 | 3 | 3 | 1.1 by 5, 1.4 by 5 | 10.3 |
|
||||
| 170 | 0.4.0 | | decision | **Re-tune `WORKER_STEPS` and `REVIEW_STEPS` once 10 chunks have shipped under them, from how well L and E + I order their review rounds and 🔴 findings.** | 2 | 2 | 3 | 5 | 2 | 1.1 by 3, 1.4 by 5 | 10.0 |
|
||||
| 170 | 0.4.0 | | decision | **Re-tune `WORKER_STEPS` and `REVIEW_STEPS` once 10 chunks have shipped under them, from how well L, E and I order their review rounds and 🔴 findings.** | 2 | 2 | 3 | 5 | 2 | 1.1 by 3, 1.4 by 5 | 10.0 |
|
||||
| 116 | 0.4.0 | | | **Report L by the share of chunks with a 🔴, once 10 chunks that each shipped one item record finding counts and most record no 🔴.** | 1 | 1 | 2 | 2 | 2 | 1.1 by 5 | 9.1 |
|
||||
| 121 | 0.4.0 | | | **Check that `bin/round_titles.py`'s `PASSES` lists exactly the passes of larv-review §3's table.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 5 | 9.1 |
|
||||
| 112 | 0.4.0 | | | **Count an item as shipped only where its chunk built it, so an item a chunk dropped from the plan gets no outcome.** | 3 | 2 | 2 | 5 | 3 | 1.1 by 6 | 8.2 |
|
||||
@@ -97,7 +97,7 @@ A defect: goal 3, keeping unattended runs' transcripts backed up, does not hold
|
||||
|
||||
### 153. Route the judge and the reviewers on the highest E + I in their chunk, an item the worker takes in included.
|
||||
|
||||
The judge reports the L of an item the worker takes in, and `routing.stepped` lifts the worker to the rung it picks, but the judge and the reviewers route on the routed item's E + I alone, which the archived route line holds. So when the worker takes in an item whose defects would escape further or break more, the judge and the reviewers stay on a lower rung than it needs: a missed interrupt lets the worker settle a decision that is lilleman's (goal 1.2), and a weaker review lets a defect through (goal 1.1). The judge must report the highest E + I taken in too, and the route must carry it, archived so a rerun keeps it; loop-claude then hands the reviewers' rung it picks to the worker's prompt and environment.
|
||||
The judge reports the L of an item the worker takes in, and `routing.stepped` lifts the worker to the rung it picks, but the judge and the reviewers route on the routed item's E + I alone, which the archived route line holds. So when the worker takes in an item whose defects would escape further or break more, the judge and the reviewers stay on a lower rung than that item needs: a missed interrupt lets the worker settle a decision that is lilleman's (goal 1.2), and a weaker review lets a defect through (goal 1.1). The judge must report the highest E + I taken in too, and the route must carry it, archived so a rerun keeps it; loop-claude then hands the reviewers' rung it picks to the worker's prompt and environment.
|
||||
|
||||
### 161. Have loop-claude run a preflight script before each fresh worker and put its findings in the worker's prompt, so the worker no longer spends tool calls finding the chunk's mechanical facts.
|
||||
|
||||
@@ -134,7 +134,7 @@ This risks goal 1.1: a worker fixing from a one-line summary can miss context it
|
||||
|
||||
lilleman asked, on 2026-10-16, for the prose pass to run on the worker's rung or lower, and the same day, answering PR #296, for every pass to drop below the project's rung only where a replay shows it may. Today `bin/loop_claude/cycle/prompts.py`'s `review_launches` keeps every reviewer at or above the project's rung whatever the item's E + I, so a docs-only chunk such as item 140's, routed to Haiku 5.5, runs its reviewers on Opus 5.5.
|
||||
|
||||
Whether a pass runs cheaper without lowering quality (goal 1.4 against 1.1) is measured, not assumed, as `docs/decisions.md` → `routing: the stability reviewer stays on the project's rung…` measures the stability reviewer: replay a pass's archived rounds on the code each reviewed, first on the project's rung, then on each rung below it, and take the lowest rung that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again. A missed prose finding breaks no behaviour and a later change fixes it cheaply, so replay the prose pass first. Generalise `bin/stability-replay` to take the pass, in place of a replay tool per pass. Record each run and the rung it chose in a `routing:` decision, valid until a new model joins the ladder or the pass's prompt changes. Give each pass a row of its own in `review_launches`' table of review kinds, rather than a branch.
|
||||
Whether a pass runs cheaper without lowering quality (goal 1.4 against 1.1) is measured, as `docs/decisions.md` → `routing: the stability reviewer runs no lower than the project's rung…` measures the stability reviewer: replay a pass's archived rounds on the code each reviewed, first on the project's rung, then on each rung below it, and take the lowest rung that, like each rung above it, finds every 🔴 and 🟡 the project's rung finds again. A missed prose finding breaks no behaviour and a later change fixes it cheaply, so replay the prose pass first. Generalise `bin/stability-replay` to take the pass, in place of a replay tool per pass. Record each run and the rung it chose in a `routing:` decision, valid until a new model joins the ladder or the pass's prompt changes. Give each pass a row of its own in `REVIEWS`, `review_launches`' table of review kinds in `bin/loop_claude/cycle/prompts.py`, rather than a branch.
|
||||
|
||||
### 80. State in README → Install that every machine taking this checkout must be a sandbox, as `settings.json`'s auto mode assumes.
|
||||
|
||||
@@ -228,7 +228,7 @@ The chunk that ships this item files a copy of it in the release after the one i
|
||||
|
||||
### 171. Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.
|
||||
|
||||
Raised by the goals audit of PR #296. `bin/routing_figures/measure.py` groups a chunk's review rounds by the rung its worker started on, and calls for a routing change when a lower rung took more of them. Since PR #296 the reviewers run on the rung their item's E + I picks, apart from the worker's, and the archive does not record it. A cheap worker on an item of high E + I gets stronger reviewers, who may raise more rounds, and a call could then blame the worker's rung for them: an interrupt of lilleman that settles nothing (goal 1.2).
|
||||
Raised by the goals audit of PR #296. `bin/routing_figures/measure.py` groups a chunk's review rounds by the rung its worker started on, and calls for a routing change when a lower rung took more of them. Since PR #296 the reviewers may run above the worker's rung, on the one their item's E + I picks, and no figure groups rounds by it, though the archived worker line names the E + I it follows from. A cheap worker on an item of high E + I gets stronger reviewers, who may raise more rounds, and a call could then blame the worker's rung for them: an interrupt of lilleman that settles nothing (goal 1.2).
|
||||
|
||||
### 165. Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.
|
||||
|
||||
@@ -246,13 +246,13 @@ All four seats of PR #257's scoring run named `bin/loop_claude/cycle/judge.py:16
|
||||
|
||||
F is 8 until then, about one S 3 change, lilleman's call in PR #283. A chunk costs more than its size: the worker reads about 130k tokens before its first commit whatever the item, and every chunk takes its review passes. Fit cost against 2**S from the finished chunks `score-outcomes` measures, in the same units `2**S` counts, and say in `docs/decisions.md` → `todo.md: …` how far the fit holds. A chunk bundling tiny items shares F between them.
|
||||
|
||||
### 170. Re-tune `WORKER_STEPS` and `REVIEW_STEPS` once 10 chunks have shipped under them, from how well L and E + I order their review rounds and 🔴 findings.
|
||||
### 170. Re-tune `WORKER_STEPS` and `REVIEW_STEPS` once 10 chunks have shipped under them, from how well L, E and I order their review rounds and 🔴 findings.
|
||||
|
||||
`docs/decisions.md` → `routing: one ladder…` calls the steps lilleman approved on 2026-10-16 a first guess. Once 10 chunks that each shipped one item scored with L, E and I have merged, read `score-outcomes`' rank correlations for L, E and I and the routed-chunks table of `routing-figures.md`, and propose new steps to lilleman, whose call they are. A change to the steps appends a `Reset` to `routing-calls-raised.md` once merged, as `AGENTS.md` says.
|
||||
`docs/decisions.md` → `routing: one ladder…` calls the steps lilleman approved on 2026-10-16 a first guess. Once 10 chunks that each shipped one item scored with L, E and I have merged, read `score-outcomes`' rank correlations for L, E and I and the routed-chunks table of `routing-figures.md`, and propose new steps to lilleman, whose call they are.
|
||||
|
||||
### 116. Report L by the share of chunks with a 🔴, once 10 chunks that each shipped one item record finding counts and most record no 🔴.
|
||||
|
||||
`score-outcomes` reports the median 🔴 count at each guessed R, and how well L orders chunks by 🔴 findings; the part of R a 🔴 tests is L. Where most chunks have no 🔴, every median reads 0 and the rank correlation reads none, which tells a scorer nothing. No round title held counts before PR #254, so how often a chunk has a 🔴 is not yet known.
|
||||
`score-outcomes` reports the median 🔴 count at each guessed R, and how well L orders chunks by 🔴 findings; the part of R a 🔴 tests is L. Where most chunks have no 🔴, every median reads 0 and the rank correlation rests on the few that have one, which tells a scorer little. No round title held counts before PR #254, so how often a chunk has a 🔴 is not yet known.
|
||||
|
||||
### 121. Check that `bin/round_titles.py`'s `PASSES` lists exactly the passes of larv-review §3's table.
|
||||
|
||||
@@ -310,7 +310,7 @@ The paired ruling on PR #265 ruled worse. Make these places easier to read: `mea
|
||||
|
||||
### 149. Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.
|
||||
|
||||
A release's capacity is a sum of its items' cost, `2**S`. Each project states its own; the human sets the first value per project. Nothing is deleted by score.
|
||||
A release's capacity is a sum of its items' cost, as the formula counts it. Each project states its own; the human sets the first value per project. Nothing is deleted by score.
|
||||
|
||||
### 12. Run the test suite in CI on every PR, or record in `docs/decisions.md` why the local run is enough.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user