Move larv-review's mechanical rules into a tested bin/larv-review, so the skill says only when to run it and what each answer means

This commit is contained in:
2026-10-09 01:02:48 +02:00
parent c7f6099945
commit 77fef49fce
18 changed files with 851 additions and 90 deletions
+3
View File
@@ -19,6 +19,9 @@
!bin/
!bin/failure.py
!bin/file_writes.py
!bin/larv-review
!bin/larv_review/
!bin/larv_review/*.py
!bin/loop-claude
!bin/loop_claude/
!bin/loop_claude/**/
+1
View File
@@ -2,6 +2,7 @@
## Unreleased
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling` and `record-scoring`, which read a seats file and append to `docs/comprehension-history.md`). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release left with no plan item is due a scoring run however its items left, moved to another release included.
- The new `bin/metric-bench` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest: cognitive complexity, length, shared state and change spread, each scored by how well it ranks the named functions above the rest. No metric enters review yet.
- larv-review blocks a merge when simulated readers rule that the PR makes the project read worse than the last measured commit: the default branch's newest merged change that added a scoring run to `docs/comprehension-history.md`, or whose last ruling there is better. A ruling of same leaves that commit in place, so a worse ruling may name places earlier chunks wrote. In each architecture pass Mid A and the inherited architect vote better, same or worse; when they disagree, the maintainability senior and Junior A vote too. A ruling needs a lead of two or more votes, else it is same, and a vote that names no place counts as same. A worse ruling never ships: after three rounds larv-review asks you for another round, a no leaves the PR unmerged, and so does an unattended run. The four-seat scoring run checks the comprehension floor only when a chunk ships or removes a release's last plan item, or ships an item naming the floor; between those runs the floor blocks nothing. A score under the floor blocks the merge until fixed or until you decide to ship; a decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
- larv-review reruns fewer passes after a fix. A fix reruns the pass whose 🔴 or 🟡 it answers, plus stability when the fix changes behaviour, security when it opens an untrusted-input boundary or touches authn/authz, secrets or crypto, and architecture when it carries an architectural decision. A rerun reviews only what changed since that pass last reviewed. A fix for a 🟢, a ❓ or any prose finding reruns no pass, and prose takes one round.
+1
View File
@@ -67,6 +67,7 @@ Rename `## Unreleased` in `CHANGELOG.md` to the version, commit, tag that commit
Install puts only `loop-claude` on `PATH`; call the others by their path, such as `~/.claude/bin/metric-bench`, from the project's checkout.
- `larv-review <command>` holds larv-review's mechanical rules: the paired ruling's base and vote count, each pass's round count, the scoring run's trigger and the comprehension history's format; `larv-review --help` states them.
- `loop-claude <task>` runs a project's next chunk cycle after cycle; `loop-claude --help` states its contract.
- `metric-bench [history]` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest; `metric-bench --help` states the metrics and the scores.
- `rank_todo.py [--check] [todo.md]` scores and ranks a project's `todo.md`; `rank_todo.py --help` states the format.
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env python3
import pathlib
import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from larv_review import main
sys.exit(main.main(sys.argv[1:]))
View File
+73
View File
@@ -0,0 +1,73 @@
import dataclasses
import subprocess
from failure import Failure
from larv_review.rules import HistoryChange
HISTORY = "docs/comprehension-history.md"
@dataclasses.dataclass(frozen=True)
class Repository:
"""The checkout the tool runs in: its top folder, and the default branch's ref after a fetch, `origin/<branch>` with a remote."""
top: str
default: str
def git(*args: str) -> str | Failure:
try:
ran = subprocess.run(["git", *args], capture_output=True, text=True)
except (OSError, ValueError) as failure:
return Failure(f"git could not start: {failure}")
return ran.stdout if ran.returncode == 0 else Failure(f"git {' '.join(args)}: {ran.stderr.strip()}")
def repository() -> Repository | Failure:
top = git("rev-parse", "--show-toplevel")
if isinstance(top, Failure):
return top
remotes = git("remote")
if isinstance(remotes, Failure):
return remotes
if "origin" in remotes.split():
if isinstance(fetched := git("fetch", "--quiet", "origin"), Failure):
return fetched
default = git("symbolic-ref", "--short", "refs/remotes/origin/HEAD")
if isinstance(default, Failure):
return Failure(f"origin names no default branch; set it with `git remote set-head origin --auto`: {default.reason}")
return Repository(top.strip(), default.strip())
for branch in ("main", "master"):
if not isinstance(git("rev-parse", "--quiet", "--verify", f"refs/heads/{branch}"), Failure):
return Repository(top.strip(), branch)
return Failure("no origin remote, and no main or master branch to take as the default branch")
def file_at(revision: str, path: str) -> str | Failure:
"""A file's text at a revision, empty where the revision does not hold it."""
if isinstance(git("rev-parse", "--quiet", "--verify", f"{revision}:{path}"), Failure):
return ""
return git("show", f"{revision}:{path}")
def history_changes(default: str) -> list[HistoryChange] | Failure:
"""Each merged change on the default branch that touched the history, newest first; a merge commit counts as one change against its first parent."""
commits = git("log", "--first-parent", "--format=%H", default, "--", HISTORY)
if isinstance(commits, Failure):
return commits
changes = []
for commit in commits.split():
before = file_at(f"{commit}^1", HISTORY)
after = file_at(commit, HISTORY)
if isinstance(before, Failure):
return before
if isinstance(after, Failure):
return after
changes.append(HistoryChange(commit, before, after))
return changes
def landed_since_release(default: str) -> bool | Failure:
"""Whether the default branch holds a commit after its newest `v*` tag, or any commit when it has none."""
tag = git("describe", "--tags", "--abbrev=0", "--match", "v*", default)
count = git("rev-list", "--count", default if isinstance(tag, Failure) else f"{tag.strip()}..{default}")
return count if isinstance(count, Failure) else int(count) > 0
+110
View File
@@ -0,0 +1,110 @@
import dataclasses
import os
import re
from markdown_sections import sections
TITLE = "# Comprehension history\n"
HEADING = re.compile(r"^([^,]+), (.+) at ([0-9a-f]{7,40})(?:, against ([0-9a-f]{7,40}))?$")
RULING = re.compile(r"^Ruling: (better|same|worse)$", re.M)
SEAT_PLACES = re.compile(r"^.+, (decided by|hardest first|would restructure first):$")
# `path:line` or `:line`, then the unit's name where one follows.
PLACE = re.compile(r"`([^`\s]*):\d+`(?: `(\w+)`)?")
@dataclasses.dataclass(frozen=True)
class Place:
path: str
name: str | None
@dataclasses.dataclass(frozen=True)
class Entry:
"""One paired ruling, which names the commit it was against and its ruling, or one scoring run, which names neither."""
heading: str
time: str
commit: str
against: str | None
ruling: str | None
places: list[Place]
@dataclasses.dataclass(frozen=True)
class ScoringRun:
heading: str
commit: str
places: list[Place]
def entries(history: str) -> list[Entry]:
"""Each paired ruling and scoring run in a comprehension history, in the order the file holds them; a section whose heading names no commit is left out."""
found = []
for heading, bodies in sections(history).items():
if named := HEADING.match(heading):
body = "\n".join(bodies)
ruling = RULING.search(body)
found.append(Entry(heading, named.group(1), named.group(3), named.group(4), ruling.group(1) if ruling else None, seat_places(body)))
return found
def scoring_runs(history: str) -> list[ScoringRun]:
"""Each scoring run, with the places its seats named, as they wrote them; paired rulings name what changed, not what is hard, and are left out."""
return [ScoringRun(entry.heading, entry.commit, entry.places) for entry in entries(history) if entry.against is None]
def seat_places(body: str) -> list[Place]:
places = []
listing = False
for line in body.splitlines():
if SEAT_PLACES.match(line):
listing = True
elif listing and line.startswith("- "):
places += bullet_places(line)
elif not line.startswith(" "):
listing = False
return places
def bullet_places(bullet: str) -> list[Place]:
"""A `:line` with no path is in the file named before it in the bullet."""
places = []
path = ""
for match in PLACE.finditer(bullet):
path = match.group(1) or path
if path:
places.append(Place(path, match.group(2)))
return places
def resolved(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
"""Entries written before paths were written from the repository's root shorten a path once they have named its folder: `cycle/run.py` after `bin/loop_claude/cycle/states.py`. Each place resolves to the one tracked file it names, or, where several match, the one in the folder of the place resolved before it; a place naming no file, or several, is dropped. Returns the resolved places and the dropped ones."""
found = []
dropped = []
before = ""
for place in places:
candidates = [place.path] if place.path in tracked else [path for path in tracked if path.endswith("/" + place.path)]
nearest = [path for path in candidates if path.startswith(os.path.dirname(before) + "/")] if len(candidates) > 1 else candidates
if len(nearest) == 1:
before = nearest[0]
found.append(Place(before, place.name))
else:
dropped.append(place)
return found, dropped
def appended(history: str, entry: str) -> str:
"""The history with entry added last; an empty history gets the file's title first."""
text = history.rstrip("\n") + "\n" if history.strip() else TITLE
return f"{text}\n{entry}"
def entry_text(heading: str, lines: list[str], table: list[list[str]], seats: list[tuple[str, list[str]]]) -> str:
"""An entry in the history's format: the heading, any lines such as `Ruling: same`, a table whose first row is its header, and under each seat's label its places, one bullet each."""
header, *rows = table
parts = [f"## {heading}", *lines, "\n".join([row_text(header), "|" + "---|" * len(header), *map(row_text, rows)])]
parts += [f"{label}\n\n" + "\n".join(f"- {place}" for place in places) for label, places in seats if places]
return "\n\n".join(parts) + "\n"
def row_text(cells: list[str]) -> str:
return "| " + " | ".join(cells) + " |"
+205
View File
@@ -0,0 +1,205 @@
"""Hold larv-review's mechanical rules: the paired ruling's base and vote count, each pass's rounds, the scoring run's trigger, and the comprehension history's format.
usage: larv-review ruling-base | round-count | needs-scoring-run <item ID>... | record-ruling <seats file> | record-scoring <seats file>
Run it inside a project checkout or one of its worktrees. It first fetches origin, when the checkout
has that remote, and takes origin's default branch, or else the local main or master, as the
default branch. The history is docs/comprehension-history.md in the checkout.
ruling-base prints the commit a paired ruling reads against, then how it was found: the default
branch's newest merged change that adds a scoring run to the history, or whose last added paired
ruling, by its heading's time, is better; a merge commit is one change against its first parent.
With none, the base the history's first paired ruling names; with no ruling either, the merge base
of HEAD and the default branch.
round-count prints, for each pass, the highest <n> among this branch's commit titles opening
`Answer <pass> round <n>:`, counting commits the default branch does not hold, and marks a pass at
the cap of 3 rounds.
needs-scoring-run prints why the chunk shipping the given todo.md item IDs takes a scoring run, or
`No scoring run due.` It is due when the chunk ships an item whose tagline names the comprehension
floor, or when a release the default branch's todo.md names has no item left in the checkout's
todo.md, less the items shipped, and work has landed towards it: the chunk ships one of its items,
or it is the earliest release the default branch's todo.md names and the default branch has a
commit after its newest v* tag, or has no such tag.
record-ruling and record-scoring read a seats file: a first line naming what the seats read, then
each seat's line, each followed by its places, one `- ` bullet each, as the history writes them:
PR #231 at abc1234, against def5678
Mid A: worse
- `bin/x.py:12` `run`: three branches to hold at once
Inherited architect: worse
- `bin/x.py:12` `run`: ...
record-ruling takes Mid A's and the inherited architect's votes, better, same or worse; a better or
worse vote naming no `path:line` place counts as same. When the two counted votes differ, it
appends nothing and says to add the maintainability senior's and Junior A's lines. Otherwise the
ruling is the vote that leads each other vote by two or more, else same, and it appends the ruling.
record-scoring takes Junior A's, Mid A's, the maintainability senior's and the inherited
architect's lines in that order, each `Navigation <n>, Locality <n>, Shape <n>, Self-sufficiency
<n>, Overall <n>`, each score from 1 to 10 and the overall at most 1 above the seat's lowest
dimension. A first line names no `, against`. It appends the seats' scores and their means, rounded
half up to two decimals, and prints the means and each one under the floor of 7.0.
Each record is appended to the history, created when missing, under a heading of the current UTC
time and the seats file's first line.
Exit 1, saying why, on unknown arguments, an unknown item ID, a seats file it cannot read or that
breaks the shape above, or when git cannot start, finds no checkout or no default branch, or the
history or todo.md cannot be read or written.
"""
import pathlib
import sys
import time
from larv_review.history import appended, entry_text, HEADING
from larv_review.rules import CAP, first_ruling, means, measured_commit, plan_items, round_counts, scoring_run_due, under_floor
from larv_review.seats import DIMENSIONS, Report, report, ruling, scores, votes
from failure import Failure
from larv_review.git import file_at, git, HISTORY, history_changes, landed_since_release, Repository, repository
from file_writes import replace_file
from loop_claude.transcript_archive import utc
USAGE = "usage: larv-review ruling-base | round-count | needs-scoring-run <item ID>... | record-ruling <seats file> | record-scoring <seats file>"
def main(argv: list[str]) -> int:
if argv in (["-h"], ["--help"]):
print(__doc__.strip())
return 0
command, *given = argv or [""]
valid = {"ruling-base": not given, "round-count": not given, "needs-scoring-run": bool(given) and all(id_.isdigit() for id_ in given), "record-ruling": len(given) == 1, "record-scoring": len(given) == 1}
if not valid.get(command):
print(f"unknown arguments {' '.join(argv)}; {USAGE}", file=sys.stderr)
return 1
repo = repository()
if isinstance(repo, Failure):
return failed(repo)
if command == "ruling-base":
done = ruling_base(repo)
elif command == "round-count":
done = round_count(repo)
elif command == "needs-scoring-run":
done = needs_scoring_run(repo, [int(id_) for id_ in given])
else:
done = record(repo, pathlib.Path(given[0]), command == "record-ruling", utc(time.time()))
return failed(done) if isinstance(done, Failure) else 0
def ruling_base(repo: Repository) -> Failure | None:
changes = history_changes(repo.default)
if isinstance(changes, Failure):
return changes
if found := measured_commit(changes):
print(f"{found}\nThe newest merged change on {repo.default} that adds a scoring run to {HISTORY}, or whose last added paired ruling is better.")
return None
history = file_at(repo.default, HISTORY)
if isinstance(history, Failure):
return history
if (first := first_ruling(history)) and first.against:
print(f"{first.against}\nNo merged change on {repo.default} adds a scoring run or a better ruling; the base its first paired ruling names.")
return None
merge_base = git("merge-base", "HEAD", repo.default)
if isinstance(merge_base, Failure):
return merge_base
print(f"{merge_base.strip()}\nNo merged change on {repo.default} adds a scoring run or a paired ruling; the merge base of HEAD and {repo.default}.")
return None
def round_count(repo: Repository) -> Failure | None:
titles = git("log", "--format=%s", f"{repo.default}..HEAD")
if isinstance(titles, Failure):
return titles
for name, count in round_counts(titles.splitlines()).items():
print(f"{name}: {count}{', at the cap' if count >= CAP else ''}")
return None
def needs_scoring_run(repo: Repository, ids: list[int]) -> Failure | None:
before = file_at(repo.default, "todo.md")
if isinstance(before, Failure):
return before
try:
after = pathlib.Path(repo.top, "todo.md").read_text(encoding="utf-8")
except FileNotFoundError:
after = ""
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read todo.md: {failure}")
planned = plan_items(before)
if isinstance(planned, Failure):
return planned
chunk = plan_items(after)
if isinstance(chunk, Failure):
return chunk
if unknown := sorted(set(ids) - {item.id for item in planned + chunk}):
return Failure(f"neither {repo.default}'s todo.md nor the checkout's holds item {', '.join(map(str, unknown))}")
landed = landed_since_release(repo.default)
if isinstance(landed, Failure):
return landed
reasons = scoring_run_due(planned, chunk, ids, landed)
print(f"Scoring run due: the chunk {' and '.join(reasons)}." if reasons else "No scoring run due.")
return None
def record(repo: Repository, seats_file: pathlib.Path, paired: bool, now: str) -> Failure | None:
try:
given = report(seats_file.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {seats_file}: {failure}")
if isinstance(given, Failure):
return given
heading = f"{now}, {given.read}"
if not (named := HEADING.match(heading)) or bool(named.group(4)) != paired:
return Failure(f"the seats file's first line is not `<PR or branch> at <commit>{', against <base>' if paired else ''}`: {given.read}")
entry = ruling_entry(heading, given) if paired else scoring_entry(heading, given)
if isinstance(entry, Failure) or entry is None:
return entry
history = pathlib.Path(repo.top, HISTORY)
try:
text = history.read_text(encoding="utf-8") if history.exists() else ""
history.parent.mkdir(parents=True, exist_ok=True)
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {history}: {failure}")
unwritten = replace_file(history, appended(text, entry).encode())
if unwritten:
return Failure(f"could not write {history}: {unwritten.reason}")
print(f"Appended to {history}.")
return None
def ruling_entry(heading: str, given: Report) -> str | Failure | None:
"""The ruling's entry, or None, after saying which seats to add, when the first two counted votes differ."""
counted = votes(given.seats)
if isinstance(counted, Failure):
return counted
tally = ", ".join(f"{seat} {vote}" for seat, vote in counted.items())
result = ruling(list(counted.values()))
if result is None:
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the seats file, and run this again.")
return None
print(f"Ruling: {result}. Votes as counted: {tally}.")
cells = [[seat.name, counted[seat.name] if counted[seat.name] == seat.said else f"same (voted {seat.said}, naming no place)"] for seat in given.seats]
return entry_text(heading, [f"Ruling: {result}"], [["Seat", "Vote"], *cells], [(f"{seat.name}, decided by:", seat.places) for seat in given.seats])
def scoring_entry(heading: str, given: Report) -> str | Failure:
found = scores(given.seats)
if isinstance(found, Failure):
return found
mean = means(found)
under = under_floor(mean)
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, mean))}. {'Under the floor of 7.0: ' + ', '.join(under) if under else 'None under the floor of 7.0'}.")
rows = [[seat, *values] for seat, values in found.items()] + [["Mean", *map(str, mean)]]
labels = [(f"{seat.name}, {'would restructure first' if seat.name == 'Inherited architect' else 'hardest first'}:", seat.places) for seat in given.seats]
return entry_text(heading, [], [["Seat", *DIMENSIONS], *rows], labels)
def failed(failure: Failure) -> int:
print(failure.reason, file=sys.stderr)
return 1
+97
View File
@@ -0,0 +1,97 @@
import dataclasses
import decimal
import re
from rank_todo import cells, ITEMS_HEADER, table_rows
from larv_review.seats import DIMENSIONS
from larv_review.history import entries, Entry
from failure import Failure
CAP = 3
FLOOR = decimal.Decimal("7.0")
PASSES = ("architecture", "full scan", "security", "stability", "product-owner", "prose")
ROUND = re.compile(r"^Answer (.+) round ([0-9]+):")
@dataclasses.dataclass(frozen=True)
class HistoryChange:
"""A merged change on the default branch that touched the history, and the history before and after it."""
commit: str
before: str
after: str
@dataclasses.dataclass(frozen=True)
class PlanItem:
id: int
release: tuple[int, int, int]
tagline: str
def means(scores: dict[str, list[str]]) -> list[decimal.Decimal]:
"""The seats' mean of each score, in DIMENSIONS' order, rounded half up to two decimals."""
return [(sum((decimal.Decimal(seat[index]) for seat in scores.values()), decimal.Decimal(0)) / len(scores)).quantize(decimal.Decimal("0.01"), decimal.ROUND_HALF_UP) for index in range(len(DIMENSIONS))]
def under_floor(found: list[decimal.Decimal]) -> list[str]:
return [f"{name} {mean}" for name, mean in zip(DIMENSIONS, found) if mean < FLOOR]
def round_counts(titles: list[str]) -> dict[str, int]:
"""Each pass's highest round among the branch's `Answer <pass> round <n>:` commit titles; a pass with none has 0."""
counts = dict.fromkeys(PASSES, 0)
for title in titles:
if named := ROUND.match(title):
counts[named.group(1)] = max(counts.get(named.group(1), 0), int(named.group(2)))
return counts
def measured_commit(changes: list[HistoryChange]) -> str | None:
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better."""
for change in changes:
held = {entry.heading for entry in entries(change.before)}
added = [entry for entry in entries(change.after) if entry.heading not in held]
rulings = sorted((entry for entry in added if entry.against), key=lambda entry: entry.time)
if any(entry.against is None for entry in added) or rulings and rulings[-1].ruling == "better":
return change.commit
return None
def first_ruling(history: str) -> Entry | None:
rulings = sorted((entry for entry in entries(history) if entry.against), key=lambda entry: entry.time)
return rulings[0] if rulings else None
def plan_items(todo: str) -> list[PlanItem] | Failure:
"""The ID, release and tagline of each row in a todo.md's items table, in the shape `rank_todo.py --help` shows; a plan with no table has no items."""
lines = todo.splitlines()
if ITEMS_HEADER not in lines:
return []
table = table_rows(lines, ITEMS_HEADER)
if isinstance(table, Failure):
return table
items = []
for row in table.rows:
found = cells(row, 10)
if isinstance(found, Failure):
return found
id_, release, _, tagline, *_ = found
if not re.fullmatch(r"[0-9]+", id_) or not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", release):
return Failure(f"a todo.md row has no whole-number ID or no semver release: {row}")
major, minor, patch = (int(part) for part in release.split("."))
items.append(PlanItem(int(id_), (major, minor, patch), tagline))
return items
def scoring_run_due(before: list[PlanItem], after: list[PlanItem], ships: list[int], landed: bool) -> list[str]:
"""Why the chunk takes a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after the newest release tag. A release has work landed towards it when the chunk ships one of its items, or when it is the earliest release before names and landed holds."""
plan = {item.id: item for item in before + after}
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
left = {item.release for item in after if item.id not in ships}
named = {item.release for item in before} | {item.release for item in shipped}
earliest = min((item.release for item in before), default=None)
for release in sorted(named - left):
if any(item.release == release for item in shipped) or release == earliest and landed:
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
return reasons
+82
View File
@@ -0,0 +1,82 @@
import dataclasses
import re
from failure import Failure
DIMENSIONS = ("Navigation", "Locality", "Shape", "Self-sufficiency", "Overall")
FIRST_SEATS = ("Mid A", "Inherited architect")
SPLIT_SEATS = ("Maintainability senior", "Junior A")
SCORING_SEATS = ("Junior A", "Mid A", "Maintainability senior", "Inherited architect")
SEAT_LINE = re.compile(r"^(Junior A|Mid A|Maintainability senior|Inherited architect): (.+)$")
SCORES = re.compile(", ".join(rf"{name} ([0-9]+(?:\.[0-9]+)?)" for name in DIMENSIONS))
VOTES = ("better", "same", "worse")
@dataclasses.dataclass(frozen=True)
class Seat:
name: str
said: str
places: list[str]
@dataclasses.dataclass(frozen=True)
class Report:
"""A seats file: what the seats read, `<PR or branch> at <commit>`, with `, against <base>` for a paired ruling, then each seat's line and its places."""
read: str
seats: list[Seat]
def report(text: str) -> Report | Failure:
lines = [line for line in text.splitlines() if line.strip()]
if not lines:
return Failure("the seats file is empty; its first line names what the seats read: <PR or branch> at <commit>")
seats: list[Seat] = []
for line in lines[1:]:
if seat := SEAT_LINE.match(line):
if seat.group(1) in [known.name for known in seats]:
return Failure(f"{seat.group(1)} appears twice in the seats file")
seats.append(Seat(seat.group(1), seat.group(2).strip(), []))
elif seats and line.startswith("- "):
seats[-1].places.append(line[2:].strip())
elif seats and seats[-1].places and line.startswith(" "):
seats[-1].places[-1] += " " + line.strip()
else:
return Failure(f"the seats file has a line that is neither a seat's line nor one of its places: {line}")
return Report(lines[0].strip(), seats)
def votes(seats: list[Seat]) -> dict[str, str] | Failure:
"""Each seat's vote as it counts: a better or worse vote naming no `path:line` place counts as same."""
names = [seat.name for seat in seats]
if names[:2] != list(FIRST_SEATS) or names[2:] not in ([], list(SPLIT_SEATS)):
return Failure(f"a paired ruling's seats are {' and '.join(FIRST_SEATS)}, then, when their votes differ, {' and '.join(SPLIT_SEATS)}, in that order; the file has {', '.join(names) or 'none'}")
if unknown := [seat for seat in seats if seat.said not in VOTES]:
return Failure(f"{unknown[0].name}'s vote {unknown[0].said!r} is not better, same or worse")
return {seat.name: seat.said if seat.said == "same" or any(re.search(r"`[^`\s]*:\d+`", place) for place in seat.places) else "same" for seat in seats}
def ruling(counted: list[str]) -> str | None:
"""The vote that leads each other vote by two or more, else same; None when only the first two seats voted and they differ, so two more must vote."""
if len(counted) == 2 and counted[0] != counted[1]:
return None
for vote in VOTES:
if all(counted.count(vote) >= counted.count(other) + 2 for other in VOTES if other != vote):
return vote
return "same"
def scores(seats: list[Seat]) -> dict[str, list[str]] | Failure:
"""Each seat's five scores as written, in DIMENSIONS' order."""
if [seat.name for seat in seats] != list(SCORING_SEATS):
return Failure(f"a scoring run's seats are {', '.join(SCORING_SEATS)}, in that order; the file has {', '.join(seat.name for seat in seats) or 'none'}")
found = {}
for seat in seats:
if not (written := SCORES.fullmatch(seat.said)):
return Failure(f"{seat.name}'s line is not {', '.join(name + ' <score>' for name in DIMENSIONS)}: {seat.said}")
values = [float(value) for value in written.groups()]
if not all(1 <= value <= 10 for value in values):
return Failure(f"{seat.name}'s scores are not each from 1 to 10: {seat.said}")
if values[-1] > min(values[:-1]) + 1:
return Failure(f"{seat.name}'s overall exceeds their lowest dimension by more than 1; ask the seat to rescore: {seat.said}")
found[seat.name] = list(written.groups())
return found
+1 -1
View File
@@ -3,7 +3,7 @@ import collections
import dataclasses
import statistics
from metric_bench.places import Place
from larv_review.history import Place
from metric_bench.python_units import Unit
METRICS: dict[str, Callable[[Unit, dict[str, int]], int]] = {
+2 -2
View File
@@ -37,7 +37,7 @@ import sys
from metric_bench.bench import bench, co_changed, medians, METRICS, RunScores, Score
from failure import Failure
from metric_bench.python_units import measured, python_units, Unit
from metric_bench.places import resolved, scoring_runs, ScoringRun
from larv_review.history import resolved, scoring_runs, ScoringRun
def main(argv: list[str]) -> int:
@@ -56,7 +56,7 @@ def main(argv: list[str]) -> int:
return failed(benched)
rows, notes = benched
if not rows:
return failed(Failure(f"no scoring run in {history} could be scored; its format: ~/.claude/skills/larv-review/SKILL.md → \"History\"{noted(notes)}"))
return failed(Failure(f"no scoring run in {history} could be scored; its format is what ~/.claude/bin/larv-review writes{noted(notes)}"))
print(table(rows) + noted(notes))
return 0
-73
View File
@@ -1,73 +0,0 @@
import dataclasses
import os
import re
from markdown_sections import sections
# The format: skills/larv-review/SKILL.md §3 → "History", which names this reader.
RUN = re.compile(r"^(.+), (.+) at ([0-9a-f]{7,40})$")
SEAT_PLACES = re.compile(r"^.+, (hardest first|would restructure first):$")
# `path:line` or `:line`, then the unit's name where one follows.
PLACE = re.compile(r"`([^`\s]*):\d+`(?: `(\w+)`)?")
@dataclasses.dataclass(frozen=True)
class Place:
path: str
name: str | None
@dataclasses.dataclass(frozen=True)
class ScoringRun:
heading: str
commit: str
places: list[Place]
def scoring_runs(history: str) -> list[ScoringRun]:
"""Each scoring run in a comprehension history, with the places its seats named, as they wrote them; paired rulings name what changed, not what is hard, and are left out."""
runs = []
for heading, bodies in sections(history).items():
if (named := RUN.match(heading)) and ", against " not in heading:
runs.append(ScoringRun(heading, named.group(3), [place for body in bodies for place in seat_places(body)]))
return runs
def seat_places(body: str) -> list[Place]:
places = []
listing = False
for line in body.splitlines():
if SEAT_PLACES.match(line):
listing = True
elif listing and line.startswith("- "):
places += bullet_places(line)
elif not line.startswith(" "):
listing = False
return places
def bullet_places(bullet: str) -> list[Place]:
"""A `:line` with no path is in the file named before it in the bullet."""
places = []
path = ""
for match in PLACE.finditer(bullet):
path = match.group(1) or path
if path:
places.append(Place(path, match.group(2)))
return places
def resolved(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
"""Entries written before paths were written from the repository's root shorten a path once they have named its folder: `cycle/run.py` after `bin/loop_claude/cycle/states.py`. Each place resolves to the one tracked file it names, or, where several match, the one in the folder of the place resolved before it; a place naming no file, or several, is dropped. Returns the resolved places and the dropped ones."""
found = []
dropped = []
before = ""
for place in places:
candidates = [place.path] if place.path in tracked else [path for path in tracked if path.endswith("/" + place.path)]
nearest = [path for path in candidates if path.startswith(os.path.dirname(before) + "/")] if len(candidates) > 1 else candidates
if len(nearest) == 1:
before = nearest[0]
found.append(Place(before, place.name))
else:
dropped.append(place)
return found, dropped
+251
View File
@@ -0,0 +1,251 @@
import decimal
import os
import pathlib
import subprocess
import sys
import tempfile
import textwrap
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from larv_review.history import appended, entries, entry_text, Place, scoring_runs
from failure import Failure
from larv_review.rules import first_ruling, HistoryChange, means, measured_commit, plan_items, PlanItem, round_counts, scoring_run_due, under_floor
from test_loop_claude_black_box import host_env
from larv_review.seats import report, ruling, scores, Seat, votes
TOOL = pathlib.Path(__file__).resolve().parent / "larv-review"
SCORING = "## 2026-10-08T12:00:00Z, PR #1 at aaaaaaa\n\n| Seat | Overall |\n|---|---|\n| Mean | 7.00 |\n"
def paired(time: str, verdict: str, commit: str = "bbbbbbb") -> str:
return f"## {time}, PR #2 at {commit}, against aaaaaaa\n\nRuling: {verdict}\n"
def item(id_: int, release: tuple[int, int, int], tagline: str = "**Do it.**") -> PlanItem:
return PlanItem(id_, release, tagline)
def todo(*rows: str) -> str:
return "## Items\n\n| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |\n|---|---|---|---|---|---|---|---|---|---|\n" + "".join(f"{row}\n" for row in rows)
class Seats(unittest.TestCase):
def test_reads_each_seats_line_and_places_and_joins_a_wrapped_place(self):
given = report("PR #2 at abc1234, against def5678\n\nMid A: worse\n\n- `a.py:1` `f`: hard\n to read\n\nInherited architect: same\n")
self.assertEqual(given.read, "PR #2 at abc1234, against def5678")
self.assertEqual(given.seats, [Seat("Mid A", "worse", ["`a.py:1` `f`: hard to read"]), Seat("Inherited architect", "same", [])])
def test_refuses_a_line_that_is_neither_a_seat_nor_a_place_and_a_seat_twice(self):
self.assertIsInstance(report("PR #2 at abc1234\n\nMid A worse\n"), Failure)
self.assertIsInstance(report("PR #2 at abc1234\n\nMid A: worse\n\nMid A: same\n"), Failure)
self.assertIsInstance(report(""), Failure)
def test_counts_a_vote_naming_no_place_as_same(self):
seats = [Seat("Mid A", "worse", ["the whole skill"]), Seat("Inherited architect", "better", ["`a.py:3` `f`: shorter"])]
self.assertEqual(votes(seats), {"Mid A": "same", "Inherited architect": "better"})
def test_takes_the_first_two_seats_then_the_other_two_in_order(self):
first = [Seat("Mid A", "same", []), Seat("Inherited architect", "same", [])]
self.assertIsInstance(votes(first[::-1]), Failure)
self.assertIsInstance(votes(first + [Seat("Junior A", "same", [])]), Failure)
self.assertIsInstance(votes([first[0], Seat("Inherited architect", "worst", [])]), Failure)
def test_rules_by_a_lead_of_two_and_splits_two_seats_that_differ(self):
cases = [
(["worse", "worse"], "worse"),
(["same", "same"], "same"),
(["worse", "same"], None),
(["worse", "better", "worse", "worse"], "worse"),
(["worse", "same", "worse", "same"], "same"),
(["worse", "same", "worse", "better"], "same"),
(["better", "same", "better", "better"], "better"),
]
for counted, expected in cases:
with self.subTest(counted):
self.assertEqual(ruling(counted), expected)
def test_reads_four_seats_scores_and_refuses_an_overall_above_the_lowest_by_more_than_one(self):
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
seats = [Seat(name, line, []) for name in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect")]
self.assertEqual(scores(seats)["Mid A"], ["7", "6.5", "7", "6.5", "7"])
self.assertIsInstance(scores(seats[:3]), Failure)
self.assertIsInstance(scores([*seats[:3], Seat("Inherited architect", line.replace("Overall 7", "Overall 8"), [])]), Failure)
self.assertIsInstance(scores([*seats[:3], Seat("Inherited architect", line.replace("Shape 7", "Shape 11"), [])]), Failure)
def test_rounds_each_mean_half_up_and_names_each_under_the_floor(self):
found = means({"a": ["7.5", "6.5", "7", "7", "7"], "b": ["7.5", "7", "7", "6.5", "7"], "c": ["7", "6.5", "7", "6.5", "6.5"], "d": ["7.5", "6.5", "7", "6.5", "7"]})
self.assertEqual([str(mean) for mean in found], ["7.38", "6.63", "7.00", "6.63", "6.88"])
self.assertEqual(under_floor(found), ["Locality 6.63", "Self-sufficiency 6.63", "Overall 6.88"])
self.assertEqual(under_floor([decimal.Decimal("7.00")] * 5), [])
class History(unittest.TestCase):
def test_reads_back_what_it_writes(self):
ruled = entry_text("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", ["Ruling: worse"], [["Seat", "Vote"], ["Mid A", "worse"]], [("Mid A, decided by:", ["`bin/a.py:3` `f`: hard, with `:9` `g`"]), ("Inherited architect, decided by:", [])])
history = appended(appended("", SCORING), ruled)
self.assertTrue(history.startswith("# Comprehension history\n\n## 2026-10-08T12:00:00Z"))
self.assertNotIn("Inherited architect, decided by:", history)
found = entries(history)
self.assertEqual([(entry.commit, entry.against, entry.ruling) for entry in found], [("aaaaaaa", None, None), ("bbbbbbb", "aaaaaaa", "worse")])
self.assertEqual(found[1].places, [Place("bin/a.py", "f"), Place("bin/a.py", "g")])
self.assertEqual([run.commit for run in scoring_runs(history)], ["aaaaaaa"])
class Base(unittest.TestCase):
def test_takes_the_newest_change_adding_a_scoring_run_or_ending_on_a_better_ruling(self):
older = HistoryChange("old", "", SCORING)
same = HistoryChange("same", SCORING, SCORING + paired("2026-10-08T13:00:00Z", "same"))
self.assertEqual(measured_commit([same, older]), "old")
# The file holds the later ruling first, as a merge conflict resolved the other way would.
better = HistoryChange("better", SCORING, SCORING + paired("2026-10-08T14:00:00Z", "better") + paired("2026-10-08T13:00:00Z", "worse"))
self.assertEqual(measured_commit([better, older]), "better")
worse_last = HistoryChange("worse", SCORING, SCORING + paired("2026-10-08T13:00:00Z", "better") + paired("2026-10-08T14:00:00Z", "worse"))
self.assertEqual(measured_commit([worse_last, older]), "old")
self.assertIsNone(measured_commit([same]))
def test_finds_the_earliest_ruling_by_its_time(self):
ruled = first_ruling(paired("2026-10-08T14:00:00Z", "same", "ccccccc") + paired("2026-10-08T13:00:00Z", "worse"))
self.assertEqual(ruled and ruled.commit, "bbbbbbb")
self.assertIsNone(first_ruling(SCORING))
class Rounds(unittest.TestCase):
def test_counts_each_passs_highest_round(self):
counts = round_counts(["Answer prose round 1: x", "Answer full scan round 2: y", "Answer full scan round 1: z", "Answer stability: no round", "Rescore: w"])
self.assertEqual((counts["prose"], counts["full scan"], counts["stability"], counts["architecture"]), (1, 2, 0, 0))
class ScoringRunDue(unittest.TestCase):
def test_reads_each_rows_id_release_and_tagline(self):
self.assertEqual(plan_items(todo("| 3 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), [item(3, (0, 1, 0))])
self.assertEqual(plan_items("# todo\n"), [])
self.assertIsInstance(plan_items(todo("| x | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), Failure)
def test_is_due_for_an_item_naming_the_floor(self):
plan = [item(1, (0, 1, 0), "**Lift the project to the Comprehension floor.**"), item(2, (0, 1, 0))]
self.assertEqual(scoring_run_due(plan, plan, [1], True), ["ships item 1, which names the comprehension floor"])
def test_is_due_when_the_chunk_ships_a_releases_last_item(self):
plan = [item(1, (0, 1, 0)), item(2, (0, 2, 0))]
self.assertEqual(scoring_run_due(plan, plan, [1], False), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(plan, plan + [item(3, (0, 1, 0))], [1], True), [])
def test_is_due_when_the_chunk_removes_the_next_releases_last_item_after_work_landed(self):
before = [item(1, (0, 1, 0)), item(2, (0, 1, 0)), item(3, (0, 2, 0))]
after = [item(2, (0, 1, 0)), item(3, (0, 2, 0))]
self.assertEqual(scoring_run_due(before, after, [2], True), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, after, [], True), [])
emptied = [item(3, (0, 2, 0))]
self.assertEqual(scoring_run_due(before, emptied, [2], False), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, emptied, [], True), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, emptied, [], False), [])
def test_is_not_due_when_a_later_release_loses_its_last_item_unshipped(self):
before = [item(1, (0, 1, 0)), item(3, (0, 2, 0))]
self.assertEqual(scoring_run_due(before, before[:1], [], True), [])
class LarvReviewTool(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
root = pathlib.Path(os.path.realpath(temp.name))
self.env = {**host_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(root)}
self.checkout = root / "proj"
subprocess.run(["git", "init", "-q", "--bare", "-b", "main", str(root / "origin.git")], check=True, env=self.env)
subprocess.run(["git", "clone", "-q", str(root / "origin.git"), str(self.checkout)], check=True, env=self.env, capture_output=True)
self.commit("start", {"todo.md": todo("| 1 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |", "| 2 | 0.2.0 | | **Then this.** | 1 | 1 | 1 | 1 | 1 | 9.0 |", "| 3 | 0.2.0 | | **And this.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")})
self.git("push", "-q", "origin", "HEAD:main")
self.git("remote", "set-head", "origin", "main")
self.git("switch", "-q", "-c", "chunk")
def git(self, *args):
return subprocess.run(["git", "-c", "user.name=t", "-c", "user.email=t@t", *args], cwd=self.checkout, check=True, env=self.env, capture_output=True, text=True).stdout
def commit(self, title, files=None):
for name, text in (files or {}).items():
(self.checkout / name).parent.mkdir(parents=True, exist_ok=True)
(self.checkout / name).write_text(text)
self.git("add", name)
self.git("commit", "-q", "--allow-empty", "-m", title)
def run_tool(self, *args):
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=self.checkout, env=self.env, text=True)
def seats_file(self, text):
path = self.checkout.parent / "seats.md"
path.write_text(textwrap.dedent(text))
return str(path)
def test_takes_the_merge_base_until_a_merged_change_records_a_scoring_run(self):
start = self.git("rev-parse", "HEAD").strip()
self.commit("work")
ran = self.run_tool("ruling-base")
self.assertEqual((ran.returncode, ran.stdout.splitlines()[0]), (0, start), ran.stderr)
self.git("switch", "-q", "-c", "measured", "main")
self.commit("Merge a scoring run", {"docs/comprehension-history.md": "# Comprehension history\n\n" + SCORING})
self.git("push", "-q", "origin", "HEAD:main")
measured = self.git("rev-parse", "HEAD").strip()
self.git("switch", "-q", "chunk")
self.assertEqual(self.run_tool("ruling-base").stdout.splitlines()[0], measured)
def test_counts_rounds_on_this_branch_alone(self):
self.commit("Answer stability round 1: a")
self.commit("Answer stability round 2: b")
self.commit("Answer prose round 1: c")
self.commit("Answer stability round 3: d")
ran = self.run_tool("round-count")
self.assertIn("stability: 3, at the cap\n", ran.stdout)
self.assertIn("prose: 1\n", ran.stdout)
self.assertIn("architecture: 0\n", ran.stdout)
def test_says_whether_a_scoring_run_is_due_and_refuses_an_unknown_item(self):
self.assertEqual(self.run_tool("needs-scoring-run", "1").stdout, "Scoring run due: the chunk leaves release 0.1.0 with no plan item.\n")
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, "No scoring run due.\n")
unknown = self.run_tool("needs-scoring-run", "9")
self.assertEqual(unknown.returncode, 1)
self.assertIn("item 9", unknown.stderr)
def test_records_a_ruling_once_four_seats_settle_a_split(self):
lines = """\
PR #2 at abc1234, against def5678
Mid A: worse
- `bin/a.py:3` `f`: three branches
Inherited architect: same
"""
split = self.run_tool("record-ruling", self.seats_file(lines))
self.assertEqual(split.returncode, 0, split.stderr)
self.assertTrue(split.stdout.startswith("Split: Mid A worse, Inherited architect same."))
history = self.checkout / "docs" / "comprehension-history.md"
self.assertFalse(history.exists())
lines += "\n\t\t\tMaintainability senior: worse\n\n\t\t\t- `bin/a.py:3` `f`: hard\n\n\t\t\tJunior A: worse\n"
ran = self.run_tool("record-ruling", self.seats_file(lines))
self.assertEqual(ran.stdout.splitlines()[0], "Ruling: same. Votes as counted: Mid A worse, Inherited architect same, Maintainability senior worse, Junior A same.")
recorded = entries(history.read_text())
self.assertEqual([(entry.commit, entry.against, entry.ruling) for entry in recorded], [("abc1234", "def5678", "same")])
self.assertIn("| Junior A | same (voted worse, naming no place) |", history.read_text())
def test_records_a_scoring_run_with_its_means(self):
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
seats = "".join(f"\n{seat}: {line}\n\n- `bin/a.py:3` `f`: long\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
ran = self.run_tool("record-scoring", self.seats_file("PR #2 at abc1234\n" + seats))
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertIn("Under the floor of 7.0: Locality 6.50, Self-sufficiency 6.50.", ran.stdout)
text = (self.checkout / "docs" / "comprehension-history.md").read_text()
self.assertIn("| Mean | 7.00 | 6.50 | 7.00 | 6.50 | 7.00 |", text)
self.assertIn("Inherited architect, would restructure first:", text)
self.assertEqual([run.places for run in scoring_runs(text)], [[Place("bin/a.py", "f")] * 4])
def test_refuses_a_ruling_file_that_names_no_base(self):
ran = self.run_tool("record-ruling", self.seats_file("PR #2 at abc1234\n\nMid A: same\n\nInherited architect: same\n"))
self.assertEqual(ran.returncode, 1)
self.assertIn("against <base>", ran.stderr)
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -13,7 +13,7 @@ from metric_bench.bench import auc, bench, co_changed, top_k
from metric_bench.python_units import cognitive, measured, python_units, Unit
from failure import Failure
from test_loop_claude_black_box import host_env
from metric_bench.places import Place, resolved, scoring_runs
from larv_review.history import Place, resolved, scoring_runs
TOOL = pathlib.Path(__file__).resolve().parent / "metric-bench"
+1 -1
View File
@@ -11,5 +11,5 @@ class Types(unittest.TestCase):
self.assertIsNotNone(floor, "README.md's Install names `Python 3.N or newer`")
if shutil.which("uv") is None:
self.fail("uv on PATH fetches mypy; see README.md's Test line")
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "-p", "loop_claude", "-p", "metric_bench", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline"], capture_output=True, cwd=ROOT / "bin", text=True)
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "-p", "larv_review", "-p", "loop_claude", "-p", "metric_bench", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline"], capture_output=True, cwd=ROOT / "bin", text=True)
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
+2 -2
View File
@@ -50,8 +50,8 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
- 2026-10-04, PR #215: 7.38, 6.63, 7.0, 6.88, 6.88.
- 2026-10-08, PR #224: 7.00, 6.50, 7.00, 6.50, 6.75. lilleman shipped it below the ratchet as well: its 20 lines of skill prose left the code as it was, so the fall is the panel's noise between runs, which the paired ruling cancels.
- 2026-10-08, PR #225: 7.25, 6.75, 7.00, 6.88, 6.88. Above PR #224's run on every score but Shape, which held.
- 2026-10-08, PR #228: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet: the seats blamed the new ratchet's own rules, which `todo.md` item 85 moves into a tested tool.
- 2026-10-08, PR #229: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet for PR #228's reason: the seats' decisive places were still PR #228's rules, which item 85 moves into a tested tool.
- 2026-10-08, PR #228: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet: the seats blamed the new ratchet's own rules, since moved into the tested `bin/larv-review`.
- 2026-10-08, PR #229: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet for PR #228's reason: the seats' decisive places were still PR #228's rules, since moved into `bin/larv-review`.
## loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
+12 -10
View File
@@ -32,13 +32,15 @@ A pass is one kind of review, a row of the table, named by its first column. Its
| Pass | Reviewer | Trigger |
|------|----------|----------------------------------------|
| Architecture | systems-architect, then principles-auditor and the comprehension check | ships or removes the last plan item of a release that has work landed towards it, or ships an item naming the comprehension floor; or carries an architectural decision: a new, restructured or moved component, module, service or boundary; a new dependency or first use of an infrastructure pattern (queue, cron, cache, websocket, …); a new or changed external contract (routes, request/response shapes, events, webhooks, published file formats, CLI flags and exit codes); a schema change beyond an additive column; a fix, only where it carries an architectural decision |
| Architecture | systems-architect, then principles-auditor and the comprehension check | is due a scoring run ("Architecture" → "Scoring run"); or carries an architectural decision: a new, restructured or moved component, module, service or boundary; a new dependency or first use of an infrastructure pattern (queue, cron, cache, websocket, …); a new or changed external contract (routes, request/response shapes, events, webhooks, published file formats, CLI flags and exit codes); a schema change beyond an additive column; a fix, only where it carries an architectural decision |
| Full scan | the `claude-security` plugin at high effort: `/claude-security scan this branch's changes at high` | touches authn/authz, secrets or crypto, opens an untrusted-input boundary, or adds an external surface |
| Security | `/security-review` | changes code or config, unless the full scan runs on the same change; a fix, only where it opens an untrusted-input boundary or touches authn/authz, secrets or crypto, and the full scan does not rerun on it |
| Stability | stability-reviewer | changes code, config or tests; a fix, only where it changes behaviour: what the code or its tests do |
| Product-owner | product-owner-reviewer | changes an external surface: API, CLI, UI, error or deprecation text a consumer reads, README and docs an outside reader lands on |
| Prose | prose-reviewer | every chunk, one round |
`~/.claude/bin/larv-review` holds this section's mechanical rules: the round count, the paired ruling's base and vote count, the scoring run's trigger and the history's format. Run it in the chunk's worktree; `larv-review --help` states each rule. On exit 1, repair what its message names and run it again.
A comment is prose, not code. Passes run in this order: line-level polish on code the architect wants restructured, or on a call the security pass will rewrite, is wasted work.
Give a reviewer without Bash the branch diff and `git ls-files`'s output in its prompt: some harnesses grant it Read alone.
@@ -55,7 +57,7 @@ Give a reviewer without Bash the branch diff and `git ls-files`'s output in its
- **Round.** A round is one launch of a pass's reviewer: the first, each re-entry, and the rerun after the architect returns Stage 0 questions ("Architecture" below). A rescore is a round only as "Architecture" → "Fresh" says.
- **Architecture round.** The architecture pass's first round is the architect (unless skipped), the principles audit and the comprehension check, which "Architecture" → "Suspended" may skip. A later round relaunches each reviewer whose 🔴 or 🟡 the previous round's fixes answer, the comprehension check only as "Architecture" → "Fresh" says. Where those fixes carry an architectural decision, the architect and the principles audit run again.
- **Auto-reviewer.** The external auto-reviewer (§4) is a pass of its own, each review it posts a round.
- **Count.** Close each round with one commit that holds the round's fixes and is titled `Answer <pass> round <n>: …`, `<pass>` being the pass's name in lower case, such as `full scan`. A round that changed nothing gets an empty commit (`git commit --allow-empty`). Push the commits when §4 runs next, where there is a remote. A pass's count is the highest `<n>` among titles holding `Answer <pass> round <n>:` in `git log origin/<default branch>..HEAD` after a `git fetch`, or in `git log <default branch>..HEAD` with no remote. The auto-reviewer's count is the number of its reviews on the PR, less its reviews of a `Rescore:` commit.
- **Count.** Close each round with one commit that holds the round's fixes and is titled `Answer <pass> round <n>: …`, `<pass>` being the pass's name in lower case, such as `full scan`. A round that changed nothing gets an empty commit (`git commit --allow-empty`). Push the commits when §4 runs next, where there is a remote. `larv-review round-count` prints each pass's count and marks a pass at the cap. The auto-reviewer's count is the number of its reviews on the PR, less its reviews of a `Rescore:` commit.
- **Later rounds.** A rerun pass reviews only what changed since it last reviewed: `git diff <the commit it last reviewed, or else the commit before this round's fixes> HEAD`, since the fixes may be unpushed. The comprehension check's seats still read the whole project. From round 2, prompt an agent under `~/.claude/agents/` with its own last round's findings verbatim too.
- **Done.** A pass is done when its latest round leaves no 🔴 or 🟡 standing: none was reported, or each was shown factually wrong, declined, filed, or named under the bar per "Findings", or, for the prose pass, fixed. Resolve that round's 🟢s and ❓s per "Findings" too.
- **Cap.** A pass takes at most three rounds per chunk. Instead of a fourth, list what is open and ask the user. For the auto-reviewer, ask before the push that would summon its fourth, and hold the fixes unpushed until the answer.
@@ -90,17 +92,17 @@ Then the comprehension check: simulated readers, launched per `~/.claude/skills/
**Paired ruling**, in every architecture pass. Each seat votes whether the project reads better, the same or worse at this branch's head than at the base.
- **Base.** The last measured commit: the newest commit on `origin/<default branch>`, after a `git fetch`, whose change to the history (below) adds a scoring run, or whose last added paired ruling is better. Read each merged change whole, a merge commit as one change: `git log --first-parent --diff-merges=first-parent -p origin/<default branch> -- docs/comprehension-history.md`; with no remote, use `<default branch>`. No such commit → the base the first paired ruling in that history names; no ruling either → the merge base of the head and the default branch.
- **Base.** The last measured commit, which `larv-review ruling-base` prints.
- **Head.** Before each ruling, merge `origin/<default branch>` into this branch unless the head already holds it, so the seats see only this branch's change on top of the base.
- **Seats.** Mid A and the inherited architect. When their votes differ, the maintainability senior and Junior A vote too, launched together.
- **Ruling.** The vote that leads each other vote by two or more; without one, same. A better or worse vote that names no place counts as same.
- **Seats.** Mid A and the inherited architect. Write what they read and their replies into a seats file in your scratchpad, in the shape `larv-review --help` shows, and run `larv-review record-ruling <file>`. On `Split:`, launch the maintainability senior and Junior A together, add their lines, and run it again.
- **Ruling.** The `Ruling:` line it prints, which it appends to the history.
**Scoring run**, when the chunk ships or removes the last plan item of a release that has work landed towards it, or ships an item naming the comprehension floor. Junior A, Mid A, the maintainability senior and the inherited architect each score the project at the head from 1 to 10 overall and on four dimensions: Navigation, Locality, Shape and Self-sufficiency. A seat's overall may not exceed its lowest dimension by more than 1. Each score is the mean of the four seats, to two decimals.
**Scoring run**, when `larv-review needs-scoring-run <the IDs of the items the chunk ships>` prints `Scoring run due:`. Run it before removing the chunk's items from `todo.md`. Junior A, Mid A, the maintainability senior and the inherited architect each score the project at the head from 1 to 10 overall and on four dimensions: Navigation, Locality, Shape and Self-sufficiency. Write their lines into a seats file and run `larv-review record-scoring <file>`, which appends the run to the history and prints the four seats' means.
The chunk does not merge while either holds:
- **Ratchet.** The paired ruling is worse.
- **Floor.** The scoring run puts any dimension or the overall below 7.0.
- **Floor.** `record-scoring` names a mean under the floor of 7.0.
Each is a 🔴 of the comprehension check. It is never filed, and stands until a later run clears it or, for the floor, a decision to ship answers it.
@@ -108,16 +110,16 @@ Fix a worse ruling at the places the seats voting worse name, and the floor at t
Two files of the project keep the records:
- **History.** `docs/comprehension-history.md` keeps every paired ruling and scoring run, appended and never edited. Each opens with a `## <UTC time, ISO 8601>, <PR or branch> at <commit>` heading naming when the seats finished and the commit they read; a paired ruling's heading ends `, against <base>`. A paired ruling holds a `Ruling: <better, same or worse>` line, a table of each seat's vote, and under each seat, labelled `<seat>, decided by:`, its places. A scoring run holds a table of each seat's five scores, with a last row holding the four seats' mean of each, and under each seat, labelled `<seat>, hardest first:` or, for the architect, `<seat>, would restructure first:`, its places. A place is `path:line` and the unit's name, the path written from the repository's root. A bare `:line` is in the path named before it in the same bullet. `bin/metric_bench/places.py` reads this format: change both together. Appending re-enters no pass. Resolve a merge conflict in the file by putting the default branch's entries first.
- **History.** `docs/comprehension-history.md` keeps every paired ruling and scoring run, which `larv-review` appends; nothing edits them. Appending re-enters no pass. Resolve a merge conflict in the file by putting the default branch's entries first.
- **Entry.** `docs/decisions.md` keeps one entry whose title opens `comprehension floor:`. It holds each decision to ship under the floor, with its scores.
A `comprehension floor:` entry written before the history may still hold a scoring run's five scores. The next scoring run moves them, and only them, into the history as its first run: a heading of the date or time the entry gives and whatever it names the run by, its commit included where named, and a table with the mean alone. The rest of the entry stays.
Put each ruling with its base and votes, and each run's five scores, in the summary, and append them to the history on this branch. List in the entry each decision to ship under the floor, and file its places in `todo.md` per `~/.claude/CLAUDE.md` → "Todo items". A project without the entry gets it at its first scoring run, titled `comprehension floor: every dimension and the overall at 7.0 or above` and applying KISS (`~/.claude/principles/technical-principles.md`).
Put each ruling with its base and votes, and each run's five scores, in the summary. List in the entry each decision to ship under the floor, and file its places in `todo.md` per `~/.claude/CLAUDE.md` → "Todo items". A project without the entry gets it at its first scoring run, titled `comprehension floor: every dimension and the overall at 7.0 or above` and applying KISS (`~/.claude/principles/technical-principles.md`).
**Fresh.** Where the chunk ran a comprehension check, the architecture pass is done only once its last paired ruling, and its last scoring run where it ran one, read the commit to merge. They may predate commits that change only `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`. Any other commit after them, a prose fix included, because the seats read the docs, calls for a rescore: the comprehension check again, once every other pass is done, the auto-reviewer included. A rescore reporting a 🔴 that no decision to ship answers is an architecture round. Any other rescore is not a round: append only its results to the history, in a commit titled `Rescore: …`. A `Rescore:` commit re-enters no pass: push it and wait for the forge's checks (§4 step 3), not for an auto-review.
**Suspended.** A decision in the project's `docs/decisions.md`, held before the chunk began, may name plan items to ship without a comprehension check. A named item counts as `todo.md` worded it before the chunk began. The decision must name the later plan item whose chunk ends the suspension with a comprehension check. A chunk may ship named items only with each other. Such a chunk skips the comprehension check, so neither the ratchet nor the floor can block it, and its summary cites the decision. A chunk that meets the scoring run's trigger runs it, even when every item it ships is named.
**Suspended.** A decision in the project's `docs/decisions.md`, held before the chunk began, may name plan items to ship without a comprehension check. A named item counts as `todo.md` worded it before the chunk began. The decision must name the later plan item whose chunk ends the suspension with a comprehension check. A chunk may ship named items only with each other. Such a chunk skips the comprehension check, so neither the ratchet nor the floor can block it, and its summary cites the decision. A chunk due a scoring run runs it, even when every item it ships is named.
### Security