Rule each dimension in the paired ruling, and count a better ruling as worse when a dimension it rules fallen is under the floor

This commit is contained in:
2026-10-09 10:51:09 +02:00
parent a984e98e0a
commit 2c6e0633cf
10 changed files with 197 additions and 88 deletions
+1 -1
View File
@@ -12,7 +12,7 @@ Decisions live in `docs/decisions.md`:
- loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
- review: a pass gets at most three rounds per chunk; instead of a fourth, the run asks the human
- review: a fix for a 🔴 or 🟡 reruns the pass that found it and each pass whose trigger names fixes like it; a fix for a 🟢, a ❓ or a prose finding reruns none
- review: a paired ruling against the last measured commit decides the ratchet; the four-seat scoring run checks the floor when a release's items run out or an item names the floor
- review: a paired ruling against the last measured commit decides the ratchet, and counts as worse when a dimension it rules fallen is under the floor; the four-seat scoring run checks the floor when a release's items run out, an item names the floor or a better ruling names a fallen dimension
- review: a project decision may suspend the comprehension check for the plan items it names
- review: a pass skips only on a precedent: a decision, or a plan item crediting the pass, that the repo held before the chunk began
- loop-claude: the archive keeps the raw transcript, and a failed archive stops the loop
+1
View File
@@ -2,6 +2,7 @@
## Unreleased
- larv-review's paired ruling takes each seat's vote on each dimension too: `Navigation same, Locality worse, Shape same, Self-sufficiency same, Overall better`. A better ruling in which a dimension fell calls a scoring run, and counts as worse, blocking the merge, when that dimension's mean is under the floor.
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling` and `record-scoring`, which read a seats file and append to `docs/comprehension-history.md`). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release whose last plan item moves to another release is due a scoring run too, once work has landed towards it. A seats file must name commits the checkout holds. Scores a `comprehension floor:` decision entry still holds move into the history through `record-scoring`, as a `Mean:` line. When origin cannot be fetched, the tool reads it as last fetched and says so.
- The new `bin/metric-bench` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest: cognitive complexity, length, shared state and change spread, each scored by how well it ranks the named functions above the rest. No metric enters review yet.
- larv-review blocks a merge when simulated readers rule that the PR makes the project read worse than the last measured commit: the default branch's newest merged change that added a scoring run to `docs/comprehension-history.md`, or whose last ruling there is better. A ruling of same leaves that commit in place, so a worse ruling may name places earlier chunks wrote. In each architecture pass Mid A and the inherited architect vote better, same or worse; when they disagree, the maintainability senior and Junior A vote too. A ruling needs a lead of two or more votes, else it is same, and a vote that names no place counts as same. A worse ruling never ships: after three rounds larv-review asks you for another round, a no leaves the PR unmerged, and so does an unattended run. The four-seat scoring run checks the comprehension floor only when a chunk ships or removes a release's last plan item, or ships an item naming the floor; between those runs the floor blocks nothing. A score under the floor blocks the merge until fixed or until you decide to ship; a decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
+15 -11
View File
@@ -8,6 +8,7 @@ from markdown_sections import sections
DATED = re.compile(r"^[0-9]{4}-[0-9]{2}-[0-9]{2}(?:T[0-9]{2}:[0-9]{2}:[0-9]{2}Z)?, ")
DECIDED_BY = "decided by"
FELL = re.compile(r"^Fell: (.+)$", re.M)
HARDEST_FIRST = "hardest first"
HEADING = re.compile(r"^([^,]+), (.+) at ([0-9a-f]{7,40})(?:, against ([0-9a-f]{7,40}))?$")
# `path:line` or `:line`, then the unit's name where one follows.
@@ -27,12 +28,13 @@ class Place:
@dataclasses.dataclass(frozen=True)
class Entry:
"""One paired ruling, which names the commit it was against and its ruling, or one scoring run, which names neither."""
"""One paired ruling, which names the commit it was against, its ruling and the dimensions that fell when it is better, or one scoring run, which names none of them."""
heading: str
time: str
commit: str
against: str | None
ruling: str | None
fell: list[str]
places: list[Place]
@property
@@ -62,7 +64,8 @@ def entries(history: str) -> list[Entry]:
if named := HEADING.match(heading):
body = "\n".join(bodies)
ruling = RULING.search(body)
found.append(Entry(heading, named.group(1), named.group(3), named.group(4), ruling.group(1) if ruling else None, seat_places(body)))
fell = FELL.search(body)
found.append(Entry(heading, named.group(1), named.group(3), named.group(4), ruling.group(1) if ruling else None, fell.group(1).split(", ") if fell else [], seat_places(body)))
return found
@@ -108,21 +111,22 @@ def appended(history: str, entry: str) -> str:
return f"{text}\n{entry}"
def ruling_entry(title: str, seats: list[Seat], counted: dict[str, str], ruled: str) -> str:
"""A paired ruling's entry; a seat's vote that counted as same for naming no place says so."""
votes = [[seat.name, vote_text(seat, counted[seat.name])] for seat in seats]
return entry_text(title, [f"Ruling: {ruled}"], [["Seat", "Vote"], *votes], [(f"{seat.name}, {DECIDED_BY}:", seat.places) for seat in seats])
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, fell: list[str]) -> str:
"""A paired ruling's entry, with a `Fell:` line when it is better and fell names dimensions; a seat's vote that counted as same for naming no place says so."""
votes = [[seat.name, *map(vote_text, said[seat.name], counted[seat.name])] for seat in seats]
lines = [f"Ruling: {ruled}"] + ([f"Fell: {', '.join(fell)}"] if ruled == "better" and fell else [])
return entry_text(title, lines, [["Seat", *DIMENSIONS], *votes], [(f"{seat.name}, {DECIDED_BY}:", seat.places) for seat in seats])
def vote_text(seat: Seat, counted: str) -> str:
return counted if counted == seat.said else f"same (voted {seat.said}, naming no place)"
def vote_text(said: str, counted: str) -> str:
return counted if counted == said else f"same (voted {said}, naming no place)"
def scoring_entry(title: str, seats: list[Seat], scores: dict[str, list[str]], means: list[str]) -> str:
"""A scoring run's entry; a run moved from a decision entry has its means alone."""
def scoring_entry(title: str, seats: list[Seat], scores: dict[str, list[str]], means: list[str], lines: list[str]) -> str:
"""A scoring run's entry, with any lines such as a `Ratchet:` line; a run moved from a decision entry has its means alone."""
rows = [[seat, *values] for seat, values in scores.items()] + [[MEAN, *means]]
labels = [(f"{seat.name}, {RESTRUCTURE_FIRST if seat.name == ARCHITECT else HARDEST_FIRST}:", seat.places) for seat in seats]
return entry_text(title, [], [["Seat", *DIMENSIONS], *rows], labels)
return entry_text(title, lines, [["Seat", *DIMENSIONS], *rows], labels)
def entry_text(title: str, lines: list[str], table: list[list[str]], seats: list[tuple[str, list[str]]]) -> str:
+52 -22
View File
@@ -20,7 +20,8 @@ cap of {cap} rounds, and each name in such a title that is not a pass.
needs-scoring-run takes the IDs of the todo.md items the chunk ships, none when it ships none, and
prints why the chunk is due a scoring run, or `No scoring run due.` It is due when the chunk ships
an item whose tagline names the comprehension floor, or when it leaves a release with no item in
an item whose tagline names the comprehension floor, when the checkout's history ends on a better
paired ruling with a `Fell:` line, or when it leaves a release with no item in
the checkout's todo.md, less the shipped items, and work has landed towards that release. Work has
landed when the chunk ships one of the release's items, or when the release is the earliest the
default branch's todo.md names and the default branch has a commit after its newest v* tag, or has
@@ -34,26 +35,30 @@ opens with its label, written exactly: `Junior A:`, `Mid A:`, `Maintainability s
PR #231 at abc1234, against def5678
Mid A: worse
Mid A: Navigation same, Locality worse, Shape same, Self-sufficiency same, Overall worse
- `bin/x.py:12` `run`: three branches to hold at once
Inherited architect: worse
Inherited architect: Navigation same, Locality worse, Shape same, Self-sufficiency same, Overall worse
- `bin/x.py:12` `run`: ...
record-ruling takes Mid A's and the inherited architect's votes, better, same or worse; a better or
worse vote naming no place in backticks, `path:line`, counts as same, and the printed votes say so.
When the two counted votes differ, it appends nothing and prints `Split:`, saying to add the
maintainability senior's and Junior A's lines. Otherwise the ruling is the vote that leads each
other vote by two or more, else same; it prints `Ruling:` and appends the ruling.
record-ruling takes Mid A's and the inherited architect's votes on each dimension and overall,
each better, same or worse. Each vote of a seat naming no place in backticks, `path:line`, counts
as same, and the printed votes say so. When the two counted overall votes differ, it appends
nothing and prints `Split:`, saying to add the maintainability senior's and Junior A's lines.
Otherwise the ruling is the overall vote that leads each other vote by two or more, else same; it
prints `Ruling:` and appends the ruling. A dimension falls when its worse votes lead each other
vote by two or more. On a better ruling, it prints and records a `Fell:` line naming each.
record-scoring takes Junior A's, Mid A's, the maintainability senior's and the inherited
architect's lines in that order, or a `Mean:` line alone for a run moved from a decision entry.
Each line is `Navigation <n>, Locality <n>, Shape <n>, Self-sufficiency <n>, Overall <n>`, each
score from 1 to 10 and the overall at most 1 above the line's lowest dimension. Its first line
names no `, against`. It appends the seats' scores and their means, rounded half up to two
decimals, and prints the means and each one under the floor of {floor}.
decimals, and prints the means and each one under the floor of {floor}. When the history ends on a
better paired ruling with a `Fell:` line, it says whether the ruling stands; when a dimension that
fell is under the floor, the ruling counts as worse, and it prints and records a `Ratchet:` line.
Each record is appended to the history, created when missing, under a heading of the current UTC
time and the seats file's first line. A file holding a `Mean:` line alone opens with the date or
@@ -71,7 +76,7 @@ import time
import typing
from larv_review.history import appended, heading, ruling_entry, scoring_entry, vote_text
from larv_review.rules import base, CAP, FLOOR, means, PASSES, plan_items, round_counts, ruling, scoring_run_due, under_floor, votes
from larv_review.rules import base, CAP, counted, fallen, fell_under_floor, FLOOR, means, PASSES, pending_fall, plan_items, round_counts, ruling, scoring_run_due, under_floor, votes
from larv_review.git import commit_exists, default_branch, file_at, git, HISTORY, history_changes, landed_since_release, top
from larv_review.seats import DIMENSIONS, MEAN, Report, report, score_line, scores
from failure import Failure
@@ -168,7 +173,10 @@ def needs_scoring_run(default: str, ids: list[int]) -> Failure | None:
landed = landed_since_release(default)
if isinstance(landed, Failure):
return landed
reasons = scoring_run_due(planned, chunk, ids, landed)
history = history_text()
if isinstance(history, Failure):
return history
reasons = scoring_run_due(planned, chunk, ids, landed, pending_fall(history[1]))
if isinstance(reasons, Failure):
return reasons
print(f"Scoring run due: the chunk {' and '.join(reasons)}." if reasons else "No scoring run due.")
@@ -180,16 +188,20 @@ def record_ruling(seats_file: pathlib.Path) -> Failure | None:
if isinstance(read, Failure):
return read
title, given = read
counted = votes(given.seats)
if isinstance(counted, Failure):
return counted
tally = ", ".join(f"{seat.name} {vote_text(seat, counted[seat.name])}" for seat in given.seats)
ruled = ruling(list(counted.values()))
said = votes(given.seats)
if isinstance(said, Failure):
return said
count = counted(given.seats, said)
tally = ", ".join(f"{seat.name} {vote_text(said[seat.name][-1], count[seat.name][-1])}" for seat in given.seats)
ruled = ruling([each[-1] for each in count.values()])
if ruled is None:
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the seats file, and run this again.")
return None
print(f"Ruling: {ruled}. Votes as counted: {tally}.")
return append(ruling_entry(title, given.seats, counted, ruled))
fell = fallen(count)
print(f"Ruling: {ruled}. Overall votes as counted: {tally}.")
if ruled == "better" and fell:
print(f"Fell: {', '.join(fell)}. A scoring run is due; the ruling counts as worse if it puts {' or '.join(fell)} under the floor of {FLOOR}.")
return append(ruling_entry(title, given.seats, said, count, ruled, fell))
def record_scoring(seats_file: pathlib.Path) -> Failure | None:
@@ -206,7 +218,14 @@ def record_scoring(seats_file: pathlib.Path) -> Failure | None:
mean = means(list(found.values()) or [moved])
under = under_floor(mean)
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, mean))}. {f'Under the floor of {FLOOR}: ' + ', '.join(under) if under else f'None under the floor of {FLOOR}'}.")
return append(scoring_entry(title, given.seats, found, [str(value) for value in mean]))
history = history_text()
if isinstance(history, Failure):
return history
fell = [] if given.moved_mean else pending_fall(history[1])
ratchet = [f"Ratchet: the paired ruling counts as worse, as {', '.join(dropped)} fell under the floor of {FLOOR}."] if (dropped := fell_under_floor(fell, mean)) else []
if fell:
print(ratchet[0] if ratchet else f"The better paired ruling stands: {', '.join(fell)} fell and stays at or above the floor of {FLOOR}.")
return append(scoring_entry(title, given.seats, found, [str(value) for value in mean], ratchet))
def read_seats(seats_file: pathlib.Path, paired: bool) -> tuple[str, Report] | Failure:
@@ -225,16 +244,27 @@ def read_seats(seats_file: pathlib.Path, paired: bool) -> tuple[str, Report] | F
return named.title, given
def append(entry: str) -> Failure | None:
def history_text() -> tuple[pathlib.Path, str] | Failure:
"""The checkout's history and its text, empty when the file is missing."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
history = pathlib.Path(checkout, HISTORY)
try:
text = history.read_text(encoding="utf-8") if history.exists() else ""
history.parent.mkdir(parents=True, exist_ok=True)
return history, history.read_text(encoding="utf-8") if history.exists() else ""
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {history}: {failure}")
def append(entry: str) -> Failure | None:
read = history_text()
if isinstance(read, Failure):
return read
history, text = read
try:
history.parent.mkdir(parents=True, exist_ok=True)
except OSError as failure:
return Failure(f"could not create {history.parent}: {failure}")
if unwritten := replace_file(history, appended(text, entry).encode()):
return Failure(f"could not write {history}: {unwritten.reason}")
print(f"Appended to {history}.")
+37 -12
View File
@@ -1,7 +1,7 @@
import decimal
import re
from larv_review.seats import DIMENSIONS, FIRST_SEATS, Seat, SPLIT_SEATS
from larv_review.seats import DIMENSIONS, FIRST_SEATS, Seat, SPLIT_SEATS, vote_line
from larv_review.history import entries, Entry, HistoryChange, PLACE
from failure import Failure
from rank_todo import Item, ITEMS_HEADER, parse_row, table_rows
@@ -13,18 +13,26 @@ ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):")
VOTES = ("better", "same", "worse")
def votes(seats: list[Seat]) -> dict[str, str] | Failure:
"""Each seat's vote as it counts: a better or worse vote naming no `path:line` place counts as same."""
def votes(seats: list[Seat]) -> dict[str, list[str]] | Failure:
"""Each seat's five votes as written, in DIMENSIONS' order."""
names = [seat.name for seat in seats]
if names[:2] != list(FIRST_SEATS) or names[2:] not in ([], list(SPLIT_SEATS)):
return Failure(f"a paired ruling's seats are {' and '.join(FIRST_SEATS)}, then, when their votes differ, {' and '.join(SPLIT_SEATS)}, in that order; the file has {', '.join(names) or 'none'}")
if unknown := [seat for seat in seats if seat.said not in VOTES]:
return Failure(f"{unknown[0].name}'s vote {unknown[0].said!r} is not better, same or worse")
counted = {seat.name: seat.said if names_a_place(seat) else "same" for seat in seats}
first, second, *more = counted.values()
return Failure(f"a paired ruling's seats are {' and '.join(FIRST_SEATS)}, then, when their overall votes differ, {' and '.join(SPLIT_SEATS)}, in that order; the file has {', '.join(names) or 'none'}")
said = {}
for seat in seats:
line = vote_line(seat.name, seat.said)
if isinstance(line, Failure):
return line
said[seat.name] = line
first, second, *more = (each[-1] for each in counted(seats, said).values())
if more and first == second:
return Failure(f"{' and '.join(FIRST_SEATS)} both count as {first}, which settles the ruling; drop the other seats' lines")
return counted
return Failure(f"{' and '.join(FIRST_SEATS)} both count as {first} overall, which settles the ruling; drop the other seats' lines")
return said
def counted(seats: list[Seat], said: dict[str, list[str]]) -> dict[str, list[str]]:
"""Each seat's votes as they count: a seat naming no `path:line` place counts as same on each."""
return {seat.name: said[seat.name] if names_a_place(seat) else ["same"] * len(DIMENSIONS) for seat in seats}
def names_a_place(seat: Seat) -> bool:
@@ -41,6 +49,21 @@ def ruling(counted: list[str]) -> str | None:
return "same"
def fallen(counted: dict[str, list[str]]) -> list[str]:
"""Each dimension, Overall aside, that the seats rule worse by a lead of two."""
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if ruling(list(column)) == "worse"]
def pending_fall(history: str) -> list[str]:
"""The dimensions that fell in the history's newest entry, when it is a better paired ruling, so no scoring run has followed it. Of entries with one time, the later in the file is newer."""
newest = max(enumerate(entries(history)), key=lambda indexed: (indexed[1].time, indexed[0]), default=None)
return newest[1].fell if newest and newest[1].ruling == "better" else []
def fell_under_floor(fell: list[str], found: list[decimal.Decimal]) -> list[str]:
return [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if name in fell and value < FLOOR]
def means(lines: list[list[str]]) -> list[decimal.Decimal]:
"""The mean of each score across the lines, in DIMENSIONS' order, rounded half up to two decimals."""
return [mean([decimal.Decimal(score) for score in column]) for column in zip(*lines)]
@@ -102,13 +125,15 @@ def plan_items(todo: str) -> list[Item] | Failure:
return items
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
"""Why the chunk takes a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag. Work has landed towards an emptied release when the chunk ships one of its items, or when it is the next release, the earliest before names, and landed holds."""
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool, fell: list[str]) -> list[str] | Failure:
"""Why the chunk takes a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, landed whether the default branch holds a commit after its newest release tag, and fell what pending_fall finds in the chunk's history. Work has landed towards an emptied release when the chunk ships one of its items, or when it is the next release, the earliest before names, and landed holds."""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
if fell:
reasons.append(f"has a better paired ruling in which {' and '.join(fell)} fell")
next_release = min((item.release for item in before), default=None)
for release in emptied(before, after, shipped):
if any(item.release == release for item in shipped) or (release == next_release and landed):
+8
View File
@@ -17,6 +17,7 @@ FIRST_SEATS = (MID, ARCHITECT)
SCORING_SEATS = (JUNIOR, MID, SENIOR, ARCHITECT)
SEAT_LINE = re.compile(rf"^({'|'.join(SCORING_SEATS)}): (.+)$")
SPLIT_SEATS = (SENIOR, JUNIOR)
VOTE_LINE = re.compile(", ".join(rf"{name} (better|same|worse)" for name in DIMENSIONS))
@dataclasses.dataclass(frozen=True)
@@ -81,3 +82,10 @@ def score_line(name: str, said: str) -> list[str] | Failure:
if values[-1] > min(values[:-1]) + 1:
return Failure(f"{name}'s overall exceeds the lowest dimension by more than 1; ask the seat to rescore: {said}")
return list(written.groups())
def vote_line(name: str, said: str) -> list[str] | Failure:
"""A paired seat's five votes, in DIMENSIONS' order."""
if not (written := VOTE_LINE.fullmatch(said)):
return Failure(f"{name}'s line is not {', '.join(vote + ' <better, same or worse>' for vote in DIMENSIONS)}: {said}")
return list(written.groups())
+75 -34
View File
@@ -10,7 +10,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from larv_review.history import appended, entries, HistoryChange, Place, ruling_entry
from larv_review.rules import base, first_ruling, means, measured_commit, plan_items, round_counts, ruling, scoring_run_due, under_floor, votes
from larv_review.rules import base, counted, fallen, fell_under_floor, first_ruling, means, measured_commit, pending_fall, plan_items, round_counts, ruling, scoring_run_due, under_floor, votes
from failure import Failure
from test_loop_claude_black_box import host_env
from rank_todo import Item
@@ -24,6 +24,10 @@ def paired(time: str, verdict: str, commit: str = "bbbbbbb") -> str:
return f"## {time}, PR #2 at {commit}, against aaaaaaa\n\nRuling: {verdict}\n"
def said(overall: str, locality: str = "same") -> str:
return f"Navigation same, Locality {locality}, Shape same, Self-sufficiency same, Overall {overall}"
def item(id_: int, release: tuple[int, int, int], tagline: str = "**Do it.**") -> Item:
return Item(id_, release, "", tagline, {"R": 1, "S": 1, "A": 1, "G": 1}, ("1",))
@@ -34,28 +38,38 @@ def todo(*rows: str) -> str:
class Seats(unittest.TestCase):
def test_reads_each_seats_line_and_places_and_joins_a_wrapped_place(self):
given = report("PR #2 at abc1234, against def5678\n\nMid A: worse\n\n- `a.py:1` `f`: hard\n to read\n\nInherited architect: same\n")
given = report(f"PR #2 at abc1234, against def5678\n\nMid A: {said('worse')}\n\n- `a.py:1` `f`: hard\n to read\n\nInherited architect: {said('same')}\n")
self.assertEqual(given.read, "PR #2 at abc1234, against def5678")
self.assertEqual(given.seats, [Seat("Mid A", "worse", ["`a.py:1` `f`: hard to read"]), Seat("Inherited architect", "same", [])])
self.assertEqual(given.seats, [Seat("Mid A", said("worse"), ["`a.py:1` `f`: hard to read"]), Seat("Inherited architect", said("same"), [])])
def test_refuses_a_line_that_is_neither_a_seat_nor_a_place_and_a_seat_twice(self):
self.assertIsInstance(report("PR #2 at abc1234\n\nMid A worse\n"), Failure)
self.assertIsInstance(report("PR #2 at abc1234\n\nMid A: worse\n\nMid A: same\n"), Failure)
self.assertIsInstance(report(""), Failure)
def test_counts_a_vote_naming_no_place_as_same(self):
seats = [Seat("Mid A", "worse", ["the whole skill"]), Seat("Inherited architect", "better", ["`a.py:3` `f`: shorter"])]
self.assertEqual(votes(seats), {"Mid A": "same", "Inherited architect": "better"})
def test_counts_each_vote_of_a_seat_naming_no_place_as_same(self):
seats = [Seat("Mid A", said("worse", "worse"), ["the whole skill"]), Seat("Inherited architect", said("better"), ["`a.py:3` `f`: shorter"])]
written = votes(seats)
self.assertEqual(written, {"Mid A": ["same", "worse", "same", "same", "worse"], "Inherited architect": ["same", "same", "same", "same", "better"]})
self.assertEqual(counted(seats, written)["Mid A"], ["same"] * 5)
def test_refuses_two_more_seats_once_the_first_two_agree(self):
seats = [Seat(name, vote, ["`f.py:1` `f`: x"]) for name, vote in (("Mid A", "worse"), ("Inherited architect", "worse"), ("Maintainability senior", "better"), ("Junior A", "better"))]
def test_refuses_two_more_seats_once_the_first_two_agree_overall(self):
seats = [Seat(name, said(vote), ["`f.py:1` `f`: x"]) for name, vote in (("Mid A", "worse"), ("Inherited architect", "worse"), ("Maintainability senior", "better"), ("Junior A", "better"))]
self.assertIsInstance(votes(seats), Failure)
def test_takes_the_first_two_seats_then_the_other_two_in_order(self):
first = [Seat("Mid A", "same", []), Seat("Inherited architect", "same", [])]
def test_takes_the_first_two_seats_then_the_other_two_in_order_and_each_dimensions_vote(self):
first = [Seat("Mid A", said("same"), []), Seat("Inherited architect", said("same"), [])]
self.assertIsInstance(votes(first[::-1]), Failure)
self.assertIsInstance(votes(first + [Seat("Junior A", "same", [])]), Failure)
self.assertIsInstance(votes([first[0], Seat("Inherited architect", "worst", [])]), Failure)
self.assertIsInstance(votes(first + [Seat("Junior A", said("same"), [])]), Failure)
self.assertIsInstance(votes([first[0], Seat("Inherited architect", said("worst"), [])]), Failure)
self.assertIsInstance(votes([first[0], Seat("Inherited architect", "same", [])]), Failure)
def test_finds_each_dimension_ruled_worse_by_a_lead_of_two(self):
self.assertEqual(fallen({"Mid A": ["same", "worse", "worse", "same", "better"], "Inherited architect": ["same", "worse", "same", "same", "better"]}), ["Locality"])
four = {name: ["same", vote, "same", "same", "better"] for name, vote in (("Mid A", "worse"), ("Inherited architect", "same"), ("Maintainability senior", "worse"), ("Junior A", "worse"))}
self.assertEqual(fallen(four), ["Locality"])
self.assertEqual(fallen({**four, "Junior A": ["same", "better", "same", "same", "better"]}), [])
self.assertEqual(fell_under_floor(["Locality", "Shape"], [decimal.Decimal(score) for score in ("6", "6.5", "7", "6", "6")]), ["Locality 6.5"])
def test_rules_by_a_lead_of_two_and_splits_two_seats_that_differ(self):
cases = [
@@ -90,17 +104,27 @@ class Seats(unittest.TestCase):
class History(unittest.TestCase):
def test_reads_back_what_it_writes(self):
seats = [Seat("Mid A", "worse", ["`bin/a.py:3` `f`: hard, with `:9` `g`"]), Seat("Inherited architect", "worse", [])]
ruled = ruling_entry("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", seats, {"Mid A": "worse", "Inherited architect": "same"}, "worse")
written = {"Mid A": ["same", "worse", "same", "same", "better"], "Inherited architect": ["same", "worse", "same", "same", "worse"]}
ruled = ruling_entry("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", seats, written, {**written, "Inherited architect": ["same"] * 5}, "better", ["Locality"])
history = appended(appended("", SCORING), ruled)
self.assertTrue(history.startswith("# Comprehension history\n\n## 2026-10-08T12:00:00Z"))
self.assertNotIn("Inherited architect, decided by:", history)
self.assertIn("| Inherited architect | same (voted worse, naming no place) |", history)
self.assertIn("| Inherited architect | same | same (voted worse, naming no place) | same | same | same (voted worse, naming no place) |", history)
found = entries(history)
self.assertEqual([(entry.commit, entry.against, entry.ruling) for entry in found], [("aaaaaaa", None, None), ("bbbbbbb", "aaaaaaa", "worse")])
self.assertEqual([(entry.commit, entry.against, entry.ruling, entry.fell) for entry in found], [("aaaaaaa", None, None, []), ("bbbbbbb", "aaaaaaa", "better", ["Locality"])])
self.assertEqual(found[1].places, [Place("bin/a.py", "f"), Place("bin/a.py", "g")])
self.assertEqual([entry.paired for entry in found], [False, True])
class PendingFall(unittest.TestCase):
def test_finds_a_fall_in_the_newest_entry_when_it_is_a_better_ruling(self):
fell = paired("2026-10-08T13:00:00Z", "better\n\nFell: Locality, Shape")
self.assertEqual(pending_fall(SCORING + fell), ["Locality", "Shape"])
self.assertEqual(pending_fall(fell + SCORING.replace("12:00:00Z", "14:00:00Z")), [])
self.assertEqual(pending_fall(SCORING + paired("2026-10-08T13:00:00Z", "better")), [])
self.assertEqual(pending_fall(""), [])
class Base(unittest.TestCase):
def test_takes_the_newest_change_adding_a_scoring_run_or_ending_on_a_better_ruling(self):
older = HistoryChange("old", "", SCORING)
@@ -139,27 +163,31 @@ class ScoringRunDue(unittest.TestCase):
def test_is_due_for_an_item_naming_the_floor(self):
plan = [item(1, (0, 1, 0), "**Lift the project to the Comprehension floor.**"), item(2, (0, 1, 0))]
self.assertEqual(scoring_run_due(plan, plan, [1], True), ["ships item 1, which names the comprehension floor"])
self.assertIsInstance(scoring_run_due(plan, plan, [9], True), Failure)
self.assertEqual(scoring_run_due(plan, plan, [1], True, []), ["ships item 1, which names the comprehension floor"])
self.assertIsInstance(scoring_run_due(plan, plan, [9], True, []), Failure)
def test_is_due_when_the_chunk_ships_a_releases_last_item(self):
plan = [item(1, (0, 1, 0)), item(2, (0, 2, 0))]
self.assertEqual(scoring_run_due(plan, plan, [1], False), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(plan, plan + [item(3, (0, 1, 0))], [1], True), [])
self.assertEqual(scoring_run_due(plan, plan, [1], False, []), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(plan, plan + [item(3, (0, 1, 0))], [1], True, []), [])
def test_is_due_when_the_chunk_removes_the_next_releases_last_item_after_work_landed(self):
before = [item(1, (0, 1, 0)), item(2, (0, 1, 0)), item(3, (0, 2, 0))]
after = [item(2, (0, 1, 0)), item(3, (0, 2, 0))]
self.assertEqual(scoring_run_due(before, after, [2], True), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, after, [], True), [])
self.assertEqual(scoring_run_due(before, after, [2], True, []), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, after, [], True, []), [])
emptied = [item(3, (0, 2, 0))]
self.assertEqual(scoring_run_due(before, emptied, [2], False), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, emptied, [], True), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, emptied, [], False), [])
self.assertEqual(scoring_run_due(before, emptied, [2], False, []), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, emptied, [], True, []), ["leaves release 0.1.0 with no plan item"])
self.assertEqual(scoring_run_due(before, emptied, [], False, []), [])
def test_is_due_when_a_better_ruling_hides_a_fall(self):
plan = [item(1, (0, 1, 0)), item(2, (0, 1, 0))]
self.assertEqual(scoring_run_due(plan, plan, [], True, ["Locality"]), ["has a better paired ruling in which Locality fell"])
def test_is_not_due_when_a_later_release_loses_its_last_item_unshipped(self):
before = [item(1, (0, 1, 0)), item(3, (0, 2, 0))]
self.assertEqual(scoring_run_due(before, before[:1], [], True), [])
self.assertEqual(scoring_run_due(before, before[:1], [], True, []), [])
class LarvReviewTool(unittest.TestCase):
@@ -230,25 +258,38 @@ class LarvReviewTool(unittest.TestCase):
lines = """\
PR #2 at {start}, against {start}
Mid A: worse
Mid A: {worse}
- `bin/a.py:3` `f`: three branches
Inherited architect: same
Inherited architect: {same}
"""
lines = lines.replace("{start}", self.start)
lines = lines.replace("{start}", self.start).replace("{worse}", said("worse")).replace("{same}", said("same"))
split = self.run_tool("record-ruling", self.seats_file(lines))
self.assertEqual(split.returncode, 0, split.stderr)
self.assertTrue(split.stdout.startswith("Split: Mid A worse, Inherited architect same."))
history = self.checkout / "docs" / "comprehension-history.md"
self.assertFalse(history.exists())
lines += "\n\t\t\tMaintainability senior: worse\n\n\t\t\t- `bin/a.py:3` `f`: hard\n\n\t\t\tJunior A: worse\n"
lines += f"\n\t\t\tMaintainability senior: {said('worse')}\n\n\t\t\t- `bin/a.py:3` `f`: hard\n\n\t\t\tJunior A: {said('worse')}\n"
ran = self.run_tool("record-ruling", self.seats_file(lines))
self.assertEqual(ran.stdout.splitlines()[0], "Ruling: same. Votes as counted: Mid A worse, Inherited architect same, Maintainability senior worse, Junior A same (voted worse, naming no place).")
self.assertEqual(ran.stdout.splitlines()[0], "Ruling: same. Overall votes as counted: Mid A worse, Inherited architect same, Maintainability senior worse, Junior A same (voted worse, naming no place).")
recorded = entries(history.read_text())
self.assertEqual([(entry.commit, entry.against, entry.ruling) for entry in recorded], [(self.start, self.start, "same")])
self.assertEqual(recorded[0].places, [Place("bin/a.py", "f")] * 2)
self.assertIn("| Junior A | same (voted worse, naming no place) |", history.read_text())
self.assertIn("| Junior A | same | same | same | same | same (voted worse, naming no place) |", history.read_text())
def test_holds_a_better_ruling_that_hides_a_fall_until_a_scoring_run_decides_it(self):
place = "\n\n- `bin/a.py:3` `f`: shorter\n\n"
ran = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}, against {self.start}\n\nMid A: {said('better', 'worse')}{place}Inherited architect: {said('better', 'worse')}{place}"))
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertIn("Fell: Locality. A scoring run is due", ran.stdout)
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, "Scoring run due: the chunk has a better paired ruling in which Locality fell.\n")
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 7, Overall 7"
seats = "".join(f"\n{seat}: {line}\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
scored = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats))
self.assertIn("Ratchet: the paired ruling counts as worse, as Locality 6.50 fell under the floor of 7.0.", scored.stdout)
self.assertIn("\n\nRatchet: the paired ruling counts as worse", (self.checkout / "docs" / "comprehension-history.md").read_text())
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, "No scoring run due.\n")
def test_records_a_scoring_run_with_its_means(self):
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
@@ -262,10 +303,10 @@ class LarvReviewTool(unittest.TestCase):
self.assertEqual([entry.places for entry in entries(text)], [[Place("bin/a.py", "f")] * 4])
def test_refuses_a_ruling_file_that_names_no_base_or_a_commit_the_checkout_lacks(self):
ran = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}\n\nMid A: same\n\nInherited architect: same\n"))
ran = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}\n\nMid A: {said('same')}\n\nInherited architect: {said('same')}\n"))
self.assertEqual(ran.returncode, 1)
self.assertIn("against <base>", ran.stderr)
lacking = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}, against abcdef1\n\nMid A: same\n\nInherited architect: same\n"))
lacking = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}, against abcdef1\n\nMid A: {said('same')}\n\nInherited architect: {said('same')}\n"))
self.assertEqual(lacking.returncode, 1)
self.assertIn("abcdef1, which is no commit", lacking.stderr)
@@ -286,7 +327,7 @@ class LarvReviewTool(unittest.TestCase):
def test_refuses_a_dated_first_line_except_for_a_mean_alone_and_a_mean_without_one(self):
mean = "Mean: Navigation 7, Locality 7, Shape 7, Self-sufficiency 7, Overall 7\n"
self.assertEqual(self.run_tool("record-scoring", self.seats_file(f"PR #1 at {self.start}\n\n{mean}")).returncode, 1)
ruled = self.run_tool("record-ruling", self.seats_file(f"2026-10-08, PR #2 at {self.start}, against {self.start}\n\nMid A: same\n\nInherited architect: same\n"))
ruled = self.run_tool("record-ruling", self.seats_file(f"2026-10-08, PR #2 at {self.start}, against {self.start}\n\nMid A: {said('same')}\n\nInherited architect: {said('same')}\n"))
self.assertEqual(ruled.returncode, 1)
self.assertIn("only a seats file holding a Mean line alone", ruled.stderr)
+2 -2
View File
@@ -65,9 +65,9 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
2026-10-08, lilleman, replacing the 2026-09-30 rule that every fix, a nit's included, re-enters review. Serves goal 1.4. An audit of the four costliest archived chunks (2026-10-08) found nearly every finding that mattered in a pass's first round; later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes keep changing nothing a user would notice.
## review: a paired ruling against the last measured commit decides the ratchet; the four-seat scoring run checks the floor when a release's items run out or an item names the floor
## review: a paired ruling against the last measured commit decides the ratchet, and counts as worse when a dimension it rules fallen is under the floor; the four-seat scoring run checks the floor when a release's items run out, an item names the floor or a better ruling names a fallen dimension
2026-10-08, lilleman. Serves goals 1.1 and 1.4. Two independent four-seat runs moved up to 0.4 on near-identical code, so a fall within that noise forced rounds that changed nothing (PR #224). A seat reading both versions judges the difference on the same material, so the noise the versions share cancels. No conventional significance level lets a sign test settle with four votes, so the human chose a lead of two votes over every other ruling, same included. Two seats that agree settle it, so a PR that leaves the reading as it was costs two seats. A third seat alone can never settle it, so the third and fourth launch together. The base is the default branch's newest merged change that adds a scoring run to the history, or whose last added paired ruling is better. The human chose the last measured commit on the architect's finding that with a merge base as the base, chunks without an architecture pass, and a suspended restructure's steps, could each lower comprehension unseen. A ruling of same leaves the base where it was, or a run of small falls each ruled same would add up unseen. A scoring run moves it too: otherwise a project never ruled better would compare against its first ruling forever, and a release's floor check is where the human sees the absolute level. Reading the default branch one merged change at a time keeps the rule working for merge commits and squash merges alike: a branch's own early ruling of better never counts, only the merged change's last ruling. The scoring run also runs on a chunk shipping an item that names the floor, so the chunk that lifts a project to the floor can prove it. A chunk that removes a release's last item without shipping it runs it too, so no release ends unmeasured. Valid while seats reading both versions agree more often than two absolute runs do.
2026-10-08, lilleman. Serves goals 1.1 and 1.4. Two independent four-seat runs moved up to 0.4 on near-identical code, so a fall within that noise forced rounds that changed nothing (PR #224). A seat reading both versions judges the difference on the same material, so the noise the versions share cancels. No conventional significance level lets a sign test settle with four votes, so the human chose a lead of two votes over every other ruling, same included. Two seats that agree settle it, so a PR that leaves the reading as it was costs two seats. A third seat alone can never settle it, so the third and fourth launch together. The base is the default branch's newest merged change that adds a scoring run to the history, or whose last added paired ruling is better. The human chose the last measured commit on the architect's finding that with a merge base as the base, chunks without an architecture pass, and a suspended restructure's steps, could each lower comprehension unseen. A ruling of same leaves the base where it was, or a run of small falls each ruled same would add up unseen. A scoring run moves it too: otherwise a project never ruled better would compare against its first ruling forever, and a release's floor check is where the human sees the absolute level. Reading the default branch one merged change at a time keeps the rule working for merge commits and squash merges alike: a branch's own early ruling of better never counts, only the merged change's last ruling. The scoring run also runs on a chunk shipping an item that names the floor, so the chunk that lifts a project to the floor can prove it. A chunk that removes a release's last item without shipping it runs it too, so no release ends unmeasured. 2026-10-09, lilleman: each seat also rules each dimension, and a dimension falls by the same lead of two. A better ruling naming a fallen dimension calls the scoring run, and counts as worse when that dimension's mean is under the floor, so between releases a project cannot trade one dimension below the floor for gains in the other three. Valid while seats reading both versions agree more often than two absolute runs do.
## review: a project decision may suspend the comprehension check for the plan items it names
+3 -3
View File
@@ -84,11 +84,11 @@ Your harness pre-loads these files into your context before your first tool call
No running anything, no builds, no tests, no edits of any kind.
Rule whether the project as a whole reads better, the same or worse at {head} than at {base}, judged on these four dimensions:
Rule whether the project reads better, the same or worse at {head} than at {base} on each of these four dimensions, and as a whole:
{dimensions}
Same means a reader of either version would find it as easy to understand and change. Do not manufacture a difference.
Report, in this shape:
1. Your ruling: better, same or worse.
2. The places that decided it, most decisive first. Each: path:line and the unit's name, in the version the place is in, and one or two sentences on what reads easier or harder there than in the other version. A better or worse ruling with no named place is discarded.
1. Your rulings, each better, same or worse, in this shape: Navigation <ruling>, Locality <ruling>, Shape <ruling>, Self-sufficiency <ruling>, Overall <ruling>
2. The places that decided them, most decisive first. Each: path:line and the unit's name, in the version the place is in, the dimension it moves, and one or two sentences on what reads easier or harder there than in the other version. Rulings with no named place are discarded.
```
+3 -3
View File
@@ -95,18 +95,18 @@ Then the comprehension check: simulated readers, launched per `~/.claude/skills/
- **Base.** The last measured commit, which `~/.claude/bin/larv-review ruling-base` prints.
- **Head.** Before each ruling, merge `origin/<default branch>` into this branch unless the head already holds it, so the seats see only this branch's change on top of the base.
- **Seats.** Mid A and the inherited architect. Write what they read and their replies into a seats file in your scratchpad, in the shape `~/.claude/bin/larv-review --help` shows, and run `~/.claude/bin/larv-review record-ruling <file>`. On `Split:`, launch the maintainability senior and Junior A together, add their lines, and run it again.
- **Ruling.** The `Ruling:` line `record-ruling` prints and appends to the history.
- **Ruling.** The `Ruling:` line `record-ruling` prints and appends to the history. A better ruling may carry a `Fell:` line naming dimensions the seats ruled worse; a scoring run then decides whether it stands.
**Scoring run**, when `~/.claude/bin/larv-review needs-scoring-run <the IDs of the items the chunk ships, if any>` prints `Scoring run due:`. Run `needs-scoring-run` before removing the chunk's items from `todo.md`. Junior A, Mid A, the maintainability senior and the inherited architect each score the project at the head from 1 to 10 overall and on four dimensions: Navigation, Locality, Shape and Self-sufficiency. Write their lines into a seats file and run `~/.claude/bin/larv-review record-scoring <file>`, which appends the run to the history and prints the four seats' means.
The chunk does not merge while either holds:
- **Ratchet.** The paired ruling is worse.
- **Ratchet.** The paired ruling is worse, or `record-scoring` prints `Ratchet:` because a dimension that fell is under the floor.
- **Floor.** `record-scoring` names a mean under the floor of 7.0.
Each is a 🔴 of the comprehension check. It is never filed, and stands until a later run clears it or, for the floor, a decision to ship answers it.
Fix a worse ruling at the places the seats voting worse name, and the floor at the places the seats name under the lowest dimension. Under the floor alone, the user may decide to ship instead. A decision that lets every chunk ship under the floor, held in the project's `docs/decisions.md` before the chunk began, counts as theirs while it stands. A decision to ship listed in the `comprehension floor:` entry covers only the chunk it names. No user reachable → fix. A worse ruling never ships: after three rounds, "Round cap" → "Cap" asks the user for another round.
Fix a worse ruling at the places the seats voting worse name, a fallen dimension at the places they name under it, and the floor at the places the seats name under the lowest dimension. Under the floor alone, the user may decide to ship instead. A decision that lets every chunk ship under the floor, held in the project's `docs/decisions.md` before the chunk began, counts as theirs while it stands. A decision to ship listed in the `comprehension floor:` entry covers only the chunk it names. No user reachable → fix. A worse ruling never ships: after three rounds, "Round cap" → "Cap" asks the user for another round.
Two files of the project keep the records: