Merge pull request 'Rule each dimension in the paired ruling, and count a ruling as worse when a dimension that falls scores under the floor' (#232) from paired-dimensions into main

This commit was merged in pull request #232.
This commit is contained in:
2026-10-09 13:16:06 +02:00
23 changed files with 876 additions and 445 deletions
+1
View File
@@ -17,6 +17,7 @@
!agents/
!agents/*.md
!bin/
!bin/comprehension_history.py
!bin/failure.py
!bin/file_writes.py
!bin/larv-review
+1 -1
View File
@@ -12,7 +12,7 @@ Decisions live in `docs/decisions.md`:
- loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
- review: a pass gets at most three rounds per chunk; instead of a fourth, the run asks the human
- review: a fix for a 🔴 or 🟡 reruns the pass that found it and each pass whose trigger names fixes like it; a fix for a 🟢, a ❓ or a prose finding reruns none
- review: a paired ruling against the last measured commit decides the ratchet; the four-seat scoring run checks the floor when a release's items run out or an item names the floor
- review: a paired ruling against the last measured commit decides the ratchet, and counts as worse when a dimension it rules fallen is under the floor; the four-seat scoring run checks the floor when a release's items run out or an item names the floor, and also runs when a ruling names a fallen dimension
- review: a project decision may suspend the comprehension check for the plan items it names
- review: a pass skips only on a precedent: a decision, or a plan item crediting the pass, that the repo held before the chunk began
- loop-claude: the archive keeps the raw transcript, and a failed archive stops the loop
+2 -2
View File
@@ -3,9 +3,9 @@
## Unreleased
- The stability reviewer keeps its scratch work in a directory it creates for the run, deletes that directory afterwards, and deletes nothing else.
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling` and `record-scoring`, which read a seats file and append to `docs/comprehension-history.md`). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release whose last plan item moves to another release is due a scoring run too, once work has landed towards it. A seats file must name commits the checkout holds. Scores a `comprehension floor:` decision entry still holds move into the history through `record-scoring`, as a `Mean:` line. When origin cannot be fetched, the tool reads it as last fetched and says so.
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling <votes file> [<scores file>]` and `record-scoring <scores file> [<item ID>...]`, which append to `docs/comprehension-history.md` and refuse a second entry on what a seats file's first line reads; `record-scoring` refuses a run the plan does not call, and prints `Floor:` naming each mean under the floor). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release whose last plan item moves to another release is due a scoring run too, once work has landed towards it. A seats file must name commits the checkout holds. When origin cannot be fetched, the tool reads it as last fetched and says so.
- The new `bin/metric-bench` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest: cognitive complexity, length, shared state and change spread, each scored by how well it ranks the named functions above the rest. No metric enters review yet.
- larv-review blocks a merge when simulated readers rule that the PR makes the project read worse than the last measured commit: the default branch's newest merged change that added a scoring run to `docs/comprehension-history.md`, or whose last ruling there is better. A ruling of same leaves that commit in place, so a worse ruling may name places earlier chunks wrote. In each architecture pass Mid A and the inherited architect vote better, same or worse; when they disagree, the maintainability senior and Junior A vote too. A ruling needs a lead of two or more votes, else it is same, and a vote that names no place counts as same. A worse ruling never ships: after three rounds larv-review asks you for another round, a no leaves the PR unmerged, and so does an unattended run. The four-seat scoring run checks the comprehension floor only when a chunk ships or removes a release's last plan item, or ships an item naming the floor; between those runs the floor blocks nothing. A score under the floor blocks the merge until fixed or until you decide to ship; a decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
- larv-review blocks a merge when simulated readers rule that the PR makes the project read worse than the last measured commit: the default branch's newest merged change that added a scoring run to `docs/comprehension-history.md`, or whose last ruling there is better. A ruling of same leaves that commit in place, so a worse ruling may name places earlier chunks wrote. In each architecture pass Mid A and the inherited architect vote better, same or worse on each dimension and overall: `Navigation same, Locality worse, Shape same, Self-sufficiency same, Overall better`. When their overall votes differ, the maintainability senior and Junior A vote too. A ruling needs a lead of two or more votes, else it is same. Each place a seat names opens with the dimensions it moves, such as `Locality, Shape: `, or the run fails, and a vote that no place opens with counts as same. A dimension falls when its worse votes lead by two. When a dimension falls in a better or same ruling, `record-ruling` asks for the four scoring seats and, given their scores file, counts the ruling as worse when that dimension's mean is under the floor. A worse ruling never ships: after three rounds larv-review asks you for another round, a no leaves the PR unmerged, and so does an unattended run. The four-seat scoring run checks the comprehension floor only when a chunk ships or removes a release's last plan item, or ships an item naming the floor; between those runs, a score under the floor blocks a merge only through a fallen dimension, and that block counts as a worse ruling. On those runs, a score under the floor blocks the merge until fixed or until you decide to ship; a decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
- larv-review reruns fewer passes after a fix. A fix reruns the pass whose 🔴 or 🟡 it answers, plus stability when the fix changes behaviour, security when it opens an untrusted-input boundary or touches authn/authz, secrets or crypto, and architecture when it carries an architectural decision. A rerun reviews only what changed since that pass last reviewed. A fix for a 🟢, a ❓ or any prose finding reruns no pass, and prose takes one round.
- larv-review runs one security pass on each change: the full scan alone when it runs, else `/security-review`.
- larv-review runs the new `bin/reading-cost`, which measures each chunk `loop-claude` finished from its archive: the new input tokens the worker and its subagents took in before the worker's first commit, and the files git tracks that they read most. It appends each figure the project's `docs/reading-cost.md` lacks. Once 10 chunks are measured, and the last 5 chunks' median is above the 5 before them, larv-review files one plan item naming the files read most, or refreshes it. The chunk that ships it runs `reading-cost --reset`, so the trend counts only chunks that finished after it.
+163
View File
@@ -0,0 +1,163 @@
import dataclasses
import decimal
import re
import time
from failure import Failure
from markdown_sections import sections
ARCHITECT = "Inherited architect"
DECIDED_BY = "decided by"
DIMENSIONS = ("Navigation", "Locality", "Shape", "Self-sufficiency", "Overall")
HARDEST_FIRST = "hardest first"
HEADING = re.compile(r"^([^,]+), (.+) at ([0-9a-f]{7,40})(?:, against ([0-9a-f]{7,40}))?$")
HISTORY = "docs/comprehension-history.md"
JUNIOR = "Junior A"
MEAN = "Mean"
MID = "Mid A"
# `path:line` or `:line`, then the unit's name where one follows.
PLACE = re.compile(r"`([^`\s]*):\d+`(?: `(\w+)`)?")
RESTRUCTURE_FIRST = "would restructure first"
RULING = re.compile(r"^Ruling: (better|same|worse)$", re.M)
RULED_PLACES = re.compile(rf"^.+, {DECIDED_BY}:$")
SCORED_PLACES = re.compile(rf"^.+, ({HARDEST_FIRST}|{RESTRUCTURE_FIRST}):$")
SENIOR = "Maintainability senior"
STAMP = "%Y-%m-%dT%H:%M:%SZ"
TITLE = "# Comprehension history\n"
@dataclasses.dataclass(frozen=True)
class Seat:
name: str
said: str
places: list[str]
@dataclasses.dataclass(frozen=True)
class Place:
path: str
name: str | None
@dataclasses.dataclass(frozen=True)
class Entry:
"""One paired ruling, which names the commit it was against and its ruling, or one scoring run, which names neither. A scoring run's places, or those of the scores a fall called into a ruling, are scored_places."""
heading: str
time: str
commit: str
against: str | None
ruling: str | None
ruled_places: list[Place]
scored_places: list[Place]
@property
def paired(self) -> bool:
return self.against is not None
@dataclasses.dataclass(frozen=True)
class Heading:
title: str
label: str
commit: str
against: str | None
@dataclasses.dataclass(frozen=True)
class Scoring:
"""The four scoring seats' lines, their scores as written and the means, in DIMENSIONS' order."""
seats: list[Seat]
scores: dict[str, list[str]]
means: list[decimal.Decimal]
def entries(history: str) -> list[Entry]:
"""Each paired ruling and scoring run in a comprehension history, in the order the file holds them; a section whose heading names no commit is left out."""
found = []
for heading, bodies in sections(history).items():
if named := HEADING.match(heading):
body = "\n".join(bodies)
ruling = RULING.search(body)
found.append(Entry(heading, named.group(1), named.group(3), named.group(4), ruling.group(1) if ruling else None, seat_places(body, RULED_PLACES), seat_places(body, SCORED_PLACES)))
return found
def recorded(history: str, read: str) -> bool:
"""Whether the history holds an entry on what a seats file's first line says the seats read."""
return any(entry.heading.partition(", ")[2] == read for entry in entries(history))
def seat_places(body: str, labels: re.Pattern[str]) -> list[Place]:
"""The places under each seat label labels matches."""
places = []
listing = False
for line in body.splitlines():
if labels.match(line):
listing = True
elif listing and line.startswith("- "):
places += bullet_places(line)
elif not line.startswith(" "):
listing = False
return places
def bullet_places(bullet: str) -> list[Place]:
"""A `:line` with no path is in the file named before it in the bullet."""
places = []
path = ""
for match in PLACE.finditer(bullet):
path = match.group(1) or path
if path:
places.append(Place(path, match.group(2)))
return places
def heading(at: float, read: str, paired: bool) -> Heading | Failure:
"""An entry's heading, matched by HEADING: when the seats finished, in UTC, and what they read."""
title = f"{time.strftime(STAMP, time.gmtime(at))}, {read}"
if (named := HEADING.match(title)) and bool(named.group(4)) == paired:
return Heading(title, named.group(2), named.group(3), named.group(4))
return Failure(f"the seats file's first line is not `<PR or branch> at <commit>{', against <base>' if paired else ''}`: {read}")
def appended(history: str, entry: str) -> str:
"""The history with entry added last; an empty history gets the file's title first."""
text = history.rstrip("\n") + "\n" if history.strip() else TITLE
return f"{text}\n{entry}"
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, notes: list[str], scoring: Scoring | None) -> str:
"""A paired ruling's entry, with any notes, such as why a fall decided it, and the scores that decided a fall; a seat's vote that counted as same for want of a place says so."""
votes = [[seat.name, *map(vote_text, DIMENSIONS, said[seat.name], counted[seat.name])] for seat in seats]
tables = [[["Seat", *DIMENSIONS], *votes]] + ([scores_table(scoring)] if scoring else [])
labels = [(f"{seat.name}, {DECIDED_BY}:", seat.places) for seat in seats] + (scoring_labels(scoring.seats) if scoring else [])
return entry_text(title, [f"Ruling: {ruled}", *notes], tables, labels)
def vote_text(dimension: str, said: str, counted: str) -> str:
if counted == said:
return counted
return f"same (voted {said}, naming no place)" if dimension == DIMENSIONS[-1] else f"same (voted {said}, no place naming {dimension})"
def scoring_entry(title: str, scoring: Scoring) -> str:
return entry_text(title, [], [scores_table(scoring)], scoring_labels(scoring.seats))
def scores_table(scoring: Scoring) -> list[list[str]]:
return [["Seat", *DIMENSIONS], *([seat, *values] for seat, values in scoring.scores.items()), [MEAN, *map(str, scoring.means)]]
def scoring_labels(seats: list[Seat]) -> list[tuple[str, list[str]]]:
return [(f"{seat.name}, {RESTRUCTURE_FIRST if seat.name == ARCHITECT else HARDEST_FIRST}:", seat.places) for seat in seats]
def entry_text(title: str, lines: list[str], tables: list[list[list[str]]], seats: list[tuple[str, list[str]]]) -> str:
"""The heading, any lines such as `Ruling: same`, each table, whose first row is its header, and under each seat's label its places, one bullet each."""
parts = [f"## {title}", *lines, *("\n".join([row_text(header), "|" + "---|" * len(header), *map(row_text, rows)]) for header, *rows in tables)]
parts += [f"{label}\n\n" + "\n".join(f"- {place}" for place in places) for label, places in seats if places]
return "\n\n".join(parts) + "\n"
def row_text(cells: list[str]) -> str:
return "| " + " | ".join(cells) + " |"
+37
View File
@@ -0,0 +1,37 @@
import dataclasses
from comprehension_history import entries, Entry
@dataclasses.dataclass(frozen=True)
class HistoryChange:
"""A merged change on the default branch that touched the history, and the history before and after it."""
commit: str
before: str
after: str
def measured_commit(changes: list[HistoryChange]) -> str | None:
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better."""
for change in changes:
held = {entry.heading for entry in entries(change.before)}
added = [entry for entry in entries(change.after) if entry.heading not in held]
rulings = sorted((entry for entry in added if entry.paired), key=lambda entry: entry.time)
ruled_better = bool(rulings) and rulings[-1].ruling == "better"
if ruled_better or any(not entry.paired for entry in added):
return change.commit
return None
def first_ruling(history: str) -> Entry | None:
rulings = sorted((entry for entry in entries(history) if entry.paired), key=lambda entry: entry.time)
return rulings[0] if rulings else None
def base(changes: list[HistoryChange], history: str) -> tuple[str, str] | None:
"""The commit a paired ruling reads against, and which fallback found it; None leaves the merge base of HEAD and the default branch. history is the default branch's."""
if found := measured_commit(changes):
return found, "the last measured commit"
if (first := first_ruling(history)) and first.against:
return first.against, "the base the history's first paired ruling names"
return None
+2 -3
View File
@@ -2,9 +2,8 @@ import re
import subprocess
from failure import Failure
from larv_review.history import HistoryChange
HISTORY = "docs/comprehension-history.md"
from comprehension_history import HISTORY
from larv_review.base import HistoryChange
def git(*args: str) -> str | Failure:
-137
View File
@@ -1,137 +0,0 @@
import dataclasses
import re
import time
from larv_review.seats import ARCHITECT, DIMENSIONS, MEAN, Seat
from failure import Failure
from markdown_sections import sections
DATED = re.compile(r"^[0-9]{4}-[0-9]{2}-[0-9]{2}(?:T[0-9]{2}:[0-9]{2}:[0-9]{2}Z)?, ")
DECIDED_BY = "decided by"
HARDEST_FIRST = "hardest first"
HEADING = re.compile(r"^([^,]+), (.+) at ([0-9a-f]{7,40})(?:, against ([0-9a-f]{7,40}))?$")
# `path:line` or `:line`, then the unit's name where one follows.
PLACE = re.compile(r"`([^`\s]*):\d+`(?: `(\w+)`)?")
RESTRUCTURE_FIRST = "would restructure first"
RULING = re.compile(r"^Ruling: (better|same|worse)$", re.M)
SEAT_PLACES = re.compile(rf"^.+, ({DECIDED_BY}|{HARDEST_FIRST}|{RESTRUCTURE_FIRST}):$")
STAMP = "%Y-%m-%dT%H:%M:%SZ"
TITLE = "# Comprehension history\n"
@dataclasses.dataclass(frozen=True)
class Place:
path: str
name: str | None
@dataclasses.dataclass(frozen=True)
class Entry:
"""One paired ruling, which names the commit it was against and its ruling, or one scoring run, which names neither."""
heading: str
time: str
commit: str
against: str | None
ruling: str | None
places: list[Place]
@property
def paired(self) -> bool:
return self.against is not None
@dataclasses.dataclass(frozen=True)
class Heading:
title: str
commit: str
against: str | None
@dataclasses.dataclass(frozen=True)
class HistoryChange:
"""A merged change on the default branch that touched the history, and the history before and after it."""
commit: str
before: str
after: str
def entries(history: str) -> list[Entry]:
"""Each paired ruling and scoring run in a comprehension history, in the order the file holds them; a section whose heading names no commit is left out."""
found = []
for heading, bodies in sections(history).items():
if named := HEADING.match(heading):
body = "\n".join(bodies)
ruling = RULING.search(body)
found.append(Entry(heading, named.group(1), named.group(3), named.group(4), ruling.group(1) if ruling else None, seat_places(body)))
return found
def seat_places(body: str) -> list[Place]:
places = []
listing = False
for line in body.splitlines():
if SEAT_PLACES.match(line):
listing = True
elif listing and line.startswith("- "):
places += bullet_places(line)
elif not line.startswith(" "):
listing = False
return places
def bullet_places(bullet: str) -> list[Place]:
"""A `:line` with no path is in the file named before it in the bullet."""
places = []
path = ""
for match in PLACE.finditer(bullet):
path = match.group(1) or path
if path:
places.append(Place(path, match.group(2)))
return places
def heading(at: float, read: str, paired: bool, moved: bool) -> Heading | Failure:
"""An entry's heading, matched by HEADING: when the seats finished, in UTC, and what they read. A run moved from a decision entry opens instead with the date or time that entry gives."""
if moved and not DATED.match(read):
return Failure(f"a seats file holding a {MEAN} line alone opens with YYYY-MM-DD or YYYY-MM-DDTHH:MM:SSZ, then `, `: {read}")
if DATED.match(read) and not moved:
return Failure(f"only a seats file holding a {MEAN} line alone opens with a date: {read}")
title = read if moved else f"{time.strftime(STAMP, time.gmtime(at))}, {read}"
if (named := HEADING.match(title)) and bool(named.group(4)) == paired:
return Heading(title, named.group(3), named.group(4))
return Failure(f"the seats file's first line is not `<PR or branch> at <commit>{', against <base>' if paired else ''}`: {read}")
def appended(history: str, entry: str) -> str:
"""The history with entry added last; an empty history gets the file's title first."""
text = history.rstrip("\n") + "\n" if history.strip() else TITLE
return f"{text}\n{entry}"
def ruling_entry(title: str, seats: list[Seat], counted: dict[str, str], ruled: str) -> str:
"""A paired ruling's entry; a seat's vote that counted as same for naming no place says so."""
votes = [[seat.name, vote_text(seat, counted[seat.name])] for seat in seats]
return entry_text(title, [f"Ruling: {ruled}"], [["Seat", "Vote"], *votes], [(f"{seat.name}, {DECIDED_BY}:", seat.places) for seat in seats])
def vote_text(seat: Seat, counted: str) -> str:
return counted if counted == seat.said else f"same (voted {seat.said}, naming no place)"
def scoring_entry(title: str, seats: list[Seat], scores: dict[str, list[str]], means: list[str]) -> str:
"""A scoring run's entry; a run moved from a decision entry has its means alone."""
rows = [[seat, *values] for seat, values in scores.items()] + [[MEAN, *means]]
labels = [(f"{seat.name}, {RESTRUCTURE_FIRST if seat.name == ARCHITECT else HARDEST_FIRST}:", seat.places) for seat in seats]
return entry_text(title, [], [["Seat", *DIMENSIONS], *rows], labels)
def entry_text(title: str, lines: list[str], table: list[list[str]], seats: list[tuple[str, list[str]]]) -> str:
"""The heading, any lines such as `Ruling: same`, a table whose first row is its header, and under each seat's label its places, one bullet each."""
header, *rows = table
parts = [f"## {title}", *lines, "\n".join([row_text(header), "|" + "---|" * len(header), *map(row_text, rows)])]
parts += [f"{label}\n\n" + "\n".join(f"- {place}" for place in places) for label, places in seats if places]
return "\n\n".join(parts) + "\n"
def row_text(cells: list[str]) -> str:
return "| " + " | ".join(cells) + " |"
+156 -80
View File
@@ -1,11 +1,11 @@
"""Hold larv-review's mechanical rules: the paired ruling's base and vote count, each pass's rounds, the scoring run's trigger, and the comprehension history's format.
usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <seats file> | record-scoring <seats file>
usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]
Run it inside a project checkout or one of its worktrees. ruling-base, round-count and
needs-scoring-run first fetch origin, when the checkout has that remote, and take origin's default
branch, or else the local main or master, as the default branch. When the fetch fails, round-count
reads origin as last fetched and says so on stderr; the other two exit 1. The history is
Run it inside a project checkout or one of its worktrees. Each command but record-ruling first
fetches origin, when the checkout has that remote, and takes origin's default branch, or else the
local main or master, as the default branch. When the fetch fails, round-count reads origin as
last fetched and says so on stderr; the other three exit 1. The history is
docs/comprehension-history.md in the checkout.
ruling-base prints the commit a paired ruling reads against, then which of these found it. The last
@@ -14,9 +14,9 @@ history, or whose last added paired ruling, by heading time, is better; a merge
one change against its first parent. With none, the base is the one the history's first paired
ruling names; with no ruling either, it is the merge base of HEAD and the default branch.
round-count prints each pass's count: the highest <n> among the titles opening
`Answer <pass> round <n>:` of commits the default branch does not hold. It marks each pass at the
cap of {cap} rounds, and each name in such a title that is not a pass.
round-count prints each pass's count: the highest <n> among the titles opening `Answer <pass> round
<n>:` of commits the default branch does not hold. It marks each pass at the cap of {cap} rounds,
and each name in such a title that is not a pass.
needs-scoring-run takes the IDs of the todo.md items the chunk ships, none when it ships none, and
prints why the chunk is due a scoring run, or `No scoring run due.` It is due when the chunk ships
@@ -26,39 +26,49 @@ landed when the chunk ships one of the release's items, or when the release is t
default branch's todo.md names and the default branch has a commit after its newest v* tag, or has
no v* tag.
record-ruling and record-scoring read a seats file: a first line naming what the seats read,
`<PR or branch> at <commit>`, ending `, against <base>` for a paired ruling, then each seat's
line, each followed by its places, one `- ` bullet each, as the history writes them. A seat's line
opens with its label, written exactly: `Junior A:`, `Mid A:`, `Maintainability senior:` or
`Inherited architect:`.
A seats file's first line names what the seats read, `<PR or branch> at <commit>`, ending
`, against <base>` in a votes file. Each seat's line follows, opening with its label written
exactly: `Junior A:`, `Mid A:`, `Maintainability senior:` or `Inherited architect:`. Its places
follow it, one `- ` bullet each, as the history writes them. A votes file:
PR #231 at abc1234, against def5678
Mid A: worse
Mid A: Navigation better, Locality worse, Shape same, Self-sufficiency same, Overall better
- `bin/x.py:12` `run`: three branches to hold at once
- Navigation, Overall: `README.md:40` "Tools": names the tool a symptom leads to
- Locality: `bin/x.py:12` `run`: three branches to hold at once
Inherited architect: worse
Inherited architect: Navigation better, Locality worse, Shape same, Self-sufficiency same, Overall better
- `bin/x.py:12` `run`: ...
- Navigation: `README.md:40` "Tools": ...
- Locality: `bin/x.py:12` `run`: ...
record-ruling takes Mid A's and the inherited architect's votes, better, same or worse; a better or
worse vote naming no place in backticks, `path:line`, counts as same, and the printed votes say so.
When the two counted votes differ, it appends nothing and prints `Split:`, saying to add the
maintainability senior's and Junior A's lines. Otherwise the ruling is the vote that leads each
other vote by two or more, else same; it prints `Ruling:` and appends the ruling.
A scores file holds Junior A's, Mid A's, the maintainability senior's and the inherited architect's
lines in that order, each `Navigation <n>, Locality <n>, Shape <n>, Self-sufficiency <n>, Overall
<n>`: each score from 1 to 10, and the overall at most 1 above the line's lowest dimension.
record-scoring takes Junior A's, Mid A's, the maintainability senior's and the inherited
architect's lines in that order, or a `Mean:` line alone for a run moved from a decision entry.
Each line is `Navigation <n>, Locality <n>, Shape <n>, Self-sufficiency <n>, Overall <n>`, each
score from 1 to 10 and the overall at most 1 above the line's lowest dimension. Its first line
names no `, against`. It appends the seats' scores and their means, rounded half up to two
decimals, and prints the means and each one under the floor of {floor}.
record-ruling takes Mid A's and the inherited architect's votes on each dimension and overall, each
better, same or worse. Each place opens with the names from the seat's line that it moves, joined by
`, ` and followed by `: `. A dimension's vote counts as same unless one of the seat's places opens
with that dimension and holds a `path:line` in backticks; the overall vote counts as same unless any
place holds one. The printed votes say so. When the two counted overall votes differ, it appends
nothing and prints `Split:`, saying to add the maintainability senior's and Junior A's lines.
Otherwise the ruling is the overall vote that leads each other vote by two or more, else same. A
dimension falls when its worse votes lead each other vote by two or more. When a better or same
ruling has a fallen dimension, it appends nothing and prints `Fell:`, saying to run the four
scoring seats on the same commit and run it again with their scores file. With that file, the
ruling counts as worse when a fallen dimension's mean is under the floor of {floor}, and stands
otherwise. It prints `Ruling:` and appends the ruling, with the scores where a fall called them.
record-scoring takes the scores file of a run needs-scoring-run calls, whose first line names no
`, against`, and the same item IDs, and exits 1 when they call no run: each scoring run moves the
ratchet's base. It appends the seats' scores and their means, rounded half up to two decimals,
prints the means and, when a mean is under the floor of {floor}, a line opening `Floor:` naming
each one, which blocks the merge.
Each record is appended to the history, created when missing, under a heading of the current UTC
time and the seats file's first line. A file holding a `Mean:` line alone opens with the date or
time its decision entry gives, and that line alone is the heading. Each commit the line names
must be in the checkout. Each run appends, so record a ruling or a run once.
time and the seats file's first line. Each commit that line names must be in the checkout. It exits 1
when the history already holds an entry on what that line reads.
Exit 1, saying why, on unknown arguments, an unknown item ID, a seats file it cannot read or that
breaks the shape above, or when git cannot start, finds no checkout or no default branch, cannot
@@ -70,15 +80,18 @@ import sys
import time
import typing
from larv_review.history import appended, heading, ruling_entry, scoring_entry, vote_text
from larv_review.rules import base, CAP, FLOOR, means, PASSES, plan_items, round_counts, ruling, scoring_run_due, under_floor, votes
from larv_review.git import commit_exists, default_branch, file_at, git, HISTORY, history_changes, landed_since_release, top
from larv_review.seats import DIMENSIONS, MEAN, Report, report, score_line, scores
from comprehension_history import appended, DIMENSIONS, heading, Heading, HISTORY, recorded, ruling_entry, Scoring, scoring_entry, vote_text
from larv_review.base import base
from larv_review.rounds import CAP, PASSES, round_counts
from larv_review.git import commit_exists, default_branch, file_at, git, history_changes, landed_since_release, top
from larv_review.ruling import counted, fallen, ruling, votes
from failure import Failure
from larv_review.scoring import fall_ruling, fall_step, FLOOR, means, plan_items, scoring_run_due, under_floor
from rank_todo import ITEMS_HEADER
from file_writes import replace_file
from larv_review.seats import scores, seats_file, SeatsFile
USAGE = "usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <seats file> | record-scoring <seats file>"
USAGE = "usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]"
def main(argv: list[str]) -> int:
@@ -92,12 +105,13 @@ def main(argv: list[str]) -> int:
done = with_default_branch(round_count, stale_ok=True)
elif command == "needs-scoring-run" and all(re.fullmatch(r"[0-9]+", id_) for id_ in given):
done = with_default_branch(lambda default: needs_scoring_run(default, [int(id_) for id_ in given]), stale_ok=False)
elif command == "record-ruling" and len(given) == 1:
done = record_ruling(pathlib.Path(given[0]))
elif command == "record-scoring" and len(given) == 1:
done = record_scoring(pathlib.Path(given[0]))
elif command == "needs-scoring-run":
print(f"needs-scoring-run takes item IDs, the numbers in todo.md's ID column: {' '.join(given)}", file=sys.stderr)
elif command == "record-ruling" and len(given) in (1, 2):
done = record_ruling(*map(pathlib.Path, given))
elif command == "record-scoring" and given and all(re.fullmatch(r"[0-9]+", id_) for id_ in given[1:]):
done = with_default_branch(lambda default: record_scoring(default, pathlib.Path(given[0]), [int(id_) for id_ in given[1:]]), stale_ok=False)
elif command in ("needs-scoring-run", "record-scoring") and given:
named = given[1:] if command == "record-scoring" else given
print(f"{command} takes item IDs, the numbers in todo.md's ID column, not {' '.join(id_ for id_ in named if not re.fullmatch(r'[0-9]+', id_))}; {USAGE}", file=sys.stderr)
return 1
elif not argv:
print(USAGE, file=sys.stderr)
@@ -148,6 +162,15 @@ def round_count(default: str) -> Failure | None:
def needs_scoring_run(default: str, ids: list[int]) -> Failure | None:
reasons = plan_reasons(default, ids)
if isinstance(reasons, Failure):
return reasons
print(f"Scoring run due: the chunk {' and '.join(reasons)}." if reasons else "No scoring run due.")
return None
def plan_reasons(default: str, ids: list[int]) -> list[str] | Failure:
"""Why the plan calls a scoring run, from the default branch's todo.md and the checkout's."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
@@ -168,73 +191,126 @@ def needs_scoring_run(default: str, ids: list[int]) -> Failure | None:
landed = landed_since_release(default)
if isinstance(landed, Failure):
return landed
reasons = scoring_run_due(planned, chunk, ids, landed)
return scoring_run_due(planned, chunk, ids, landed)
def record_ruling(votes_file: pathlib.Path, scores_file: pathlib.Path | None = None) -> Failure | None:
read = read_seats(votes_file, paired=True)
if isinstance(read, Failure):
return read
named, given = read
said = votes(given.seats)
if isinstance(said, Failure):
return said
count = counted(given.seats, said)
tally = ", ".join(f"{seat.name} {vote_text(DIMENSIONS[-1], said[seat.name][-1], count[seat.name][-1])}" for seat in given.seats)
ruled = ruling([each[-1] for each in count.values()])
if ruled is None:
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the votes file, and run this again.")
return None
fell = fallen(count) if ruled != "worse" else []
scoring = None
notes = []
match fall_step(fell, scores_file is not None):
case "ask":
print(f"Fell: {', '.join(fell)}, in a {ruled} ruling. Launch the four scoring seats on {named.label} at {named.commit}, write their lines to a scores file, and run this again with both files.")
return None
case "refuse":
return Failure(f"no dimension fell in this {ruled} ruling, so it takes no scores file")
if scores_file:
scored = read_scoring(scores_file, named)
if isinstance(scored, Failure):
return scored
scoring = scored
ruled, why = fall_ruling(ruled, fell, scoring.means)
notes = [why]
print(f"Ruling: {ruled}. {' '.join(notes + [f'Overall votes as counted: {tally}.'])}")
return append(given.read, ruling_entry(named.title, given.seats, said, count, ruled, notes, scoring))
def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> Failure | None:
read = read_seats(scores_file, paired=False)
if isinstance(read, Failure):
return read
named, given = read
scoring = scored_seats(given)
if isinstance(scoring, Failure):
return scoring
reasons = plan_reasons(default, ids)
if isinstance(reasons, Failure):
return reasons
print(f"Scoring run due: the chunk {' and '.join(reasons)}." if reasons else "No scoring run due.")
return None
if not reasons:
return Failure("no plan item or release calls this scoring run; name the item IDs the chunk ships, as needs-scoring-run took them, or record a fall's scores with record-ruling")
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, scoring.means))}.")
if under := under_floor(scoring.means):
print(f"Floor: {', '.join(under)} under the floor of {FLOOR} block the merge, as the chunk {' and '.join(reasons)}.")
return append(given.read, scoring_entry(named.title, scoring))
def record_ruling(seats_file: pathlib.Path) -> Failure | None:
read = read_seats(seats_file, paired=True)
def read_scoring(scores_file: pathlib.Path, votes: Heading) -> Scoring | Failure:
"""The scores a fall called, once their file names what the votes file read."""
read = read_seats(scores_file, paired=False, fresh=False)
if isinstance(read, Failure):
return read
title, given = read
counted = votes(given.seats)
if isinstance(counted, Failure):
return counted
tally = ", ".join(f"{seat.name} {vote_text(seat, counted[seat.name])}" for seat in given.seats)
ruled = ruling(list(counted.values()))
if ruled is None:
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the seats file, and run this again.")
return None
print(f"Ruling: {ruled}. Votes as counted: {tally}.")
return append(ruling_entry(title, given.seats, counted, ruled))
named, given = read
if (named.label, named.commit) != (votes.label, votes.commit):
return Failure(f"the scores file's first line is not `{votes.label} at {votes.commit}`, as the votes file reads: {given.read}")
return scored_seats(given)
def record_scoring(seats_file: pathlib.Path) -> Failure | None:
read = read_seats(seats_file, paired=False)
if isinstance(read, Failure):
return read
title, given = read
found = scores(given.seats) if given.moved_mean is None else {}
def scored_seats(given: SeatsFile) -> Scoring | Failure:
found = scores(given.seats)
if isinstance(found, Failure):
return found
moved = [] if given.moved_mean is None else score_line(MEAN, given.moved_mean)
if isinstance(moved, Failure):
return moved
mean = means(list(found.values()) or [moved])
under = under_floor(mean)
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, mean))}. {f'Under the floor of {FLOOR}: ' + ', '.join(under) if under else f'None under the floor of {FLOOR}'}.")
return append(scoring_entry(title, given.seats, found, [str(value) for value in mean]))
return Scoring(given.seats, found, means(list(found.values())))
def read_seats(seats_file: pathlib.Path, paired: bool) -> tuple[str, Report] | Failure:
"""The seats file, and the heading its record goes under, once each commit it names is in the checkout."""
def read_seats(path: pathlib.Path, paired: bool, fresh: bool = True) -> tuple[Heading, SeatsFile] | Failure:
"""The seats file, and the heading its record goes under, once each commit it names is in the checkout and, when fresh, before anything is printed, once the history holds no entry on what it reads."""
try:
given = report(seats_file.read_text(encoding="utf-8"))
given = seats_file(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {seats_file}: {failure}")
return Failure(f"could not read {path}: {failure}")
if isinstance(given, Failure):
return given
named = heading(time.time(), given.read, paired, given.moved_mean is not None)
named = heading(time.time(), given.read, paired)
if isinstance(named, Failure):
return named
if unknown := [commit for commit in (named.commit, named.against) if commit and not commit_exists(commit)]:
return Failure(f"the seats file names {', '.join(unknown)}, which is no commit in this checkout")
return named.title, given
if fresh:
history = history_file()
if isinstance(history, Failure):
return history
if recorded(history[1], given.read):
return Failure(f"{history[0]} already holds an entry on {given.read}; record each ruling or run once")
return named, given
def append(entry: str) -> Failure | None:
def history_file() -> tuple[pathlib.Path, str] | Failure:
"""The checkout's history and its text, empty when the file is missing."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
history = pathlib.Path(checkout, HISTORY)
try:
text = history.read_text(encoding="utf-8") if history.exists() else ""
history.parent.mkdir(parents=True, exist_ok=True)
return history, history.read_text(encoding="utf-8") if history.exists() else ""
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {history}: {failure}")
def append(read: str, entry: str) -> Failure | None:
"""Append entry unless the history already holds one on what the seats read."""
read_history = history_file()
if isinstance(read_history, Failure):
return read_history
history, text = read_history
if recorded(text, read):
return Failure(f"{history} already holds an entry on {read}; record each ruling or run once")
try:
history.parent.mkdir(parents=True, exist_ok=True)
except OSError as failure:
return Failure(f"could not create {history.parent}: {failure}")
if unwritten := replace_file(history, appended(text, entry).encode()):
return Failure(f"could not write {history}: {unwritten.reason}")
print(f"Appended to {history}.")
+14
View File
@@ -0,0 +1,14 @@
import re
CAP = 3
PASSES = ("architecture", "full scan", "security", "stability", "product-owner", "prose")
ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):")
def round_counts(titles: list[str]) -> dict[str, int]:
"""Each pass's highest round among the branch's `Answer <pass> round <n>:` commit titles; a pass with none has 0, and a title naming no known pass adds its name."""
counts = dict.fromkeys(PASSES, 0)
for title in titles:
if named := ROUND.match(title):
counts[named.group(1)] = max(counts.get(named.group(1), 0), int(named.group(2)))
return counts
-123
View File
@@ -1,123 +0,0 @@
import decimal
import re
from larv_review.seats import DIMENSIONS, FIRST_SEATS, Seat, SPLIT_SEATS
from larv_review.history import entries, Entry, HistoryChange, PLACE
from failure import Failure
from rank_todo import Item, ITEMS_HEADER, parse_row, table_rows
CAP = 3
FLOOR = decimal.Decimal("7.0")
PASSES = ("architecture", "full scan", "security", "stability", "product-owner", "prose")
ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):")
VOTES = ("better", "same", "worse")
def votes(seats: list[Seat]) -> dict[str, str] | Failure:
"""Each seat's vote as it counts: a better or worse vote naming no `path:line` place counts as same."""
names = [seat.name for seat in seats]
if names[:2] != list(FIRST_SEATS) or names[2:] not in ([], list(SPLIT_SEATS)):
return Failure(f"a paired ruling's seats are {' and '.join(FIRST_SEATS)}, then, when their votes differ, {' and '.join(SPLIT_SEATS)}, in that order; the file has {', '.join(names) or 'none'}")
if unknown := [seat for seat in seats if seat.said not in VOTES]:
return Failure(f"{unknown[0].name}'s vote {unknown[0].said!r} is not better, same or worse")
counted = {seat.name: seat.said if names_a_place(seat) else "same" for seat in seats}
first, second, *more = counted.values()
if more and first == second:
return Failure(f"{' and '.join(FIRST_SEATS)} both count as {first}, which settles the ruling; drop the other seats' lines")
return counted
def names_a_place(seat: Seat) -> bool:
return any(PLACE.search(place) for place in seat.places)
def ruling(counted: list[str]) -> str | None:
"""The vote that leads each other vote by two or more, else same; None when only the first two seats voted and they differ, so two more must vote."""
if len(counted) == 2 and counted[0] != counted[1]:
return None
for vote in VOTES:
if all(counted.count(vote) >= counted.count(other) + 2 for other in VOTES if other != vote):
return vote
return "same"
def means(lines: list[list[str]]) -> list[decimal.Decimal]:
"""The mean of each score across the lines, in DIMENSIONS' order, rounded half up to two decimals."""
return [mean([decimal.Decimal(score) for score in column]) for column in zip(*lines)]
def mean(values: list[decimal.Decimal]) -> decimal.Decimal:
return (sum(values, decimal.Decimal(0)) / len(values)).quantize(decimal.Decimal("0.01"), decimal.ROUND_HALF_UP)
def under_floor(found: list[decimal.Decimal]) -> list[str]:
return [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if value < FLOOR]
def round_counts(titles: list[str]) -> dict[str, int]:
"""Each pass's highest round among the branch's `Answer <pass> round <n>:` commit titles; a pass with none has 0, and a title naming no known pass adds its name."""
counts = dict.fromkeys(PASSES, 0)
for title in titles:
if named := ROUND.match(title):
counts[named.group(1)] = max(counts.get(named.group(1), 0), int(named.group(2)))
return counts
def measured_commit(changes: list[HistoryChange]) -> str | None:
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better."""
for change in changes:
held = {entry.heading for entry in entries(change.before)}
added = [entry for entry in entries(change.after) if entry.heading not in held]
rulings = sorted((entry for entry in added if entry.paired), key=lambda entry: entry.time)
if any(not entry.paired for entry in added) or rulings and rulings[-1].ruling == "better":
return change.commit
return None
def first_ruling(history: str) -> Entry | None:
rulings = sorted((entry for entry in entries(history) if entry.paired), key=lambda entry: entry.time)
return rulings[0] if rulings else None
def base(changes: list[HistoryChange], history: str) -> tuple[str, str] | None:
"""The commit a paired ruling reads against, and which fallback found it; None leaves the merge base of HEAD and the default branch. history is the default branch's."""
if found := measured_commit(changes):
return found, "the last measured commit"
if (first := first_ruling(history)) and first.against:
return first.against, "the base the history's first paired ruling names"
return None
def plan_items(todo: str) -> list[Item] | Failure:
"""Each row of a todo.md's items table, read as `rank_todo.py` reads it."""
table = table_rows(todo.splitlines(), ITEMS_HEADER)
if isinstance(table, Failure):
return table
items = []
for row in table.rows:
item = parse_row(row)
if isinstance(item, Failure):
return item
items.append(item)
return items
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
"""Why the chunk takes a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag. Work has landed towards an emptied release when the chunk ships one of its items, or when it is the next release, the earliest before names, and landed holds."""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
next_release = min((item.release for item in before), default=None)
for release in emptied(before, after, shipped):
if any(item.release == release for item in shipped) or (release == next_release and landed):
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
return reasons
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
named = {item.release for item in before + shipped}
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
return sorted(named - left)
+54
View File
@@ -0,0 +1,54 @@
from comprehension_history import DIMENSIONS, PLACE, Seat
from failure import Failure
from larv_review.seats import FIRST_SEATS, SPLIT_SEATS, vote_line
VOTES = ("better", "same", "worse")
def votes(seats: list[Seat]) -> dict[str, list[str]] | Failure:
"""Each seat's five votes as written, in DIMENSIONS' order."""
names = [seat.name for seat in seats]
if names[:2] != list(FIRST_SEATS) or names[2:] not in ([], list(SPLIT_SEATS)):
return Failure(f"a paired ruling's seats are {' and '.join(FIRST_SEATS)}, then, when their overall votes differ, {' and '.join(SPLIT_SEATS)}, in that order; the file has {', '.join(names) or 'none'}")
said = {}
for seat in seats:
line = vote_line(seat.name, seat.said)
if isinstance(line, Failure):
return line
if unnamed := [place for place in seat.places if not place_dimensions(place)]:
return Failure(f"{seat.name}'s place does not open with the dimensions it moves, each one of {', '.join(DIMENSIONS)}, then `: `: {unnamed[0]}")
said[seat.name] = line
first, second, *more = (each[-1] for each in counted(seats, said).values())
if more and first == second:
return Failure(f"{' and '.join(FIRST_SEATS)} both count as {first} overall, which settles the ruling; drop the other seats' lines")
return said
def counted(seats: list[Seat], said: dict[str, list[str]]) -> dict[str, list[str]]:
"""Each seat's votes as they count: a dimension's vote counts as same unless one of the seat's `path:line` places opens with that dimension, and the overall vote unless the seat names any such place."""
return {seat.name: [vote if names_a_place(seat, name) else "same" for name, vote in zip(DIMENSIONS, said[seat.name])] for seat in seats}
def names_a_place(seat: Seat, dimension: str) -> bool:
return any(PLACE.search(place) and (dimension == DIMENSIONS[-1] or dimension in place_dimensions(place)) for place in seat.places)
def place_dimensions(place: str) -> list[str]:
"""The dimensions a place opens with, as in `Locality, Shape: `path:line` ...`; empty when any name before the first `: ` is no dimension."""
named = place.partition(": ")[0].split(", ")
return named if all(name in DIMENSIONS for name in named) else []
def ruling(counted: list[str]) -> str | None:
"""The vote that leads each other vote by two or more, else same; None when only the first two seats voted and they differ, so two more must vote."""
if len(counted) == 2 and counted[0] != counted[1]:
return None
for vote in VOTES:
if all(counted.count(vote) >= counted.count(other) + 2 for other in VOTES if other != vote):
return vote
return "same"
def fallen(counted: dict[str, list[str]]) -> list[str]:
"""Each dimension, Overall aside, that the seats rule worse by a lead of two."""
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if ruling(list(column)) == "worse"]
+72
View File
@@ -0,0 +1,72 @@
import decimal
import typing
from comprehension_history import DIMENSIONS
from failure import Failure
from rank_todo import Item, ITEMS_HEADER, parse_row, table_rows
FLOOR = decimal.Decimal("7.0")
def fall_step(fell: list[str], scored: bool) -> typing.Literal["ask", "refuse", "rule"]:
"""What record-ruling does next: ask for the scoring seats when a dimension fell and no scores file came, refuse a scores file no fall calls, or rule."""
if fell and not scored:
return "ask"
if scored and not fell:
return "refuse"
return "rule"
def fall_ruling(ruled: str, fell: list[str], found: list[decimal.Decimal]) -> tuple[str, str]:
"""The ruling once the scoring run has scored the dimensions that fell, and the line that says why: worse when one of their means is under the floor."""
if under := [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if name in fell and value < FLOOR]:
return "worse", f"Fell: {', '.join(fell)}. Under the floor of {FLOOR}: {', '.join(under)}, so the {ruled} ruling counts as worse."
return ruled, f"Fell: {', '.join(fell)}, each at or above the floor of {FLOOR}, so the {ruled} ruling stands."
def means(lines: list[list[str]]) -> list[decimal.Decimal]:
"""The mean of each score across the lines, in DIMENSIONS' order, rounded half up to two decimals."""
return [mean([decimal.Decimal(score) for score in column]) for column in zip(*lines)]
def mean(values: list[decimal.Decimal]) -> decimal.Decimal:
return (sum(values, decimal.Decimal(0)) / len(values)).quantize(decimal.Decimal("0.01"), decimal.ROUND_HALF_UP)
def under_floor(found: list[decimal.Decimal]) -> list[str]:
return [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if value < FLOOR]
def plan_items(todo: str) -> list[Item] | Failure:
"""Each row of a todo.md's items table, read as `rank_todo.py` reads it."""
table = table_rows(todo.splitlines(), ITEMS_HEADER)
if isinstance(table, Failure):
return table
items = []
for row in table.rows:
item = parse_row(row)
if isinstance(item, Failure):
return item
items.append(item)
return items
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
"""Why the plan calls a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag. Work has landed towards an emptied release when the chunk ships one of its items, or when it is the next release, the earliest before names, and landed holds."""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
next_release = min((item.release for item in before), default=None)
for release in emptied(before, after, shipped):
if any(item.release == release for item in shipped) or (release == next_release and landed):
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
return reasons
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
named = {item.release for item in before + shipped}
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
return sorted(named - left)
+15 -26
View File
@@ -2,50 +2,32 @@ import dataclasses
import decimal
import re
from comprehension_history import ARCHITECT, DIMENSIONS, JUNIOR, MID, Seat, SENIOR
from failure import Failure
ARCHITECT = "Inherited architect"
DIMENSIONS = ("Navigation", "Locality", "Shape", "Self-sufficiency", "Overall")
JUNIOR = "Junior A"
# A scoring run moved from a `comprehension floor:` decision entry kept only the seats' means.
MEAN = "Mean"
MID = "Mid A"
SCORES = re.compile(", ".join(rf"{name} ([0-9]+(?:\.[0-9]+)?)" for name in DIMENSIONS))
SENIOR = "Maintainability senior"
FIRST_SEATS = (MID, ARCHITECT)
SCORING_SEATS = (JUNIOR, MID, SENIOR, ARCHITECT)
SEAT_LINE = re.compile(rf"^({'|'.join(SCORING_SEATS)}): (.+)$")
SPLIT_SEATS = (SENIOR, JUNIOR)
VOTE_LINE = re.compile(", ".join(rf"{name} (better|same|worse)" for name in DIMENSIONS))
@dataclasses.dataclass(frozen=True)
class Seat:
name: str
said: str
places: list[str]
@dataclasses.dataclass(frozen=True)
class Report:
"""A seats file: what the seats read, `<PR or branch> at <commit>`, with `, against <base>` for a paired ruling, then each seat's line and its places, or, for a run moved from a decision entry, its Mean line alone."""
class SeatsFile:
"""What the seats read, `<PR or branch> at <commit>`, with `, against <base>` for a paired ruling, then each seat's line and its places."""
read: str
seats: list[Seat]
moved_mean: str | None
def report(text: str) -> Report | Failure:
def seats_file(text: str) -> SeatsFile | Failure:
lines = [line for line in text.splitlines() if line.strip()]
if not lines:
return Failure("the seats file is empty; its first line names what the seats read: <PR or branch> at <commit>")
seats: list[Seat] = []
moved_mean = None
for line in lines[1:]:
if line.startswith(f"{MEAN}: ") and not seats and moved_mean is None:
moved_mean = line[len(MEAN) + 2:].strip()
elif moved_mean is not None:
return Failure(f"a {MEAN} line stands alone in its seats file: {line}")
elif seat := SEAT_LINE.match(line):
if seat := SEAT_LINE.match(line):
if seat.group(1) in [known.name for known in seats]:
return Failure(f"{seat.group(1)} appears twice in the seats file")
seats.append(Seat(seat.group(1), seat.group(2).strip(), []))
@@ -55,13 +37,13 @@ def report(text: str) -> Report | Failure:
seats[-1].places[-1] += " " + line.strip()
else:
return Failure(f"the seats file has a line that is neither a seat's line, opening with one of {', '.join(SCORING_SEATS)} and a colon, nor one of its places: {line}")
return Report(lines[0].strip(), seats, moved_mean)
return SeatsFile(lines[0].strip(), seats)
def scores(seats: list[Seat]) -> dict[str, list[str]] | Failure:
"""Each seat's five scores as written, in DIMENSIONS' order."""
if [seat.name for seat in seats] != list(SCORING_SEATS):
return Failure(f"a scoring run's seats are {', '.join(SCORING_SEATS)}, in that order, or a {MEAN} line alone; the file has {', '.join(seat.name for seat in seats) or 'none'}")
return Failure(f"a scoring run's seats are {', '.join(SCORING_SEATS)}, in that order; the file has {', '.join(seat.name for seat in seats) or 'none'}")
found = {}
for seat in seats:
values = score_line(seat.name, seat.said)
@@ -81,3 +63,10 @@ def score_line(name: str, said: str) -> list[str] | Failure:
if values[-1] > min(values[:-1]) + 1:
return Failure(f"{name}'s overall exceeds the lowest dimension by more than 1; ask the seat to rescore: {said}")
return list(written.groups())
def vote_line(name: str, said: str) -> list[str] | Failure:
"""A paired seat's five votes, in DIMENSIONS' order."""
if not (written := VOTE_LINE.fullmatch(said)):
return Failure(f"{name}'s line is not {', '.join(vote + ' <better, same or worse>' for vote in DIMENSIONS)}: {said}")
return list(written.groups())
+1 -1
View File
@@ -4,7 +4,7 @@ import dataclasses
import os
import statistics
from larv_review.history import Place
from comprehension_history import Place
from metric_bench.python_units import Unit
METRICS: dict[str, Callable[[Unit, dict[str, int]], int]] = {
+13 -6
View File
@@ -3,9 +3,10 @@
usage: metric-bench [history]
history defaults to docs/comprehension-history.md in the checkout it runs in, and the bench reads
the checkout that holds the history. For each scoring run in it, at the commit the panel read, it
measures each module-level function and each method of a module-level class in the Python files git
tracks, leaving out test_*.py, *_test.py and anything under a tests/ folder:
the checkout that holds the history. For each scoring run in it, those a fall called inside a paired
ruling included, at the commit the panel read, it measures each module-level function and each
method of a module-level class in the Python files git tracks, leaving out test_*.py, *_test.py and
anything under a tests/ folder:
- cognitive complexity: SonarSource's white paper, read for Python;
- length: lines;
@@ -35,7 +36,7 @@ import subprocess
import sys
from metric_bench.bench import bench, co_changed, medians, METRICS, resolved, RunScores, Score
from larv_review.history import entries, Entry
from comprehension_history import entries, Entry
from failure import Failure
from metric_bench.python_units import measured, python_units, Unit
@@ -77,11 +78,17 @@ def read_history(given: pathlib.Path | None) -> tuple[str, pathlib.Path, str] |
return Failure(f"could not read {history}: {failure}")
def scored_runs(found: list[Entry]) -> list[Entry]:
"""Each scoring run, and each paired ruling holding the scores a fall called unless a scoring run on its commit records them too."""
runs = {entry.commit for entry in found if not entry.paired}
return [entry for entry in found if not entry.paired or entry.scored_places and entry.commit not in runs]
def bench_runs(text: str, top: str) -> tuple[list[tuple[str, RunScores]], list[str]] | Failure:
"""Each scored run's heading and scores, and a note for each run skipped or not scored and each file left out."""
rows = []
notes = []
for run in [entry for entry in entries(text) if not entry.paired]:
for run in scored_runs(entries(text)):
if isinstance(missing := git(["cat-file", "-e", f"{run.commit}^{{commit}}"], top), Failure):
notes.append(f"Skipped, its commit is not in {top}: {run.heading} ({missing.reason})")
continue
@@ -121,7 +128,7 @@ def run_scores(run: Entry, top: str) -> tuple[RunScores | None, list[str]] | Fai
log = git(["-c", "core.quotePath=false", "log", "--no-merges", "--format=tformat:@@", "--name-only", run.commit], top)
if isinstance(log, Failure):
return log
places, dropped = resolved(run.places, tracked)
places, dropped = resolved(run.scored_places, tracked)
return bench(units, places, len(set(dropped)), co_changed(log)), unparsed
+116 -50
View File
@@ -9,12 +9,15 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from larv_review.history import appended, entries, HistoryChange, Place, ruling_entry
from larv_review.rules import base, first_ruling, means, measured_commit, plan_items, round_counts, ruling, scoring_run_due, under_floor, votes
from comprehension_history import appended, entries, Place, recorded, ruling_entry, Scoring, Seat
from larv_review.base import base, first_ruling, HistoryChange, measured_commit
from larv_review.ruling import counted, fallen, ruling, votes
from failure import Failure
from larv_review.scoring import fall_ruling, fall_step, means, plan_items, scoring_run_due, under_floor
from test_loop_claude_black_box import host_env
from rank_todo import Item
from larv_review.seats import report, score_line, scores, Seat
from larv_review.rounds import round_counts
from larv_review.seats import score_line, scores, seats_file
TOOL = pathlib.Path(__file__).resolve().parent / "larv-review"
SCORING = "## 2026-10-08T12:00:00Z, PR #1 at aaaaaaa\n\n| Seat | Overall |\n|---|---|\n| Mean | 7.00 |\n"
@@ -24,6 +27,10 @@ def paired(time: str, verdict: str, commit: str = "bbbbbbb") -> str:
return f"## {time}, PR #2 at {commit}, against aaaaaaa\n\nRuling: {verdict}\n"
def said(overall: str, locality: str = "same") -> str:
return f"Navigation same, Locality {locality}, Shape same, Self-sufficiency same, Overall {overall}"
def item(id_: int, release: tuple[int, int, int], tagline: str = "**Do it.**") -> Item:
return Item(id_, release, "", tagline, {"R": 1, "S": 1, "A": 1, "G": 1}, ("1",))
@@ -34,28 +41,45 @@ def todo(*rows: str) -> str:
class Seats(unittest.TestCase):
def test_reads_each_seats_line_and_places_and_joins_a_wrapped_place(self):
given = report("PR #2 at abc1234, against def5678\n\nMid A: worse\n\n- `a.py:1` `f`: hard\n to read\n\nInherited architect: same\n")
given = seats_file(f"PR #2 at abc1234, against def5678\n\nMid A: {said('worse')}\n\n- `a.py:1` `f`: hard\n to read\n\nInherited architect: {said('same')}\n")
self.assertEqual(given.read, "PR #2 at abc1234, against def5678")
self.assertEqual(given.seats, [Seat("Mid A", "worse", ["`a.py:1` `f`: hard to read"]), Seat("Inherited architect", "same", [])])
self.assertEqual(given.seats, [Seat("Mid A", said("worse"), ["`a.py:1` `f`: hard to read"]), Seat("Inherited architect", said("same"), [])])
def test_refuses_a_line_that_is_neither_a_seat_nor_a_place_and_a_seat_twice(self):
self.assertIsInstance(report("PR #2 at abc1234\n\nMid A worse\n"), Failure)
self.assertIsInstance(report("PR #2 at abc1234\n\nMid A: worse\n\nMid A: same\n"), Failure)
self.assertIsInstance(report(""), Failure)
self.assertIsInstance(seats_file("PR #2 at abc1234\n\nMid A worse\n"), Failure)
self.assertIsInstance(seats_file("PR #2 at abc1234\n\nMid A: worse\n\nMid A: same\n"), Failure)
self.assertIsInstance(seats_file(""), Failure)
def test_counts_a_vote_naming_no_place_as_same(self):
seats = [Seat("Mid A", "worse", ["the whole skill"]), Seat("Inherited architect", "better", ["`a.py:3` `f`: shorter"])]
self.assertEqual(votes(seats), {"Mid A": "same", "Inherited architect": "better"})
def test_counts_each_vote_of_a_seat_naming_no_place_as_same(self):
seats = [Seat("Mid A", said("worse", "worse"), ["Locality: the whole skill"]), Seat("Inherited architect", said("better"), ["Overall: `a.py:3` `f`: shorter"])]
written = votes(seats)
self.assertEqual(written, {"Mid A": ["same", "worse", "same", "same", "worse"], "Inherited architect": ["same", "same", "same", "same", "better"]})
self.assertEqual(counted(seats, written)["Mid A"], ["same"] * 5)
def test_refuses_two_more_seats_once_the_first_two_agree(self):
seats = [Seat(name, vote, ["`f.py:1` `f`: x"]) for name, vote in (("Mid A", "worse"), ("Inherited architect", "worse"), ("Maintainability senior", "better"), ("Junior A", "better"))]
def test_counts_a_dimensions_vote_only_where_a_place_names_that_dimension(self):
seats = [Seat("Mid A", said("better", "worse"), ["Shape: `a.py:3` `f`: the Locality of it, reshaped"]), Seat("Inherited architect", said("better", "worse"), ["Locality, Shape: `a.py:3` `f`: held at once"])]
written = {seat.name: ["same", "worse", "same", "same", "better"] for seat in seats}
self.assertEqual(counted(seats, written), {"Mid A": ["same", "same", "same", "same", "better"], "Inherited architect": written["Inherited architect"]})
self.assertIsInstance(votes([Seat("Mid A", said("better"), ["Locality, self sufficiency: `a.py:3` `f`: x"]), seats[1]]), Failure)
self.assertIsInstance(votes([Seat("Mid A", said("better"), ["`a.py:3` `f`: Locality: x"]), seats[1]]), Failure)
self.assertEqual(votes(seats), {seat.name: written[seat.name] for seat in seats})
def test_refuses_two_more_seats_once_the_first_two_agree_overall(self):
seats = [Seat(name, said(vote), ["Overall: `f.py:1` `f`: x"]) for name, vote in (("Mid A", "worse"), ("Inherited architect", "worse"), ("Maintainability senior", "better"), ("Junior A", "better"))]
self.assertIsInstance(votes(seats), Failure)
def test_takes_the_first_two_seats_then_the_other_two_in_order(self):
first = [Seat("Mid A", "same", []), Seat("Inherited architect", "same", [])]
def test_takes_the_first_two_seats_then_the_other_two_in_order_and_each_dimensions_vote(self):
first = [Seat("Mid A", said("same"), []), Seat("Inherited architect", said("same"), [])]
self.assertIsInstance(votes(first[::-1]), Failure)
self.assertIsInstance(votes(first + [Seat("Junior A", "same", [])]), Failure)
self.assertIsInstance(votes([first[0], Seat("Inherited architect", "worst", [])]), Failure)
self.assertIsInstance(votes(first + [Seat("Junior A", said("same"), [])]), Failure)
self.assertIsInstance(votes([first[0], Seat("Inherited architect", said("worst"), [])]), Failure)
self.assertIsInstance(votes([first[0], Seat("Inherited architect", "same", [])]), Failure)
def test_finds_each_dimension_ruled_worse_by_a_lead_of_two(self):
self.assertEqual(fallen({"Mid A": ["same", "worse", "worse", "same", "better"], "Inherited architect": ["same", "worse", "same", "same", "better"]}), ["Locality"])
four = {name: ["same", vote, "same", "same", "better"] for name, vote in (("Mid A", "worse"), ("Inherited architect", "same"), ("Maintainability senior", "worse"), ("Junior A", "worse"))}
self.assertEqual(fallen(four), ["Locality"])
self.assertEqual(fallen({**four, "Junior A": ["same", "better", "same", "same", "better"]}), [])
def test_rules_by_a_lead_of_two_and_splits_two_seats_that_differ(self):
cases = [
@@ -89,18 +113,33 @@ class Seats(unittest.TestCase):
class History(unittest.TestCase):
def test_reads_back_what_it_writes(self):
seats = [Seat("Mid A", "worse", ["`bin/a.py:3` `f`: hard, with `:9` `g`"]), Seat("Inherited architect", "worse", [])]
ruled = ruling_entry("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", seats, {"Mid A": "worse", "Inherited architect": "same"}, "worse")
seats = [Seat("Mid A", said("better", "worse"), ["`bin/a.py:3` `f`: hard, with `:9` `g`"]), Seat("Inherited architect", said("worse", "worse"), [])]
written = {"Mid A": ["same", "worse", "same", "same", "better"], "Inherited architect": ["same", "worse", "same", "same", "worse"]}
scoring = Scoring([Seat("Junior A", "", ["`bin/b.py:4` `h`: long"])], {"Junior A": ["7", "6.5", "7", "7", "7"]}, [decimal.Decimal(mean) for mean in ("7.00", "6.50", "7.00", "7.00", "7.00")])
ruled = ruling_entry("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", seats, written, {**written, "Inherited architect": ["same"] * 5}, "worse", ["Fell: Locality, under the floor."], scoring)
history = appended(appended("", SCORING), ruled)
self.assertTrue(history.startswith("# Comprehension history\n\n## 2026-10-08T12:00:00Z"))
self.assertNotIn("Inherited architect, decided by:", history)
self.assertIn("| Inherited architect | same (voted worse, naming no place) |", history)
self.assertIn("| Inherited architect | same | same (voted worse, no place naming Locality) | same | same | same (voted worse, naming no place) |", history)
found = entries(history)
self.assertEqual([(entry.commit, entry.against, entry.ruling) for entry in found], [("aaaaaaa", None, None), ("bbbbbbb", "aaaaaaa", "worse")])
self.assertEqual(found[1].places, [Place("bin/a.py", "f"), Place("bin/a.py", "g")])
self.assertEqual((found[1].ruled_places, found[1].scored_places), ([Place("bin/a.py", "f"), Place("bin/a.py", "g")], [Place("bin/b.py", "h")]))
self.assertTrue(recorded(history, "PR #2 at bbbbbbb, against aaaaaaa"))
self.assertFalse(recorded(history, "PR #2 at bbbbbbb"))
self.assertIn("| Mean | 7.00 | 6.50 | 7.00 | 7.00 | 7.00 |\n\nMid A, decided by:", history)
self.assertEqual([entry.paired for entry in found], [False, True])
class FallRuling(unittest.TestCase):
def test_counts_a_ruling_worse_when_a_fallen_dimensions_mean_is_under_the_floor(self):
found = [decimal.Decimal(score) for score in ("6", "6.5", "7", "6", "6")]
self.assertEqual(fall_ruling("better", ["Locality", "Shape"], found), ("worse", "Fell: Locality, Shape. Under the floor of 7.0: Locality 6.5, so the better ruling counts as worse."))
self.assertEqual(fall_ruling("same", ["Shape"], found), ("same", "Fell: Shape, each at or above the floor of 7.0, so the same ruling stands."))
def test_asks_for_scores_on_a_fall_refuses_them_without_one_and_rules_otherwise(self):
self.assertEqual([fall_step(["Locality"], False), fall_step([], True), fall_step(["Locality"], True), fall_step([], False)], ["ask", "refuse", "rule", "rule"])
class Base(unittest.TestCase):
def test_takes_the_newest_change_adding_a_scoring_run_or_ending_on_a_better_ruling(self):
older = HistoryChange("old", "", SCORING)
@@ -112,6 +151,8 @@ class Base(unittest.TestCase):
worse_last = HistoryChange("worse", SCORING, SCORING + paired("2026-10-08T13:00:00Z", "better") + paired("2026-10-08T14:00:00Z", "worse"))
self.assertEqual(measured_commit([worse_last, older]), "old")
self.assertIsNone(measured_commit([same]))
scored_fall = HistoryChange("same fell", SCORING, SCORING + paired("2026-10-08T13:00:00Z", "same\n\nFell: Locality, each at or above the floor.\n\n| Seat | Overall |\n|---|---|\n| Mean | 7.00 |"))
self.assertEqual(measured_commit([scored_fall, older]), "old")
def test_falls_back_to_the_first_rulings_base_then_to_none(self):
ruled = paired("2026-10-08T13:00:00Z", "worse")
@@ -190,8 +231,8 @@ class LarvReviewTool(unittest.TestCase):
def run_tool(self, *args):
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=self.checkout, env=self.env, text=True)
def seats_file(self, text):
path = self.checkout.parent / "seats.md"
def seats_file(self, text, name="seats.md"):
path = self.checkout.parent / name
path.write_text(textwrap.dedent(text))
return str(path)
@@ -230,52 +271,84 @@ class LarvReviewTool(unittest.TestCase):
lines = """\
PR #2 at {start}, against {start}
Mid A: worse
Mid A: {worse}
- `bin/a.py:3` `f`: three branches
- Overall: `bin/a.py:3` `f`: three branches
Inherited architect: same
Inherited architect: {same}
"""
lines = lines.replace("{start}", self.start)
lines = lines.replace("{start}", self.start).replace("{worse}", said("worse")).replace("{same}", said("same"))
split = self.run_tool("record-ruling", self.seats_file(lines))
self.assertEqual(split.returncode, 0, split.stderr)
self.assertTrue(split.stdout.startswith("Split: Mid A worse, Inherited architect same."))
history = self.checkout / "docs" / "comprehension-history.md"
self.assertFalse(history.exists())
lines += "\n\t\t\tMaintainability senior: worse\n\n\t\t\t- `bin/a.py:3` `f`: hard\n\n\t\t\tJunior A: worse\n"
lines += f"\n\t\t\tMaintainability senior: {said('worse')}\n\n\t\t\t- Overall: `bin/a.py:3` `f`: hard\n\n\t\t\tJunior A: {said('worse')}\n"
ran = self.run_tool("record-ruling", self.seats_file(lines))
self.assertEqual(ran.stdout.splitlines()[0], "Ruling: same. Votes as counted: Mid A worse, Inherited architect same, Maintainability senior worse, Junior A same (voted worse, naming no place).")
self.assertEqual(ran.stdout.splitlines()[0], "Ruling: same. Overall votes as counted: Mid A worse, Inherited architect same, Maintainability senior worse, Junior A same (voted worse, naming no place).")
recorded = entries(history.read_text())
self.assertEqual([(entry.commit, entry.against, entry.ruling) for entry in recorded], [(self.start, self.start, "same")])
self.assertEqual(recorded[0].places, [Place("bin/a.py", "f")] * 2)
self.assertIn("| Junior A | same (voted worse, naming no place) |", history.read_text())
self.assertEqual(recorded[0].ruled_places, [Place("bin/a.py", "f")] * 2)
self.assertIn("| Junior A | same | same | same | same | same (voted worse, naming no place) |", history.read_text())
def test_asks_for_the_scoring_seats_when_a_dimension_falls_and_rules_with_their_scores(self):
place = "\n\n- Locality, Overall: `bin/a.py:3` `f`: shorter\n\n"
votes_file = self.seats_file(f"PR #2 at {self.start}, against {self.start}\n\nMid A: {said('better', 'worse')}{place}Inherited architect: {said('better', 'worse')}{place}", "votes.md")
asked = self.run_tool("record-ruling", votes_file)
self.assertEqual(asked.returncode, 0, asked.stderr)
self.assertTrue(asked.stdout.startswith(f"Fell: Locality, in a better ruling. Launch the four scoring seats on PR #2 at {self.start},"))
history = self.checkout / "docs" / "comprehension-history.md"
self.assertFalse(history.exists())
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 7, Overall 7"
scores_text = f"PR #2 at {self.start}\n" + "".join(f"\n{seat}: {line}\n\n- `bin/b.py:4` `h`: long\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
ruled = self.run_tool("record-ruling", votes_file, self.seats_file(scores_text, "scores.md"))
self.assertEqual(ruled.returncode, 0, ruled.stderr)
self.assertTrue(ruled.stdout.startswith("Ruling: worse. Fell: Locality. Under the floor of 7.0: Locality 6.50, so the better ruling counts as worse."))
again = self.run_tool("record-ruling", votes_file, self.seats_file(scores_text, "scores.md"))
self.assertIn("already holds an entry on", again.stderr)
self.assertEqual(self.run_tool("record-ruling", votes_file).stdout, "")
recorded = entries(history.read_text())
self.assertEqual([entry.ruling for entry in recorded], ["worse"])
self.assertIn("| Mean | 7.00 | 6.50 | 7.00 | 7.00 | 7.00 |", history.read_text())
votes_file = self.seats_file(pathlib.Path(votes_file).read_text().replace("PR #2", "PR #3"), "votes.md")
scores_text = scores_text.replace("PR #2", "PR #3")
stands = self.run_tool("record-ruling", votes_file, self.seats_file(scores_text.replace("Locality 6.5", "Locality 7"), "scores.md"))
self.assertTrue(stands.stdout.startswith("Ruling: better. Fell: Locality, each at or above the floor of 7.0, so the better ruling stands."), stands.stderr)
votes_file = self.seats_file(pathlib.Path(votes_file).read_text().replace("PR #3", "PR #4"), "votes.md")
other = self.run_tool("record-ruling", votes_file, self.seats_file(scores_text.replace("PR #3", "PR #5"), "scores.md"))
self.assertIn("the scores file's first line is not", other.stderr)
def test_names_no_fall_on_a_worse_ruling_and_takes_no_scores_file_without_one(self):
place = "\n\n- Locality, Overall: `bin/a.py:3` `f`: longer\n\n"
ran = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}, against {self.start}\n\nMid A: {said('worse', 'worse')}{place}Inherited architect: {said('worse', 'worse')}{place}"))
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertNotIn("Fell:", ran.stdout + (self.checkout / "docs" / "comprehension-history.md").read_text())
extra = self.run_tool("record-ruling", self.seats_file(f"PR #3 at {self.start}, against {self.start}\n\nMid A: {said('worse')}{place}Inherited architect: {said('worse')}{place}"), self.seats_file("PR #2", "scores.md"))
self.assertIn("takes no scores file", extra.stderr)
def test_records_a_scoring_run_with_its_means(self):
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
seats = "".join(f"\n{seat}: {line}\n\n- `bin/a.py:3` `f`: long\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
ran = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats))
uncalled = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats))
self.assertEqual(uncalled.returncode, 1)
self.assertIn("no plan item or release calls this scoring run", uncalled.stderr)
ran = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats), "1")
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertIn("Under the floor of 7.0: Locality 6.50, Self-sufficiency 6.50.", ran.stdout)
self.assertIn("\nFloor: Locality 6.50, Self-sufficiency 6.50 under the floor of 7.0 block the merge, as the chunk leaves release 0.1.0 with no plan item.\n", ran.stdout)
self.assertIn("not /", self.run_tool("record-scoring", "1", self.seats_file(f"PR #2 at {self.start}\n" + seats)).stderr)
text = (self.checkout / "docs" / "comprehension-history.md").read_text()
self.assertIn("| Mean | 7.00 | 6.50 | 7.00 | 6.50 | 7.00 |", text)
self.assertIn("Inherited architect, would restructure first:", text)
self.assertEqual([entry.places for entry in entries(text)], [[Place("bin/a.py", "f")] * 4])
self.assertEqual(entries(text)[0].scored_places, [Place("bin/a.py", "f")] * 4)
def test_refuses_a_ruling_file_that_names_no_base_or_a_commit_the_checkout_lacks(self):
ran = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}\n\nMid A: same\n\nInherited architect: same\n"))
ran = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}\n\nMid A: {said('same')}\n\nInherited architect: {said('same')}\n"))
self.assertEqual(ran.returncode, 1)
self.assertIn("against <base>", ran.stderr)
lacking = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}, against abcdef1\n\nMid A: same\n\nInherited architect: same\n"))
lacking = self.run_tool("record-ruling", self.seats_file(f"PR #2 at {self.start}, against abcdef1\n\nMid A: {said('same')}\n\nInherited architect: {said('same')}\n"))
self.assertEqual(lacking.returncode, 1)
self.assertIn("abcdef1, which is no commit", lacking.stderr)
def test_moves_a_dated_mean_from_a_decision_entry_as_a_scoring_run(self):
mean = "Mean: Navigation 7.38, Locality 6.63, Shape 7.0, Self-sufficiency 6.88, Overall 6.88\n"
ran = self.run_tool("record-scoring", self.seats_file(f"2026-10-04, PR #1 merged at {self.start}\n\n{mean}"))
self.assertEqual(ran.returncode, 0, ran.stderr)
text = (self.checkout / "docs" / "comprehension-history.md").read_text()
self.assertIn(f"## 2026-10-04, PR #1 merged at {self.start}\n\n| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |\n|---|---|---|---|---|---|\n| Mean | 7.38 | 6.63 | 7.00 | 6.88 | 6.88 |\n", text)
def test_states_its_rules_and_finds_origins_default_branch_where_the_clone_recorded_none(self):
helped = self.run_tool("--help")
self.assertEqual(helped.returncode, 0, helped.stderr)
@@ -283,13 +356,6 @@ class LarvReviewTool(unittest.TestCase):
self.git("remote", "set-head", "origin", "--delete")
self.assertEqual(self.run_tool("needs-scoring-run", "2").stdout, "No scoring run due.\n")
def test_refuses_a_dated_first_line_except_for_a_mean_alone_and_a_mean_without_one(self):
mean = "Mean: Navigation 7, Locality 7, Shape 7, Self-sufficiency 7, Overall 7\n"
self.assertEqual(self.run_tool("record-scoring", self.seats_file(f"PR #1 at {self.start}\n\n{mean}")).returncode, 1)
ruled = self.run_tool("record-ruling", self.seats_file(f"2026-10-08, PR #2 at {self.start}, against {self.start}\n\nMid A: same\n\nInherited architect: same\n"))
self.assertEqual(ruled.returncode, 1)
self.assertIn("only a seats file holding a Mean line alone", ruled.stderr)
def test_takes_a_local_main_without_origin_and_reads_origin_as_last_fetched_when_the_fetch_fails(self):
self.commit("Answer prose round 1: a")
self.git("remote", "set-url", "origin", str(self.checkout.parent / "gone.git"))
+8 -4
View File
@@ -11,9 +11,10 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from metric_bench.bench import auc, bench, co_changed, resolved, top_k
from metric_bench.python_units import cognitive, measured, python_units, Unit
from larv_review.history import entries, Place
from comprehension_history import entries, Place
from failure import Failure
from test_loop_claude_black_box import host_env
from metric_bench.main import scored_runs
TOOL = pathlib.Path(__file__).resolve().parent / "metric-bench"
@@ -141,10 +142,13 @@ class Places(unittest.TestCase):
- `bin/tool/cycle/run.py:1` `step`
""")
def test_reads_each_scoring_runs_places_and_leaves_paired_rulings_out(self):
runs = [entry for entry in entries(self.HISTORY) if not entry.paired]
def test_reads_each_scoring_runs_places_and_leaves_rulings_out_unless_they_hold_a_falls_scores(self):
runs = scored_runs(entries(self.HISTORY))
self.assertEqual([(run.heading, run.commit) for run in runs], [("2026-10-08T12:50:49Z, PR #224 at a58a0a8", "a58a0a8")])
self.assertEqual(runs[0].places, [Place("skills/x/SKILL.md", None), Place("bin/tool/cycle/states.py", "transition"), Place("cycle/run.py", "step"), Place("cycle/run.py", "close"), Place("bin/tool/cycle/judge.py", "run_judge")])
fall = "\n## 2026-10-09T09:00:00Z, PR #232 at {commit}, against 12d6d75\n\nRuling: worse\n\nJunior A, hardest first:\n\n- `bin/a.py:3` `f`\n"
self.assertEqual([run.commit for run in scored_runs(entries(self.HISTORY + fall.format(commit="bbbbbbb")))], ["a58a0a8", "bbbbbbb"])
self.assertEqual([run.commit for run in scored_runs(entries(self.HISTORY + fall.format(commit="a58a0a8")))], ["a58a0a8"])
self.assertEqual(runs[0].scored_places, [Place("skills/x/SKILL.md", None), Place("bin/tool/cycle/states.py", "transition"), Place("cycle/run.py", "step"), Place("cycle/run.py", "close"), Place("bin/tool/cycle/judge.py", "run_judge")])
def test_resolves_a_shortened_path_to_the_file_nearest_the_one_before_it(self):
tracked = ["bin/tool/cycle/states.py", "bin/tool/cycle/run.py", "bin/other/cycle/run.py"]
+1 -1
View File
@@ -11,5 +11,5 @@ class Types(unittest.TestCase):
self.assertIsNotNone(floor, "README.md's Install names `Python 3.N or newer`")
if shutil.which("uv") is None:
self.fail("uv on PATH fetches mypy; see README.md's Test line")
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "-p", "larv_review", "-p", "loop_claude", "-p", "metric_bench", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline"], capture_output=True, cwd=ROOT / "bin", text=True)
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "-m", "comprehension_history", "-p", "larv_review", "-p", "loop_claude", "-p", "metric_bench", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline"], capture_output=True, cwd=ROOT / "bin", text=True)
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
+204
View File
@@ -528,3 +528,207 @@ Junior A, decided by:
- `bin/larv_review/git.py:5` and `bin/metric_bench/main.py:38`: the git layer depends on the history type, and metric-bench on larv-review's package
- `bin/larv_review/main.py:1` the module docstring: one home, but a wall of rules leaning on terms defined elsewhere
- `bin/larv_review/rules.py:16` `votes`: a seat protocol reconstructed only from `skills/larv-review/SKILL.md:97`
## 2026-10-09T08:54:36Z, PR #232 at 2c6e063, against 12d6d75
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | same | same | better | better |
| Inherited architect | better | worse | same | better | better |
Mid A, decided by:
- `bin/larv_review/main.py:146` `round_count`: Navigation, better: the fourth-round symptom leads to a named unit, where the base had only `skills/larv-review/SKILL.md:58` "Count"
- `bin/larv_review/rules.py:42` `ruling`, `:52` `fallen` and `:128` `scoring_run_due`: Self-sufficiency, better: vote arithmetic and the scoring trigger are pure functions with docstrings, where the base had prose alone
- `README.md:68` "Tools": Navigation, better: the README names larv-review and metric-bench
- `bin/larv_review/main.py:1` the module docstring: Self-sufficiency, better, with a cost: Split, Fell, Ratchet and the moved Mean line arrive in long sentences
- `bin/larv_review/history.py:96` `heading`: Locality, against: two flags with DATED and HEADING, traced through `bin/larv_review/seats.py:38` `report` and `bin/larv_review/main.py:207` `record_scoring`
- `bin/larv_review/history.py:18` `SEAT_PLACES`, `:114` `ruling_entry` and `:125` `scoring_entry`: Locality, against: writers and parser must agree on label strings
- `bin/larv_review/rules.py:1` the module: Shape, against: one vague name over five concerns
- `bin/metric_bench/bench.py:87` `resolved` and `bin/metric_bench/main.py:38`: Shape and Locality, against: metric-bench depends on larv-review's history parser
Inherited architect, decided by:
- `bin/larv_review/main.py:146` `round_count`: Navigation: the 3am question lands on a function, where the base had `skills/larv-review/SKILL.md:58` "Count"
- `bin/larv_review/rules.py:89` `measured_commit` and `:105` `base`: Self-sufficiency: the ratchet's base is executable, where the base had `skills/larv-review/SKILL.md:100`
- `bin/larv_review/history.py:60` `entries`, `:114` `ruling_entry` and `:125` `scoring_entry`: Self-sufficiency and Navigation: the history's format is a regex plus writers, where the base had `skills/larv-review/SKILL.md:92`
- `bin/larv_review/main.py:207` `record_scoring`, with `bin/larv_review/seats.py:45` `report` and `bin/larv_review/history.py:96` `heading`: Locality: the moved Mean case threads through three modules
- `bin/larv_review/rules.py:57` `pending_fall`, called from `bin/larv_review/main.py:179` and `:224`: Locality: commands depend on state an earlier command appended to the history; `history[1]` hides it at the call site
- `bin/metric_bench/main.py:38` and `bin/metric_bench/bench.py:7`: Locality: metric-bench parses the history with larv-review's `entries`, and keeps its own `git()` at `bin/metric_bench/main.py:143`
- `bin/larv_review/rules.py:1` the module: Shape: a grab-bag name
- `bin/larv_review/git.py:56` `file_at` and `bin/larv_review/main.py:247` `history_text`: Locality: two histories, the default branch's and the checkout's
- `skills/larv-review/SKILL.md:97` "Seats": Self-sufficiency, against the skill: it needs the tool's help open beside it
- `bin/rank_todo.py:386` `parse_row`: Shape, a small gain
## 2026-10-09T09:35:29Z, PR #232 at 6a8bbcf, against 12d6d75
Ruling: same
Fell: Locality, Shape
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | worse | same | better | better |
| Inherited architect | better | worse | worse | same | worse |
| Maintainability senior | better | worse | worse | better | same |
| Junior A | better | worse | worse | better | worse |
Mid A, decided by:
- Navigation, Self-sufficiency, Overall: `bin/larv_review/rules.py:118` `round_counts`, with `bin/larv_review/main.py:155` `round_count`: the fourth-round symptom leads from the README's Tools list to a named, tested function, where the base had `skills/larv-review/SKILL.md:58` "Count"
- Self-sufficiency, Overall: `bin/larv_review/history.py:117` `ruling_entry` and `:130` `scoring_entry`: the history format is produced by code and checked by a regex, where the base had `skills/larv-review/SKILL.md:92` "History"
- Locality: `bin/larv_review/main.py:234` `record_scoring`: five conditionals on `given.moved_mean` weave a one-off migration into the main scoring path
- Locality: `bin/metric_bench/main.py:38` and `bin/larv_review/history.py:31` `Entry`: places are parsed in larv_review but used only by metric_bench; `bin/metric_bench/bench.py:87` `resolved` copes with old paths
- Locality: `bin/larv_review/main.py:1` the module docstring: rules compressed into one dense block; the fall, Ratchet and Still standing chain reaches across `bin/larv_review/rules.py:95` `fall_outcome`, `:76` `standing_ratchet` and `bin/larv_review/main.py:164` `needs_scoring_run`
- Navigation: `README.md:66` "Tools": lists larv-review and metric-bench; `skills/larv-review/SKILL.md:46` points at `--help`
- Shape: `bin/larv_review/rules.py:155` `plan_items` and `:169` `scoring_run_due`: a catch-all module, offset by the clean package split and the truthful `skills/comprehension-panel/seat-prompts.md:1` name
Inherited architect, decided by:
- Locality: `bin/larv_review/main.py:234` `record_scoring`: whether a run is allowed and whether the floor blocks depend on the plan diff, a pending fall read from the history, and `moved_mean`, checked five times in 25 lines
- Locality: `bin/larv_review/rules.py:71` `pending_fall` and `:76` `standing_ratchet`: commands talk only through the history file, so the outcome depends on the order they ran in
- Locality, Shape: `bin/larv_review/rules.py:127` `measured_commit`: re-parses the history around every merged change and applies a two-clause rule; where it rots first
- Shape: `bin/larv_review/rules.py:1` the module: vote counting, round counting, base search, plan parsing and release emptying under a name that says as little as utils; `HistoryChange` defined in history but built in `bin/larv_review/git.py:66` `history_changes`
- Shape: `bin/metric_bench/main.py:38`: a second tool no review runs, depending on larv_review's history parser; `bin/` gains two packages
- Shape: `bin/metric_bench/bench.py:87` `resolved`: a shim for old path formats
- Navigation: `bin/larv_review/rules.py:118` `round_counts` and `bin/larv_review/main.py:155` `round_count`: the 3am symptom lands on a named command, where the base had `skills/larv-review/SKILL.md:58` "Count"
- Navigation: `README.md:66` "Tools": lists every tool
- Navigation: `skills/comprehension-panel/seat-prompts.md:1`: a truthful name
- Self-sufficiency: `bin/larv_review/main.py:1` the module docstring and `skills/larv-review/SKILL.md:93` "Paired ruling": the code explains itself, but the skill's ratchet text needs the docstring open beside it; ruled same
Maintainability senior, decided by:
- Locality: `bin/larv_review/main.py:234` `record_scoring`: tests `given.moved_mean` five times, each branch tied to `chunk_history`, `due_reasons` and `pending_fall`
- Locality: `bin/larv_review/rules.py:71` `pending_fall`: its docstring holds only because `bin/larv_review/history.py:62` `entries` gives scoring runs an empty `fell`; the Fell, Ratchet and Still standing chain spans four files
- Locality, Shape: `bin/metric_bench/main.py:38` `main`: metric-bench imports larv_review's history internals; `bin/metric_bench/main.py:143` `git` repeats `bin/larv_review/git.py:10` `git`
- Shape: `bin/larv_review/rules.py:35` `counted`: a function and a parameter share the name; the module name says nothing of its contents
- Shape: `bin/larv_review/main.py:177` `due_reasons`: main holds commands plus I/O helpers, and the vocabulary grew to seven concepts
- Navigation: `README.md:66` "Tools": every tool listed with its `--help`
- Self-sufficiency, Navigation: `bin/larv_review/rules.py:118` `round_counts`: the round count, vote count and trigger state their rules in code, where the base had `skills/larv-review/SKILL.md:58` "Count"
- Self-sufficiency: `bin/larv_review/main.py:1` the module docstring: an 80-line block of every rule at once, partly cancelling the gain
Junior A, decided by:
- Locality, Overall: `bin/larv_review/main.py:234` `record_scoring`: the moved-Mean case is checked five times and decides four things
- Locality, Overall: `bin/larv_review/rules.py:76` `standing_ratchet` and `:71` `pending_fall`: a merge block depends on state earlier commands wrote into the history
- Locality, Shape: `bin/larv_review/rules.py:127` `measured_commit`: walks every merged change's history; restated at `bin/larv_review/main.py:11`, the hardest sentence in the repo
- Shape: `bin/larv_review/rules.py:1` the module: a grab-bag name beside truthfully named siblings
- Shape: `bin/larv_review/seats.py:31` `Report`: the field `read` looks like a verb; `said`, `counted`, `given` and `found` at `bin/larv_review/main.py:217` `record_ruling` need tracing
- Shape: `bin/metric_bench/main.py:38`: the history format is a contract between two tool packages
- Navigation, Self-sufficiency: `bin/larv_review/rules.py:118` `round_counts`: the 3am question lands on one named function, where the base had `skills/larv-review/SKILL.md:58` "Count"
- Navigation: `README.md:66` "Tools": the README lists larv-review and metric-bench
- Self-sufficiency: `bin/larv_review/history.py:62` `entries` and `:117` `ruling_entry`: the history format is enforced in code; the skill needs `--help` open beside it
## 2026-10-09T09:46:21Z, PR #232 at 324d650, against 12d6d75
Ruling: same
Fell: Locality
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | worse | worse | better | same |
| Inherited architect | better | worse | worse | better | better |
| Maintainability senior | better | worse | same | better | better |
| Junior A | better | worse | same | same | same |
Mid A, decided by:
- Navigation, Self-sufficiency: `bin/larv_review/rules.py:125` `round_counts`: the fourth-round symptom leads to a named function and `CAP`, where the base had prose at `skills/larv-review/SKILL.md:58` "Count"
- Navigation: `README.md:70` "Tools": names larv-review and metric-bench and the rules each holds
- Locality: `bin/larv_review/main.py:238` `record_scoring`: its result depends on which entry is newest in the history; the whole fall, scoring run, Ratchet, Still standing sequence must be held
- Locality: `bin/larv_review/rules.py:71` `pending_fall` and `:83` `standing_ratchet`: the verdict depends on entry order in a file no single function shows
- Locality, Shape: `bin/metric_bench/bench.py:7` and `bin/metric_bench/main.py:38`: metric-bench depends on larv-review's history parser and keeps its own `git` at `bin/metric_bench/main.py:143`
- Shape: `bin/larv_review/rules.py:16` `votes` through `:176` `scoring_run_due`: one module named rules holds four concepts
- Shape: `bin/larv_review/rules.py:35` `counted` and `bin/larv_review/history.py:68` `entries`: one name over two things
- Self-sufficiency: `bin/larv_review/seats.py:38` `report` and `bin/larv_review/history.py:99` `heading`: the seats file and history entry shapes are enforced in code
Inherited architect, decided by:
- Navigation, Self-sufficiency, Overall: `bin/larv_review/rules.py:125` `round_counts`, with `bin/larv_review/main.py:159` `round_count`: the 3am question leads to one tested function, where the base had `skills/larv-review/SKILL.md:58` "Count"
- Navigation, Overall: `README.md:70` "Tools": one hop from a symptom to a file
- Locality: `bin/larv_review/main.py:238` `record_scoring`: four helpers over four sources; why `fell` is empty here and not at `bin/larv_review/main.py:168` is stated nowhere at the call
- Locality: `bin/larv_review/rules.py:83` `standing_ratchet` with `bin/larv_review/history.py:12` `RATCHET`: state passes between commands through a markdown file's wording
- Locality, Shape: `bin/larv_review/rules.py:176` `scoring_run_due` and `:192` `emptied`: the due rule packs five conditions
- Shape: `bin/larv_review/rules.py:1` the module: one name over many concepts
- Shape: `bin/metric_bench/main.py:38` and `bin/metric_bench/bench.py:7`: the history format has no home of its own; `bin/larv_review/git.py:7` `HISTORY` sits in a module named for git
- Self-sufficiency: `bin/larv_review/main.py:1` the module docstring: dense, but it ships with the code and states every rule
Maintainability senior, decided by:
- Navigation, Self-sufficiency, Overall: `bin/larv_review/rules.py:125` `round_counts`, with `bin/larv_review/main.py:159` `round_count`: the rule is one small tested function, where the base had `skills/larv-review/SKILL.md:58` "Count"
- Navigation: `README.md:66` "Tools": lists both tools with a pointer to each `--help`
- Self-sufficiency: `bin/larv_review/rules.py:35` `counted`, `:50` `ruling` and `:60` `fallen`: each rule stated in a docstring and enforced beside it, where the base had `skills/larv-review/SKILL.md:100` "Ratchet"
- Locality: `bin/larv_review/main.py:238` `record_scoring`: `due_reasons` with an empty fall, then `pending_fall` separately; four places to hold
- Locality: `bin/larv_review/rules.py:71` `pending_fall` and `:83` `standing_ratchet`: order dependence between commands through a file
- Locality, Shape: `bin/larv_review/seats.py:38` `report`, `bin/larv_review/history.py:99` `heading` and `bin/larv_review/main.py:267` `record_moved_mean`: a one-time migration threaded through three modules
- Shape: `bin/metric_bench/main.py:38` and `bin/metric_bench/bench.py:7`: a second tool depends on the first one's parser; `Entry.places` at `bin/larv_review/history.py:40` exists only for it
- Shape: `bin/larv_review/rules.py:176` `scoring_run_due`: a grab-bag module, offset by the truer `skills/comprehension-panel/seat-prompts.md:1` name
- Self-sufficiency: `bin/larv_review/main.py:11` the `ruling-base` paragraph: took three reads, resolved only by `bin/larv_review/rules.py:134` `measured_commit`
Junior A, decided by:
- Navigation: `bin/larv_review/rules.py:125` `round_counts`: the symptom leads to the `round-count` command, where the base had `skills/larv-review/SKILL.md:58` "Count"
- Navigation: `README.md:68` "Tools": names both tools; `skills/larv-review/SKILL.md:46` points at the same tool
- Locality: `bin/larv_review/main.py:238` `record_scoring`: whether the run blocks, ratchets or fails depends on three outside sources
- Locality: `bin/larv_review/rules.py:134` `measured_commit`: the base depends on what each past merged change added, by heading time
- Locality: `bin/metric_bench/main.py:38` `entries, Entry`: a second tool parses the history through larv-review's internals
- Shape: `bin/larv_review/rules.py:176` `scoring_run_due`: a catch-all module, offset by the clean split and the truer `skills/comprehension-panel/seat-prompts.md:1`
- Self-sufficiency: `skills/larv-review/SKILL.md:98` "Ruling": the skill needs `bin/larv_review/main.py:1` open beside it, offset by rules now in code
## 2026-10-09T09:47:25Z, PR #232 at 324d650
Ratchet: the paired ruling counts as worse, as Locality fell and Locality is 6.50, under the floor of 7.0.
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Junior A | 7 | 6 | 6.5 | 6 | 6.5 |
| Mid A | 7 | 7 | 6.5 | 6.5 | 7 |
| Maintainability senior | 7.5 | 6.5 | 7 | 7 | 7 |
| Inherited architect | 7 | 6.5 | 6.5 | 7 | 7 |
| Mean | 7.13 | 6.50 | 6.63 | 6.63 | 6.88 |
Junior A, hardest first:
- `bin/larv_review/main.py:238` `record_scoring`: `due_reasons` with an empty fall means "reasons besides a fall", stated only at `bin/larv_review/rules.py:93` `floor_verdict`
- `bin/larv_review/main.py:1` the module docstring: 80 lines of interlocking rules that never say why
- `bin/larv_review/rules.py:134` `measured_commit`: `not (last and last.fell)` is true with no ruling at all
- `bin/larv_review/rules.py:16` `votes`, with `:35` `counted` and `:40` `names_a_place`: the counting rule spans three files
- `bin/larv_review/history.py:62` `entries` and `:75` `seat_places`: the parser and writer share a format by eye
- `bin/loop_claude/cycle/judge.py:31` `run_judge`: five outcomes, the spec a prompt string
- `bin/loop_claude/transcript_archive.py:30` `archive`: reading-cost parses its outcome text
- `bin/metric_bench/bench.py:87` `resolved`: order-dependent legacy shim
Mid A, hardest first:
- `bin/larv_review/main.py:238` `record_scoring`: six exits; `due_reasons` with an empty fall list on purpose
- `bin/larv_review/rules.py:134` `measured_commit`: states what, never why
- `bin/larv_review/rules.py:176` `scoring_run_due`: five inputs in one dense sentence
- `bin/loop_claude/cycle/judge.py:31` `run_judge`: two text checks against two texts
- `bin/loop_claude/transcript_archive.py:30` `archive`: a prose format that is a contract with another tool
- `bin/loop_claude/cycle/states.py:69` `transition`: case order decides behaviour
- `bin/larv_review/rules.py:16` `votes`: parse also enforces the ruling protocol; `counted` shadowed
- `bin/loop_claude/claude_code/entries.py:45` `string_by_human`: heuristics on who wrote an entry
Maintainability senior, hardest first:
- `bin/larv_review/main.py:238` `record_scoring`: ten lines, six ideas; `due_reasons`'s third argument means different things to its two callers
- `bin/larv_review/rules.py:134` `measured_commit`: the no-ruling case deduced only from the help
- `bin/larv_review/rules.py:176` `scoring_run_due`: a 70-word docstring and a second document open
- `bin/larv_review/rules.py:16` `votes`: Overall's rule hidden in `DIMENSIONS[-1]`; `None` means split
- `bin/larv_review/history.py:62` `entries`: eight positional arguments counted against a regex
- `bin/larv_review/history.py:99` `heading`: a one-off migration threaded through three files
- `bin/metric_bench/bench.py:87` `resolved`: depends on earlier bullets' order
- `bin/loop_claude/cycle/states.py:69` `transition`: case order, resolved by comments
Inherited architect, would restructure first:
- `bin/loop_claude/transcript_archive.py:111` `finished_chunks`, `bin/reading_cost/archive.py:10` `project_archive` and `bin/loop_claude/claude_code/entries.py:5` `parse_entry`: lift the shared formats out of loop_claude
- `bin/larv_review/rules.py:125` `round_counts` and `:176` `scoring_run_due`: split rules into rounds, ruling, base and scoring
- `bin/loop_claude/cycle/judge_reply.py:12` `JUDGE_RULES` and `bin/loop_claude/cycle/worker.py:41` `worker_args`: give the prompt text one home
- `bin/failure.py:5` `Failure`, `bin/file_writes.py:21` `replace_file` and `bin/markdown_sections.py:6` `sections`: tidy the bin root
- `bin/larv_review/git.py:10` `git`, `bin/metric_bench/main.py:143` `git` and `bin/reading_cost/main.py:113` `located`: one git runner
- `bin/loop_claude/project/checkout.py:32` `read_utf8`: a generic reader under a lying name
- `bin/larv_review/main.py:101` `main`: an if/elif dispatch where the next command lands
+4 -1
View File
@@ -52,6 +52,7 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
- 2026-10-08, PR #225: 7.25, 6.75, 7.00, 6.88, 6.88. Above PR #224's run on every score but Shape, which held.
- 2026-10-08, PR #228: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet: the seats blamed the new ratchet's own rules, since moved into the tested `bin/larv-review`.
- 2026-10-08, PR #229: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet for PR #228's reason: the seats' decisive places were still PR #228's rules, since moved into `bin/larv-review`.
- 2026-10-09, PR #232: 7.13, 6.50, 6.63, 6.63, 6.88, at 324d650, whose ruling the fall made worse. Its last paired ruling, at 53014f8 against 12d6d75, split: Mid A better, the inherited architect worse, both with Locality worse. lilleman shipped it below the ratchet: after a restructure the seats still weigh PRs #229 to #231's code against the base, and their Locality places are filed under items 62 and 83.
## loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
@@ -65,10 +66,12 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
2026-10-08, lilleman, replacing the 2026-09-30 rule that every fix, a nit's included, re-enters review. Serves goal 1.4. An audit of the four costliest archived chunks (2026-10-08) found nearly every finding that mattered in a pass's first round; later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes keep changing nothing a user would notice.
## review: a paired ruling against the last measured commit decides the ratchet; the four-seat scoring run checks the floor when a release's items run out or an item names the floor
## review: a paired ruling against the last measured commit decides the ratchet, and counts as worse when a dimension it rules fallen is under the floor; the four-seat scoring run checks the floor when a release's items run out or an item names the floor, and also runs when a ruling names a fallen dimension
2026-10-08, lilleman. Serves goals 1.1 and 1.4. Two independent four-seat runs moved up to 0.4 on near-identical code, so a fall within that noise forced rounds that changed nothing (PR #224). A seat reading both versions judges the difference on the same material, so the noise the versions share cancels. No conventional significance level lets a sign test settle with four votes, so the human chose a lead of two votes over every other ruling, same included. Two seats that agree settle it, so a PR that leaves the reading as it was costs two seats. A third seat alone can never settle it, so the third and fourth launch together. The base is the default branch's newest merged change that adds a scoring run to the history, or whose last added paired ruling is better. The human chose the last measured commit on the architect's finding that with a merge base as the base, chunks without an architecture pass, and a suspended restructure's steps, could each lower comprehension unseen. A ruling of same leaves the base where it was, or a run of small falls each ruled same would add up unseen. A scoring run moves it too: otherwise a project never ruled better would compare against its first ruling forever, and a release's floor check is where the human sees the absolute level. Reading the default branch one merged change at a time keeps the rule working for merge commits and squash merges alike: a branch's own early ruling of better never counts, only the merged change's last ruling. The scoring run also runs on a chunk shipping an item that names the floor, so the chunk that lifts a project to the floor can prove it. A chunk that removes a release's last item without shipping it runs it too, so no release ends unmeasured. Valid while seats reading both versions agree more often than two absolute runs do.
2026-10-09, lilleman: each seat also rules each dimension, and a dimension falls by the same lead of two. A better or same ruling naming a fallen dimension calls the scoring run, and counts as worse when that dimension's mean is under the floor, so between releases a project cannot trade one dimension below the floor for gains in the other three. The four scoring seats' scores go into the ruling itself, which `record-ruling` decides in one call, so no command reads a fall another command recorded. Those scores block on the fallen dimensions alone, the floor's own block staying with a release's end and an item naming the floor, and they leave the base where the ruling leaves it: a run of falls each ruled same must not add up unseen. PR #232's own history holds one fall-called scoring run recorded on its own, before this rule; it moves the base like any scoring run, which is safe under it, since a later fall of a dimension under the floor is always caught. A dimension's vote counts only where one of the seat's places opens with that dimension, so a fall rests on places the seats name.
## review: a project decision may suspend the comprehension check for the plan items it names
2026-10-04, lilleman. Serves goal 1.4. In a restructure, each step's scoring run measures code the next step deletes. It costs a run and, below the floor, blocks the step, yet says nothing about what the restructure buys. The scoring run after the last step measures that. 2026-10-08: the paired ruling replaced the scoring run in most chunks, and the suspension covers it too; the ruling after the last step compares with the commit measured before the first step. Valid while such a decision names its items and the plan item that ends it.
+3 -3
View File
@@ -84,11 +84,11 @@ Your harness pre-loads these files into your context before your first tool call
No running anything, no builds, no tests, no edits of any kind.
Rule whether the project as a whole reads better, the same or worse at {head} than at {base}, judged on these four dimensions:
Rule whether the project reads better, the same or worse at {head} than at {base} on each of these four dimensions, and as a whole:
{dimensions}
Same means a reader of either version would find it as easy to understand and change. Do not manufacture a difference.
Report, in this shape:
1. Your ruling: better, same or worse.
2. The places that decided it, most decisive first. Each: path:line and the unit's name, in the version the place is in, and one or two sentences on what reads easier or harder there than in the other version. A better or worse ruling with no named place is discarded.
1. Your rulings, each better, same or worse, in this shape: Navigation <ruling>, Locality <ruling>, Shape <ruling>, Self-sufficiency <ruling>, Overall <ruling>
2. The places that decided them, most decisive first. Each opens with the names from your rulings line that it moves, joined by ", " and followed by ": ", such as `Locality, Shape: `. Then `path:line` and the unit's name, each in backticks, in the version the place is in, and one or two sentences on what reads easier or harder there than in the other version. A dimension's ruling that no place opens with counts as same, and the overall ruling does unless you name a place.
```
+5 -7
View File
@@ -94,30 +94,28 @@ Then the comprehension check: simulated readers, launched per `~/.claude/skills/
- **Base.** The last measured commit, which `~/.claude/bin/larv-review ruling-base` prints.
- **Head.** Before each ruling, merge `origin/<default branch>` into this branch unless the head already holds it, so the seats see only this branch's change on top of the base.
- **Seats.** Mid A and the inherited architect. Write what they read and their replies into a seats file in your scratchpad, in the shape `~/.claude/bin/larv-review --help` shows, and run `~/.claude/bin/larv-review record-ruling <file>`. On `Split:`, launch the maintainability senior and Junior A together, add their lines, and run it again.
- **Seats.** Mid A and the inherited architect. Write what they read and their replies into a votes file in your scratchpad, in the shape `~/.claude/bin/larv-review --help` shows, and run `~/.claude/bin/larv-review record-ruling <votes file>`. On `Split:`, launch the maintainability senior and Junior A together, add their lines, and run it again. On `Fell:`, launch the four scoring seats, below, on the same commit, write their lines into a scores file, and run `record-ruling <votes file> <scores file>`.
- **Ruling.** The `Ruling:` line `record-ruling` prints and appends to the history.
**Scoring run**, when `~/.claude/bin/larv-review needs-scoring-run <the IDs of the items the chunk ships, if any>` prints `Scoring run due:`. Run `needs-scoring-run` before removing the chunk's items from `todo.md`. Junior A, Mid A, the maintainability senior and the inherited architect each score the project at the head from 1 to 10 overall and on four dimensions: Navigation, Locality, Shape and Self-sufficiency. Write their lines into a seats file and run `~/.claude/bin/larv-review record-scoring <file>`, which appends the run to the history and prints the four seats' means.
**Scoring run**, when `~/.claude/bin/larv-review needs-scoring-run <the IDs of the items the chunk ships, if any>` prints `Scoring run due:`. Run `needs-scoring-run` before removing the chunk's items from `todo.md`. Junior A, Mid A, the maintainability senior and the inherited architect each score the project at the head from 1 to 10 overall and on four dimensions: Navigation, Locality, Shape and Self-sufficiency. Write their lines into a scores file and run `~/.claude/bin/larv-review record-scoring <scores file> <the same item IDs>`, which appends the run to the history and prints the four seats' means. Where a fall's scores file already read the head, run `record-scoring` on that file.
The chunk does not merge while either holds:
- **Ratchet.** The paired ruling is worse.
- **Floor.** `record-scoring` names a mean under the floor of 7.0.
- **Floor.** `record-scoring` prints `Floor:`, naming means under the floor of 7.0.
Each is a 🔴 of the comprehension check. It is never filed, and stands until a later run clears it or, for the floor, a decision to ship answers it.
Fix a worse ruling at the places the seats voting worse name, and the floor at the places the seats name under the lowest dimension. Under the floor alone, the user may decide to ship instead. A decision that lets every chunk ship under the floor, held in the project's `docs/decisions.md` before the chunk began, counts as theirs while it stands. A decision to ship listed in the `comprehension floor:` entry covers only the chunk it names. No user reachable → fix. A worse ruling never ships: after three rounds, "Round cap" → "Cap" asks the user for another round.
Fix a worse ruling at the places the seats voting worse name, or, where a fallen dimension made it worse, at the places that open with that dimension, and the floor at the places the seats name under the lowest dimension. When only the Floor bullet holds, the user may decide to ship instead. A decision that lets every chunk ship under the floor, held in the project's `docs/decisions.md` before the chunk began, counts as theirs while it stands. A decision to ship listed in the `comprehension floor:` entry covers only the chunk it names. No user reachable → fix. A worse ruling never ships: after three rounds, "Round cap" → "Cap" asks the user for another round.
Two files of the project keep the records:
- **History.** `docs/comprehension-history.md` keeps every paired ruling and scoring run, which `larv-review` appends; never edit one. Appending re-enters no pass. Resolve a merge conflict in the file by putting the default branch's entries first.
- **Entry.** `docs/decisions.md` keeps one entry whose title opens `comprehension floor:`. It holds each decision to ship under the floor, with its scores.
A `comprehension floor:` entry written before the history may still hold a scoring run's five scores. The next scoring run moves them, and only them, into the history as its first run. Write a seats file of two lines and run `~/.claude/bin/larv-review record-scoring` on it. The first line is `<the date or time the entry gives>, <what the entry names the run by> at <commit>`, the commit being the one the entry names, or else the one that wrote those scores into it. The second is `Mean:` and the five scores, in the shape `--help` shows. The rest of the entry stays.
Put each ruling with its base and votes, and each run's five scores, in the summary. List in the entry each decision to ship under the floor, and file its places in `todo.md` per `~/.claude/CLAUDE.md` → "Todo items". A project without the entry gets it at its first scoring run, titled `comprehension floor: every dimension and the overall at 7.0 or above` and applying KISS (`~/.claude/principles/technical-principles.md`).
**Fresh.** Where the chunk ran a comprehension check, the architecture pass is done only once its last paired ruling, and its last scoring run where it ran one, read the commit to merge. They may predate commits that change only `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`. Any other commit after them, a prose fix included, because the seats read the docs, calls for a rescore: the comprehension check again, once every other pass is done, the auto-reviewer included. A rescore reporting a 🔴 that no decision to ship answers is an architecture round. Any other rescore is not a round: append only its results to the history, in a commit titled `Rescore: …`. A `Rescore:` commit re-enters no pass: push it and wait for the forge's checks (§4 step 3), not for an auto-review.
**Fresh.** Where the chunk ran a comprehension check, the architecture pass is done only once its last paired ruling, and its last scoring run where one is due, read the commit to merge. They may predate commits that change only `docs/decisions.md`, `docs/comprehension-history.md`, `docs/reading-cost.md`, `todo.md` or `CHANGELOG.md`. Any other commit after them, a prose fix included, because the seats read the docs, calls for a rescore: the comprehension check again, once every other pass is done, the auto-reviewer included. A rescore reporting a 🔴 that no decision to ship answers is an architecture round. Any other rescore is not a round: append only its results to the history, in a commit titled `Rescore: …`. A `Rescore:` commit re-enters no pass: push it and wait for the forge's checks (§4 step 3), not for an auto-review.
**Suspended.** A decision in the project's `docs/decisions.md`, held before the chunk began, may name plan items to ship without a comprehension check. A named item counts as `todo.md` worded it before the chunk began. The decision must name the later plan item whose chunk ends the suspension with a comprehension check. A chunk may ship named items only with each other. Such a chunk skips the comprehension check, so neither the ratchet nor the floor can block it, and its summary cites the decision. A chunk due a scoring run runs it, even when every item it ships is named.
+4
View File
@@ -66,6 +66,8 @@ Raised by the principles audit of PR #215 against KISS, in `principles/technical
PR #225's last run, at 8c9bf09, put Locality at 6.75 and Self-sufficiency and the overall at 6.88. Its seats named: the judge's policy spread over `loop/main.py`'s docstring, `cycle/judge_reply.py`'s `JUDGE_RULES`, `cycle/judge.py`'s `judge_prompt` and `cycle/worker.py`'s `worker_args`; `loop_claude/claude_code/` and the read side of `transcript_archive.py`, which `bin/reading_cost/` also imports; two meanings of a checkout's root in `project/checkout.py`'s `checkout_root` and `reading_cost/main.py`'s `located`; and `skills/larv-review/SKILL.md` §3's round count, which only prose enforces.
2026-10-09, from two architecture studies on PR #232: give the judge's contract one home. `cycle/judge.py` takes `judge_reply.py`'s rules, schema and `verdict_from`; `QUOTE_SOURCES`, `ANSWER_DESTINATIONS` and `ANSWERS_RECORDED` move to `project/records.py`, so `worker.py` imports the records, not the judge; `states.py`'s two `Judging` cases get exclusive guards, so their order carries no meaning. PR #232's scoring run at 324d650 put Locality at 6.50.
### 21. Give `settings.json`'s model one writer, and sort its keys.
Claude Code rewrites `"model"` in this tracked file whenever a model is picked, so one `/model` dirties the checkout and overrides the synced choice.
@@ -92,6 +94,8 @@ Raised by the principles audit of PR #224. The `settings.json` decision is valid
Raised by the architect on PR #225 and by the inherited architect of PR #224's scoring run: `bin/` is one flat folder of tools, the modules they share and the repo's checks, with `loop_claude`'s tests beside them instead of beside the package. `loop_claude/claude_code/` parses Claude Code's own formats and `bin/reading_cost/measure.py` imports it too, so it is a package of its own.
2026-10-09, from two architecture studies on PR #232: lift `loop_claude/claude_code/` and the archive's layout out of `loop_claude/` into packages of their own, `claude_code/` and `transcript_archive/`, so `reading_cost` imports no tool. `transcript_archive` takes the write side (`archive`), the read side (`finished_chunks`, `FinishedChunk`, the stamps) and one `project_archive(checkout_root, home, env)` that both tools call, which derives the archive key one way, as item 5 asks. Move each tool's tests beside its package, and `test_loop_claude_black_box.host_env`, which three other tools' tests import, into a test helper of its own.
### 5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
The key names the project's session, archive folder and models entry. `project_key` drops the leading `/`, so `~/code/a` and `/a` share all three, and one loop can resume or delete the other's session.