Answer stability round 1: 0 🔴, 6 🟡; pin the averaged overall with a second pair and a run with no Mean row, test a plan record-scoring reads but cannot rank, share rank_file between rank_todo.py and record-scoring, read the history path from HISTORY, make the comments and larv-review's help true

This commit is contained in:
2026-10-18 21:12:19 +02:00
parent 1605d80cc5
commit eafcefe0da
8 changed files with 131 additions and 70 deletions
+1 -2
View File
@@ -2,8 +2,7 @@
## Unreleased
- Every project now shares one achieve goal, code health: the code is easy to understand and change safely. `CLAUDE.md` → "Shared goal" states it, with its signal and counter-signal. `rank_todo.py` ranks it first among a project's achieve goals. It weighs four times as much while the project's overall is under 7.0, or while no run has measured the project: the overall mean of the newest scoring run in `docs/comprehension-history.md`, averaged with every other run on its commit. An item's Moves cell names it as in `code health by 6`. A README voids it with `**Void:** code health, since <reason>.` under `## Goals`, and `rank_todo.py` then refuses an item that names it. `larv-review record-scoring` re-ranks `todo.md` after it appends a run. Expect items that make code easier to read to rank higher in every project. This re-scores every item in a project that keeps code health, so each project's `todo.md` fails `rank_todo.py --check` until `rank_todo.py` ranks it again, and rule-hook holds its next `todo.md` commit until then. Move every machine to this release before ranking a plan another machine shares, since an older `rank_todo.py` scores it without code health and rejects a `code health` cell. A project that opts out of the comprehension check must void code health, or `rank_todo.py` refuses its plan. Then drop each README goal that restates code health, and point the items that cite that goal at `code health`.
- Every project now shares one achieve goal, code health: the code is easy to understand and change safely. `CLAUDE.md` → "Shared goal" states it, with its signal and counter-signal. `rank_todo.py` ranks it first among a project's achieve goals. It weighs four times as much while the project's overall is under 7.0, or while no run has measured the project: the overall mean of the newest scoring run in `docs/comprehension-history.md`, averaged with every other run on its commit. An item's Moves cell names it as in `code health by 6`. A README voids it with `**Void:** code health, since <reason>.` under `## Goals`, and `rank_todo.py` then refuses an item that names it. `larv-review record-scoring` re-ranks `todo.md` after it appends a run, and exits 1 where `rank_todo.py` would, keeping the run. Expect items that make code easier to read to rank higher in every project. This re-scores every item in a project that keeps code health, so each project's `todo.md` fails `rank_todo.py --check` until `rank_todo.py` ranks it again, and rule-hook holds its next `todo.md` commit until then. Move every machine to this release before ranking a plan another machine shares, since an older `rank_todo.py` scores it without code health and rejects a `code health` cell. A project that opts out of the comprehension check must void code health, or `rank_todo.py` refuses its plan. Then, in each project whose README restates code health as a goal of its own, file an item to drop that goal and point the Moves cells citing it at `code health`. Dropping it renumbers the goals after it, so that item also repoints every Moves cell and decision citing them; `rank_todo.py` cannot tell a cell left pointing at the wrong goal.
- `stability-replay <model>` is now `review-replay <pass> <model>`, and replays any pass an agent under `agents/` reviews, such as `review-replay prose claude-opus-5-5`. It now tries every rung below the project's, down to the ladder's lowest. Its runs on the prose and stability passes of Opus 5.5 projects found no rung below the project's that finds every 🔴 and 🟡 the project's rung finds: Sonnet 5.5 at high missed 12 of 27 prose findings and 11 of 20 stability findings. So every pass stays where it runs. A rung a future run finds lowers only its pass, once `REPLAYED` in `bin/model_routing/ladder.py` names it, and only where the chunk's E + I steps no review up and the worker runs no higher than the project's rung; there it may run below the worker's rung.
- `chunk-preflight` names the release it finds finished and ready to cut, or to be planned first, wherever todo.md has named that release since the last cut.
- Each reviewer under `agents/` now writes its full report to a file that larv-review names in the reviewer's prompt, in a folder per chunk under `~/.local/state/larv-review/`. It returns only its verdict and one line per finding, so a review takes less of the worker's context. The worker reads a finding's full entry before it fixes or answers a 🔴 or 🟡. Each reviewer now holds Write. `rule-hook` refuses a reviewer's Edit and MultiEdit, and any Write outside `~/.local/state/larv-review/`; no override waives this rule. Each chunk's folder stays after the merge, so `review-replay` can replay a round that reads an earlier round's report. It replays on a copy of the folder.
+16 -13
View File
@@ -108,30 +108,33 @@ def entries(history: str) -> list[Entry]:
def latest_places(history: str) -> str | None:
"""The places seats found hard since the newest scored commit, under each entry's heading, oldest first; None when no place counts.
The newest scored commit is the commit of the newest scored entry, by time. Its places start at the first scored entry on that commit, so a rerun of one head adds to the run before it. From there, each entry gives every scoring seat's place, and each ruled place opening with a dimension its seat voted worse. With no scored entry, they start at the oldest entry."""
They start at the first of newest_scored's entries, so a rerun of one head adds to the run before it, or at the oldest entry where no entry counts as scoring the project. From there, each entry gives every scoring seat's place, and each ruled place opening with a dimension its seat voted worse."""
found = by_time(entries(history))
newest = next((entry.commit for entry in reversed(found) if scored(entry)), None)
start = next((index for index, entry in enumerate(found) if scored(entry) and entry.commit == newest), 0)
newest = newest_scored(found)
start = found.index(newest[0]) if newest else 0
parts = [placed(entry, worse_places(entry) + labelled_bullets(entry.body, SCORED_PLACES)) for entry in found[start:]]
return "\n".join(part for part in parts if part) or None
def latest_overall(history: str) -> decimal.Decimal | Failure | None:
"""The mean of the overall means of the scoring runs on the newest run's commit, since a rerun of one head measures it again; None when the history holds no run."""
runs = scoring_runs(entries(history))
if not runs:
return None
"""The mean of the overall means of newest_scored's entries, since a rerun of one head measures it again; None where no entry counts as scoring the project."""
overalls = []
for run in (run for run in runs if run.commit == runs[-1].commit):
overall = run_overall(run)
for entry in newest_scored(by_time(entries(history))):
overall = run_overall(entry)
if isinstance(overall, Failure):
return overall
overalls.append(overall)
return sum(overalls, decimal.Decimal(0)) / len(overalls)
return sum(overalls, decimal.Decimal(0)) / len(overalls) if overalls else None
def newest_scored(found: list[Entry]) -> list[Entry]:
"""In found's order, the entries that count as scoring the project on the newest scored commit, the commit of the last such entry; empty where none counts. A commit matches by prefix, since a heading may write it at any length from 7 characters."""
scoring = [entry for entry in found if scored(entry)]
return [entry for entry in scoring if entry.commit.startswith(scoring[-1].commit) or scoring[-1].commit.startswith(entry.commit)] if scoring else []
def run_overall(run: Entry) -> decimal.Decimal | Failure:
"""The overall cell of a scoring run's Mean row."""
"""The overall cell of the Mean row of an entry that counts as scoring the project."""
try:
overall = decimal.Decimal(overall_cells(run).get(MEAN, ""))
except decimal.InvalidOperation:
@@ -145,7 +148,7 @@ def overall_cells(entry: Entry) -> dict[str, str]:
def scoring_runs(found: list[Entry]) -> list[Entry]:
"""The scoring runs among the entries, oldest first. Unlike scored, it leaves out a ruling from before PR #255 that holds scoring seats' places, since such a ruling has no Mean row."""
"""The scoring runs among the entries, oldest first. Unlike scored, it leaves out a ruling from before PR #255 that holds scoring seats' places: there overall_cells reads each voting seat's vote, not its score."""
return [entry for entry in by_time(found) if not entry.paired]
@@ -154,7 +157,7 @@ def by_time(found: list[Entry]) -> list[Entry]:
def scored(entry: Entry) -> bool:
"""Whether the entry scored the project: a scoring run, or a ruling from before PR #255 that holds scoring seats' places."""
"""Whether the entry counts as scoring the project: a scoring run, or a ruling from before PR #255 that holds scoring seats' places."""
return not entry.paired or bool(entry.scored_places)
+7 -5
View File
@@ -72,8 +72,9 @@ prints `Ruling:` and appends the ruling.
record-scoring takes the scores file of a run needs-scoring-run calls, whose first line names no
`, against`, and the same item IDs, and exits 1 when they call no run. It appends the seats' scores
and their means, rounded half up to two decimals, prints the means and, when a mean is under the
floor of {floor}, a line opening `Floor:` naming each one. Then it runs rank_todo.py on the
checkout's todo.md, since the overall sets the weight of code health, the goal every project shares.
floor of {floor}, a line opening `Floor:` naming each one. Then it ranks the checkout's todo.md as
rank_todo.py does, since the overall sets the weight of code health, the goal every project shares,
and exits 1 where rank_todo.py would, with the run recorded.
places prints the places seats found hard since the newest scored commit: the places of each scoring
run on it, and,
@@ -92,9 +93,10 @@ time and the seats file's first line. Each commit that line names must be in the
when the history already holds an entry on what that line reads.
Exit 1, saying why, on unknown arguments, an unknown item ID, a seats file it cannot read or that
breaks the shape above, or when git cannot start, finds no checkout or no default branch, cannot
fetch origin for ruling-base, needs-scoring-run or record-scoring, or cannot read or write the
history, todo.md or the review folder.
breaks the shape above, a todo.md that fails rank_todo.py after record-scoring records a run, or
when git cannot start, finds no checkout or no default branch, cannot fetch origin for ruling-base,
needs-scoring-run or record-scoring, or cannot read or write the history, todo.md or the review
folder.
"""
import pathlib
import re
+5 -14
View File
@@ -54,28 +54,19 @@ def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> F
def rerank() -> Failure | None:
"""Rank the checkout's todo.md again, since the overall sets code health's weight. The run stays recorded whatever happens, so a failure says to rank todo.md, not to record the run again."""
"""Rank the checkout's todo.md again as rank_todo.py does, since the overall sets code health's weight, and fail where rank_todo.py would. The run stays recorded whatever happens, so those failures say to settle todo.md and run rank_todo.py on it, not to record the run again."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
todo = pathlib.Path(checkout, "todo.md")
texts = rank_todo.read_all(todo)
if isinstance(texts, Failure):
return unranked(todo, texts)
ranked = rank_todo.rank(*texts)
ranked = rank_todo.rank_file(todo)
if isinstance(ranked, Failure):
return unranked(todo, ranked)
if ranked.text != texts[0] and (unwritten := rank_todo.write(todo, ranked.text)):
return Failure(f"recorded the run, but {unwritten.reason}; run rank_todo.py on {todo}")
for line in ranked.notes + ranked.under_bar:
print(f"{todo}: {line}")
return Failure(f"recorded the run, but rank_todo.py cannot rank {todo}: {ranked.reason}. Repair it and run rank_todo.py on it, not record-scoring again")
if ranked.under_bar:
return Failure(f"recorded the run and ranked {todo}, but the lines above name items now under the bar. Settle each as its line says, then run rank_todo.py on it, not record-scoring again")
return None
def unranked(todo: pathlib.Path, why: Failure) -> Failure:
return Failure(f"recorded the run, but rank_todo.py cannot rank {todo}: {why.reason}. Repair it and run rank_todo.py on it, not record-scoring again")
def scored_seats(given: SeatsFile) -> ScoringRun | Failure:
found = scores(given.seats)
if isinstance(found, Failure):
+34 -10
View File
@@ -161,6 +161,8 @@ class History(unittest.TestCase):
first, rerun, other = (run.format(time=time, pr=pr, commit=commit, line=line) for time, pr, commit, line in (("2026-10-08T12:00:00Z", "PR #1", "aaaaaaa", 1), ("2026-10-08T13:00:00Z", "PR #1 rerun", "aaaaaaa", 2), ("2026-10-09T12:00:00Z", "PR #2", "bbbbbbb", 3)))
self.assertEqual(latest_places(appended(appended("", first), rerun)), first + "\n" + rerun)
self.assertEqual(latest_places(appended(appended(appended("", first), rerun), other)), other)
in_full = rerun.replace("aaaaaaa", "aaaaaaa" + "0" * 33)
self.assertEqual(latest_places(appended(appended("", first), in_full)), first + "\n" + in_full)
def scored(time: str, pr: str, commit: str, mid: str, architect: str) -> str:
@@ -448,32 +450,54 @@ class LarvReviewTool(unittest.TestCase):
self.assertIn("Inherited architect, would restructure first:", text)
self.assertEqual(entries(text)[0].scored_places, [Place("bin/a.py", "f")] * 4)
def plan(self):
"""Commit and push a README.md and a todo.md that rank, item 1 alone in release 0.1.0, as ranked before any scoring run."""
def plan(self, readme="# P\n\n## Goals\n\n1. **Achieve:** A.\n", bar="0.1"):
"""Commit and push the README.md and a todo.md whose item 1, alone in release 0.1.0, moves code health, each item scored as before any scoring run."""
header = "| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |\n|---|---|---|---|---|---|---|---|---|---|---|---|\n"
# Before any scoring run code health weighs four times its weight at an overall of 7.0: 4 / 1.97 against goal 1's 0.97 / 1.97.
rows = "| 1 | 0.1.0 | | | **Read easier.** | 1 | 1 | 1 | 1 | 1 | code health by 1 | 101.5 |\n| 2 | 0.2.0 | | | **Do A.** | 1 | 1 | 1 | 1 | 1 | 1 by 1 | 24.6 |\n"
self.commit("Plan", {"README.md": "# P\n\n## Goals\n\n1. **Achieve:** A.\n", "todo.md": "# todo\n\n## Scoring\n\n`Score = 100 * V / 2**S`\n\n`Bar = 0.1`\n\n`Next ID = 3`\n\n## Items\n\n" + header + rows})
rows = "| 1 | 0.1.0 | | | **Read easier.** | 1 | 1 | 1 | 1 | 1 | code health by 1 | 101.5 |\n| 2 | 0.2.0 | | | **Do A.** | 1 | 1 | 1 | 1 | 1 | 1 by 3 | 73.9 |\n"
self.commit("Plan", {"README.md": readme, "todo.md": f"# todo\n\n## Scoring\n\n`Score = 100 * V / 2**S`\n\n`Bar = {bar}`\n\n`Next ID = 3`\n\n## Items\n\n" + header + rows})
self.git("push", "-q", "origin", "HEAD:main")
def test_reports_a_plan_it_cannot_rank_yet_keeps_the_run(self):
def record_sevens(self):
"""Run record-scoring for item 1 on a scores file in which every seat scores 7 throughout."""
line = "Navigation 7, Locality 7, Shape 7, Self-sufficiency 7, Overall 7"
seats = "".join(f"\n{seat}: {line}\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
# This checkout's todo.md, beside no README.md, does not rank.
ran = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats), "1")
return self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats), "1")
def test_reports_a_plan_it_cannot_read_yet_keeps_the_run(self):
# This checkout has no README.md beside its todo.md.
ran = self.record_sevens()
self.assertEqual(ran.returncode, 1)
self.assertIn(f"recorded the run, but rank_todo.py cannot rank {self.checkout / 'todo.md'}: {self.checkout / 'README.md'}: not found", ran.stderr)
self.assertIn("not record-scoring again", ran.stderr)
self.assertEqual(len(entries((self.checkout / "docs" / "comprehension-history.md").read_text())), 1)
def test_reports_a_plan_it_reads_but_cannot_rank_yet_keeps_the_run(self):
self.plan(readme="# P\n\n## Goals\n\n**Void:** code health, since it is a probe.\n\n1. **Achieve:** A.\n")
ran = self.record_sevens()
self.assertEqual(ran.returncode, 1)
self.assertIn(f"recorded the run, but rank_todo.py cannot rank {self.checkout / 'todo.md'}: item 1 names code health, which README.md voids", ran.stderr)
self.assertIn("not record-scoring again", ran.stderr)
self.assertEqual(len(entries((self.checkout / "docs" / "comprehension-history.md").read_text())), 1)
def test_reranks_todo_md_by_the_overall_it_records(self):
self.plan()
line = "Navigation 7, Locality 7, Shape 7, Self-sufficiency 7, Overall 7"
seats = "".join(f"\n{seat}: {line}\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
ran = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats), "1")
ran = self.record_sevens()
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertIn("| code health by 1 | 25.4 |", (self.checkout / "todo.md").read_text())
def test_fails_once_its_re_rank_leaves_an_item_under_the_bar_yet_keeps_the_run(self):
self.plan(bar="30")
ran = self.record_sevens()
self.assertEqual(ran.returncode, 1)
todo = self.checkout / "todo.md"
# At an overall of 7.0 item 1 falls from 101.5 to 25.4.
self.assertIn(f"{todo}: item 1 scores 25.4, under the bar 30; delete it, or mark it with a reason the bar does not apply\n", ran.stderr)
self.assertIn("recorded the run", ran.stderr)
self.assertIn("not record-scoring again", ran.stderr)
self.assertIn("| code health by 1 | 25.4 |", todo.read_text())
self.assertEqual(len(entries((self.checkout / "docs" / "comprehension-history.md").read_text())), 1)
def test_prints_the_places_seats_found_hard_since_the_newest_scoring_run(self):
self.assertEqual(self.run_tool("places").stdout, "Seats found no places hard.\n")
self.commit("Record a scoring run", {"docs/comprehension-history.md": appended("", SCORING + "\nJunior A, hardest first:\n\n- `bin/b.py:4` `h`: long\n")})
+24 -16
View File
@@ -77,8 +77,8 @@ A valid todo.md; ## Details and each item's detail are optional:
release is cut. A release whose items all moved elsewhere without a line reads as finished.
Exit 1, saying why, on bad arguments, a file that cannot be read or written, a todo.md, README.md,
docs/decisions.md or comprehension history that breaks the rules above, an item that is not exempt scoring under the bar, or
--check finding the file unranked.
docs/decisions.md or comprehension history that breaks the rules above, an item that is not exempt
scoring under the bar, or --check finding the file unranked.
"""
import ast
import dataclasses
@@ -91,7 +91,7 @@ import typing
from todo_table import BLOCKED_BY, cells, CODE_HEALTH, DETAILS, HEADER_START, Item, item_details, items_table, MOVES, OLD_SHAPE, parse_row, renames, risk, Shape, SHAPE, shape_of, Table, table_rows, version
from failure import Failure
from comprehension_history import FLOOR, latest_overall
from comprehension_history import FLOOR, HISTORY, latest_overall
from markdown_text import listed, sections
from file_writes import replace_file
@@ -130,7 +130,7 @@ class Scoring:
class Ranked:
text: str
under_bar: list[str]
# What ranking prints and --check does not: an old-shape table's call to reshape, a README with no achieve goal, and each dropped blocker ID, since a mistyped ID drops the same way.
# What ranking prints and --check does not: an old-shape table's call to reshape, a README that voids code health and marks no goal **Achieve:**, and each dropped blocker ID, since a mistyped ID drops the same way.
notes: list[str]
@@ -147,19 +147,25 @@ def main(argv: list[str]) -> int:
for line in failures:
print(line, file=sys.stderr)
return 1 if failures else 0
texts = read_all(todo)
if isinstance(texts, Failure):
return fail(texts)
ranked = rank(*texts)
ranked = rank_file(todo)
if isinstance(ranked, Failure):
return fail(ranked)
if ranked.text != texts[0]:
unwritten = write(todo, ranked.text)
if unwritten:
return fail(unwritten)
return 1 if ranked.under_bar else 0
def rank_file(todo: pathlib.Path) -> Ranked | Failure:
"""Rank todo in place: rank what read_all reads, write todo where that changed it, and print the notes and under-bar lines on stderr."""
texts = read_all(todo)
if isinstance(texts, Failure):
return texts
ranked = rank(*texts)
if isinstance(ranked, Failure):
return ranked
if ranked.text != texts[0] and (unwritten := write(todo, ranked.text)):
return unwritten
for line in ranked.notes + ranked.under_bar:
print(f"{todo}: {line}", file=sys.stderr)
return 1 if ranked.under_bar else 0
return ranked
def check_failures(todo: pathlib.Path) -> list[str]:
@@ -189,7 +195,7 @@ def parse_args(argv: list[str]) -> tuple[bool, pathlib.Path] | Failure:
def read_all(todo: pathlib.Path) -> tuple[str, str, str, str] | Failure:
"""todo.md, the README.md beside it, and its docs/decisions.md and docs/comprehension-history.md, each empty when there is none."""
texts = []
for path, missing in ((todo, "every project keeps a todo.md beside its README.md, and rank_todo.py --help shows one"), (todo.parent / "README.md", "rank_todo.py weighs goals from the README.md beside todo.md"), (todo.parent / "docs" / "decisions.md", None), (todo.parent / "docs" / "comprehension-history.md", None)):
for path, missing in ((todo, "every project keeps a todo.md beside its README.md, and rank_todo.py --help shows one"), (todo.parent / "README.md", "rank_todo.py weighs goals from the README.md beside todo.md"), (todo.parent / "docs" / "decisions.md", None), (todo.parent / HISTORY, None)):
try:
texts.append(path.read_text(encoding="utf-8"))
except FileNotFoundError:
@@ -306,12 +312,14 @@ def read_scoring(text: str, lines: list[str], readme: str, decisions: str, histo
def ranked_goals(readme: str, decisions: str) -> list[tuple[str, Mark]] | Failure:
"""Every goal in rank order, each with its mark: code health first, unless README.md voids it, then README.md's goals in reading order. A void of anything else fails, and so does a docs/decisions.md that opts out of the comprehension check where README.md keeps code health, since the goal then has no signal."""
"""Every goal in rank order, each with its mark: code health first, unless README.md voids it, then README.md's goals in reading order. A void spelled any other way fails, and so does a docs/decisions.md that opts out of the comprehension check where README.md keeps code health, since the goal then has no signal."""
goals = readme_goals(readme)
if isinstance(goals, Failure):
return goals
voids = VOID.findall("".join(sections(readme).get("Goals", [])))
if other := [void for void in voids if not void.startswith(f"{CODE_HEALTH}, since ")]:
if other[0].lower().startswith(CODE_HEALTH):
return Failure(f"README.md's ## Goals voids code health as `{other[0]}`; spell the void exactly `**Void:** {CODE_HEALTH}, since <reason>.`")
return Failure(f"README.md's ## Goals voids `{other[0]}`; only code health may be voided, as `**Void:** {CODE_HEALTH}, since <reason>.`")
if voids:
return goals
@@ -468,7 +476,7 @@ def readme_goals(readme: str) -> list[tuple[str, Mark]] | Failure:
def goal_weights(goals: list[tuple[str, Mark]], overall: decimal.Decimal | None) -> dict[str, float] | Failure:
"""Each achieve goal's weight: the one ranked k of n weighs GOAL_STEP ** (k - 1) / (1 + GOAL_STEP + ... + GOAL_STEP ** (n - 1)), so the weights sum to 1. Code health, where goals holds it, then weighs UNDER_FLOOR times that while the overall is under FLOOR or None, and the other goals keep their weights. Each maintain goal weighs as the heaviest achieve goal, since a broken one outranks any achieve goal; only a defect that restores it may name it. Every weight is 0 where README.md marks no goal **Achieve:**, so every item scores V 0 and only an exempt one passes the bar."""
"""Each achieve goal's weight: the one ranked k of n weighs GOAL_STEP ** (k - 1) / (1 + GOAL_STEP + ... + GOAL_STEP ** (n - 1)), so the weights sum to 1. Code health, where goals holds it, then weighs UNDER_FLOOR times that while the overall is under FLOOR or None, and the other goals keep their weights. Each maintain goal weighs as the heaviest achieve goal, since a broken one outranks any achieve goal; only a defect that restores it may name it. Every weight is 0 where goals holds no achieve goal, as where README.md voids code health and marks no goal **Achieve:**, so every item scores V 0 and only an exempt one passes the bar."""
if unmarked := [number for number, mark in goals if mark is None]:
return Failure(f"README.md's goal {unmarked[0]} is marked neither **Maintain:** nor **Achieve:**; mark it, as ~/.claude/rules/goals-and-audience.md says, so rank_todo.py knows which goals score")
achieve = [number for number, mark in goals if mark == "Achieve"]
+43 -9
View File
@@ -256,6 +256,10 @@ class CodeHealth(unittest.TestCase):
ranked = rank_todo.rank(self.todo, readme, "", history)
return ranked if isinstance(ranked, Failure) else {line.split(" | ")[0]: line.rsplit(" | ", 1)[1].rstrip(" |") for line in ranked.text.splitlines() if line.startswith("| ") and " 0.1.0 " in line}
def entry(self, overall, at="2026-10-08T12", read="PR #1 at aaaaaaa"):
"""A scoring run's entry, without the history's title."""
return SCORED.format(overall=overall).removeprefix("# Comprehension history\n\n").replace("2026-10-08T12", at).replace("PR #1 at aaaaaaa", read)
def test_ranks_code_health_first_among_the_achieve_goals_at_an_overall_of_7_or_above(self):
# Code health, then goal 1: weights 1 / 1.97 and 0.97 / 1.97, an item moving one by 1 at S 1 scoring 50 times its weight.
self.assertEqual(self.scores(SCORED.format(overall="7.00")), {"| 1": "25.4", "| 2": "24.6"})
@@ -266,22 +270,36 @@ class CodeHealth(unittest.TestCase):
self.assertEqual(self.scores(history), {"| 1": "101.5", "| 2": "24.6"})
def test_weighs_by_the_newest_scored_commit_wherever_the_file_holds_it(self):
older = SCORED.format(overall="6.50").removeprefix("# Comprehension history\n\n")
newer = SCORED.format(overall="7.25").removeprefix("# Comprehension history\n\n").replace("2026-10-08", "2026-10-10").replace("aaaaaaa", "bbbbbbb")
older = self.entry("6.50")
newer = self.entry("7.25", "2026-10-10T12", "PR #1 at bbbbbbb")
for history in (f"{older}\n{newer}", f"{newer}\n{older}"):
with self.subTest(history=history):
self.assertEqual(self.scores(history)["| 1"], "25.4")
def test_weighs_by_the_mean_overall_of_every_run_on_the_newest_scored_commit(self):
first = SCORED.format(overall="7.13").removeprefix("# Comprehension history\n\n")
rerun = SCORED.format(overall="6.88").removeprefix("# Comprehension history\n\n").replace("2026-10-08T12", "2026-10-08T13").replace("PR #1 at", "PR #1 rerun at")
# 7.13 and 6.88 average 7.005, at the floor or above.
self.assertEqual(self.scores(f"{first}\n{rerun}")["| 1"], "25.4")
later = SCORED.format(overall="6.50").removeprefix("# Comprehension history\n\n").replace("2026-10-08", "2026-10-09").replace("aaaaaaa", "bbbbbbb")
self.assertEqual(self.scores(f"{first}\n{rerun}\n{later}")["| 1"], "101.5")
# 7.13 and 6.88 average 7.005, at the floor or above; 7.10 and 6.80 average 6.95, under it.
for first, rerun, value in (("7.13", "6.88", "25.4"), ("7.10", "6.80", "101.5")):
with self.subTest(first=first, rerun=rerun):
self.assertEqual(self.scores(f"{self.entry(first)}\n{self.entry(rerun, '2026-10-08T13', 'PR #1 rerun at aaaaaaa')}")["| 1"], value)
later = self.entry("6.50", "2026-10-09T12", "PR #2 at bbbbbbb")
self.assertEqual(self.scores(f"{self.entry('7.13')}\n{self.entry('6.88', '2026-10-08T13', 'PR #1 rerun at aaaaaaa')}\n{later}")["| 1"], "101.5")
def test_matches_a_commit_whichever_length_a_heading_writes_it_at(self):
full = "aaaaaaa" + "0" * 33
# As runs of one commit, 7.13 and 6.88 average 7.005; the newer alone is under the floor.
for first, rerun in (("aaaaaaa", full), (full, "aaaaaaa")):
with self.subTest(first=first, rerun=rerun):
self.assertEqual(self.scores(f"{self.entry('7.13', read=f'PR #1 at {first}')}\n{self.entry('6.88', '2026-10-08T13', f'PR #1 rerun at {rerun}')}")["| 1"], "25.4")
def test_weighs_by_a_ruling_that_holds_scoring_seats_places(self):
# A ruling recorded before PR #255 scored the project too: below its votes, a scores table with a Mean row, and each scoring seat's places.
ruling = "## 2026-10-10T12:00:00Z, PR #2 at bbbbbbb, against aaaaaaa\n\nRuling: better\n\n| Seat | Overall |\n|---|---|\n| Mid A | better |\n\n| Seat | Overall |\n|---|---|\n| Mid A | 7.5 |\n| Mean | 7.25 |\n\nJunior A, hardest first:\n\n- `bin/a.py:1` `f`: long\n"
self.assertEqual(self.scores(f"{self.entry('6.50')}\n{ruling}")["| 1"], "25.4")
def test_refuses_a_newest_scoring_run_with_no_overall_mean(self):
self.assertEqual(self.scores(SCORED.format(overall="high")), Failure("docs/comprehension-history.md's scoring run 2026-10-08T12:00:00Z, PR #1 at aaaaaaa has no Mean row ending in its overall"))
for history in (SCORED.format(overall="high"), SCORED.split("| Mean")[0]):
with self.subTest(history=history):
self.assertEqual(self.scores(history), Failure("docs/comprehension-history.md's scoring run 2026-10-08T12:00:00Z, PR #1 at aaaaaaa has no Mean row ending in its overall"))
def test_drops_code_health_where_readme_voids_it(self):
voided = "## Goals\n\n**Void:** code health, since it is a one-off script.\n\n1. **Achieve:** A.\n"
@@ -310,6 +328,11 @@ class CodeHealth(unittest.TestCase):
def test_refuses_a_void_of_anything_but_code_health(self):
self.assertEqual(self.scores("", "## Goals\n\n**Void:** speed, since nobody waits.\n\n1. **Achieve:** A.\n"), Failure("README.md's ## Goals voids `speed, since nobody waits.`; only code health may be voided, as `**Void:** code health, since <reason>.`"))
def test_shows_the_spelling_of_a_void_of_code_health_that_misses_it(self):
for void in ("code health.", "Code health, since it is a one-off script.", "code health since it is a one-off script."):
with self.subTest(void=void):
self.assertEqual(self.scores("", f"## Goals\n\n**Void:** {void}\n\n1. **Achieve:** A.\n"), Failure(f"README.md's ## Goals voids code health as `{void}`; spell the void exactly `**Void:** code health, since <reason>.`"))
def test_parses_code_health_in_a_moves_cell(self):
self.assertEqual(todo_table.parse_moves("1", "code health by 4, 1.1 by 2"), (("code health", 4), ("1.1", 2)))
self.assertIsInstance(todo_table.parse_moves("1", "code health by 11"), Failure)
@@ -559,6 +582,17 @@ class Main(unittest.TestCase):
pathlib.Path(temp, "README.md").unlink()
self.assertEqual(self.run_main(str(todo)), (1, f"{temp}/README.md: not found; rank_todo.py weighs goals from the README.md beside todo.md\n"))
def test_writes_the_ranked_file_yet_fails_on_an_item_under_the_bar(self):
under = "item 2 scores 3.1, under the bar 4; delete it, or mark it with a reason the bar does not apply"
with tempfile.TemporaryDirectory() as temp:
todo = pathlib.Path(temp, "todo.md")
todo.write_text(TODO.replace("`Bar = 2`", "`Bar = 4`"))
pathlib.Path(temp, "README.md").write_text(README)
pathlib.Path(temp, "docs").mkdir()
pathlib.Path(temp, "docs", "decisions.md").write_text(DECISIONS)
self.assertEqual(self.run_main(str(todo)), (1, f"{todo}: {under}\n"))
self.assertEqual(self.run_main("--check", str(todo)), (1, f"{todo}: {under}\n"))
def test_says_to_reshape_an_old_shape_table_but_passes_the_check(self):
with tempfile.TemporaryDirectory() as temp:
todo = pathlib.Path(temp, "todo.md")
+1 -1
View File
@@ -7,5 +7,5 @@ paths:
- **Goals** state the project's aim and intent, not promises: one numbered line each, like slogans, in priority order; a sub-goal is indented under its goal — "1. Blazing fast performance. 2. Correctness is more important than many features." A goal names no step of the work, such as a check to run first, and rarely an implementation. How a goal is met goes in a method section further down the README and names the goal it serves. BANNED as a goal: `Kill weak candidates early, with the cheap checks first.` GOOD: the goal `Keep research on a budget, so it scales in both time and cost.`, with the cheap checks moved to the method section, naming that goal.
- **Mark each goal `**Maintain:**` or `**Achieve:**`.** A maintain goal is a state to hold: a change that breaks it is redesigned or dropped, whatever its value to other goals. An achieve goal is a direction to pursue, weighed against the other achieve goals. Under each achieve goal, three indented bullets: `Signal:` what moving towards it looks like, `Counter-signal:` what shows it is bought at a wrong price, and `Why:` one line on how the two connect. A goal with sub-goals states its own signal. A maintain goal has no bullets. BANNED: an unmarked goal; an achieve goal with no signal; `**Achieve:** Delivery speed.` with the signal `faster`. GOOD: `**Achieve:** Delivery speed.` with `Signal: hours from a chunk's start to its merge fall.`, `Counter-signal: 🔴 findings per chunk rise.` and `Why: speed bought by skipping review shows up as defects.`
- **Leave code health out of the README.** `~/.claude/CLAUDE.md` → "Shared goal" states it for every project, ranked first, and says when and how a README voids it. Drop a README goal that restates it.
- **Leave code health out of the README.** `~/.claude/CLAUDE.md` → "Shared goal" states it for every project, ranked first, and says when and how a README voids it. File an item to drop a README goal that restates it: the goals after it renumber, and every Moves cell and decision citing them must follow.
- **Audience** names who depends on the project and what they may rely on.