Answer principles round 1: 0 🔴, 6 🟡; average the overall over the newest scored commit's runs, re-rank through rank_todo's functions and say not to record again, tie the help to UNDER_FLOOR and FLOOR, add code health to the working loop's premises, loop-claude's first prompt and two more agents, and fix items 114, 124 and 125's details
This commit is contained in:
+1
-1
@@ -2,7 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Every project now shares one achieve goal, code health: the code is easy to understand and change safely. `CLAUDE.md` → "Shared goal" states it, with its signal and counter-signal. `rank_todo.py` ranks it first among a project's achieve goals. It weighs four times as much while the overall of the project's newest scoring run in `docs/comprehension-history.md` is under 7.0, or while no run has measured the project. An item's Moves cell names it `code health by 4`. A README voids it with `**Void:** code health, since <reason>.` under `## Goals`, and `rank_todo.py` then refuses an item that names it. `larv-review record-scoring` re-ranks `todo.md` after it appends a run. Expect items that make code easier to read to rank higher in every project. This re-scores every item in a project that keeps code health, so each project's `todo.md` fails `rank_todo.py --check` until `rank_todo.py` ranks it again, and rule-hook holds its next `todo.md` commit until then. Move every machine to this release before ranking a plan another machine shares, since an older `rank_todo.py` scores it without code health and rejects a `code health` cell. A project that opts out of the comprehension check must void code health, or `rank_todo.py` refuses its plan. Then drop each README goal that restates code health, and point the items that cite that goal at `code health`.
|
||||
- Every project now shares one achieve goal, code health: the code is easy to understand and change safely. `CLAUDE.md` → "Shared goal" states it, with its signal and counter-signal. `rank_todo.py` ranks it first among a project's achieve goals. It weighs four times as much while the project's overall is under 7.0, or while no run has measured the project: the overall mean of the newest scoring run in `docs/comprehension-history.md`, averaged with every other run on its commit. An item's Moves cell names it as in `code health by 6`. A README voids it with `**Void:** code health, since <reason>.` under `## Goals`, and `rank_todo.py` then refuses an item that names it. `larv-review record-scoring` re-ranks `todo.md` after it appends a run. Expect items that make code easier to read to rank higher in every project. This re-scores every item in a project that keeps code health, so each project's `todo.md` fails `rank_todo.py --check` until `rank_todo.py` ranks it again, and rule-hook holds its next `todo.md` commit until then. Move every machine to this release before ranking a plan another machine shares, since an older `rank_todo.py` scores it without code health and rejects a `code health` cell. A project that opts out of the comprehension check must void code health, or `rank_todo.py` refuses its plan. Then drop each README goal that restates code health, and point the items that cite that goal at `code health`.
|
||||
|
||||
- `stability-replay <model>` is now `review-replay <pass> <model>`, and replays any pass an agent under `agents/` reviews, such as `review-replay prose claude-opus-5-5`. It now tries every rung below the project's, down to the ladder's lowest. Its runs on the prose and stability passes of Opus 5.5 projects found no rung below the project's that finds every 🔴 and 🟡 the project's rung finds: Sonnet 5.5 at high missed 12 of 27 prose findings and 11 of 20 stability findings. So every pass stays where it runs. A rung a future run finds lowers only its pass, once `REPLAYED` in `bin/model_routing/ladder.py` names it, and only where the chunk's E + I steps no review up and the worker runs no higher than the project's rung; there it may run below the worker's rung.
|
||||
- `chunk-preflight` names the release it finds finished and ready to cut, or to be planned first, wherever todo.md has named that release since the last cut.
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
How work proceeds in any repo, unless the repo's `AGENTS.md` says otherwise. The *next release* is the earliest one not yet cut. Other words, such as *chunk*, are defined under "Vocabulary" at the end of this file.
|
||||
|
||||
- **Until `README.md` has a `## Goals` and an `## Audience` section, writing them is the only work.** Not even planning in `todo.md`. Both are public: an outside reader judges a project by where it is heading and who it is for. Goals or an audience written anywhere else, such as `AGENTS.md` or `docs/`, do not count.
|
||||
- **Until `README.md` has a `## Goals` and an `## Audience` section, writing them is the only work.** Not even planning in `todo.md`. Both are public: an outside reader judges a project by where it is heading and who it is for. Goals or an audience written anywhere else, such as `AGENTS.md` or `docs/`, do not count; the one exception is code health, which "Shared goal" below states for every project.
|
||||
- **Plan `todo.md` as one scored table, in the shape whose example `~/.claude/bin/rank_todo.py --help` shows**: every item names the release that ships it. Until `todo.md` has that shape, reshaping it is the only work, and the human approves it.
|
||||
- **When the next release has no items and no work has landed towards it, plan it as this chunk.** Move the highest-scoring items of later releases into it, and get the human's approval of that set before writing any code.
|
||||
- **When the next release has no items left and work has landed towards it, it is finished: stop.** Report what it holds and that it is ready to cut. Under `/loop`, this ends the loop. Until the human cuts it, only a security fix lands, into that release; never start a later release's items or plan a new one.
|
||||
@@ -22,7 +22,7 @@ How work proceeds in any repo, unless the repo's `AGENTS.md` says otherwise. The
|
||||
|
||||
## Shared goal
|
||||
|
||||
Every project pursues this achieve goal, ranked first among its README's achieve goals. A project whose code nobody will maintain voids it with one line under `## Goals`, before the numbered goals: `**Void:** code health, since <reason>.` So does a project that opts out of the comprehension check, since the goal then has no signal. An item's Moves cell names it as `code health by 4`, and `rank_todo.py --help` says how much it weighs.
|
||||
Every project pursues this achieve goal, ranked first among its README's achieve goals. A project whose code nobody will maintain voids it with one line under `## Goals`, before the numbered goals: `**Void:** code health, since <reason>.` So does a project that opts out of the comprehension check, since the goal then has no signal. An item's Moves cell names it as `code health by 6`, and `rank_todo.py --help` says how much it weighs.
|
||||
|
||||
**Achieve:** Code health: the code is easy to understand and change safely.
|
||||
- Signal: the comprehension panel's overall score rises, always. Under 7.0 it is very important.
|
||||
|
||||
@@ -46,7 +46,7 @@ Before anything else, **read the shared principles** at `~/.claude/principles/`.
|
||||
|
||||
## Your Output Format
|
||||
|
||||
Report only a defect a persona hits, a requirement of the plan or the README's goals that the change breaks, or a breach of a technical principle. Friction that is none of these is not a finding.
|
||||
Report only a defect a persona hits, a requirement of the plan, the README's goals or code health (`~/.claude/CLAUDE.md` → "Shared goal", unless the README voids it) that the change breaks, or a breach of a technical principle. Friction that is none of these is not a finding.
|
||||
|
||||
Verdict, severities and rounds: ~/.claude/skills/larv-review/review-findings.md. After the verdict:
|
||||
|
||||
|
||||
@@ -129,7 +129,7 @@ Every PR gets at least this sweep.
|
||||
|
||||
## Output Format
|
||||
|
||||
Report only a defect, a requirement of the plan or the README's goals that the change breaks, a breach of a technical principle, and the places the prompt hands you. A finding that is none of these is noise, however true: leave it out.
|
||||
Report only a defect, a requirement of the plan, the README's goals or code health (`~/.claude/CLAUDE.md` → "Shared goal", unless the README voids it) that the change breaks, a breach of a technical principle, and the places the prompt hands you. A finding that is none of these is noise, however true: leave it out.
|
||||
|
||||
Verdict, severities and rounds: ~/.claude/skills/larv-review/review-findings.md. After the verdict, a short **Coverage & Verification** block: what you ran (tests, static analysis, mutation — and their results), what you could not verify and why, and any files that got a lighter pass — never let silence imply coverage. Then the findings, most severe first, each with:
|
||||
- **Location**: File and line/function
|
||||
|
||||
@@ -117,15 +117,26 @@ def latest_places(history: str) -> str | None:
|
||||
|
||||
|
||||
def latest_overall(history: str) -> decimal.Decimal | Failure | None:
|
||||
"""The overall mean of the newest scoring run, by time; None when the history holds none."""
|
||||
"""The mean of the overall means of the scoring runs on the newest run's commit, since a rerun of one head measures it again; None when the history holds no run."""
|
||||
runs = scoring_runs(entries(history))
|
||||
if not runs:
|
||||
return None
|
||||
overalls = []
|
||||
for run in (run for run in runs if run.commit == runs[-1].commit):
|
||||
overall = run_overall(run)
|
||||
if isinstance(overall, Failure):
|
||||
return overall
|
||||
overalls.append(overall)
|
||||
return sum(overalls, decimal.Decimal(0)) / len(overalls)
|
||||
|
||||
|
||||
def run_overall(run: Entry) -> decimal.Decimal | Failure:
|
||||
"""The overall cell of a scoring run's Mean row."""
|
||||
try:
|
||||
overall = decimal.Decimal(overall_cells(runs[-1]).get(MEAN, ""))
|
||||
overall = decimal.Decimal(overall_cells(run).get(MEAN, ""))
|
||||
except decimal.InvalidOperation:
|
||||
overall = None
|
||||
return overall if overall is not None and overall.is_finite() else Failure(f"{HISTORY}'s newest scoring run, {runs[-1].heading}, has no Mean row ending in its overall")
|
||||
return overall if overall is not None and overall.is_finite() else Failure(f"{HISTORY}'s scoring run {run.heading} has no Mean row ending in its overall")
|
||||
|
||||
|
||||
def overall_cells(entry: Entry) -> dict[str, str]:
|
||||
@@ -134,7 +145,7 @@ def overall_cells(entry: Entry) -> dict[str, str]:
|
||||
|
||||
|
||||
def scoring_runs(found: list[Entry]) -> list[Entry]:
|
||||
"""The scoring runs among the entries, oldest first."""
|
||||
"""The scoring runs among the entries, oldest first. Unlike scored, it leaves out a ruling from before PR #255 that holds scoring seats' places, since such a ruling has no Mean row."""
|
||||
return [entry for entry in by_time(found) if not entry.paired]
|
||||
|
||||
|
||||
|
||||
@@ -54,16 +54,28 @@ def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> F
|
||||
|
||||
|
||||
def rerank() -> Failure | None:
|
||||
"""Rank the checkout's todo.md again, since the overall sets code health's weight."""
|
||||
"""Rank the checkout's todo.md again, since the overall sets code health's weight. The run stays recorded whatever happens, so a failure says to rank todo.md, not to record the run again."""
|
||||
checkout = top()
|
||||
if isinstance(checkout, Failure):
|
||||
return checkout
|
||||
todo = pathlib.Path(checkout, "todo.md")
|
||||
if rank_todo.main([str(todo)]):
|
||||
return Failure(f"recorded the run, but rank_todo.py exited 1 on {todo}, saying why above; settle that before committing the run")
|
||||
texts = rank_todo.read_all(todo)
|
||||
if isinstance(texts, Failure):
|
||||
return unranked(todo, texts)
|
||||
ranked = rank_todo.rank(*texts)
|
||||
if isinstance(ranked, Failure):
|
||||
return unranked(todo, ranked)
|
||||
if ranked.text != texts[0] and (unwritten := rank_todo.write(todo, ranked.text)):
|
||||
return Failure(f"recorded the run, but {unwritten.reason}; run rank_todo.py on {todo}")
|
||||
for line in ranked.notes + ranked.under_bar:
|
||||
print(f"{todo}: {line}")
|
||||
return None
|
||||
|
||||
|
||||
def unranked(todo: pathlib.Path, why: Failure) -> Failure:
|
||||
return Failure(f"recorded the run, but rank_todo.py cannot rank {todo}: {why.reason}. Repair it and run rank_todo.py on it, not record-scoring again")
|
||||
|
||||
|
||||
def scored_seats(given: SeatsFile) -> ScoringRun | Failure:
|
||||
found = scores(given.seats)
|
||||
if isinstance(found, Failure):
|
||||
|
||||
@@ -438,10 +438,9 @@ class LarvReviewTool(unittest.TestCase):
|
||||
uncalled = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats))
|
||||
self.assertEqual(uncalled.returncode, 1)
|
||||
self.assertIn("no plan item or release calls this scoring run", uncalled.stderr)
|
||||
self.plan()
|
||||
ran = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats), "1")
|
||||
# This checkout's todo.md, beside no README.md, does not rank, so the run is recorded and the re-rank fails.
|
||||
self.assertEqual(ran.returncode, 1)
|
||||
self.assertIn(f"recorded the run, but rank_todo.py exited 1 on {self.checkout / 'todo.md'}", ran.stderr)
|
||||
self.assertEqual(ran.returncode, 0, ran.stderr)
|
||||
self.assertIn("\nFloor: Locality 6.50, Self-sufficiency 6.50 under the floor of 7.0.\n", ran.stdout)
|
||||
self.assertIn("not /", self.run_tool("record-scoring", "1", self.seats_file(f"PR #2 at {self.start}\n" + seats)).stderr)
|
||||
text = (self.checkout / "docs" / "comprehension-history.md").read_text()
|
||||
@@ -449,12 +448,26 @@ class LarvReviewTool(unittest.TestCase):
|
||||
self.assertIn("Inherited architect, would restructure first:", text)
|
||||
self.assertEqual(entries(text)[0].scored_places, [Place("bin/a.py", "f")] * 4)
|
||||
|
||||
def test_reranks_todo_md_by_the_overall_it_records(self):
|
||||
def plan(self):
|
||||
"""Commit and push a README.md and a todo.md that rank, item 1 alone in release 0.1.0, as ranked before any scoring run."""
|
||||
header = "| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |\n|---|---|---|---|---|---|---|---|---|---|---|---|\n"
|
||||
# Before any scoring run code health weighs four times its weight at an overall of 7.0: 4 / 1.97 against goal 1's 0.97 / 1.97.
|
||||
rows = "| 1 | 0.1.0 | | | **Read easier.** | 1 | 1 | 1 | 1 | 1 | code health by 1 | 101.5 |\n| 2 | 0.2.0 | | | **Do A.** | 1 | 1 | 1 | 1 | 1 | 1 by 1 | 24.6 |\n"
|
||||
self.commit("Plan", {"README.md": "# P\n\n## Goals\n\n1. **Achieve:** A.\n", "todo.md": "# todo\n\n## Scoring\n\n`Score = 100 * V / 2**S`\n\n`Bar = 0.1`\n\n`Next ID = 3`\n\n## Items\n\n" + header + rows})
|
||||
self.git("push", "-q", "origin", "HEAD:main")
|
||||
|
||||
def test_reports_a_plan_it_cannot_rank_yet_keeps_the_run(self):
|
||||
line = "Navigation 7, Locality 7, Shape 7, Self-sufficiency 7, Overall 7"
|
||||
seats = "".join(f"\n{seat}: {line}\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
|
||||
# This checkout's todo.md, beside no README.md, does not rank.
|
||||
ran = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats), "1")
|
||||
self.assertEqual(ran.returncode, 1)
|
||||
self.assertIn(f"recorded the run, but rank_todo.py cannot rank {self.checkout / 'todo.md'}: {self.checkout / 'README.md'}: not found", ran.stderr)
|
||||
self.assertIn("not record-scoring again", ran.stderr)
|
||||
self.assertEqual(len(entries((self.checkout / "docs" / "comprehension-history.md").read_text())), 1)
|
||||
|
||||
def test_reranks_todo_md_by_the_overall_it_records(self):
|
||||
self.plan()
|
||||
line = "Navigation 7, Locality 7, Shape 7, Self-sufficiency 7, Overall 7"
|
||||
seats = "".join(f"\n{seat}: {line}\n" for seat in ("Junior A", "Mid A", "Maintainability senior", "Inherited architect"))
|
||||
ran = self.run_tool("record-scoring", self.seats_file(f"PR #2 at {self.start}\n" + seats), "1")
|
||||
|
||||
@@ -14,11 +14,15 @@ ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could
|
||||
REVIEWS: tuple[tuple[str, ReviewKind], ...] = (*((f"each {agent}", kind) for kind, agent in REVIEWERS.items()), ("each comprehension seat", "seat"), ("the security pass's subagent", "security"))
|
||||
|
||||
|
||||
# The premises rank code health, the goal every project shares, first among the goals the first prompt quotes.
|
||||
AFTER_CODE_HEALTH = 'after code health (~/.claude/CLAUDE.md → "Shared goal") unless README.md voids it'
|
||||
|
||||
|
||||
def fresh(task: str, premises: str, route: Route, anchor: Rung | None, facts: str) -> str:
|
||||
"""A chunk's first prompt; facts is preflight_facts' or preflight_failed's text."""
|
||||
escalated = f"{escalation(route, anchor)}{reruns(route, anchor)}".lstrip()
|
||||
routed = f"{routed_by(route, anchor)}{recount(route)}\n\n" if route.item else (f"{escalated}\n\n" if escalated else "")
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.\n\n{routed}{facts}\n\nThe chunk's todo.md items, or else the task above, set its scope. The goals below choose between ways to deliver that scope, and never widen it.\n\nFrom README.md:\n{premises}"
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.\n\n{routed}{facts}\n\nThe chunk's todo.md items, or else the task above, set its scope. The goals below, {AFTER_CODE_HEALTH}, choose between ways to deliver that scope, and never widen it.\n\nFrom README.md:\n{premises}"
|
||||
|
||||
|
||||
def routed_by(route: Route, anchor: Rung | None) -> str:
|
||||
|
||||
@@ -3,9 +3,10 @@
|
||||
usage: loop-claude <task>
|
||||
|
||||
Run it anywhere inside a project checkout whose README.md has ## Goals and ## Audience, the
|
||||
premises; the task runs at the checkout root. Each cycle a worker runs the task. A chunk's first
|
||||
worker prompt also quotes the premises, which choose between ways to deliver the chunk's todo.md
|
||||
items, or else the task, and never widen them. A judge then reads the transcript's end, the
|
||||
premises, which rank code health (~/.claude/CLAUDE.md → "Shared goal") first among the goals unless
|
||||
README.md voids it; the task runs at the checkout root. Each cycle a worker runs the task. A chunk's
|
||||
first worker prompt also quotes the premises, which choose between ways to deliver the chunk's
|
||||
todo.md items, or else the task, and never widen them. A judge then reads the transcript's end, the
|
||||
premises, and the records: the project's AGENTS.md and docs/decisions.md, and the human's
|
||||
~/.claude/CLAUDE.md, ~/.claude/rules/*.md and
|
||||
~/.claude/skills/larv-review/standards/*.md, where they exist. It rules whether the chunk, the work
|
||||
|
||||
@@ -371,7 +371,7 @@ class Cycles(unittest.TestCase):
|
||||
self.assertIsNone(argument(workers[1], "--name"))
|
||||
self.assertEqual(argument(workers[2], "--session-id"), session)
|
||||
self.assertTrue(argument(workers[0], "-p").startswith(f"{TASK}\n\nContinue any open draft PR"))
|
||||
self.assertTrue(argument(workers[0], "-p").endswith("The chunk's todo.md items, or else the task above, set its scope. The goals below choose between ways to deliver that scope, and never widen it.\n\nFrom README.md:\n## Goals\n1. Work.\n\n## Audience\nMe."))
|
||||
self.assertTrue(argument(workers[0], "-p").endswith("The chunk's todo.md items, or else the task above, set its scope. The goals below, " 'after code health (~/.claude/CLAUDE.md → "Shared goal") unless README.md voids it' ", choose between ways to deliver that scope, and never widen it.\n\nFrom README.md:\n## Goals\n1. Work.\n\n## Audience\nMe."))
|
||||
self.assertIn("main or master", argument(workers[0], "-p"))
|
||||
self.assertNotIn(TASK, argument(workers[1], "-p"))
|
||||
self.assertIn("Even then, it is finished only once every answer the human typed in it", argument(workers[1], "-p"))
|
||||
|
||||
+14
-13
@@ -4,12 +4,12 @@
|
||||
usage: rank_todo.py [--check] [todo.md]
|
||||
|
||||
Reads todo.md (default: ./todo.md), the README.md beside it, and docs/decisions.md and
|
||||
docs/comprehension-history.md when there are. Rewrites each item's Score from the formula and sorts the items by release; within a release,
|
||||
question rows come first, then security rows, then the rest, each group by score from highest, then
|
||||
by ID. Then it moves each blocker, an item another's Blocked by cell names, that ranks below an item
|
||||
it blocks to just above the first such item. Orders the details to match. Writes each item's R, and
|
||||
adds the Blocked by and R columns to a table without them. --check writes nothing and fails when the
|
||||
file is not ranked as it would be.
|
||||
docs/comprehension-history.md where they exist. Rewrites each item's Score from the formula and
|
||||
sorts the items by release; within a release, question rows come first, then security rows, then
|
||||
the rest, each group by score from highest, then by ID. Then it moves each blocker, an item
|
||||
another's Blocked by cell names, that ranks below an item it blocks to just above the first such
|
||||
item. Orders the details to match. Writes each item's R, and adds the Blocked by and R columns to a
|
||||
table without them. --check writes nothing and fails when the file is not ranked as it would be.
|
||||
|
||||
A valid todo.md; ## Details and each item's detail are optional:
|
||||
|
||||
@@ -38,15 +38,16 @@ A valid todo.md; ## Details and each item's detail are optional:
|
||||
- The formula holds numbers, + - * / ** and parentheses over L, E, I, R, S and V.
|
||||
- V sums, over the goals in an item's Moves cell, each goal's weight times how far the item moves it.
|
||||
README.md marks each goal **Maintain:** or **Achieve:**. The achieve goals rank code health first,
|
||||
the goal every project shares, which a Moves cell names as `code health by 4`. README.md's achieve
|
||||
the goal every project shares, which a Moves cell names as `code health by 6`. README.md's achieve
|
||||
goals follow in reading order, a sub-goal such as 1.1 included. Each weighs 0.97 times the one
|
||||
before, and the weights sum to 1. With 5 achieve goals the first weighs 0.212 and the last 0.188.
|
||||
With 15 they weigh 0.082 and 0.053. Code health then weighs 4 times that while the overall of
|
||||
docs/comprehension-history.md's newest scoring run is under 7.0, or there is none. A line
|
||||
With 15 they weigh 0.082 and 0.053. Code health then weighs 4 times that while the overall is
|
||||
under 7.0, and while docs/comprehension-history.md holds no scoring run. The overall is the mean
|
||||
of the overall means of every run on the newest run's commit. A line
|
||||
`**Void:** code health, since <reason>.` in README.md's ## Goals drops code health, and an item
|
||||
may not name it. A project whose docs/decisions.md has a `## review opt-out:` title naming the
|
||||
comprehension check must void it. Only an item marked defect may name a maintain goal, the one it restores, which
|
||||
weighs as the heaviest achieve goal.
|
||||
may not name it. A project must void it where its docs/decisions.md has a `## review opt-out:`
|
||||
title naming the comprehension check. Only an item marked defect may name a maintain goal, the
|
||||
one it restores, which weighs as the heaviest achieve goal.
|
||||
- The bar is judged on the Score as shown, to one decimal.
|
||||
- A new item takes Next ID, and Next ID then goes up by one; every ID stays below it.
|
||||
- An item's L, E and I are levels that ~/.claude/rules/todo-items.md defines: L from 1 to 4, E from 1
|
||||
@@ -320,7 +321,7 @@ def ranked_goals(readme: str, decisions: str) -> list[tuple[str, Mark]] | Failur
|
||||
|
||||
|
||||
def weigh(lines: list[str], history: str, goals: list[tuple[str, Mark]], shape: Shape) -> dict[str, float] | Failure:
|
||||
"""Each goal's weight. In OLD_SHAPE, README.md's goals weigh what todo.md states, and code health nothing. Otherwise goal_weights gives each its weight, at the overall of the history's newest scoring run where the goals hold code health."""
|
||||
"""Each goal's weight. In OLD_SHAPE, README.md's goals weigh what todo.md states, and code health nothing. Otherwise goal_weights gives each its weight, at the overall latest_overall reads where the goals hold code health."""
|
||||
if shape is OLD_SHAPE:
|
||||
return parse_weights(lines, [number for number, _ in goals if number != CODE_HEALTH])
|
||||
if WEIGHTS_HEADER in lines:
|
||||
|
||||
+12
-3
@@ -130,6 +130,7 @@ class Rank(unittest.TestCase):
|
||||
self.assertEqual((round(weights[0], 3), round(weights[-1], 3), round(sum(weights), 6)), (first, last, 1))
|
||||
for figure in (rank_todo.GOAL_STEP, first, last):
|
||||
self.assertIn(str(figure), rank_todo.__doc__)
|
||||
self.assertIn(f"Code health then weighs {rank_todo.UNDER_FLOOR} times that while the overall is\n under {rank_todo.FLOOR},", rank_todo.__doc__)
|
||||
|
||||
def test_scores_an_item_by_the_weights_of_the_goals_it_moves(self):
|
||||
readme = "## Goals\n\n" + VOID + "1. **Achieve:** A.\n 1. **Maintain:** B.\n 2. **Achieve:** C.\n2. **Achieve:** D.\n"
|
||||
@@ -264,15 +265,23 @@ class CodeHealth(unittest.TestCase):
|
||||
with self.subTest(history=history):
|
||||
self.assertEqual(self.scores(history), {"| 1": "101.5", "| 2": "24.6"})
|
||||
|
||||
def test_weighs_by_the_newest_scoring_run_wherever_the_file_holds_it(self):
|
||||
def test_weighs_by_the_newest_scored_commit_wherever_the_file_holds_it(self):
|
||||
older = SCORED.format(overall="6.50").removeprefix("# Comprehension history\n\n")
|
||||
newer = SCORED.format(overall="7.25").removeprefix("# Comprehension history\n\n").replace("2026-10-08", "2026-10-10")
|
||||
newer = SCORED.format(overall="7.25").removeprefix("# Comprehension history\n\n").replace("2026-10-08", "2026-10-10").replace("aaaaaaa", "bbbbbbb")
|
||||
for history in (f"{older}\n{newer}", f"{newer}\n{older}"):
|
||||
with self.subTest(history=history):
|
||||
self.assertEqual(self.scores(history)["| 1"], "25.4")
|
||||
|
||||
def test_weighs_by_the_mean_overall_of_every_run_on_the_newest_scored_commit(self):
|
||||
first = SCORED.format(overall="7.13").removeprefix("# Comprehension history\n\n")
|
||||
rerun = SCORED.format(overall="6.88").removeprefix("# Comprehension history\n\n").replace("2026-10-08T12", "2026-10-08T13").replace("PR #1 at", "PR #1 rerun at")
|
||||
# 7.13 and 6.88 average 7.005, at the floor or above.
|
||||
self.assertEqual(self.scores(f"{first}\n{rerun}")["| 1"], "25.4")
|
||||
later = SCORED.format(overall="6.50").removeprefix("# Comprehension history\n\n").replace("2026-10-08", "2026-10-09").replace("aaaaaaa", "bbbbbbb")
|
||||
self.assertEqual(self.scores(f"{first}\n{rerun}\n{later}")["| 1"], "101.5")
|
||||
|
||||
def test_refuses_a_newest_scoring_run_with_no_overall_mean(self):
|
||||
self.assertEqual(self.scores(SCORED.format(overall="high")), Failure("docs/comprehension-history.md's newest scoring run, 2026-10-08T12:00:00Z, PR #1 at aaaaaaa, has no Mean row ending in its overall"))
|
||||
self.assertEqual(self.scores(SCORED.format(overall="high")), Failure("docs/comprehension-history.md's scoring run 2026-10-08T12:00:00Z, PR #1 at aaaaaaa has no Mean row ending in its overall"))
|
||||
|
||||
def test_drops_code_health_where_readme_voids_it(self):
|
||||
voided = "## Goals\n\n**Void:** code health, since it is a one-off script.\n\n1. **Achieve:** A.\n"
|
||||
|
||||
+1
-1
@@ -142,7 +142,7 @@ Valid while the ladder orders its rungs by competence.
|
||||
|
||||
2026-10-16, lilleman, answering the review of PR #283. An item that restores a maintain goal the project breaks today is a defect: exempt as `defect`. Its Moves name that maintain goal and how much of it is broken, and the goal weighs as the heaviest achieve goal, since a broken maintain goal outranks any achieve goal. Every defect ranks by its score, so a trivial one waits behind valuable work. Only questions and then security findings rank first in their release: a question's answer may change other items, and a question waits for the human anyway, so ranking it first delays no chunk. An item names the most specific goal it moves, and a parent only for a move none of its sub-goals covers, so a sub-goal's move does not count twice; `rank_todo.py` refuses a goal named beside its own sub-goal. Each project reshapes its own `todo.md`; `CLAUDE.md` → "Working loop" makes the reshape the only work until done, as it makes writing missing premises. An item that makes this project easier to read moves code health, as the 2026-10-18 paragraph below says, and goal 1.4, since reading cost is measured in tokens per chunk. Cost gains a fixed part, F in `Score = 100 * V / (2**S + F)`: every chunk carries a cost its size does not shrink, the reading before its first commit, about 130k tokens, and its review passes, so `2**S` alone overrated tiny items. F is 8, about one S 3 change, until `todo.md` item 158 measures it.
|
||||
|
||||
2026-10-18, lilleman. Serves goal 1.1: every project shares one achieve goal, code health, which `CLAUDE.md` → "Shared goal" states once and which replaces each project's own readability goal. It ranks first among a project's achieve goals, since under 7.0 it is very important. While the overall of the newest scoring run in `docs/comprehension-history.md` is under 7.0, and while no scoring run has measured the project, it weighs four times its weight at 7.0 or above, as `UNDER_FLOOR` sets. The other achieve goals keep their weights. Crossing 7.0 changes the scores of the items that move code health, and of the defects, since a broken maintain goal weighs as the heaviest achieve goal; `larv-review record-scoring` re-ranks `todo.md` after it appends a run. A project that opts out of the comprehension check voids the goal, as `review: a project decision may opt every chunk out …` records. A README voids the goal with `**Void:** code health, since <reason>.` under `## Goals`, and an item's Moves cell names it `code health by 4`. A panel of three readers seated as lilleman picked both 3-0: `code health` over `code`, `health`, `C` and `0`, and the bold `Void:` line over a sentence under `## Goals` or a line under `## Audience`. The counter-signal is stated now and counted once `todo.md` item 198 ships, since no tool yet records why a finding arose. Valid while the comprehension panel's overall is how a project measures how hard it is to read.
|
||||
2026-10-18, lilleman. Serves goal 1.1: every project shares one achieve goal, code health, which `CLAUDE.md` → "Shared goal" states once and which replaces each project's own readability goal. It ranks first among a project's achieve goals, since under 7.0 it is very important. While the overall is under 7.0, and while no scoring run has measured the project, it weighs four times its weight at 7.0 or above, as `UNDER_FLOOR` sets. The overall is the mean of the overall means of every run on the commit of the newest run in `docs/comprehension-history.md`: two runs of one head measure it twice, and the newest alone would let their noise flip the weight, as 7.13 and 6.88 on one plainpages-plugin-fastjira head would have, a choice the review of PR #324 raised. The other achieve goals keep their weights. Crossing 7.0 changes the scores of the items that move code health, and of the defects, since a broken maintain goal weighs as the heaviest achieve goal; `larv-review record-scoring` re-ranks `todo.md` after it appends a run. A project that opts out of the comprehension check voids the goal, as `review: a project decision may opt every chunk out …` records. A README voids the goal with `**Void:** code health, since <reason>.` under `## Goals`, and an item's Moves cell names it as in `code health by 6`. A panel of three readers seated as lilleman picked both 3-0: `code health` over `code`, `health`, `C` and `0`, and the bold `Void:` line over a sentence under `## Goals` or a line under `## Audience`. The counter-signal is stated now and counted once `todo.md` item 198 ships, since no tool yet records why a finding arose. Valid while the comprehension panel's overall is how a project measures how hard it is to read.
|
||||
|
||||
2026-10-16, lilleman, ruling on the plan in PR #283: release 0.2.0 takes plan items 146, 152, 73, 50, 65, 84, 153 and 67, the split of `routing: one ladder`, and the question of how many chunks `score-outcomes` and the reading-cost trend wait for, ahead of their scores, to put budget and efficiency in the other projects first. A standing goal cannot capture that priority for now, so it is a one-time ruling: the later releases, and every release after 0.2.0, fill from the score again.
|
||||
|
||||
|
||||
+1
-1
@@ -20,7 +20,7 @@ paths:
|
||||
| 5 | — | — | irreversible: data lost for good, or a one-way door |
|
||||
|
||||
- **I's reach counts every project and session that runs the change.** A change to a hook or a tool that every session runs is I level 3, every consumer affected, even where one person uses it.
|
||||
- **Name in Moves each achieve goal in `README.md` → `## Goals` the item moves, and how far,** from 1 to 10, judged by its value to the audience in `## Audience`: `1.1 by 8, 1.2 by 5`. Name code health, the goal in `CLAUDE.md` → "Shared goal", as `code health by 4`, unless the README voids it. Name the most specific goal it moves, and a parent goal only for a move none of its sub-goals covers, never beside its own sub-goal. Leave out every maintain goal, which vetoes, except on an item marked `defect` that restores it: name that goal, and how much of it is broken. `rank_todo.py --help` says how it weighs each goal.
|
||||
- **Name in Moves each achieve goal in `README.md` → `## Goals` the item moves, and how far,** from 1 to 10, judged by its value to the audience in `## Audience`: `1.1 by 8, 1.2 by 5`. Name code health, the goal in `CLAUDE.md` → "Shared goal", as `code health by 6`, unless the README voids it. Name the most specific goal it moves, and a parent goal only for a move none of its sub-goals covers, never beside its own sub-goal. Leave out every maintain goal, which vetoes, except on an item marked `defect` that restores it: name that goal, and how much of it is broken. `rank_todo.py --help` says how it weighs each goal.
|
||||
- **A table with C, A, G and Goals in place of L, E, I and Moves is in the old shape.** Reshaping it is the project's only work until done, as `CLAUDE.md` → "Working loop" says, and `rank_todo.py` says how. Until then, A is the item's value to the audience and G its value to the goals it cites.
|
||||
- **Score L, E and I against the project's anchors, and check every score against finished chunks.** Guess the scores, then run `~/.claude/bin/score-outcomes <L> <E> <I> <S>` in the project. It lists the finished items guessed nearest and the project's anchors, the shipped items its `docs/score-anchors.md` names for each level, each with what its chunk measured; then the median outcome at each guessed R and S, and how well L, E and I each order the chunks by review rounds and 🔴 findings. Compare the item with each anchor in turn: is it more or less likely to hold a defect, to let one escape, or to do damage? Give it the level whose anchors it matches; where a level has no anchor, judge by the level's words in the table above. Set S to the value guessed for the finished items whose outcome matches what you expect of this one. Keep your guess when no listed item resembles the one you are scoring.
|
||||
- **Anchor each level of L, E and I with two or three items the project shipped,** in `docs/score-anchors.md`: a table whose Level column names a level, such as `L 1`, and whose Items column lists the anchors' IDs, such as `12, 14`. Choose an L anchor whose chunk, as `score-outcomes` measured it, fits the level's words: its lines changed and the components it touched. Choose an E or I anchor by the level's words and what followed its release, since no tool measures either yet. Leave out a level no shipped item shows. Replace a level's anchors when the items scored at it turn out about ten times more likely, or ten times less likely, than its words say: to hold a defect, to let one escape, or to do damage.
|
||||
|
||||
@@ -106,7 +106,7 @@ The maintainability senior of PR #257's scoring run named `bin/rule_hook/rules.p
|
||||
|
||||
### 114. Run git through `git_repo.py` in `rule_hook`, `metric_bench`, `reading_cost`, `review_replay` and `loop_claude`, keeping rule-hook's guards as an option, so one module owns how a tool runs git.
|
||||
|
||||
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per `docs/decisions.md` → `larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension`.
|
||||
Shape is the lowest dimension, so this item moves code health by 9, per `docs/decisions.md` → `larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension`.
|
||||
|
||||
The comprehension seats on PR #254 found five git runners with different contracts: `git_repo.git`, `rule_hook/git.py`'s guarded one, `metric_bench/main.py`'s, `reading_cost/main.py`'s `located` and `loop_claude/effects.py`'s. PR #276 added a sixth, `review_replay/main.py`'s `git`. The inherited architect of PR #257's scoring run named `bin/git_repo.py:8` `git` again. Make `git_repo.git` easier to read.
|
||||
|
||||
@@ -276,7 +276,7 @@ Mid A of PR #257's scoring run named `bin/larv_review/ruling.py:8` `votes`. It c
|
||||
|
||||
### 125. Give "record", "outcome" and "places" one meaning each across `bin/`, and rename `bin/larv_review/git.py` by what it holds.
|
||||
|
||||
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per `docs/decisions.md` → `larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension`.
|
||||
Shape is the lowest dimension, so this item moves code health by 9, per `docs/decisions.md` → `larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension`.
|
||||
|
||||
The inherited architect of PR #257's scoring run named `bin/reading_cost/measure.py:16` `Reading` and `bin/larv_review/git.py:6` `history_changes`, and found each word used for several concepts. "Record": the history writer `larv_review/record.py`, the judge's `loop_claude/project/records.py`, and `reading_cost/figures.py`'s `RECORD`. "Outcome": `transcript_archive/outcomes.py`, `score_outcomes/outcomes.py` `Outcome` and `rule_hook/rules.py` `Outcome`. "Places": `comprehension_history.Place` and `Reading.places`. `larv_review/git.py` holds history logic, not a git runner. Make `Reading` and `history_changes` easier to read.
|
||||
|
||||
@@ -346,7 +346,7 @@ Three seats of PR #257's scoring run named `bin/comprehension_history.py:106` `l
|
||||
|
||||
### 124. Group the shared modules at `bin/`'s root into packages named by purpose, so the root holds only launchers and stand-alone tools.
|
||||
|
||||
Shape is the lowest dimension, so this item moves goal 1.1 by 9, per `docs/decisions.md` → `larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension`.
|
||||
Shape is the lowest dimension, so this item moves code health by 9, per `docs/decisions.md` → `larv-review: a project lifting itself to the comprehension floor starts at the lowest dimension`.
|
||||
|
||||
PR #257's scoring run put Shape at 6.25, the lowest dimension. `bin/` holds about 30 entries: 6 launchers, 6 tool packages, 3 shared packages and 15 root modules. Two root modules are tools (`rank_todo.py`, `statusline.py`) and 13 are shared libraries, and nothing in the layout tells them apart; only README's import-root paragraph maps them. Candidate packages: git running and parsing, the todo table with `rank_todo.py`'s formula language and goal weighing, and the comprehension history with round titles. `rank_todo.py` keeps its path, which every project's rules name. The `.gitignore` allowlist and README follow the move.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user