Answer architecture round 1: give loop-claude's prompt text one home, split larv-review's plan trigger and ruling base out, share todo.md's table and the project key, and name the scoring run, the checkout roots and the test env for what they are

This commit is contained in:
2026-10-09 14:43:19 +02:00
parent b1913ed65d
commit dde06fc465
38 changed files with 406 additions and 259 deletions
+2 -1
View File
@@ -22,7 +22,6 @@
!bin/comprehension_history.py
!bin/failure.py
!bin/file_writes.py
!bin/host_env.py
!bin/larv-review
!bin/larv_review/
!bin/larv_review/*.py
@@ -34,6 +33,7 @@
!bin/metric-bench
!bin/metric_bench/
!bin/metric_bench/*.py
!bin/project_key.py
!bin/rank_todo.py
!bin/reading-cost
!bin/reading_cost/
@@ -42,6 +42,7 @@
!bin/repo_checks/*.py
!bin/statusline.py
!bin/test_*.py
!bin/todo_table.py
!bin/transcript_archive/
!bin/transcript_archive/*.py
!docs/
+1 -1
View File
@@ -74,7 +74,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `reading-cost [--reset] [record]` appends a figure for each chunk `loop-claude` finished to a project's `docs/reading-cost.md`, and says whether the figures rise; `reading-cost --help` states the measure.
- `statusline.py` renders Claude Code's status line; `settings.json` → `statusLine` runs it.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests. `claude_code/` reads Claude Code's files and output, and `transcript_archive/` writes and reads `loop-claude`'s archive; both serve more than one tool. `repo_checks/` tests the repo itself.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them. The other modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool: `claude_code/` reads and writes Claude Code's files and reads its output, and `transcript_archive/` writes and reads `loop-claude`'s archive. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`
+17 -4
View File
@@ -64,13 +64,26 @@ class Heading:
@dataclasses.dataclass(frozen=True)
class Scoring:
class ScoringRun:
"""The four scoring seats' lines, their scores as written and the means, in DIMENSIONS' order."""
seats: list[Seat]
scores: dict[str, list[str]]
means: list[decimal.Decimal]
# A paired ruling as entry_text writes it. A scoring run's heading has no `, against`; it has no Ruling line, ends its table with a Mean row, and labels each seat's places `<seat>, hardest first:` or, for the architect, `<seat>, would restructure first:`.
#
# ## 2026-10-09T09:47:25Z, PR #232 at 324d650, against 12d6d75
#
# Ruling: same
#
# | Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
# |---|---|---|---|---|---|
# | Mid A | better | worse | same | same | better |
#
# Mid A, decided by:
#
# - Locality: `bin/x.py:12` `run`: three branches to hold at once
def entries(history: str) -> list[Entry]:
"""Each paired ruling and scoring run in a comprehension history, in the order the file holds them; a section whose heading names no commit is left out."""
found = []
@@ -126,7 +139,7 @@ def appended(history: str, entry: str) -> str:
return f"{text}\n{entry}"
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, notes: list[str], scoring: Scoring | None) -> str:
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, notes: list[str], scoring: ScoringRun | None) -> str:
"""A paired ruling's entry, with any notes, such as why a fall decided it, and the scores that decided a fall; a seat's vote that counted as same for want of a place says so."""
votes = [[seat.name, *map(vote_text, DIMENSIONS, said[seat.name], counted[seat.name])] for seat in seats]
tables = [[["Seat", *DIMENSIONS], *votes]] + ([scores_table(scoring)] if scoring else [])
@@ -140,11 +153,11 @@ def vote_text(dimension: str, said: str, counted: str) -> str:
return f"same (voted {said}, naming no place)" if dimension == DIMENSIONS[-1] else f"same (voted {said}, no place naming {dimension})"
def scoring_entry(title: str, scoring: Scoring) -> str:
def scoring_entry(title: str, scoring: ScoringRun) -> str:
return entry_text(title, [], [scores_table(scoring)], scoring_labels(scoring.seats))
def scores_table(scoring: Scoring) -> list[list[str]]:
def scores_table(scoring: ScoringRun) -> list[list[str]]:
return [["Seat", *DIMENSIONS], *([seat, *values] for seat, values in scoring.scores.items()), [MEAN, *map(str, scoring.means)]]
+1 -1
View File
@@ -3,7 +3,7 @@ import subprocess
from failure import Failure
from comprehension_history import HISTORY
from larv_review.base import HistoryChange
from larv_review.ruling_base import HistoryChange
def git(*args: str) -> str | Failure:
+27 -20
View File
@@ -2,6 +2,11 @@
usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]
The comprehension check keeps a project from reading worse, two ways. In a paired ruling, seats read
the branch's head beside a base and vote better, same or worse, overall and on each dimension; a
worse ruling blocks the merge, which is the ratchet. In a scoring run, four seats score the project
from 1 to 10; a mean under the floor of {floor} blocks the merge.
Run it inside a project checkout or one of its worktrees. Each command but record-ruling first
fetches origin, when the checkout has that remote, and takes origin's default branch, or else the
local main or master, as the default branch. When the fetch fails, round-count reads origin as
@@ -16,7 +21,8 @@ ruling names; with no ruling either, it is the merge base of HEAD and the defaul
round-count prints each pass's count: the highest <n> among the titles opening `Answer <pass> round
<n>:` of commits the default branch does not hold. It marks each pass at the cap of {cap} rounds,
and each name in such a title that is not a pass.
and each name in such a title that is not a pass. It only reports: larv-review asks the human in
place of a further round.
needs-scoring-run takes the IDs of the todo.md items the chunk ships, none when it ships none, and
prints why the chunk is due a scoring run, or `No scoring run due.` It is due when the chunk ships
@@ -80,16 +86,17 @@ import sys
import time
import typing
from comprehension_history import appended, DIMENSIONS, heading, Heading, HISTORY, recorded, ruling_entry, Scoring, scoring_entry, vote_text
from larv_review.base import base
from comprehension_history import appended, DIMENSIONS, heading, Heading, HISTORY, recorded, ruling_entry, scoring_entry, ScoringRun, vote_text
from larv_review.ruling_base import base
from larv_review.rounds import CAP, PASSES, round_counts
from larv_review.git import commit_exists, default_branch, file_at, git, history_changes, landed_since_release, top
from larv_review.ruling import counted, fallen, ruling, votes
from larv_review.ruling import counted, fallen, ruling, split, votes
from failure import Failure
from larv_review.scoring import fall_ruling, fall_step, FLOOR, means, plan_items, scoring_run_due, under_floor
from rank_todo import ITEMS_HEADER
from larv_review.scoring import fall_ruling, FLOOR, means, under_floor
from todo_table import items, ITEMS_HEADER
from file_writes import replace_file
from larv_review.seats import scores, seats_file, SeatsFile
from larv_review.plan import scoring_run_due
USAGE = "usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]"
@@ -182,10 +189,10 @@ def plan_reasons(default: str, ids: list[int]) -> list[str] | Failure:
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read todo.md: {failure}")
# While a chunk reshapes todo.md into one table, the default branch's todo.md has no items table.
planned = plan_items(before) if ITEMS_HEADER in before.splitlines() else []
planned = items(before) if ITEMS_HEADER in before.splitlines() else []
if isinstance(planned, Failure):
return Failure(f"{default}'s todo.md: {planned.reason}")
chunk = plan_items(after)
chunk = items(after)
if isinstance(chunk, Failure):
return Failure(f"the checkout's todo.md: {chunk.reason}")
landed = landed_since_release(default)
@@ -204,21 +211,21 @@ def record_ruling(votes_file: pathlib.Path, scores_file: pathlib.Path | None = N
return said
count = counted(given.seats, said)
tally = ", ".join(f"{seat.name} {vote_text(DIMENSIONS[-1], said[seat.name][-1], count[seat.name][-1])}" for seat in given.seats)
ruled = ruling([each[-1] for each in count.values()])
if ruled is None:
overall = [each[-1] for each in count.values()]
if split(overall):
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the votes file, and run this again.")
return None
ruled = ruling(overall)
fell = fallen(count) if ruled != "worse" else []
if fell and not scores_file:
print(f"Fell: {', '.join(fell)}, in a {ruled} ruling. Launch the four scoring seats on {named.label} at {named.commit}, write their lines to a scores file, and run this again with both files.")
return None
if scores_file and not fell:
return Failure(f"no dimension fell in this {ruled} ruling, so it takes no scores file")
scoring = None
notes = []
match fall_step(fell, scores_file is not None):
case "ask":
print(f"Fell: {', '.join(fell)}, in a {ruled} ruling. Launch the four scoring seats on {named.label} at {named.commit}, write their lines to a scores file, and run this again with both files.")
return None
case "refuse":
return Failure(f"no dimension fell in this {ruled} ruling, so it takes no scores file")
if scores_file:
scored = read_scoring(scores_file, named)
scored = read_fall_scores(scores_file, named)
if isinstance(scored, Failure):
return scored
scoring = scored
@@ -247,7 +254,7 @@ def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> F
return append(given.read, scoring_entry(named.title, scoring))
def read_scoring(scores_file: pathlib.Path, votes: Heading) -> Scoring | Failure:
def read_fall_scores(scores_file: pathlib.Path, votes: Heading) -> ScoringRun | Failure:
"""The scores a fall called, once their file names what the votes file read."""
read = read_seats(scores_file, paired=False, fresh=False)
if isinstance(read, Failure):
@@ -258,11 +265,11 @@ def read_scoring(scores_file: pathlib.Path, votes: Heading) -> Scoring | Failure
return scored_seats(given)
def scored_seats(given: SeatsFile) -> Scoring | Failure:
def scored_seats(given: SeatsFile) -> ScoringRun | Failure:
found = scores(given.seats)
if isinstance(found, Failure):
return found
return Scoring(given.seats, found, means(list(found.values())))
return ScoringRun(given.seats, found, means(list(found.values())))
def read_seats(path: pathlib.Path, paired: bool, fresh: bool = True) -> tuple[Heading, SeatsFile] | Failure:
+26
View File
@@ -0,0 +1,26 @@
from failure import Failure
from todo_table import Item
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
"""Why the plan calls a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag.
A chunk that ships an item naming the floor is due one, so it can prove the floor met. So is a chunk that empties a release towards which work has landed, so no release ends unmeasured: it ships one of the release's items, or the release is the next one, the earliest before names, and landed holds. That covers a release whose last item moved to another release.
"""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
next_release = min((item.release for item in before), default=None)
for release in emptied(before, after, shipped):
if any(item.release == release for item in shipped) or (release == next_release and landed):
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
return reasons
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
named = {item.release for item in before + shipped}
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
return sorted(named - left)
+13 -9
View File
@@ -39,16 +39,20 @@ def place_dimensions(place: str) -> list[str]:
return named if all(name in DIMENSIONS for name in named) else []
def ruling(counted: list[str]) -> str | None:
"""The vote that leads each other vote by two or more, else same; None when only the first two seats voted and they differ, so two more must vote."""
if len(counted) == 2 and counted[0] != counted[1]:
return None
for vote in VOTES:
if all(counted.count(vote) >= counted.count(other) + 2 for other in VOTES if other != vote):
return vote
return "same"
def split(overall: list[str]) -> bool:
"""Whether only the first two seats voted overall and they differ, so two more must vote."""
return len(overall) == 2 and overall[0] != overall[1]
def ruling(overall: list[str]) -> str:
return lead(overall) or "same"
def fallen(counted: dict[str, list[str]]) -> list[str]:
"""Each dimension, Overall aside, that the seats rule worse by a lead of two."""
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if ruling(list(column)) == "worse"]
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if lead(list(column)) == "worse"]
def lead(votes: list[str]) -> str | None:
"""The vote that leads each other vote by two or more, if one does."""
return next((vote for vote in VOTES if all(votes.count(vote) >= votes.count(other) + 2 for other in VOTES if other != vote)), None)
@@ -12,7 +12,10 @@ class HistoryChange:
def measured_commit(changes: list[HistoryChange]) -> str | None:
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better."""
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better.
It is the ratchet's base. A ruling of same leaves the base where it was, so a run of small falls each ruled same still adds up against one commit. Entry times are UTC stamps in ISO order, so they sort as text.
"""
for change in changes:
held = {entry.heading for entry in entries(change.before)}
added = [entry for entry in entries(change.after) if entry.heading not in held]
-47
View File
@@ -1,22 +1,10 @@
import decimal
import typing
from comprehension_history import DIMENSIONS
from failure import Failure
from rank_todo import Item, ITEMS_HEADER, parse_row, table_rows
FLOOR = decimal.Decimal("7.0")
def fall_step(fell: list[str], scored: bool) -> typing.Literal["ask", "refuse", "rule"]:
"""What record-ruling does next: ask for the scoring seats when a dimension fell and no scores file came, refuse a scores file no fall calls, or rule."""
if fell and not scored:
return "ask"
if scored and not fell:
return "refuse"
return "rule"
def fall_ruling(ruled: str, fell: list[str], found: list[decimal.Decimal]) -> tuple[str, str]:
"""The ruling once the scoring run has scored the dimensions that fell, and the line that says why: worse when one of their means is under the floor."""
if under := [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if name in fell and value < FLOOR]:
@@ -35,38 +23,3 @@ def mean(values: list[decimal.Decimal]) -> decimal.Decimal:
def under_floor(found: list[decimal.Decimal]) -> list[str]:
return [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if value < FLOOR]
def plan_items(todo: str) -> list[Item] | Failure:
"""Each row of a todo.md's items table, read as `rank_todo.py` reads it."""
table = table_rows(todo.splitlines(), ITEMS_HEADER)
if isinstance(table, Failure):
return table
items = []
for row in table.rows:
item = parse_row(row)
if isinstance(item, Failure):
return item
items.append(item)
return items
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
"""Why the plan calls a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag. Work has landed towards an emptied release when the chunk ships one of its items, or when it is the next release, the earliest before names, and landed holds."""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
next_release = min((item.release for item in before), default=None)
for release in emptied(before, after, shipped):
if any(item.release == release for item in shipped) or (release == next_release and landed):
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
return reasons
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
named = {item.release for item in before + shipped}
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
return sorted(named - left)
+13 -15
View File
@@ -9,15 +9,16 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from comprehension_history import appended, entries, Place, recorded, ruling_entry, Scoring, Seat
from larv_review.base import base, first_ruling, HistoryChange, measured_commit
from larv_review.ruling import counted, fallen, ruling, votes
from comprehension_history import appended, entries, Place, recorded, ruling_entry, ScoringRun, Seat
from larv_review.ruling_base import base, first_ruling, HistoryChange, measured_commit
from larv_review.ruling import counted, fallen, ruling, split, votes
from failure import Failure
from larv_review.scoring import fall_ruling, fall_step, means, plan_items, scoring_run_due, under_floor
from host_env import host_env
from rank_todo import Item
from larv_review.scoring import fall_ruling, means, under_floor
from test_env import isolated_env
from todo_table import Item, items
from larv_review.rounds import round_counts
from larv_review.seats import score_line, scores, seats_file
from larv_review.plan import scoring_run_due
TOOL = pathlib.Path(__file__).resolve().parents[1] / "larv-review"
SCORING = "## 2026-10-08T12:00:00Z, PR #1 at aaaaaaa\n\n| Seat | Overall |\n|---|---|\n| Mean | 7.00 |\n"
@@ -93,7 +94,7 @@ class Seats(unittest.TestCase):
]
for counted, expected in cases:
with self.subTest(counted):
self.assertEqual(ruling(counted), expected)
self.assertEqual(None if split(counted) else ruling(counted), expected)
def test_reads_four_seats_scores_and_refuses_an_overall_above_the_lowest_by_more_than_one(self):
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
@@ -115,7 +116,7 @@ class History(unittest.TestCase):
def test_reads_back_what_it_writes(self):
seats = [Seat("Mid A", said("better", "worse"), ["`bin/a.py:3` `f`: hard, with `:9` `g`"]), Seat("Inherited architect", said("worse", "worse"), [])]
written = {"Mid A": ["same", "worse", "same", "same", "better"], "Inherited architect": ["same", "worse", "same", "same", "worse"]}
scoring = Scoring([Seat("Junior A", "", ["`bin/b.py:4` `h`: long"])], {"Junior A": ["7", "6.5", "7", "7", "7"]}, [decimal.Decimal(mean) for mean in ("7.00", "6.50", "7.00", "7.00", "7.00")])
scoring = ScoringRun([Seat("Junior A", "", ["`bin/b.py:4` `h`: long"])], {"Junior A": ["7", "6.5", "7", "7", "7"]}, [decimal.Decimal(mean) for mean in ("7.00", "6.50", "7.00", "7.00", "7.00")])
ruled = ruling_entry("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", seats, written, {**written, "Inherited architect": ["same"] * 5}, "worse", ["Fell: Locality, under the floor."], scoring)
history = appended(appended("", SCORING), ruled)
self.assertTrue(history.startswith("# Comprehension history\n\n## 2026-10-08T12:00:00Z"))
@@ -136,9 +137,6 @@ class FallRuling(unittest.TestCase):
self.assertEqual(fall_ruling("better", ["Locality", "Shape"], found), ("worse", "Fell: Locality, Shape. Under the floor of 7.0: Locality 6.5, so the better ruling counts as worse."))
self.assertEqual(fall_ruling("same", ["Shape"], found), ("same", "Fell: Shape, each at or above the floor of 7.0, so the same ruling stands."))
def test_asks_for_scores_on_a_fall_refuses_them_without_one_and_rules_otherwise(self):
self.assertEqual([fall_step(["Locality"], False), fall_step([], True), fall_step(["Locality"], True), fall_step([], False)], ["ask", "refuse", "rule", "rule"])
class Base(unittest.TestCase):
def test_takes_the_newest_change_adding_a_scoring_run_or_ending_on_a_better_ruling(self):
@@ -174,9 +172,9 @@ class Rounds(unittest.TestCase):
class ScoringRunDue(unittest.TestCase):
def test_reads_each_rows_id_release_and_tagline(self):
self.assertEqual(plan_items(todo("| 3 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), [item(3, (0, 1, 0))])
self.assertIsInstance(plan_items("# todo\n"), Failure)
self.assertIsInstance(plan_items(todo("| 0 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), Failure)
self.assertEqual(items(todo("| 3 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), [item(3, (0, 1, 0))])
self.assertIsInstance(items("# todo\n"), Failure)
self.assertIsInstance(items(todo("| 0 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), Failure)
def test_is_due_for_an_item_naming_the_floor(self):
plan = [item(1, (0, 1, 0), "**Lift the project to the Comprehension floor.**"), item(2, (0, 1, 0))]
@@ -208,7 +206,7 @@ class LarvReviewTool(unittest.TestCase):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
root = pathlib.Path(os.path.realpath(temp.name))
self.env = {**host_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(root)}
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(root)}
self.checkout = root / "proj"
subprocess.run(["git", "init", "-q", "--bare", "-b", "main", str(root / "origin.git")], check=True, env=self.env)
subprocess.run(["git", "clone", "-q", str(root / "origin.git"), str(self.checkout)], check=True, env=self.env, capture_output=True)
+1 -32
View File
@@ -3,46 +3,15 @@ import json
import re
import typing
from loop_claude.project.records import ANSWER_DESTINATIONS, ANSWERS_RECORDED, HUMAN_RULES
from loop_claude.effects import Effects
from failure import Failure
from claude_code.entries import hit_limit, limit_said, parse_entry, structured_output
from loop_claude.cycle.prompts import JUDGE_RULES
from loop_claude.cycle.retry import Limited, retried
from loop_claude.stop import Stop
# At 3 to 4 characters a token, 100,000 to 133,000 tokens, which leaves a 200,000-token context room for the reply; the largest session rendered here was 364,000 characters on 2026-10-03.
JUDGE_PROMPT_CHARS = 400000
JUDGE_RULES = (
"Decide whether the human must now be interrupted. interrupt=true when any of these appear:\n"
"- the agent asks the human anything, seeks approval, or says a decision is the human's, and neither "
"README.md nor the records settle the matter\n"
"- it asks again a question a [loop] line already answered\n"
"- it carried on under an assumption that decides the deliverable, that the human could "
"reasonably reject and that neither README.md nor the records settle; a caveat about "
"transient state it then handled is not one\n"
"- it is blocked on something only the human can supply: a credential, a version bump, a choice "
"outside the repo, or a release ready to cut, which only the human may publish\n"
"- it flags that the goals, the plan in todo.md or the task itself is unclear, missing or in need of change\n"
"- it chose between alternatives that neither README.md nor the records discriminate "
"between - neither rules the rejected one out - and the choice materially changes the deliverable\n"
"interrupt=false otherwise. Bookkeeping the agent settled silently - what to document, wording, "
"where a note goes - is not on its own a reason to interrupt. The human may have replied inside the transcript, on a [human] line: judge only what "
"is still open, never a question they already answered there. A [loop] line is what loop-claude told "
"the agent, such as an answer quoted from README.md or the records.\n"
"Where the agent asked or flagged something itself, the question is that one, in the agent's own "
"wording; never construct another from facts it stated.\n"
"Where README.md or the records answer the agent's own question and no [loop] line has "
"quoted that answer yet, set interrupt=false and quote the passage that answers it; the agent is "
"then given that passage in place of the human's reply. An interrupt outranks a quoted answer, and either "
"outranks finished.\n"
"The chunk is the piece of work the agent is on, shipped as one pull request. "
"finished=true when the chunk is complete: merged, or nothing remains that the agent could do on "
f"its own. Even then, it is finished only once {ANSWERS_RECORDED}. An answer counts as recorded "
f"where README.md or the records above hold it, {HUMAN_RULES} included. It also counts where "
f"the transcript shows the agent writing it into {ANSWER_DESTINATIONS}; writing it into "
f"{HUMAN_RULES} does not count. finished=false when work remains that a further run can "
"continue: an open PR, a review not yet run, CI not yet reported, an answer not yet recorded."
)
@dataclasses.dataclass(frozen=True)
+61
View File
@@ -0,0 +1,61 @@
from loop_claude.project.records import HUMAN_RULES, RECORD_FILES
# Where the judge may quote an answer from, named as the human and the worker see them; of README.md the judge sees only the premises.
QUOTE_SOURCES = "README.md's Goals and Audience, " + ", ".join(RECORD_FILES) + " or " + HUMAN_RULES
# Where the worker may write an answer the human typed, so it counts as recorded.
ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES) + ", todo.md or a comment"
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
JUDGE_RULES = (
"Decide whether the human must now be interrupted. interrupt=true when any of these appear:\n"
"- the agent asks the human anything, seeks approval, or says a decision is the human's, and neither "
"README.md nor the records settle the matter\n"
"- it asks again a question a [loop] line already answered\n"
"- it carried on under an assumption that decides the deliverable, that the human could "
"reasonably reject and that neither README.md nor the records settle; a caveat about "
"transient state it then handled is not one\n"
"- it is blocked on something only the human can supply: a credential, a version bump, a choice "
"outside the repo, or a release ready to cut, which only the human may publish\n"
"- it flags that the goals, the plan in todo.md or the task itself is unclear, missing or in need of change\n"
"- it chose between alternatives that neither README.md nor the records discriminate "
"between - neither rules the rejected one out - and the choice materially changes the deliverable\n"
"interrupt=false otherwise. Bookkeeping the agent settled silently - what to document, wording, "
"where a note goes - is not on its own a reason to interrupt. The human may have replied inside the transcript, on a [human] line: judge only what "
"is still open, never a question they already answered there. A [loop] line is what loop-claude told "
"the agent, such as an answer quoted from README.md or the records.\n"
"Where the agent asked or flagged something itself, the question is that one, in the agent's own "
"wording; never construct another from facts it stated.\n"
"Where README.md or the records answer the agent's own question and no [loop] line has "
"quoted that answer yet, set interrupt=false and quote the passage that answers it; the agent is "
"then given that passage in place of the human's reply. An interrupt outranks a quoted answer, and either "
"outranks finished.\n"
"The chunk is the piece of work the agent is on, shipped as one pull request. "
"finished=true when the chunk is complete: merged, or nothing remains that the agent could do on "
f"its own. Even then, it is finished only once {ANSWERS_RECORDED}. An answer counts as recorded "
f"where README.md or the records above hold it, {HUMAN_RULES} included. It also counts where "
f"the transcript shows the agent writing it into {ANSWER_DESTINATIONS}; writing it into "
f"{HUMAN_RULES} does not count. finished=false when work remains that a further run can "
"continue: an open PR, a review not yet run, CI not yet reported, an answer not yet recorded."
)
def fresh(task: str) -> str:
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one."
def resumed(unfinished_evidence: str) -> str:
ruled = f" A judge ruled the chunk unfinished, quoting this passage of your transcript as the work remaining:\n\n{quoted(unfinished_evidence)}\n\n" if unfinished_evidence else " "
return f"Continue the task; the previous run ended before the chunk was finished.{ruled}The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}."
def answered(question: str, answer: str) -> str:
return f"Your question: {question}\n\n{QUOTE_SOURCES} answers it:\n\n{answer}\n\nContinue the task."
def past_the_cap(question: str, answer: str, cap: int) -> str:
"""The question a quoted answer puts to the human once the cycle's quoted answers reach the cap."""
return f"{question}\n\n{QUOTE_SOURCES} says:\n{answer}\n\nThose files have answered {cap} questions this cycle, the most allowed, so this one comes to you."
# Blockquoted, so a passage holding blank lines still ends visibly where the text after it starts.
def quoted(passage: str) -> str:
return "\n".join(f"> {line}" for line in passage.splitlines())
+14 -6
View File
@@ -12,7 +12,7 @@ from loop_claude.cycle.interrupt import interrupt
from loop_claude.project.model_choice import Models
from transcript_archive import outcomes
from loop_claude.paths import Paths
from loop_claude.project.records import QUOTE_SOURCES
from loop_claude.cycle import prompts
from claude_code.transcript import read_transcript, render
from loop_claude.cycle.retry import retried
from loop_claude.cycle.worker import run_worker, worker_args
@@ -70,19 +70,27 @@ def step(cycle: Cycle, state: Active) -> Event | Stop:
def work(cycle: Cycle, answered: QuotedAnswer | None) -> WorkerRan | Stop:
if answered is not None:
print(f"\n=== {QUOTE_SOURCES} answered: {answered.question} ===\n{answered.answer}", flush=True)
print(f"\n=== {prompts.QUOTE_SOURCES} answered: {answered.question} ===\n{answered.answer}", flush=True)
if stop := keep_answer(cycle, answered):
return stop
# session_open runs inside the lambda: a failed attempt may have opened the session.
if stop := retried("worker", lambda: run_worker(cycle.root, worker_args(cycle.task, cycle.session, cycle.models.worker, session_open(cycle.session), answered, cycle.unfinished_evidence), cycle.effects), cycle.effects):
# worker_start runs inside the lambda: a failed attempt may have opened the session.
if stop := retried("worker", lambda: run_worker(cycle.root, worker_start(cycle, answered), cycle.effects), cycle.effects):
return stop
path = transcript_file(cycle.session)
return WorkerRan(path) if path is not None else Stop(1, f"claude left no transcript for session {cycle.session.id}")
def worker_start(cycle: Cycle, answered: QuotedAnswer | None) -> list[str]:
if answered is not None:
return worker_args(cycle.session, cycle.models.worker, prompts.answered(answered.question, answered.answer), resume=True)
if session_open(cycle.session):
return worker_args(cycle.session, cycle.models.worker, prompts.resumed(cycle.unfinished_evidence), resume=True)
return worker_args(cycle.session, cycle.models.worker, prompts.fresh(cycle.task), resume=False)
def keep_answer(cycle: Cycle, answered: QuotedAnswer) -> Stop | None:
path = transcript_file(cycle.session)
if path and (failure := archive(cycle.project_archive, path, cycle.task, outcomes.answered(answered.question, QUOTE_SOURCES, answered.answer, answered.evidence), cycle.effects.now())):
if path and (failure := archive(cycle.project_archive, path, cycle.task, outcomes.answered(answered.question, prompts.QUOTE_SOURCES, answered.answer, answered.evidence), cycle.effects.now())):
return Stop(1, failure.reason)
return None
@@ -106,7 +114,7 @@ def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) ->
def question_for(ruling: Interrupt | QuotedAnswer) -> str:
if isinstance(ruling, Interrupt):
return ruling.question
return f"{ruling.question}\n\n{QUOTE_SOURCES} says:\n{ruling.answer}\n\nThose files have answered {QUOTED_ANSWERS_PER_CYCLE} questions this cycle, the most allowed, so this one comes to you."
return prompts.past_the_cap(ruling.question, ruling.answer, QUOTED_ANSWERS_PER_CYCLE)
def close(cycle: Cycle, transcript_path: str, verdict: Finished | Unfinished) -> Archived | Stop:
+4 -12
View File
@@ -1,11 +1,8 @@
from loop_claude.project.records import ANSWERS_RECORDED, QUOTE_SOURCES
from claude_code.entries import assistant_parts, ends_stream, hit_limit, limit_said, parse_entry, rejected_until
from loop_claude.effects import Effects
from failure import Failure
from loop_claude.cycle.retry import Limited
from claude_code.transcript import part_line
from transcript_archive.outcomes import quoted
from loop_claude.cycle.judge import QuotedAnswer
from claude_code.session import Session
PRINTED_TOOL_INPUT = 160
@@ -40,17 +37,12 @@ def worker_outcome(code: int, result: dict | None, resets: list[int]) -> Failure
return None if result is not None else Failure("claude exited 0 without a result event")
def worker_args(task: str, session: Session, model: str, resuming: bool, answered: QuotedAnswer | None = None, unfinished_evidence: str = "") -> list[str]:
if answered is not None:
start = ["-p", f"Your question: {answered.question}\n\n{QUOTE_SOURCES} answers it:\n\n{answered.answer}\n\nContinue the task.", "--resume", str(session.id)]
elif resuming:
ruled = f" A judge ruled the chunk unfinished, quoting this passage of your transcript as the work remaining:\n\n{quoted(unfinished_evidence)}\n\n" if unfinished_evidence else " "
start = ["-p", f"Continue the task; the previous run ended before the chunk was finished.{ruled}The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}.", "--resume", str(session.id)]
else:
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
start = ["-p", f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.", "--name", session.name, "--session-id", str(session.id)]
def worker_args(session: Session, model: str, prompt: str, resume: bool) -> list[str]:
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
start = ["--resume", str(session.id)] if resume else ["--name", session.name, "--session-id", str(session.id)]
# -p fails without --verbose under stream-json.
return [
"-p", prompt,
*start,
"--output-format", "stream-json",
"--permission-mode", "bypassPermissions",
+2 -1
View File
@@ -83,7 +83,8 @@ from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint,
from loop_claude.effects import Effects
from loop_claude.project.model_choice import get_or_ask_models
from loop_claude.paths import Paths
from transcript_archive.location import project_archive, project_key
from transcript_archive.location import project_archive
from project_key import project_key
from loop_claude.project.premises import read_premises
from loop_claude.project.records import read_records
from claude_code.session import session_named
-5
View File
@@ -6,11 +6,6 @@ from loop_claude.stop import Stop
# The records: these project files and HUMAN_RULES; the judge reads them beside the README's premises.
RECORD_FILES = ("AGENTS.md", "docs/decisions.md")
HUMAN_RULES = "~/.claude/CLAUDE.md"
# Where the judge may quote an answer from, named as the human and the worker see them; of README.md the judge sees only the premises.
QUOTE_SOURCES = "README.md's Goals and Audience, " + ", ".join(RECORD_FILES) + " or " + HUMAN_RULES
# Where the worker may write an answer the human typed, so it counts as recorded.
ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES) + ", todo.md or a comment"
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
def read_records(root: str, claude_md: pathlib.Path) -> str | Stop:
+2 -2
View File
@@ -17,7 +17,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from host_env import host_env
from test_env import isolated_env
BIN = pathlib.Path(__file__).resolve().parents[1]
TASK = "Do the next chunk."
@@ -160,7 +160,7 @@ class Checkout:
self.root.mkdir(parents=True)
(self.root / "README.md").write_text(PREMISES)
self.script = BIN / "loop-claude"
self.env = host_env()
self.env = isolated_env()
fake = self.temp / "fake-bin"
fake.mkdir()
(fake / "claude").write_text(FAKE_CLAUDE.format(python=sys.executable))
+6 -6
View File
@@ -20,7 +20,7 @@ from loop_claude.effects import Effects
from claude_code import entries, session
from failure import Failure
from loop_claude.cycle.judge import Finished, Interrupt, QuotedAnswer, Unfinished, verdict_from
from loop_claude.cycle import interrupt, judge, retry, run, states, worker
from loop_claude.cycle import interrupt, judge, prompts, retry, run, states, worker
from loop_claude.cycle.retry import Limited
from loop_claude.loop import main, reload
from loop_claude import paths
@@ -384,12 +384,12 @@ class ModelPrompt(unittest.TestCase):
class WorkerArgs(unittest.TestCase):
def test_opens_a_fresh_session_by_name_and_resumes_an_open_one(self):
opened = session.session_named("A proj loop-claude", pathlib.Path("/h/.claude/projects"))
fresh = worker.worker_args("Do it.", opened, "m", False)
fresh = worker.worker_args(opened, "m", prompts.fresh("Do it."), resume=False)
self.assertEqual(fresh[fresh.index("--session-id") + 1], str(opened.id))
self.assertEqual(fresh[fresh.index("--name") + 1], "A proj loop-claude")
resumed = worker.worker_args("Do it.", opened, "m", True)
resumed = worker.worker_args(opened, "m", prompts.resumed(""), resume=True)
self.assertEqual(resumed[resumed.index("--resume") + 1], str(opened.id))
self.assertNotIn("Do it.", " ".join(resumed))
self.assertNotIn("--session-id", resumed)
self.assertTrue(resumed[resumed.index("-p") + 1].startswith("Continue the task; the previous run ended before the chunk was finished. The chunk is finished once"))
@@ -466,7 +466,7 @@ class Verdict(unittest.TestCase):
class LoopRuns(unittest.TestCase):
def test_stops_naming_the_reload_when_it_hands_over_anything_but_the_state_with_a_task(self):
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": ""})], [json.dumps({"cycle_number": "3", "remaining": "", "root": "/r", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": "do it", "turns": 1})], [json.dumps({"cycle_number": 3, "failures": 0, "remaining": "", "root": "/r", "task": "do it"})]):
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "root": "/r", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "", "unfinished_evidence": ""})], [json.dumps({"cycle_number": "3", "root": "/r", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "do it", "turns": 1, "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "failures": 0, "root": "/r", "task": "do it", "unfinished_evidence": ""})]):
with self.subTest(handed=handed):
self.assertEqual(main.loop_run_from([main.RELOAD_FLAG, *handed], CWD, FakeEffects().effects()), Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(handed)}"))
@@ -483,7 +483,7 @@ class LoopRuns(unittest.TestCase):
launcher = pathlib.Path(root, "loop-claude")
launcher.write_text("")
fake = FakeEffects(python=[ran()])
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(1, "", "/r", "do it"), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence=""), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
class HelpLimits(unittest.TestCase):
+2 -2
View File
@@ -84,8 +84,8 @@ def medians(runs: list[RunScores]) -> dict[str, Score]:
return {metric: Score(statistics.median(run.scores[metric].auc for run in runs), statistics.median(run.scores[metric].top_k for run in runs), all(run.scores[metric].constant for run in runs)) for metric in METRICS}
def resolved(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
"""Entries written before paths were written from the repository's root shorten a path once they have named its folder: `cycle/run.py` after `bin/loop_claude/cycle/states.py`. Each place resolves to the one tracked file it names, or, where several match, the one in the folder of the place resolved before it; a place naming no file, or several, is dropped. Returns the resolved places and the dropped ones."""
def tracked_places(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
"""Entries written before paths were written from the repository's root shorten a path once they have named its folder: `cycle/run.py` after `bin/loop_claude/cycle/states.py`. Each place resolves to the one tracked file it names, or, where several match, the one in the folder of the place tracked_places before it; a place naming no file, or several, is dropped. Returns the tracked_places places and the dropped ones."""
found = []
dropped = []
before = ""
+2 -2
View File
@@ -35,7 +35,7 @@ import statistics
import subprocess
import sys
from metric_bench.bench import bench, co_changed, medians, METRICS, resolved, RunScores, Score
from metric_bench.bench import bench, co_changed, medians, METRICS, RunScores, Score, tracked_places
from comprehension_history import entries, Entry
from failure import Failure
from metric_bench.python_units import measured, python_units, Unit
@@ -128,7 +128,7 @@ def run_scores(run: Entry, top: str) -> tuple[RunScores | None, list[str]] | Fai
log = git(["-c", "core.quotePath=false", "log", "--no-merges", "--format=tformat:@@", "--name-only", run.commit], top)
if isinstance(log, Failure):
return log
places, dropped = resolved(run.scored_places, tracked)
places, dropped = tracked_places(run.scored_places, tracked)
return bench(units, places, len(set(dropped)), co_changed(log)), unparsed
+8 -8
View File
@@ -9,11 +9,11 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from metric_bench.bench import auc, bench, co_changed, resolved, top_k
from metric_bench.bench import auc, bench, co_changed, top_k, tracked_places
from metric_bench.python_units import cognitive, measured, python_units, Unit
from comprehension_history import entries, Place
from failure import Failure
from host_env import host_env
from test_env import isolated_env
from metric_bench.main import scored_runs
TOOL = pathlib.Path(__file__).resolve().parents[1] / "metric-bench"
@@ -153,18 +153,18 @@ class Places(unittest.TestCase):
def test_resolves_a_shortened_path_to_the_file_nearest_the_one_before_it(self):
tracked = ["bin/tool/cycle/states.py", "bin/tool/cycle/run.py", "bin/other/cycle/run.py"]
places = [Place("bin/tool/cycle/states.py", "transition"), Place("cycle/run.py", "step"), Place("gone.py", "x")]
self.assertEqual(resolved(places, tracked), ([Place("bin/tool/cycle/states.py", "transition"), Place("bin/tool/cycle/run.py", "step")], [Place("gone.py", "x")]))
self.assertEqual(tracked_places(places, tracked), ([Place("bin/tool/cycle/states.py", "transition"), Place("bin/tool/cycle/run.py", "step")], [Place("gone.py", "x")]))
def test_a_full_path_wins_over_a_file_in_the_folder_before_it(self):
self.assertEqual(resolved([Place("pkg/x.py", "a"), Place("main.py", "f")], ["pkg/x.py", "main.py", "pkg/main.py"])[0][-1], Place("main.py", "f"))
self.assertEqual(tracked_places([Place("pkg/x.py", "a"), Place("main.py", "f")], ["pkg/x.py", "main.py", "pkg/main.py"])[0][-1], Place("main.py", "f"))
def test_resolves_a_shortened_path_after_a_repeated_place_by_the_repeat(self):
tracked = ["bin/a/main.py", "bin/b/main.py", "bin/b/bench.py"]
places = [Place("bin/a/main.py", "main"), Place("bin/b/bench.py", "bench"), Place("bin/a/main.py", "main"), Place("main.py", "figures")]
self.assertEqual(resolved(places, tracked)[0][-1], Place("bin/a/main.py", "figures"))
self.assertEqual(tracked_places(places, tracked)[0][-1], Place("bin/a/main.py", "figures"))
def test_drops_a_shortened_path_two_files_match_with_no_place_before_it(self):
self.assertEqual(resolved([Place("cycle/run.py", "step")], ["bin/tool/cycle/run.py", "bin/other/cycle/run.py"]), ([], [Place("cycle/run.py", "step")]))
self.assertEqual(tracked_places([Place("cycle/run.py", "step")], ["bin/tool/cycle/run.py", "bin/other/cycle/run.py"]), ([], [Place("cycle/run.py", "step")]))
class Bench(unittest.TestCase):
@@ -198,7 +198,7 @@ class BlackBox(unittest.TestCase):
directory = tempfile.TemporaryDirectory()
self.addCleanup(directory.cleanup)
top = directory.name
env = host_env()
env = isolated_env()
for args in (["init", "-q"], ["config", "user.email", "t@t"], ["config", "user.name", "t"]):
subprocess.run(["git", *args], cwd=top, check=True, env=env)
pathlib.Path(top, "a.py").write_text("def hard(a):\n\tif a:\n\t\tfor x in a:\n\t\t\tif x:\n\t\t\t\treturn x\n\n\ndef easy():\n\treturn 1\n")
@@ -211,7 +211,7 @@ class BlackBox(unittest.TestCase):
return top
def run_tool(self, top: str, *args: str) -> subprocess.CompletedProcess:
return subprocess.run([sys.executable, str(TOOL), *args], cwd=top, capture_output=True, text=True, env=host_env())
return subprocess.run([sys.executable, str(TOOL), *args], cwd=top, capture_output=True, text=True, env=isolated_env())
def test_prints_each_metrics_score_on_each_run_and_the_medians(self):
top = self.project("# Comprehension history\n\n## 2026-10-08T12:50:49Z, PR #1 at COMMIT\n\nJunior A, hardest first:\n\n- `a.py:1` `hard`\n")
+6
View File
@@ -0,0 +1,6 @@
import pathlib
def project_key(checkout_root: str, home: pathlib.Path) -> str:
"""The checkout's path relative to ~/code, or to / outside it: what names a project's loop-claude session, its model and its archive folder."""
return checkout_root.removeprefix(f"{home / 'code'}/").lstrip("/")
+1 -60
View File
@@ -57,12 +57,11 @@ import pathlib
import re
import sys
from todo_table import cells, Item, ITEMS_HEADER, parse_row, table_rows
from failure import Failure
from markdown_sections import FENCE, sections
from file_writes import replace_file
EXEMPTIONS = ("", "decision", "defect", "principle", "question", "security")
ITEMS_HEADER = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |"
LARGEST_FORMULA = 200
LONGEST_FORMULA = 4 * LARGEST_FORMULA
OPERATORS = (ast.Add, ast.BinOp, ast.Div, ast.Expression, ast.Load, ast.Mult, ast.Sub, ast.UAdd, ast.UnaryOp, ast.USub)
@@ -70,16 +69,6 @@ VARIABLES = ("A", "G", "R", "S", "W")
WEIGHTS_HEADER = "| Goal | W |"
@dataclasses.dataclass(frozen=True)
class Item:
id: int
release: tuple[int, int, int]
exempt: str
tagline: str
scores: dict[str, int]
goals: tuple[str, ...]
@dataclasses.dataclass(frozen=True)
class Scoring:
formula: ast.Expression
@@ -88,13 +77,6 @@ class Scoring:
weights: dict[str, float]
@dataclasses.dataclass(frozen=True)
class Table:
"""Body rows of a markdown table, and the index of the first row in the file's lines."""
start: int
rows: list[str]
@dataclasses.dataclass(frozen=True)
class Ranked:
text: str
@@ -357,23 +339,6 @@ def parse_weights(lines: list[str], goals: list[str]) -> dict[str, float] | Fail
return weights
def table_rows(lines: list[str], header: str) -> Table | Failure:
if lines.count(header) != 1:
return Failure(f"todo.md needs the table header {header} once")
start = lines.index(header) + 2
if start > len(lines) or not re.fullmatch(r"\|(-+\|)+", lines[start - 1].replace(" ", "")):
return Failure(f"the table under {header} has no |---| divider line")
end = start
while end < len(lines) and lines[end].startswith("|"):
end += 1
return Table(start, lines[start:end])
def cells(row: str, count: int) -> list[str] | Failure:
found = [cell.strip() for cell in re.split(r"(?<!\\)\|", row.strip())[1:-1]]
return found if len(found) == count else Failure(f"this row needs {count} cells: {row}")
def parse_item(row: str, weights: dict[str, float]) -> Item | Failure:
item = parse_row(row)
if isinstance(item, Failure):
@@ -383,30 +348,6 @@ def parse_item(row: str, weights: dict[str, float]) -> Item | Failure:
return item
def parse_row(row: str) -> Item | Failure:
"""An item row, every cell checked but the goals it cites."""
found = cells(row, 10)
if isinstance(found, Failure):
return found
id_, release, exempt, tagline, r, s, a, g, goals, _ = found
if not re.fullmatch(r"[0-9]+", id_) or int(id_) < 1:
return Failure(f"the ID {id_!r} is not a positive whole number: {row}")
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", release):
return Failure(f"item {id_}'s release {release!r} is not a semver version")
if exempt not in EXEMPTIONS:
return Failure(f"item {id_}'s Exempt {exempt!r} is not empty or one of {', '.join(EXEMPTIONS[1:])}")
if not re.fullmatch(r"\*\*[^*].*[^*]\*\*", tagline):
return Failure(f"item {id_}'s Item cell is not one bold tagline: {tagline}")
scores = {}
for name, value in zip("RSAG", (r, s, a, g)):
if not re.fullmatch(r"[0-9]+", value) or not 1 <= int(value) <= 10:
return Failure(f"item {id_}'s {name} {value!r} is not a whole number from 1 to 10")
scores[name] = int(value)
cited = tuple(goal.strip() for goal in goals.split(","))
major, minor, patch = (int(part) for part in release.split("."))
return Item(int(id_), (major, minor, patch), exempt, tagline, scores, cited)
def render_item(item: Item, value: float) -> str:
release = ".".join(str(part) for part in item.release)
scores = " | ".join(str(item.scores[name]) for name in "RSAG")
+2 -2
View File
@@ -9,7 +9,7 @@ from markdown_sections import sections
from transcript_archive.outcomes import STAMPED
FIGURE = re.compile(rf"^Reading cost, chunk (\w+), finished ({STAMPED})$")
PLACE = re.compile(r"^- `(.+)` (\d+)$")
FIGURE_PLACE = re.compile(r"^- `(.+)` (\d+)$")
PLACES_KEPT = 10
RESET = "Reading cost reset, "
RESET_AT = re.compile(rf"^{RESET}({STAMPED})$")
@@ -43,7 +43,7 @@ def readings_of(record: str) -> list[Reading] | Failure:
tokens = [int(match.group(1)) for line in lines if (match := TOKENS.match(line))]
if len(tokens) != 1:
return Failure(f"the figure under \"## {heading}\" holds no single tokens line; restore it from git history")
dated.append((named.group(2), heading, Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := PLACE.match(line))}))))
dated.append((named.group(2), heading, Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := FIGURE_PLACE.match(line))}))))
elif heading.startswith("Reading cost"):
return Failure(f"\"## {heading}\" names no chunk and UTC time, or no reset time; restore it from git history")
return [reading for finished, _, reading in sorted(dated, key=lambda item: item[:2]) if finished > max(resets)]
+4 -4
View File
@@ -56,8 +56,8 @@ def main(argv: list[str]) -> int:
if isinstance(checkout, Failure):
return failed(checkout)
home = pathlib.Path.home()
archive = project_archive(checkout.root, home, os.environ)
record = pathlib.Path(given[0]) if given else pathlib.Path(checkout.top, "docs", "reading-cost.md")
archive = project_archive(checkout.main_root, home, os.environ)
record = pathlib.Path(given[0]) if given else pathlib.Path(checkout.current_root, "docs", "reading-cost.md")
updated = update(record, archive, checkout, str(home), utc(time.time()) if reset else None)
if isinstance(updated, Failure):
return failed(updated)
@@ -122,8 +122,8 @@ def located() -> Checkout | Failure:
return Failure(f"could not find the checkout: {failure}")
if listed.returncode != 0:
return Failure(listed.stderr.strip())
# The common dir is the checkout's .git, whichever worktree runs this.
return Checkout(str(pathlib.Path(common).parent), top, frozenset(filter(None, listed.stdout.split("\0"))))
# The common dir is the main checkout's .git, whichever worktree runs this.
return Checkout(top, str(pathlib.Path(common).parent), frozenset(filter(None, listed.stdout.split("\0"))))
def read_record(record: pathlib.Path) -> str | Failure:
+4 -4
View File
@@ -16,11 +16,11 @@ class Reading:
places: collections.Counter[str]
@dataclasses.dataclass(frozen=True)
class Checkout:
root: str
top: str
"""main_root is the main checkout, whose path keys the archive; current_root is the checkout or worktree this runs in."""
current_root: str
main_root: str
tracked: frozenset[str]
@@ -31,7 +31,7 @@ def measure(threads: list[bytes], checkout: Checkout, home: str) -> Reading | No
return None
before = [[entry for entry in thread if entry.get("type") == "assistant" and str(entry.get("timestamp", "")) < commit] for thread in (worker, *subagents)]
reads = [reads_of(part, str(entry.get("cwd") or "/"), home) for thread in before for entry in thread for part in assistant_parts(entry)]
roots = [checkout.root, *(added for found in reads for added in found.added)]
roots = [checkout.main_root, *(added for found in reads for added in found.added)]
places = collections.Counter(place for found in reads for path in found.read if (place := relative(path, roots)) in checkout.tracked)
return Reading(sum(map(tokens_in, before)), places)
+3 -3
View File
@@ -17,7 +17,7 @@ from reading_cost.measure import Checkout, Reading
from failure import Failure
from reading_cost import figures, measure
from transcript_archive.read import finished_chunks, FinishedChunk
from host_env import host_env
from test_env import isolated_env
from transcript_archive import outcomes
from transcript_archive.location import project_archive
from reading_cost.shell_reads import Reads, shell_reads
@@ -61,7 +61,7 @@ class Measure(unittest.TestCase):
said("6", "m5", tokens=100, name="Read", input={"file_path": "/c/p/late.py"}),
]
subagent = [said("2", "s1", tokens=1, name="Grep", input={"path": "/c/w/src/a.py"}), said("7", "s2", tokens=100, name="Read", input={"file_path": "/c/p/late.py"})]
checkout = Checkout("/c/p", "/c/w", frozenset({"README.md", "late.py", "src/a.py", "src/b.py", "x.py"}))
checkout = Checkout("/c/w", "/c/p", frozenset({"README.md", "late.py", "src/a.py", "src/b.py", "x.py"}))
self.assertEqual(measure.measure([jsonl(worker), jsonl(subagent)], checkout, "/h"), Reading(33, collections.Counter({"src/a.py": 2, "README.md": 1, "src/b.py": 1})))
def test_measures_nothing_for_a_worker_that_never_committed(self):
@@ -102,7 +102,7 @@ class ReadingCostTool(unittest.TestCase):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.home = pathlib.Path(os.path.realpath(temp.name))
self.env = {**host_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.home)}
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.home)}
self.checkout = self.home / "code" / "proj"
self.checkout.mkdir(parents=True)
(self.checkout / "a.py").write_text("")
+4
View File
@@ -1,6 +1,10 @@
import pathlib
import subprocess
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import ROOT
PRIVATE = [
+4
View File
@@ -1,7 +1,11 @@
import ast
import pathlib
import subprocess
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import ROOT
+4
View File
@@ -1,6 +1,10 @@
import json
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import ROOT
+5 -1
View File
@@ -1,7 +1,11 @@
import pathlib
import shutil
import subprocess
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import readme_floor, ROOT
@@ -11,5 +15,5 @@ class Types(unittest.TestCase):
self.assertIsNotNone(floor, "README.md's Install names `Python 3.N or newer`")
if shutil.which("uv") is None:
self.fail("uv on PATH fetches mypy; see README.md's Test line")
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "--exclude", "/test_", "-p", "claude_code", "-m", "comprehension_history", "-m", "host_env", "-p", "larv_review", "-p", "loop_claude", "-p", "metric_bench", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline", "-p", "transcript_archive"], capture_output=True, cwd=ROOT / "bin", text=True)
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "--exclude", "/test_", "-p", "claude_code", "-m", "comprehension_history", "-p", "larv_review", "-p", "loop_claude", "-p", "metric_bench", "-m", "project_key", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline", "-m", "todo_table", "-p", "transcript_archive"], capture_output=True, cwd=ROOT / "bin", text=True)
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
+1 -1
View File
@@ -2,5 +2,5 @@ import os
# A git hook's GIT_DIR, the host's LOOP_CLAUDE_* or PYTHONPATH would point the program under test at the host's own state.
def host_env() -> dict[str, str]:
def isolated_env() -> dict[str, str]:
return {name: value for name, value in os.environ.items() if not name.startswith(("GIT_", "LOOP_CLAUDE_", "PYTHON"))}
+1 -2
View File
@@ -9,8 +9,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from failure import Failure
import rank_todo
ROOT = pathlib.Path(__file__).resolve().parents[1]
from repo_checks.test_python_floor import ROOT
README = "# P\n\n## Goals\n\n1. Fast.\n2. Correct.\n 1. Tested.\n\n## Audience\n\nMe.\n"
+79
View File
@@ -0,0 +1,79 @@
import dataclasses
import re
from failure import Failure
EXEMPTIONS = ("", "decision", "defect", "principle", "question", "security")
ITEMS_HEADER = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |"
@dataclasses.dataclass(frozen=True)
class Item:
id: int
release: tuple[int, int, int]
exempt: str
tagline: str
scores: dict[str, int]
goals: tuple[str, ...]
@dataclasses.dataclass(frozen=True)
class Table:
"""Body rows of a markdown table, and the index of the first row in the file's lines."""
start: int
rows: list[str]
def items(todo: str) -> list[Item] | Failure:
"""Each row of a todo.md's items table."""
table = table_rows(todo.splitlines(), ITEMS_HEADER)
if isinstance(table, Failure):
return table
items = []
for row in table.rows:
item = parse_row(row)
if isinstance(item, Failure):
return item
items.append(item)
return items
def table_rows(lines: list[str], header: str) -> Table | Failure:
if lines.count(header) != 1:
return Failure(f"todo.md needs the table header {header} once")
start = lines.index(header) + 2
if start > len(lines) or not re.fullmatch(r"\|(-+\|)+", lines[start - 1].replace(" ", "")):
return Failure(f"the table under {header} has no |---| divider line")
end = start
while end < len(lines) and lines[end].startswith("|"):
end += 1
return Table(start, lines[start:end])
def cells(row: str, count: int) -> list[str] | Failure:
found = [cell.strip() for cell in re.split(r"(?<!\\)\|", row.strip())[1:-1]]
return found if len(found) == count else Failure(f"this row needs {count} cells: {row}")
def parse_row(row: str) -> Item | Failure:
"""An item row, every cell checked but the goals it cites."""
found = cells(row, 10)
if isinstance(found, Failure):
return found
id_, release, exempt, tagline, r, s, a, g, goals, _ = found
if not re.fullmatch(r"[0-9]+", id_) or int(id_) < 1:
return Failure(f"the ID {id_!r} is not a positive whole number: {row}")
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", release):
return Failure(f"item {id_}'s release {release!r} is not a semver version")
if exempt not in EXEMPTIONS:
return Failure(f"item {id_}'s Exempt {exempt!r} is not empty or one of {', '.join(EXEMPTIONS[1:])}")
if not re.fullmatch(r"\*\*[^*].*[^*]\*\*", tagline):
return Failure(f"item {id_}'s Item cell is not one bold tagline: {tagline}")
scores = {}
for name, value in zip("RSAG", (r, s, a, g)):
if not re.fullmatch(r"[0-9]+", value) or not 1 <= int(value) <= 10:
return Failure(f"item {id_}'s {name} {value!r} is not a whole number from 1 to 10")
scores[name] = int(value)
cited = tuple(goal.strip() for goal in goals.split(","))
major, minor, patch = (int(part) for part in release.split("."))
return Item(int(id_), (major, minor, patch), exempt, tagline, scores, cited)
+2 -5
View File
@@ -1,12 +1,9 @@
import pathlib
import typing
from project_key import project_key
def project_archive(checkout_root: str, home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
archive = pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else home / ".claude" / "loop-claude-transcripts"
return archive / project_key(checkout_root, home)
def project_key(checkout_root: str, home: pathlib.Path) -> str:
"""The checkout's path relative to ~/code, or to / outside it."""
return checkout_root.removeprefix(f"{home / 'code'}/").lstrip("/")
@@ -9,7 +9,8 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from transcript_archive import outcomes, write
from transcript_archive.location import project_archive, project_key
from transcript_archive.location import project_archive
from project_key import project_key
NOW = 1790000000
+77
View File
@@ -732,3 +732,80 @@ Inherited architect, would restructure first:
- `bin/larv_review/git.py:10` `git`, `bin/metric_bench/main.py:143` `git` and `bin/reading_cost/main.py:113` `located`: one git runner
- `bin/loop_claude/project/checkout.py:32` `read_utf8`: a generic reader under a lying name
- `bin/larv_review/main.py:101` `main`: an if/elif dispatch where the next command lands
## 2026-10-09T12:31:47Z, PR #234 at b1913ed, against 2157f59
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | better | same | better | better |
| Inherited architect | better | better | better | better | better |
Mid A, decided by:
- Locality, Navigation: `bin/transcript_archive/outcomes.py:4` `FINISHED_AT` and `finished`: one module now writes and parses the outcome lines, which `bin/loop_claude/cycle/run.py:83` built inline and another file parsed
- Locality, Navigation: `bin/reading_cost/main.py:35` imports: reading-cost depends only on the shared `transcript_archive`, no longer on loop-claude's internals
- Locality: `bin/loop_claude/paths.py:15` `paths_from`: no longer reads the environment; `LOOP_CLAUDE_TRANSCRIPTS` has one home in `bin/transcript_archive/location.py:5` `project_archive`
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the import root, the shared packages and where tests live
- Self-sufficiency: `bin/transcript_archive/write.py:13` `archive`: states the archive's layout at the write
- Shape: `bin/loop_claude/loop/main.py:86` `run_loop`: imports `project_key`, which names the session and the models entry, from the archive package; `bin/loop_claude/cycle/worker.py:7` takes `quoted` from the archive's outcomes
- Shape: `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES`: judge_reply merged in, 183 lines
Inherited architect, decided by:
- Navigation, Shape: `bin/reading_cost/main.py:35` `main`: imports one shared package named for what it holds, in place of loop-claude's private packages
- Locality, Self-sufficiency: `bin/transcript_archive/outcomes.py:10` `finished`: the outcome texts sit beside `FINISHED_AT`, the regex that reads them back
- Shape: `bin/claude_code/session.py:21` `session_named`: the Claude Code package depends on nothing in loop-claude
- Locality: `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES` and `:93` `verdict_from`: in one file with `run_judge`
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the cap no longer depends on case order
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the layout and puts each test beside its package
- Self-sufficiency: `bin/loop_claude/cycle/run.py:34` `Cycle.unfinished_evidence`: the name says what `remaining` needed a comment for
- Shape: `bin/loop_claude/loop/main.py:86` `run_loop`, `bin/loop_claude/cycle/worker.py:7` `worker_args` and `bin/host_env.py:5` `host_env`: small seams against the decoupling
## 2026-10-09T12:32:00Z, PR #234 at b1913ed
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Junior A | 7 | 6 | 6.5 | 6 | 6.5 |
| Mid A | 7 | 6 | 6.5 | 6 | 6.5 |
| Maintainability senior | 7 | 6.5 | 7 | 6.5 | 7 |
| Inherited architect | 7 | 6.5 | 6 | 6.5 | 6.5 |
| Mean | 7.00 | 6.25 | 6.50 | 6.25 | 6.63 |
Junior A, hardest first:
- `skills/larv-review/SKILL.md:55` "Round cap": the counting rule spread across Round cap, Re-entry, Fresh and Suspended
- `bin/larv_review/main.py:197` `record_ruling`: split, fell, refuse and stand branches; `ruling` returns None for a split and `fallen` reuses it
- `bin/larv_review/scoring.py:54` `scoring_run_due`: why the next-release fallback holds is stated nowhere
- `bin/larv_review/base.py:14` `measured_commit`: mechanics without purpose
- `bin/loop_claude/cycle/judge.py:126` `run_judge`: the prompt assembled from `records.py` constants; two texts, two matchers
- `bin/reading_cost/measure.py:21` `Checkout`: `root` and `top` read as synonyms
Mid A, hardest first:
- `bin/larv_review/main.py:197` `record_ruling`: said and counted, None as split, three fall steps
- `bin/comprehension_history.py:74` `entries`: the history format is defined only by its writer
- `bin/loop_claude/cycle/judge.py:126` `run_judge`: elided evidence fails, a misquoted answer becomes an interrupt
- `bin/larv_review/scoring.py:54` `scoring_run_due`: one dense sentence over two plans; the module also parses the plan
- `bin/larv_review/base.py:14` `measured_commit`: the why lives in the help and the skill
- `bin/loop_claude/cycle/states.py:69` `transition`: `quoted_answers` threaded through three states and two files
Maintainability senior, hardest first:
- `bin/larv_review/main.py:197` `record_ruling`: said and count, ruled reassigned, a silent fall-through; `ruling`'s None means two things
- `bin/larv_review/base.py:14` `measured_commit`: a three-step fallback across three files, ordered by an ISO string unsaid
- `bin/comprehension_history.py:74` `entries`: the format is never stated in one place
- `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES`: the prompt interpolates four constants from another module
- `bin/larv_review/scoring.py:54` `scoring_run_due`: five inputs; `scoring` also parses the plan
- `bin/rank_todo.py:416` `rank_details`: regex surgery in a script larv-review imports as a library
Inherited architect, would restructure first:
- `bin/loop_claude/project/records.py:6` `QUOTE_SOURCES`: prompt fragments spliced into the judge, worker and interrupt texts
- `bin/transcript_archive/location.py:10` `project_key`: the project's identity filed under the archive
- `bin/host_env.py:5` `host_env`: a test helper among the production modules
- `bin/larv_review/rounds.py:8` `round_counts`: only counts; the cap lives in the skill
- `bin/larv_review/base.py:1` the module: the ruling base, named like a base class
- `bin/larv_review/scoring.py:6` imports: one tool imports another tool's internals from `bin/rank_todo.py`
- `bin/loop_claude/loop/main.py:1` the docstring: restates constants it cannot keep true
+1 -1
View File
@@ -6,7 +6,7 @@ A scoring-run seat's prompt is the opening, the seat's background, the shared br
- `{main}`: the project's main checkout.
- `{forbidden}`: `todo.md, docs/decisions.md, docs/comprehension-history.md or docs/reading-cost.md (the plan, the decision log, past panels' records and past chunks' reading cost)`.
- `{git}`: `git -C {checkout} ls-files`, and for a paired seat also `git -C {base checkout} ls-files`, each in backticks.
- `{files}`: every tracked non-test file but those `{forbidden}` names, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, such as `bin/loop_claude/**/*.py`.
- `{files}`: every tracked non-test file but those `{forbidden}` names, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, with the test files it would take in named as left out, such as `bin/**/*.py` except `test_*.py`.
- `{preloaded}`: the files the harness pre-loads into a seat, found per `SKILL.md` → "What makes the result trustworthy".
- `{anchors}` and `{dimensions}`: the six anchor bullets and the four dimension bullets in `SKILL.md` → "The score", verbatim, without bold and each on one line.
- `{language}`, `{kind}`, `{domain}`, `{domain docs}` and `{3am question}`: the project's `AGENTS.md` → "Scoring run".