Merge pull request 'Group bin/ by tool, and lift its comprehension scores towards the floor' (#234) from floor-restructure into main

This commit was merged in pull request #234.
This commit is contained in:
2026-10-09 15:23:55 +02:00
72 changed files with 1412 additions and 818 deletions
+10 -1
View File
@@ -17,6 +17,8 @@
!agents/
!agents/*.md
!bin/
!bin/claude_code/
!bin/claude_code/*.py
!bin/comprehension_history.py
!bin/failure.py
!bin/file_writes.py
@@ -27,16 +29,23 @@
!bin/loop_claude/
!bin/loop_claude/**/
!bin/loop_claude/**/*.py
!bin/markdown_sections.py
!bin/markdown_text.py
!bin/metric-bench
!bin/metric_bench/
!bin/metric_bench/*.py
!bin/project_key.py
!bin/rank_todo.py
!bin/reading-cost
!bin/reading_cost/
!bin/reading_cost/*.py
!bin/repo_checks/
!bin/repo_checks/*.py
!bin/statusline.py
!bin/test_*.py
!bin/todo_table.py
!bin/transcript_archive/
!bin/transcript_archive/*.py
!bin/utc_stamp.py
!docs/
!docs/*.md
!principles/
+1 -1
View File
@@ -3,7 +3,7 @@
## Unreleased
- The stability reviewer keeps its scratch work in a directory it creates for the run, deletes that directory afterwards, and deletes nothing else.
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling <votes file> [<scores file>]` and `record-scoring <scores file> [<item ID>...]`, which append to `docs/comprehension-history.md` and refuse a second entry on what a seats file's first line reads; `record-scoring` refuses a run the plan does not call, and prints `Floor:` naming each mean under the floor). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release whose last plan item moves to another release is due a scoring run too, once work has landed towards it. A seats file must name commits the checkout holds. When origin cannot be fetched, the tool reads it as last fetched and says so.
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`, which ignores case in a round's commit title, counts a repeated round number as a round of its own, and says to ask you before another round once a pass is at or past the cap), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling <votes file> [<scores file>]` and `record-scoring <scores file> [<item ID>...]`, which append to `docs/comprehension-history.md` and refuse a second entry on what a seats file's first line reads; `record-scoring` refuses a run the plan does not call, and prints `Floor:` naming each mean under the floor). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release whose last plan item moves to another release is due a scoring run too, once work has landed towards it. A seats file must name commits the checkout holds. When origin cannot be fetched, the tool reads it as last fetched and says so.
- The new `bin/metric-bench` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest: cognitive complexity, length, shared state and change spread, each scored by how well it ranks the named functions above the rest. No metric enters review yet.
- larv-review blocks a merge when simulated readers rule that the PR makes the project read worse than the last measured commit: the default branch's newest merged change that added a scoring run to `docs/comprehension-history.md`, or whose last ruling there is better. A ruling of same leaves that commit in place, so a worse ruling may name places earlier chunks wrote. In each architecture pass Mid A and the inherited architect vote better, same or worse on each dimension and overall: `Navigation same, Locality worse, Shape same, Self-sufficiency same, Overall better`. When their overall votes differ, the maintainability senior and Junior A vote too. A ruling needs a lead of two or more votes, else it is same. Each place a seat names opens with the dimensions it moves, such as `Locality, Shape: `, or the run fails, and a vote that no place opens with counts as same. A dimension falls when its worse votes lead by two. When a dimension falls in a better or same ruling, `record-ruling` asks for the four scoring seats and, given their scores file, counts the ruling as worse when that dimension's mean is under the floor. A worse ruling never ships: after three rounds larv-review asks you for another round, a no leaves the PR unmerged, and so does an unattended run. The four-seat scoring run checks the comprehension floor only when a chunk ships or removes a release's last plan item, or ships an item naming the floor; between those runs, a score under the floor blocks a merge only through a fallen dimension, and that block counts as a worse ruling. On those runs, a score under the floor blocks the merge until fixed or until you decide to ship; a decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
- larv-review reruns fewer passes after a fix. A fix reruns the pass whose 🔴 or 🟡 it answers, plus stability when the fix changes behaviour, security when it opens an untrusted-input boundary or touches authn/authz, secrets or crypto, and architecture when it carries an architectural decision. A rerun reviews only what changed since that pass last reviewed. A fix for a 🟢, a ❓ or any prose finding reruns no pass, and prose takes one round.
+2
View File
@@ -76,6 +76,8 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
## Tracking a new file
`.gitignore` is an allowlist. Track a file by negating it and its parent directories, then confirm with `git check-ignore -v <path>`:
@@ -6,7 +6,6 @@ import shutil
import uuid
from failure import Failure
from loop_claude.paths import Paths
NAMESPACE = uuid.UUID("64fb586b-d926-4bb3-a44c-c903e86d3eec")
@@ -19,10 +18,8 @@ class Session:
name: str
def session_for(project_key: str, paths: Paths) -> Session:
# "A " is the human's cue in the session picker that the session is the loop's.
name = f"A {project_key} loop-claude"
return Session(id=uuid.uuid5(NAMESPACE, name), claude_code_transcripts=paths.claude_code_transcripts, name=name)
def session_named(name: str, claude_code_transcripts: pathlib.Path) -> Session:
return Session(id=uuid.uuid5(NAMESPACE, name), claude_code_transcripts=claude_code_transcripts, name=name)
# <claude_code_transcripts>/-home-me-code-log/<session id>.jsonl
+93
View File
@@ -0,0 +1,93 @@
import json
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from claude_code import entries, transcript, trust
def assistant(*parts):
return {"type": "assistant", "message": {"content": list(parts)}}
def human(content):
return {"type": "user", "message": {"content": content}}
def queued(prompt, kind):
return {"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "prompt", "origin": {"kind": kind}, "prompt": prompt}}
class Render(unittest.TestCase):
def test_joins_assistant_text_tool_calls_human_turns_and_loop_prompts_and_skips_the_rest(self):
said = transcript.render([
human("do it"),
assistant({"type": "text", "text": "Starting."}, {"type": "tool_use", "name": "Bash", "input": {"command": "ls"}}),
assistant({"type": "text", "text": " "}),
human([{"type": "tool_result", "content": "not typed"}]),
human("<command-name>/exit</command-name>"),
human([{"type": "text", "text": "yes, "}, {"type": "text", "text": "merge it"}]),
queued("B, while you work", "human"),
queued("origin/main moved", "coordinator"),
{**human("Continue the task; the previous run stopped before it was finished."), "promptSource": "sdk"},
])
self.assertEqual(said, '[human] do it\n\nStarting.\n\n[tool Bash] {"command": "ls"}\n\n[human] yes, merge it\n\n[human] B, while you work\n\n[loop] Continue the task; the previous run stopped before it was finished.')
class FirstTimestampedLine(unittest.TestCase):
def test_skips_the_lines_every_chunk_shares_and_else_takes_the_first(self):
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\nnot JSON\n{"timestamp": "t"}\n{"timestamp": "u"}\n'), b'{"timestamp": "t"}')
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\n[1]\n'), b'{"type": "custom-title"}')
self.assertEqual(transcript.first_timestamped_line(b""), b"")
class HumanTurns(unittest.TestCase):
def test_counts_typed_turns_and_not_tool_results(self):
self.assertEqual(transcript.human_turns([
human("typed"),
human([{"type": "text", "text": "typed as a block"}]),
human([{"type": "tool_result", "content": "not typed"}]),
human("<command-name>/model</command-name>"),
{**human("<3 merge it"), "promptSource": "typed"},
{**human("<task-notification>done</task-notification>"), "promptSource": "system"},
{**human([{"type": "text", "text": "<task-notification>done</task-notification>"}]), "promptSource": "system"},
{**human("peer message"), "isMeta": True},
{**human("the loop's own prompt"), "promptSource": "sdk"},
human([{"type": "text", "text": "[Request interrupted by user]"}]),
human([]),
queued("typed while claude worked", "human"),
queued("<task-notification>done</task-notification>", "task-notification"),
{"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "task-notification", "prompt": "<task-notification>done</task-notification>"}},
assistant({"type": "text", "text": "said"}),
]), 4)
class PartLine(unittest.TestCase):
def test_renders_text_and_tool_calls_and_nothing_else(self):
self.assertEqual(transcript.part_line({"type": "text", "text": "hi"}, 20), "hi")
self.assertEqual(transcript.part_line({"type": "text", "text": " \n"}, 20), "")
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Bash", "input": {"command": "x" * 500}}, 20), '[tool Bash] {"command": "xxxxxxx')
self.assertEqual(transcript.part_line("not a part", 20), "")
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Write", "input": {"content": "Hälsa på"}}, 40), '[tool Write] {"content": "Hälsa på"}')
class Parse(unittest.TestCase):
def test_yields_only_json_objects(self):
self.assertEqual(entries.parse_entry('{"type": "user"}'), {"type": "user"})
self.assertIsNone(entries.parse_entry("not json"))
self.assertIsNone(entries.parse_entry("[1, 2]"))
class MarkTrusted(unittest.TestCase):
def test_returns_the_updated_config_and_leaves_the_one_it_was_handed(self):
config = {"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"kept": True}}}
handed = json.dumps(config)
self.assertEqual(trust.mark_trusted(config, ["/a", "/b", "/c"]), ({"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"hasTrustDialogAccepted": True, "kept": True}, "/c": {"hasTrustDialogAccepted": True}}}, ["/b", "/c"]))
self.assertEqual(json.dumps(config), handed)
self.assertEqual(trust.mark_trusted({}, ["/a"]), ({"projects": {"/a": {"hasTrustDialogAccepted": True}}}, ["/a"]))
if __name__ == "__main__":
unittest.main()
@@ -1,6 +1,6 @@
import json
from loop_claude.claude_code.entries import assistant_parts, human_text, loop_text, parse_entry, part_text, tool_call
from claude_code.entries import assistant_parts, human_text, loop_text, parse_entry, part_text, tool_call
from failure import Failure
JUDGED_TOOL_INPUT = 400
+28 -8
View File
@@ -1,10 +1,10 @@
import dataclasses
import decimal
import re
import time
from failure import Failure
from markdown_sections import sections
from markdown_text import sections
from utc_stamp import utc
ARCHITECT = "Inherited architect"
DECIDED_BY = "decided by"
@@ -22,7 +22,6 @@ RULING = re.compile(r"^Ruling: (better|same|worse)$", re.M)
RULED_PLACES = re.compile(rf"^.+, {DECIDED_BY}:$")
SCORED_PLACES = re.compile(rf"^.+, ({HARDEST_FIRST}|{RESTRUCTURE_FIRST}):$")
SENIOR = "Maintainability senior"
STAMP = "%Y-%m-%dT%H:%M:%SZ"
TITLE = "# Comprehension history\n"
@@ -55,6 +54,14 @@ class Entry:
return self.against is not None
@dataclasses.dataclass(frozen=True)
class HistoryChange:
"""A merged change on the default branch that touched the history, and the history before and after it."""
commit: str
before: str
after: str
@dataclasses.dataclass(frozen=True)
class Heading:
title: str
@@ -64,13 +71,26 @@ class Heading:
@dataclasses.dataclass(frozen=True)
class Scoring:
class ScoringRun:
"""The four scoring seats' lines, their scores as written and the means, in DIMENSIONS' order."""
seats: list[Seat]
scores: dict[str, list[str]]
means: list[decimal.Decimal]
# A paired ruling as entry_text writes it. A scoring run's heading has no `, against`; it has no Ruling line, ends its table with a Mean row, and labels each seat's places `<seat>, hardest first:` or, for the architect, `<seat>, would restructure first:`.
#
# ## 2026-10-09T09:47:25Z, PR #232 at 324d650, against 12d6d75
#
# Ruling: same
#
# | Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
# |---|---|---|---|---|---|
# | Mid A | better | worse | same | same | better |
#
# Mid A, decided by:
#
# - Locality: `bin/x.py:12` `run`: three branches to hold at once
def entries(history: str) -> list[Entry]:
"""Each paired ruling and scoring run in a comprehension history, in the order the file holds them; a section whose heading names no commit is left out."""
found = []
@@ -114,7 +134,7 @@ def bullet_places(bullet: str) -> list[Place]:
def heading(at: float, read: str, paired: bool) -> Heading | Failure:
"""An entry's heading, matched by HEADING: when the seats finished, in UTC, and what they read."""
title = f"{time.strftime(STAMP, time.gmtime(at))}, {read}"
title = f"{utc(at)}, {read}"
if (named := HEADING.match(title)) and bool(named.group(4)) == paired:
return Heading(title, named.group(2), named.group(3), named.group(4))
return Failure(f"the seats file's first line is not `<PR or branch> at <commit>{', against <base>' if paired else ''}`: {read}")
@@ -126,7 +146,7 @@ def appended(history: str, entry: str) -> str:
return f"{text}\n{entry}"
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, notes: list[str], scoring: Scoring | None) -> str:
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, notes: list[str], scoring: ScoringRun | None) -> str:
"""A paired ruling's entry, with any notes, such as why a fall decided it, and the scores that decided a fall; a seat's vote that counted as same for want of a place says so."""
votes = [[seat.name, *map(vote_text, DIMENSIONS, said[seat.name], counted[seat.name])] for seat in seats]
tables = [[["Seat", *DIMENSIONS], *votes]] + ([scores_table(scoring)] if scoring else [])
@@ -140,11 +160,11 @@ def vote_text(dimension: str, said: str, counted: str) -> str:
return f"same (voted {said}, naming no place)" if dimension == DIMENSIONS[-1] else f"same (voted {said}, no place naming {dimension})"
def scoring_entry(title: str, scoring: Scoring) -> str:
def scoring_entry(title: str, scoring: ScoringRun) -> str:
return entry_text(title, [], [scores_table(scoring)], scoring_labels(scoring.seats))
def scores_table(scoring: Scoring) -> list[list[str]]:
def scores_table(scoring: ScoringRun) -> list[list[str]]:
return [["Seat", *DIMENSIONS], *([seat, *values] for seat, values in scoring.scores.items()), [MEAN, *map(str, scoring.means)]]
+1 -2
View File
@@ -2,8 +2,7 @@ import re
import subprocess
from failure import Failure
from comprehension_history import HISTORY
from larv_review.base import HistoryChange
from comprehension_history import HISTORY, HistoryChange
def git(*args: str) -> str | Failure:
+28 -165
View File
@@ -2,6 +2,12 @@
usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]
The comprehension check keeps a project from reading worse, two ways. In a paired ruling, simulated
readers, the seats, read
the branch's head beside a base and vote better, same or worse, overall and on each dimension; a
worse ruling blocks the merge, which is the ratchet. In a scoring run, four seats score the project
from 1 to 10; a mean under the floor of {floor} blocks the merge, unless you decide to ship.
Run it inside a project checkout or one of its worktrees. Each command but record-ruling first
fetches origin, when the checkout has that remote, and takes origin's default branch, or else the
local main or master, as the default branch. When the fetch fails, round-count reads origin as
@@ -14,9 +20,10 @@ history, or whose last added paired ruling, by heading time, is better; a merge
one change against its first parent. With none, the base is the one the history's first paired
ruling names; with no ruling either, it is the merge base of HEAD and the default branch.
round-count prints each pass's count: the highest <n> among the titles opening `Answer <pass> round
<n>:` of commits the default branch does not hold. It marks each pass at the cap of {cap} rounds,
and each name in such a title that is not a pass.
round-count prints each pass's round count among the commits not yet on the default branch whose
titles open `Answer <pass> round <n>:`, ignoring case. The count is the highest <n>, or the number of
such commits when that is more. At or past the cap of {cap} rounds, it says to ask the human before
another round. It also lists each name in such a title that is not a pass.
needs-scoring-run takes the IDs of the todo.md items the chunk ships, none when it ships none, and
prints why the chunk is due a scoring run, or `No scoring run due.` It is due when the chunk ships
@@ -71,25 +78,23 @@ time and the seats file's first line. Each commit that line names must be in the
when the history already holds an entry on what that line reads.
Exit 1, saying why, on unknown arguments, an unknown item ID, a seats file it cannot read or that
breaks the shape above, or when git cannot start, finds no checkout or no default branch, cannot
fetch origin for ruling-base or needs-scoring-run, or cannot read or write the history or todo.md.
breaks the shape above, a scores file record-ruling did not call for or one naming another commit,
or when git cannot start, finds no checkout or no default branch, cannot
fetch origin for ruling-base, needs-scoring-run or record-scoring, or cannot read or write the history or todo.md.
"""
import pathlib
import re
import sys
import time
import typing
from comprehension_history import appended, DIMENSIONS, heading, Heading, HISTORY, recorded, ruling_entry, Scoring, scoring_entry, vote_text
from larv_review.base import base
from larv_review.ruling_base import base
from larv_review.rounds import CAP, PASSES, round_counts
from larv_review.git import commit_exists, default_branch, file_at, git, history_changes, landed_since_release, top
from larv_review.ruling import counted, fallen, ruling, votes
from larv_review.git import default_branch, file_at, git, history_changes
from failure import Failure
from larv_review.scoring import fall_ruling, fall_step, FLOOR, means, plan_items, scoring_run_due, under_floor
from rank_todo import ITEMS_HEADER
from file_writes import replace_file
from larv_review.seats import scores, seats_file, SeatsFile
from larv_review.scoring import FLOOR
from comprehension_history import HISTORY
from larv_review.plan import plan_reasons
from larv_review.record import record_ruling, record_scoring
USAGE = "usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]"
@@ -100,7 +105,7 @@ def main(argv: list[str]) -> int:
return 0
command, *given = argv or [""]
if command == "ruling-base" and not given:
done = with_default_branch(ruling_base, stale_ok=False)
done = with_default_branch(show_ruling_base, stale_ok=False)
elif command == "round-count" and not given:
done = with_default_branch(round_count, stale_ok=True)
elif command == "needs-scoring-run" and all(re.fullmatch(r"[0-9]+", id_) for id_ in given):
@@ -135,7 +140,7 @@ def with_default_branch(command: typing.Callable[[str], Failure | None], stale_o
return command(branch)
def ruling_base(default: str) -> Failure | None:
def show_ruling_base(default: str) -> Failure | None:
changes = history_changes(default)
if isinstance(changes, Failure):
return changes
@@ -157,161 +162,19 @@ def round_count(default: str) -> Failure | None:
if isinstance(titles, Failure):
return titles
for name, count in round_counts(titles.splitlines()).items():
print(f"{name}: {count}{', at the cap' if count >= CAP else ''}{'' if name in PASSES else ', not a pass: check the commit title'}")
print(f"{name}: {count}{cap_note(count) if name in PASSES else ', not a pass: check the commit title'}")
return None
def cap_note(count: int) -> str:
if count < CAP:
return ""
return f", {'at' if count == CAP else 'past'} the cap: ask the human before another round"
def needs_scoring_run(default: str, ids: list[int]) -> Failure | None:
reasons = plan_reasons(default, ids)
if isinstance(reasons, Failure):
return reasons
print(f"Scoring run due: the chunk {' and '.join(reasons)}." if reasons else "No scoring run due.")
return None
def plan_reasons(default: str, ids: list[int]) -> list[str] | Failure:
"""Why the plan calls a scoring run, from the default branch's todo.md and the checkout's."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
before = file_at(default, "todo.md")
if isinstance(before, Failure):
return before
try:
after = pathlib.Path(checkout, "todo.md").read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read todo.md: {failure}")
# While a chunk reshapes todo.md into one table, the default branch's todo.md has no items table.
planned = plan_items(before) if ITEMS_HEADER in before.splitlines() else []
if isinstance(planned, Failure):
return Failure(f"{default}'s todo.md: {planned.reason}")
chunk = plan_items(after)
if isinstance(chunk, Failure):
return Failure(f"the checkout's todo.md: {chunk.reason}")
landed = landed_since_release(default)
if isinstance(landed, Failure):
return landed
return scoring_run_due(planned, chunk, ids, landed)
def record_ruling(votes_file: pathlib.Path, scores_file: pathlib.Path | None = None) -> Failure | None:
read = read_seats(votes_file, paired=True)
if isinstance(read, Failure):
return read
named, given = read
said = votes(given.seats)
if isinstance(said, Failure):
return said
count = counted(given.seats, said)
tally = ", ".join(f"{seat.name} {vote_text(DIMENSIONS[-1], said[seat.name][-1], count[seat.name][-1])}" for seat in given.seats)
ruled = ruling([each[-1] for each in count.values()])
if ruled is None:
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the votes file, and run this again.")
return None
fell = fallen(count) if ruled != "worse" else []
scoring = None
notes = []
match fall_step(fell, scores_file is not None):
case "ask":
print(f"Fell: {', '.join(fell)}, in a {ruled} ruling. Launch the four scoring seats on {named.label} at {named.commit}, write their lines to a scores file, and run this again with both files.")
return None
case "refuse":
return Failure(f"no dimension fell in this {ruled} ruling, so it takes no scores file")
if scores_file:
scored = read_scoring(scores_file, named)
if isinstance(scored, Failure):
return scored
scoring = scored
ruled, why = fall_ruling(ruled, fell, scoring.means)
notes = [why]
print(f"Ruling: {ruled}. {' '.join(notes + [f'Overall votes as counted: {tally}.'])}")
return append(given.read, ruling_entry(named.title, given.seats, said, count, ruled, notes, scoring))
def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> Failure | None:
read = read_seats(scores_file, paired=False)
if isinstance(read, Failure):
return read
named, given = read
scoring = scored_seats(given)
if isinstance(scoring, Failure):
return scoring
reasons = plan_reasons(default, ids)
if isinstance(reasons, Failure):
return reasons
if not reasons:
return Failure("no plan item or release calls this scoring run; name the item IDs the chunk ships, as needs-scoring-run took them, or record a fall's scores with record-ruling")
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, scoring.means))}.")
if under := under_floor(scoring.means):
print(f"Floor: {', '.join(under)} under the floor of {FLOOR} block the merge, as the chunk {' and '.join(reasons)}.")
return append(given.read, scoring_entry(named.title, scoring))
def read_scoring(scores_file: pathlib.Path, votes: Heading) -> Scoring | Failure:
"""The scores a fall called, once their file names what the votes file read."""
read = read_seats(scores_file, paired=False, fresh=False)
if isinstance(read, Failure):
return read
named, given = read
if (named.label, named.commit) != (votes.label, votes.commit):
return Failure(f"the scores file's first line is not `{votes.label} at {votes.commit}`, as the votes file reads: {given.read}")
return scored_seats(given)
def scored_seats(given: SeatsFile) -> Scoring | Failure:
found = scores(given.seats)
if isinstance(found, Failure):
return found
return Scoring(given.seats, found, means(list(found.values())))
def read_seats(path: pathlib.Path, paired: bool, fresh: bool = True) -> tuple[Heading, SeatsFile] | Failure:
"""The seats file, and the heading its record goes under, once each commit it names is in the checkout and, when fresh, before anything is printed, once the history holds no entry on what it reads."""
try:
given = seats_file(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {path}: {failure}")
if isinstance(given, Failure):
return given
named = heading(time.time(), given.read, paired)
if isinstance(named, Failure):
return named
if unknown := [commit for commit in (named.commit, named.against) if commit and not commit_exists(commit)]:
return Failure(f"the seats file names {', '.join(unknown)}, which is no commit in this checkout")
if fresh:
history = history_file()
if isinstance(history, Failure):
return history
if recorded(history[1], given.read):
return Failure(f"{history[0]} already holds an entry on {given.read}; record each ruling or run once")
return named, given
def history_file() -> tuple[pathlib.Path, str] | Failure:
"""The checkout's history and its text, empty when the file is missing."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
history = pathlib.Path(checkout, HISTORY)
try:
return history, history.read_text(encoding="utf-8") if history.exists() else ""
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {history}: {failure}")
def append(read: str, entry: str) -> Failure | None:
"""Append entry unless the history already holds one on what the seats read."""
read_history = history_file()
if isinstance(read_history, Failure):
return read_history
history, text = read_history
if recorded(text, read):
return Failure(f"{history} already holds an entry on {read}; record each ruling or run once")
try:
history.parent.mkdir(parents=True, exist_ok=True)
except OSError as failure:
return Failure(f"could not create {history.parent}: {failure}")
if unwritten := replace_file(history, appended(text, entry).encode()):
return Failure(f"could not write {history}: {unwritten.reason}")
print(f"Appended to {history}.")
return None
+54
View File
@@ -0,0 +1,54 @@
import pathlib
from failure import Failure
from larv_review.git import file_at, landed_since_release, top
from todo_table import Item, items, ITEMS_HEADER
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
"""Why the plan calls a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag.
A chunk that ships an item naming the floor is due one, so it can prove the floor met. So is a chunk that empties a release after work has landed towards it, so no release ends unmeasured. Work has landed when the chunk ships one of the release's items, or when the release is the earliest one `before` names and `landed` holds. That covers a release whose last item moved to another release.
"""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
next_release = min((item.release for item in before), default=None)
for release in emptied(before, after, shipped):
if any(item.release == release for item in shipped) or (release == next_release and landed):
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
return reasons
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
named = {item.release for item in before + shipped}
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
return sorted(named - left)
def plan_reasons(default: str, ids: list[int]) -> list[str] | Failure:
"""Why the plan calls a scoring run, from the default branch's todo.md and the checkout's."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
before = file_at(default, "todo.md")
if isinstance(before, Failure):
return before
try:
after = pathlib.Path(checkout, "todo.md").read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read todo.md: {failure}")
# While a chunk reshapes todo.md into one table, the default branch's todo.md has no items table.
planned = items(before) if ITEMS_HEADER in before.splitlines() else []
if isinstance(planned, Failure):
return Failure(f"{default}'s todo.md: {planned.reason}")
chunk = items(after)
if isinstance(chunk, Failure):
return Failure(f"the checkout's todo.md: {chunk.reason}")
landed = landed_since_release(default)
if isinstance(landed, Failure):
return landed
return scoring_run_due(planned, chunk, ids, landed)
+135
View File
@@ -0,0 +1,135 @@
import pathlib
import time
from comprehension_history import appended, DIMENSIONS, heading, Heading, HISTORY, recorded, ruling_entry, scoring_entry, ScoringRun, vote_text
from larv_review.git import commit_exists, top
from larv_review.ruling import counted, fallen, overall, ruling, split, votes
from failure import Failure
from larv_review.scoring import fall_ruling, FLOOR, means, under_floor
from larv_review.plan import plan_reasons
from file_writes import replace_file
from larv_review.seats import scores, seats_file, SeatsFile
def record_ruling(votes_file: pathlib.Path, scores_file: pathlib.Path | None = None) -> Failure | None:
read = read_seats(votes_file, paired=True)
if isinstance(read, Failure):
return read
heading_read, seats_read = read
written = votes(seats_read.seats)
if isinstance(written, Failure):
return written
counts = counted(seats_read.seats, written)
tally = ", ".join(f"{seat.name} {vote_text(DIMENSIONS[-1], overall(written[seat.name]), overall(counts[seat.name]))}" for seat in seats_read.seats)
overall_votes = [overall(each) for each in counts.values()]
if split(overall_votes):
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the votes file, and run this again.")
return None
ruled = ruling(overall_votes)
fell = fallen(counts) if ruled != "worse" else []
scoring = None
notes = []
if not fell and scores_file:
why = "a worse ruling takes no scores file" if ruled == "worse" else f"no dimension fell in this {ruled} ruling, so it takes no scores file"
return Failure(f"{why}; run record-ruling again with the votes file alone")
if fell and not scores_file:
print(f"Fell: {', '.join(fell)}, in a {ruled} ruling. Launch the four scoring seats on {heading_read.label} at {heading_read.commit}, write their lines to a scores file, and run this again with both files.")
return None
if fell and scores_file:
scored = read_fall_scores(scores_file, heading_read)
if isinstance(scored, Failure):
return scored
scoring = scored
ruled, why = fall_ruling(ruled, fell, scoring.means)
notes = [why]
print(f"Ruling: {ruled}. {' '.join(notes + [f'Overall votes as counted: {tally}.'])}")
return append(seats_read.read, ruling_entry(heading_read.title, seats_read.seats, written, counts, ruled, notes, scoring))
def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> Failure | None:
read = read_seats(scores_file, paired=False)
if isinstance(read, Failure):
return read
named, given = read
scoring = scored_seats(given)
if isinstance(scoring, Failure):
return scoring
reasons = plan_reasons(default, ids)
if isinstance(reasons, Failure):
return reasons
if not reasons:
return Failure("no plan item or release calls this scoring run; name the item IDs the chunk ships, as needs-scoring-run took them, or record a fall's scores with record-ruling")
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, scoring.means))}.")
if under := under_floor(scoring.means):
print(f"Floor: {', '.join(under)} under the floor of {FLOOR} block the merge, as the chunk {' and '.join(reasons)}.")
return append(given.read, scoring_entry(named.title, scoring))
def read_fall_scores(scores_file: pathlib.Path, votes: Heading) -> ScoringRun | Failure:
"""The scores a fall called, once their file names what the votes file read."""
read = read_seats(scores_file, paired=False, refuse_recorded=False)
if isinstance(read, Failure):
return read
named, given = read
if (named.label, named.commit) != (votes.label, votes.commit):
return Failure(f"the scores file's first line is not `{votes.label} at {votes.commit}`, as the votes file reads: {given.read}")
return scored_seats(given)
def scored_seats(given: SeatsFile) -> ScoringRun | Failure:
found = scores(given.seats)
if isinstance(found, Failure):
return found
return ScoringRun(given.seats, found, means(list(found.values())))
def read_seats(path: pathlib.Path, paired: bool, refuse_recorded: bool = True) -> tuple[Heading, SeatsFile] | Failure:
"""The seats file and the heading its record goes under. Fails when the file names a commit the checkout lacks, or, when refuse_recorded, when the history already holds an entry on what it reads; it runs before anything is printed."""
try:
given = seats_file(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {path}: {failure}")
if isinstance(given, Failure):
return given
named = heading(time.time(), given.read, paired)
if isinstance(named, Failure):
return named
if unknown := [commit for commit in (named.commit, named.against) if commit and not commit_exists(commit)]:
return Failure(f"the seats file names {', '.join(unknown)}, which is no commit in this checkout")
if refuse_recorded:
history = history_file()
if isinstance(history, Failure):
return history
if recorded(history[1], given.read):
return Failure(f"{history[0]} already holds an entry on {given.read}; record each ruling or run once")
return named, given
def history_file() -> tuple[pathlib.Path, str] | Failure:
"""The checkout's history and its text, empty when the file is missing."""
checkout = top()
if isinstance(checkout, Failure):
return checkout
history = pathlib.Path(checkout, HISTORY)
try:
return history, history.read_text(encoding="utf-8") if history.exists() else ""
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read {history}: {failure}")
def append(read: str, entry: str) -> Failure | None:
"""Append entry unless the history already holds one on what the seats read."""
read_history = history_file()
if isinstance(read_history, Failure):
return read_history
history, text = read_history
if recorded(text, read):
return Failure(f"{history} already holds an entry on {read}; record each ruling or run once")
try:
history.parent.mkdir(parents=True, exist_ok=True)
except OSError as failure:
return Failure(f"could not create {history.parent}: {failure}")
if unwritten := replace_file(history, appended(text, entry).encode()):
return Failure(f"could not write {history}: {unwritten.reason}")
print(f"Appended to {history}.")
return None
+8 -5
View File
@@ -2,13 +2,16 @@ import re
CAP = 3
PASSES = ("architecture", "full scan", "security", "stability", "product-owner", "prose")
ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):")
ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):", re.IGNORECASE)
def round_counts(titles: list[str]) -> dict[str, int]:
"""Each pass's highest round among the branch's `Answer <pass> round <n>:` commit titles; a pass with none has 0, and a title naming no known pass adds its name."""
counts = dict.fromkeys(PASSES, 0)
"""Each pass's round count among the branch's `Answer <pass> round <n>:` commit titles: the highest <n>, or the number of such commits where that is more, so a repeated number still counts. A pass with none has 0, and a title naming no known pass adds its name. Case is ignored."""
highest: dict[str, int] = dict.fromkeys(PASSES, 0)
commits: dict[str, int] = dict.fromkeys(PASSES, 0)
for title in titles:
if named := ROUND.match(title):
counts[named.group(1)] = max(counts.get(named.group(1), 0), int(named.group(2)))
return counts
name = named.group(1).lower()
highest[name] = max(highest.get(name, 0), int(named.group(2)))
commits[name] = commits.get(name, 0) + 1
return {name: max(highest[name], commits[name]) for name in highest}
+19 -10
View File
@@ -18,12 +18,17 @@ def votes(seats: list[Seat]) -> dict[str, list[str]] | Failure:
if unnamed := [place for place in seat.places if not place_dimensions(place)]:
return Failure(f"{seat.name}'s place does not open with the dimensions it moves, each one of {', '.join(DIMENSIONS)}, then `: `: {unnamed[0]}")
said[seat.name] = line
first, second, *more = (each[-1] for each in counted(seats, said).values())
first, second, *more = (overall(each) for each in counted(seats, said).values())
if more and first == second:
return Failure(f"{' and '.join(FIRST_SEATS)} both count as {first} overall, which settles the ruling; drop the other seats' lines")
return said
def overall(votes: list[str]) -> str:
"""The overall vote, which DIMENSIONS lists last."""
return votes[-1]
def counted(seats: list[Seat], said: dict[str, list[str]]) -> dict[str, list[str]]:
"""Each seat's votes as they count: a dimension's vote counts as same unless one of the seat's `path:line` places opens with that dimension, and the overall vote unless the seat names any such place."""
return {seat.name: [vote if names_a_place(seat, name) else "same" for name, vote in zip(DIMENSIONS, said[seat.name])] for seat in seats}
@@ -39,16 +44,20 @@ def place_dimensions(place: str) -> list[str]:
return named if all(name in DIMENSIONS for name in named) else []
def ruling(counted: list[str]) -> str | None:
"""The vote that leads each other vote by two or more, else same; None when only the first two seats voted and they differ, so two more must vote."""
if len(counted) == 2 and counted[0] != counted[1]:
return None
for vote in VOTES:
if all(counted.count(vote) >= counted.count(other) + 2 for other in VOTES if other != vote):
return vote
return "same"
def split(overall: list[str]) -> bool:
"""Whether only the first two seats voted overall and they differ, so two more must vote: two votes that differ cannot reach a lead of two."""
return len(overall) == 2 and overall[0] != overall[1]
def ruling(overall: list[str]) -> str:
return lead(overall) or "same"
def fallen(counted: dict[str, list[str]]) -> list[str]:
"""Each dimension, Overall aside, that the seats rule worse by a lead of two."""
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if ruling(list(column)) == "worse"]
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if lead(list(column)) == "worse"]
def lead(votes: list[str]) -> str | None:
"""The vote that leads each other vote by two or more, if one does."""
return next((vote for vote in VOTES if all(votes.count(vote) >= votes.count(other) + 2 for other in VOTES if other != vote)), None)
@@ -1,18 +1,11 @@
import dataclasses
from comprehension_history import entries, Entry
@dataclasses.dataclass(frozen=True)
class HistoryChange:
"""A merged change on the default branch that touched the history, and the history before and after it."""
commit: str
before: str
after: str
from comprehension_history import entries, Entry, HistoryChange
def measured_commit(changes: list[HistoryChange]) -> str | None:
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better."""
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better.
It is the ratchet's base. A ruling of same leaves the base where it was, so a run of small falls each ruled same still adds up against one commit. Entry times are UTC stamps in ISO order, so they sort as text.
"""
for change in changes:
held = {entry.heading for entry in entries(change.before)}
added = [entry for entry in entries(change.after) if entry.heading not in held]
+1 -48
View File
@@ -1,24 +1,12 @@
import decimal
import typing
from comprehension_history import DIMENSIONS
from failure import Failure
from rank_todo import Item, ITEMS_HEADER, parse_row, table_rows
FLOOR = decimal.Decimal("7.0")
def fall_step(fell: list[str], scored: bool) -> typing.Literal["ask", "refuse", "rule"]:
"""What record-ruling does next: ask for the scoring seats when a dimension fell and no scores file came, refuse a scores file no fall calls, or rule."""
if fell and not scored:
return "ask"
if scored and not fell:
return "refuse"
return "rule"
def fall_ruling(ruled: str, fell: list[str], found: list[decimal.Decimal]) -> tuple[str, str]:
"""The ruling once the scoring run has scored the dimensions that fell, and the line that says why: worse when one of their means is under the floor."""
"""The ruling once the scoring run has scored the dimensions that fell, and the line that says why: worse when one of their means is under the floor, so a chunk cannot trade one dimension below the floor for gains in the others."""
if under := [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if name in fell and value < FLOOR]:
return "worse", f"Fell: {', '.join(fell)}. Under the floor of {FLOOR}: {', '.join(under)}, so the {ruled} ruling counts as worse."
return ruled, f"Fell: {', '.join(fell)}, each at or above the floor of {FLOOR}, so the {ruled} ruling stands."
@@ -35,38 +23,3 @@ def mean(values: list[decimal.Decimal]) -> decimal.Decimal:
def under_floor(found: list[decimal.Decimal]) -> list[str]:
return [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if value < FLOOR]
def plan_items(todo: str) -> list[Item] | Failure:
"""Each row of a todo.md's items table, read as `rank_todo.py` reads it."""
table = table_rows(todo.splitlines(), ITEMS_HEADER)
if isinstance(table, Failure):
return table
items = []
for row in table.rows:
item = parse_row(row)
if isinstance(item, Failure):
return item
items.append(item)
return items
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
"""Why the plan calls a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag. Work has landed towards an emptied release when the chunk ships one of its items, or when it is the next release, the earliest before names, and landed holds."""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
next_release = min((item.release for item in before), default=None)
for release in emptied(before, after, shipped):
if any(item.release == release for item in shipped) or (release == next_release and landed):
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
return reasons
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
named = {item.release for item in before + shipped}
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
return sorted(named - left)
@@ -1,4 +1,5 @@
import decimal
import itertools
import os
import pathlib
import subprocess
@@ -7,19 +8,21 @@ import tempfile
import textwrap
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from comprehension_history import appended, entries, Place, recorded, ruling_entry, Scoring, Seat
from larv_review.base import base, first_ruling, HistoryChange, measured_commit
from larv_review.ruling import counted, fallen, ruling, votes
from comprehension_history import appended, entries, HistoryChange, Place, recorded, ruling_entry, ScoringRun, Seat
from larv_review.ruling_base import base, first_ruling, measured_commit
import comprehension_history
from larv_review.ruling import counted, fallen, ruling, split, votes
from failure import Failure
from larv_review.scoring import fall_ruling, fall_step, means, plan_items, scoring_run_due, under_floor
from test_loop_claude_black_box import host_env
from rank_todo import Item
from larv_review.scoring import fall_ruling, means, under_floor
from test_env import isolated_env
from todo_table import Item, items
from larv_review.rounds import round_counts
from larv_review.seats import score_line, scores, seats_file
from larv_review.plan import scoring_run_due
TOOL = pathlib.Path(__file__).resolve().parent / "larv-review"
TOOL = pathlib.Path(__file__).resolve().parents[1] / "larv-review"
SCORING = "## 2026-10-08T12:00:00Z, PR #1 at aaaaaaa\n\n| Seat | Overall |\n|---|---|\n| Mean | 7.00 |\n"
@@ -93,7 +96,7 @@ class Seats(unittest.TestCase):
]
for counted, expected in cases:
with self.subTest(counted):
self.assertEqual(ruling(counted), expected)
self.assertEqual(None if split(counted) else ruling(counted), expected)
def test_reads_four_seats_scores_and_refuses_an_overall_above_the_lowest_by_more_than_one(self):
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
@@ -112,10 +115,18 @@ class Seats(unittest.TestCase):
class History(unittest.TestCase):
def test_writes_the_sample_entry_its_parser_shows(self):
source = pathlib.Path(comprehension_history.__file__).read_text(encoding="utf-8").splitlines()
start = next(index for index, line in enumerate(source) if line.startswith("# ## "))
sample = [line.removeprefix("#").removeprefix(" ") for line in itertools.takewhile(lambda line: line.startswith("#"), source[start:])]
seat = Seat("Mid A", said("better", "worse"), ["Locality: `bin/x.py:12` `run`: three branches to hold at once"])
votes = {"Mid A": ["better", "worse", "same", "same", "better"]}
self.assertEqual(ruling_entry("2026-10-09T09:47:25Z, PR #232 at 324d650, against 12d6d75", [seat], votes, votes, "same", [], None), "\n".join(sample) + "\n")
def test_reads_back_what_it_writes(self):
seats = [Seat("Mid A", said("better", "worse"), ["`bin/a.py:3` `f`: hard, with `:9` `g`"]), Seat("Inherited architect", said("worse", "worse"), [])]
written = {"Mid A": ["same", "worse", "same", "same", "better"], "Inherited architect": ["same", "worse", "same", "same", "worse"]}
scoring = Scoring([Seat("Junior A", "", ["`bin/b.py:4` `h`: long"])], {"Junior A": ["7", "6.5", "7", "7", "7"]}, [decimal.Decimal(mean) for mean in ("7.00", "6.50", "7.00", "7.00", "7.00")])
scoring = ScoringRun([Seat("Junior A", "", ["`bin/b.py:4` `h`: long"])], {"Junior A": ["7", "6.5", "7", "7", "7"]}, [decimal.Decimal(mean) for mean in ("7.00", "6.50", "7.00", "7.00", "7.00")])
ruled = ruling_entry("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", seats, written, {**written, "Inherited architect": ["same"] * 5}, "worse", ["Fell: Locality, under the floor."], scoring)
history = appended(appended("", SCORING), ruled)
self.assertTrue(history.startswith("# Comprehension history\n\n## 2026-10-08T12:00:00Z"))
@@ -136,9 +147,6 @@ class FallRuling(unittest.TestCase):
self.assertEqual(fall_ruling("better", ["Locality", "Shape"], found), ("worse", "Fell: Locality, Shape. Under the floor of 7.0: Locality 6.5, so the better ruling counts as worse."))
self.assertEqual(fall_ruling("same", ["Shape"], found), ("same", "Fell: Shape, each at or above the floor of 7.0, so the same ruling stands."))
def test_asks_for_scores_on_a_fall_refuses_them_without_one_and_rules_otherwise(self):
self.assertEqual([fall_step(["Locality"], False), fall_step([], True), fall_step(["Locality"], True), fall_step([], False)], ["ask", "refuse", "rule", "rule"])
class Base(unittest.TestCase):
def test_takes_the_newest_change_adding_a_scoring_run_or_ending_on_a_better_ruling(self):
@@ -171,12 +179,18 @@ class Rounds(unittest.TestCase):
counts = round_counts(["Answer prose round 1: x", "Answer full scan round 2: y", "Answer full scan round 1: z", "Answer stability: no round", "Answer stabilty round 1: typo", "Answer security round 1: rename round 2: x", "Rescore: w"])
self.assertEqual((counts["prose"], counts["full scan"], counts["stability"], counts["architecture"], counts["stabilty"], counts["security"]), (1, 2, 0, 0, 1, 1))
def test_counts_a_mis_cased_title_under_its_pass(self):
self.assertEqual(round_counts(["Answer Architecture round 4: x", "Answer architecture round 3: y"])["architecture"], 4)
def test_counts_a_repeated_round_number_as_a_round_of_its_own(self):
self.assertEqual(round_counts(["Answer stability round 3: x", "Answer stability round 3: y", "Answer stability round 2: z", "Answer stability round 1: w"])["stability"], 4)
class ScoringRunDue(unittest.TestCase):
def test_reads_each_rows_id_release_and_tagline(self):
self.assertEqual(plan_items(todo("| 3 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), [item(3, (0, 1, 0))])
self.assertIsInstance(plan_items("# todo\n"), Failure)
self.assertIsInstance(plan_items(todo("| 0 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), Failure)
self.assertEqual(items(todo("| 3 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), [item(3, (0, 1, 0))])
self.assertIsInstance(items("# todo\n"), Failure)
self.assertIsInstance(items(todo("| 0 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), Failure)
def test_is_due_for_an_item_naming_the_floor(self):
plan = [item(1, (0, 1, 0), "**Lift the project to the Comprehension floor.**"), item(2, (0, 1, 0))]
@@ -208,7 +222,7 @@ class LarvReviewTool(unittest.TestCase):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
root = pathlib.Path(os.path.realpath(temp.name))
self.env = {**host_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(root)}
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(root)}
self.checkout = root / "proj"
subprocess.run(["git", "init", "-q", "--bare", "-b", "main", str(root / "origin.git")], check=True, env=self.env)
subprocess.run(["git", "clone", "-q", str(root / "origin.git"), str(self.checkout)], check=True, env=self.env, capture_output=True)
@@ -254,7 +268,9 @@ class LarvReviewTool(unittest.TestCase):
self.commit("Answer prose round 1: c")
self.commit("Answer stability round 3: d")
ran = self.run_tool("round-count")
self.assertIn("stability: 3, at the cap\n", ran.stdout)
self.assertIn("stability: 3, at the cap: ask the human before another round\n", ran.stdout)
self.commit("Answer stability round 4: e")
self.assertIn("stability: 4, past the cap: ask the human before another round\n", self.run_tool("round-count").stdout)
self.assertIn("prose: 1\n", ran.stdout)
self.assertIn("architecture: 0\n", ran.stdout)
@@ -324,7 +340,9 @@ class LarvReviewTool(unittest.TestCase):
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertNotIn("Fell:", ran.stdout + (self.checkout / "docs" / "comprehension-history.md").read_text())
extra = self.run_tool("record-ruling", self.seats_file(f"PR #3 at {self.start}, against {self.start}\n\nMid A: {said('worse')}{place}Inherited architect: {said('worse')}{place}"), self.seats_file("PR #2", "scores.md"))
self.assertIn("takes no scores file", extra.stderr)
self.assertIn("a worse ruling takes no scores file; run record-ruling again with the votes file alone", extra.stderr)
better = self.run_tool("record-ruling", self.seats_file(f"PR #5 at {self.start}, against {self.start}\n\nMid A: {said('better')}{place}Inherited architect: {said('better')}{place}"), self.seats_file("PR #5", "scores.md"))
self.assertIn("no dimension fell in this better ruling, so it takes no scores file", better.stderr)
def test_records_a_scoring_run_with_its_means(self):
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
+1 -1
View File
@@ -9,4 +9,4 @@ sys.path.insert(0, str(LAUNCHER.parent))
from loop_claude import effects, paths
from loop_claude.loop import main
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home(), os.environ), LAUNCHER, pathlib.Path.cwd(), effects.system_effects()))
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home()), LAUNCHER, pathlib.Path.cwd(), effects.system_effects()))
+2 -2
View File
@@ -3,10 +3,10 @@ import uuid
from loop_claude.effects import Effects
from failure import Failure
from loop_claude.claude_code.transcript import human_turns, read_transcript
from claude_code.transcript import human_turns, read_transcript
from loop_claude.stop import needs_human, needs_terminal, Stop
from loop_claude.paths import Paths
from loop_claude.claude_code.trust import trust
from claude_code.trust import trust
from loop_claude.project.checkout import worktrees
ANSWER_QUESTION = "answer the question above"
+98 -2
View File
@@ -1,15 +1,111 @@
import dataclasses
import json
import re
import typing
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, ANSWERS_RECORDED
from loop_claude.effects import Effects
from failure import Failure
from loop_claude.claude_code.entries import hit_limit, limit_said, parse_entry, structured_output
from loop_claude.cycle.judge_reply import Interrupt, JUDGE_RULES, JUDGE_SCHEMA, QuotedAnswer, Verdict, verdict_from
from claude_code.entries import hit_limit, limit_said, parse_entry, structured_output
from loop_claude.project.records import HUMAN_RULES
from loop_claude.cycle.retry import Limited, retried
from loop_claude.stop import Stop
# At 3 to 4 characters a token, 100,000 to 133,000 tokens, which leaves a 200,000-token context room for the reply; the largest session rendered here was 364,000 characters on 2026-10-03.
JUDGE_PROMPT_CHARS = 400000
JUDGE_RULES = (
"Decide whether the human must now be interrupted. interrupt=true when any of these appear:\n"
"- the agent asks the human anything, seeks approval, or says a decision is the human's, and neither "
"README.md nor the records settle the matter\n"
"- it asks again a question a [loop] line already answered\n"
"- it carried on under an assumption that decides the deliverable, that the human could "
"reasonably reject and that neither README.md nor the records settle; a caveat about "
"transient state it then handled is not one\n"
"- it is blocked on something only the human can supply: a credential, a version bump, a choice "
"outside the repo, or a release ready to cut, which only the human may publish\n"
"- it flags that the goals, the plan in todo.md or the task itself is unclear, missing or in need of change\n"
"- it chose between alternatives that neither README.md nor the records discriminate "
"between - neither rules the rejected one out - and the choice materially changes the deliverable\n"
"interrupt=false otherwise. Bookkeeping the agent settled silently - what to document, wording, "
"where a note goes - is not on its own a reason to interrupt. The human may have replied inside the transcript, on a [human] line: judge only what "
"is still open, never a question they already answered there. A [loop] line is what loop-claude told "
"the agent, such as an answer quoted from README.md or the records.\n"
"Where the agent asked or flagged something itself, the question is that one, in the agent's own "
"wording; never construct another from facts it stated.\n"
"Where README.md or the records answer the agent's own question and no [loop] line has "
"quoted that answer yet, set interrupt=false and quote the passage that answers it; the agent is "
"then given that passage in place of the human's reply. An interrupt outranks a quoted answer, and either "
"outranks finished.\n"
"The chunk is the piece of work the agent is on, shipped as one pull request. "
"finished=true when the chunk is complete: merged, or nothing remains that the agent could do on "
f"its own. Even then, it is finished only once {ANSWERS_RECORDED}. An answer counts as recorded "
f"where README.md or the records above hold it, {HUMAN_RULES} included. It also counts where "
f"the transcript shows the agent writing it into {ANSWER_DESTINATIONS}; writing it into "
f"{HUMAN_RULES} does not count. finished=false when work remains that a further run can "
"continue: an open PR, a review not yet run, CI not yet reported, an answer not yet recorded."
)
@dataclasses.dataclass(frozen=True)
class Reply:
answer: str = dataclasses.field(metadata={"description": "With interrupt false, the passage of README.md or the records that answers the agent's question, quoted verbatim, without elisions. Empty otherwise."})
evidence: str = dataclasses.field(metadata={"description": "A verbatim quote of the one transcript passage the verdict rests on, without elisions: what triggers the interrupt; for a quoted answer, the agent's question; else what shows the chunk finished or the work remaining."})
finished: bool = dataclasses.field(metadata={"description": "Whether the chunk is finished."})
interrupt: bool = dataclasses.field(metadata={"description": "Whether the human must now be interrupted."})
question: str = dataclasses.field(metadata={"description": "With interrupt true, the single question to put to the human. With a quoted answer, the agent's question that the answer field answers. Empty otherwise."})
REPLY_TYPES = typing.get_type_hints(Reply)
JSON_TYPES = {bool: "boolean", str: "string"}
JUDGE_SCHEMA = {
"properties": {field.name: {**field.metadata, "type": JSON_TYPES[REPLY_TYPES[field.name]]} for field in dataclasses.fields(Reply)},
"required": list(REPLY_TYPES),
"type": "object",
}
@dataclasses.dataclass(frozen=True)
class Interrupt:
evidence: str
question: str
@dataclasses.dataclass(frozen=True)
class QuotedAnswer:
"""The agent asked a question that README.md or the records answer. The answer field quotes the passage."""
evidence: str
question: str
answer: str
@dataclasses.dataclass(frozen=True)
class Finished:
evidence: str
@dataclasses.dataclass(frozen=True)
class Unfinished:
evidence: str
Verdict = Interrupt | QuotedAnswer | Finished | Unfinished
def verdict_from(output: object) -> Verdict | Failure:
if not isinstance(output, dict):
return Failure("the judge returned no reply")
for name, kind in REPLY_TYPES.items():
if not isinstance(output.get(name), kind):
return Failure(f"the judge's reply has no {name} of type {JSON_TYPES[kind]}")
if output["interrupt"]:
if not output["question"].strip():
return Failure("the judge interrupted without a question")
return Interrupt(output["evidence"], output["question"])
if output["answer"].strip():
if not output["question"].strip():
return Failure("the judge quoted an answer without a question")
return QuotedAnswer(output["evidence"], output["question"], output["answer"])
return Finished(output["evidence"]) if output["finished"] else Unfinished(output["evidence"])
def judge(root: str, task: str, premises: str, records: str, rendered_transcript: str, model: str, effects: Effects) -> Verdict | Stop:
-104
View File
@@ -1,104 +0,0 @@
import dataclasses
import typing
from failure import Failure
from loop_claude.project.records import HUMAN_RULES, RECORD_FILES
# Where the judge may quote an answer from, named as the human and the worker see them; of README.md the judge sees only the premises.
QUOTE_SOURCES = "README.md's Goals and Audience, " + ", ".join(RECORD_FILES) + " or " + HUMAN_RULES
# Where the worker may write an answer the human typed, so it counts as recorded.
ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES) + ", todo.md or a comment"
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
JUDGE_RULES = (
"Decide whether the human must now be interrupted. interrupt=true when any of these appear:\n"
"- the agent asks the human anything, seeks approval, or says a decision is the human's, and neither "
"README.md nor the records settle the matter\n"
"- it asks again a question a [loop] line already answered\n"
"- it carried on under an assumption that decides the deliverable, that the human could "
"reasonably reject and that neither README.md nor the records settle; a caveat about "
"transient state it then handled is not one\n"
"- it is blocked on something only the human can supply: a credential, a version bump, a choice "
"outside the repo, or a release ready to cut, which only the human may publish\n"
"- it flags that the goals, the plan in todo.md or the task itself is unclear, missing or in need of change\n"
"- it chose between alternatives that neither README.md nor the records discriminate "
"between - neither rules the rejected one out - and the choice materially changes the deliverable\n"
"interrupt=false otherwise. Bookkeeping the agent settled silently - what to document, wording, "
"where a note goes - is not on its own a reason to interrupt. The human may have replied inside the transcript, on a [human] line: judge only what "
"is still open, never a question they already answered there. A [loop] line is what loop-claude told "
"the agent, such as an answer quoted from README.md or the records.\n"
"Where the agent asked or flagged something itself, the question is that one, in the agent's own "
"wording; never construct another from facts it stated.\n"
"Where README.md or the records answer the agent's own question and no [loop] line has "
"quoted that answer yet, set interrupt=false and quote the passage that answers it; the agent is "
"then given that passage in place of the human's reply. An interrupt outranks a quoted answer, and either "
"outranks finished.\n"
"The chunk is the piece of work the agent is on, shipped as one pull request. "
"finished=true when the chunk is complete: merged, or nothing remains that the agent could do on "
f"its own. Even then, it is finished only once {ANSWERS_RECORDED}. An answer counts as recorded "
f"where README.md or the records above hold it, {HUMAN_RULES} included. It also counts where "
f"the transcript shows the agent writing it into {ANSWER_DESTINATIONS}; writing it into "
f"{HUMAN_RULES} does not count. finished=false when work remains that a further run can "
"continue: an open PR, a review not yet run, CI not yet reported, an answer not yet recorded."
)
@dataclasses.dataclass(frozen=True)
class Reply:
answer: str = dataclasses.field(metadata={"description": "With interrupt false, the passage of README.md or the records that answers the agent's question, quoted verbatim, without elisions. Empty otherwise."})
evidence: str = dataclasses.field(metadata={"description": "A verbatim quote of the one transcript passage the verdict rests on, without elisions: what triggers the interrupt; for a quoted answer, the agent's question; else what shows the chunk finished or the work remaining."})
finished: bool = dataclasses.field(metadata={"description": "Whether the chunk is finished."})
interrupt: bool = dataclasses.field(metadata={"description": "Whether the human must now be interrupted."})
question: str = dataclasses.field(metadata={"description": "With interrupt true, the single question to put to the human. With a quoted answer, the agent's question that the answer field answers. Empty otherwise."})
REPLY_TYPES = typing.get_type_hints(Reply)
JSON_TYPES = {bool: "boolean", str: "string"}
JUDGE_SCHEMA = {
"properties": {field.name: {**field.metadata, "type": JSON_TYPES[REPLY_TYPES[field.name]]} for field in dataclasses.fields(Reply)},
"required": list(REPLY_TYPES),
"type": "object",
}
@dataclasses.dataclass(frozen=True)
class Interrupt:
evidence: str
question: str
@dataclasses.dataclass(frozen=True)
class QuotedAnswer:
"""The agent asked a question that README.md or the records answer. The answer field quotes the passage."""
evidence: str
question: str
answer: str
@dataclasses.dataclass(frozen=True)
class Finished:
evidence: str
@dataclasses.dataclass(frozen=True)
class Unfinished:
evidence: str
Verdict = Interrupt | QuotedAnswer | Finished | Unfinished
def verdict_from(output: object) -> Verdict | Failure:
if not isinstance(output, dict):
return Failure("the judge returned no reply")
for name, kind in REPLY_TYPES.items():
if not isinstance(output.get(name), kind):
return Failure(f"the judge's reply has no {name} of type {JSON_TYPES[kind]}")
if output["interrupt"]:
if not output["question"].strip():
return Failure("the judge interrupted without a question")
return Interrupt(output["evidence"], output["question"])
if output["answer"].strip():
if not output["question"].strip():
return Failure("the judge quoted an answer without a question")
return QuotedAnswer(output["evidence"], output["question"], output["answer"])
return Finished(output["evidence"]) if output["finished"] else Unfinished(output["evidence"])
+26
View File
@@ -0,0 +1,26 @@
from loop_claude.project.records import HUMAN_RULES, RECORD_FILES
from markdown_text import quoted
# Where the judge may quote an answer from, named as the human and the worker see them; of README.md the judge sees only the premises.
QUOTE_SOURCES = "README.md's Goals and Audience, " + ", ".join(RECORD_FILES) + " or " + HUMAN_RULES
# Where the worker may write an answer the human typed, so it counts as recorded.
ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES) + ", todo.md or a comment"
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
def fresh(task: str) -> str:
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one."
def resumed(unfinished_evidence: str) -> str:
ruled = f" A judge ruled the chunk unfinished, quoting this passage of your transcript as the work remaining:\n\n{quoted(unfinished_evidence)}\n\n" if unfinished_evidence else " "
return f"Continue the task; the previous run ended before the chunk was finished.{ruled}The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}."
def answered(question: str, answer: str) -> str:
return f"Your question: {question}\n\n{QUOTE_SOURCES} answers it:\n\n{answer}\n\nContinue the task."
def past_the_cap(question: str, answer: str, cap: int) -> str:
"""The question a quoted answer puts to the human once the cycle's quoted answers reach the cap."""
return f"{question}\n\n{QUOTE_SOURCES} says:\n{answer}\n\nThose files have answered {cap} questions this cycle, the most allowed, so this one comes to you."
+26 -21
View File
@@ -1,17 +1,19 @@
import dataclasses
import pathlib
import typing
from loop_claude.cycle.states import Active, Archived, Closing, Ended, Event, HumanAnswered, Interrupting, Judging, QUOTED_ANSWERS_PER_CYCLE, Ruled, State, transition, WorkerRan, Working
from loop_claude.transcript_archive import archive, FINISHED, UNFINISHED
from loop_claude.claude_code.session import delete_session, Session, session_open, transcript_file
from transcript_archive.write import archive
from claude_code.session import delete_session, Session, session_open, transcript_file
from loop_claude.effects import Effects
from failure import Failure
from loop_claude.cycle.judge_reply import Finished, Interrupt, QUOTE_SOURCES, QuotedAnswer, Unfinished
from loop_claude.cycle.judge import Finished, Interrupt, judge, QuotedAnswer, Unfinished
from loop_claude.cycle.interrupt import interrupt
from loop_claude.cycle.judge import judge
from loop_claude.project.model_choice import Models
from transcript_archive import outcomes
from loop_claude.paths import Paths
from loop_claude.claude_code.transcript import read_transcript, render
from loop_claude.cycle import prompts
from claude_code.transcript import read_transcript, render
from loop_claude.cycle.retry import retried
from loop_claude.cycle.worker import run_worker, worker_args
from loop_claude.stop import Stop
@@ -23,13 +25,13 @@ class Cycle:
models: Models
paths: Paths
premises: str
project_key: str
project_archive: pathlib.Path
records: str
# The evidence of the previous cycle's unfinished verdict, empty after any other.
remaining: str
root: str
session: Session
task: str
# Empty when the previous cycle's verdict was not Unfinished.
unfinished_evidence: str
def run_cycle(cycle: Cycle) -> Stop | Finished | Unfinished:
@@ -47,7 +49,7 @@ def run_cycle(cycle: Cycle) -> Stop | Finished | Unfinished:
def stopped(cycle: Cycle, stop: Stop, why: str) -> Stop:
path = transcript_file(cycle.session)
failure = archive(cycle.project_key, path, cycle.task, f"Stopped: {why}", cycle.effects.now(), cycle.paths) if path else None
failure = archive(cycle.project_archive, path, cycle.task, outcomes.stopped(why), cycle.effects.now()) if path else None
# A failed archive exits 1 whatever the stop; when the stop was this archive failing, its reason is said once.
return Stop(1, f"{failure.reason}\n{stop.reason}".rstrip()) if failure and failure.reason != stop.reason else stop
@@ -68,19 +70,27 @@ def step(cycle: Cycle, state: Active) -> Event | Stop:
def work(cycle: Cycle, answered: QuotedAnswer | None) -> WorkerRan | Stop:
if answered is not None:
print(f"\n=== {QUOTE_SOURCES} answered: {answered.question} ===\n{answered.answer}", flush=True)
print(f"\n=== {prompts.QUOTE_SOURCES} answered: {answered.question} ===\n{answered.answer}", flush=True)
if stop := keep_answer(cycle, answered):
return stop
# session_open runs inside the lambda: a failed attempt may have opened the session.
if stop := retried("worker", lambda: run_worker(cycle.root, worker_args(cycle.task, cycle.session, cycle.models.worker, session_open(cycle.session), answered, quoted(cycle.remaining)), cycle.effects), cycle.effects):
# worker_start runs inside the lambda: a failed attempt may have opened the session.
if stop := retried("worker", lambda: run_worker(cycle.root, worker_start(cycle, answered), cycle.effects), cycle.effects):
return stop
path = transcript_file(cycle.session)
return WorkerRan(path) if path is not None else Stop(1, f"claude left no transcript for session {cycle.session.id}")
def worker_start(cycle: Cycle, answered: QuotedAnswer | None) -> list[str]:
if answered is not None:
return worker_args(cycle.session, cycle.models.worker, prompts.answered(answered.question, answered.answer), resume=True)
if session_open(cycle.session):
return worker_args(cycle.session, cycle.models.worker, prompts.resumed(cycle.unfinished_evidence), resume=True)
return worker_args(cycle.session, cycle.models.worker, prompts.fresh(cycle.task), resume=False)
def keep_answer(cycle: Cycle, answered: QuotedAnswer) -> Stop | None:
path = transcript_file(cycle.session)
if path and (failure := archive(cycle.project_key, path, cycle.task, f"Verdict: quoted answer\n\n{answered.question}\n\n{QUOTE_SOURCES} says:\n{quoted(answered.answer)}\n\nEvidence:\n{quoted(answered.evidence)}", cycle.effects.now(), cycle.paths)):
if path and (failure := archive(cycle.project_archive, path, cycle.task, outcomes.answered(answered.question, prompts.QUOTE_SOURCES, answered.answer, answered.evidence), cycle.effects.now())):
return Stop(1, failure.reason)
return None
@@ -96,7 +106,7 @@ def rule(cycle: Cycle, transcript_path: str) -> Ruled | Stop:
def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) -> HumanAnswered | Stop:
question = question_for(ruling)
# Archived now, so the transcript the question was asked on is kept with it; the human's answer then grows the transcript.
if failure := archive(cycle.project_key, transcript_path, cycle.task, f"Verdict: interrupt\n\n{question}\n\nEvidence:\n{quoted(ruling.evidence)}", cycle.effects.now(), cycle.paths):
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.interrupted(question, ruling.evidence), cycle.effects.now()):
return Stop(1, failure.reason)
return interrupt(cycle.root, cycle.session.id, cycle.models.worker, transcript_path, question, cycle.paths, cycle.effects) or HumanAnswered()
@@ -104,18 +114,13 @@ def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) ->
def question_for(ruling: Interrupt | QuotedAnswer) -> str:
if isinstance(ruling, Interrupt):
return ruling.question
return f"{ruling.question}\n\n{QUOTE_SOURCES} says:\n{ruling.answer}\n\nThose files have answered {QUOTED_ANSWERS_PER_CYCLE} questions this cycle, the most allowed, so this one comes to you."
return prompts.past_the_cap(ruling.question, ruling.answer, QUOTED_ANSWERS_PER_CYCLE)
def close(cycle: Cycle, transcript_path: str, verdict: Finished | Unfinished) -> Archived | Stop:
finished = isinstance(verdict, Finished)
if failure := archive(cycle.project_key, transcript_path, cycle.task, f"{FINISHED if finished else UNFINISHED}\n\nEvidence:\n{quoted(verdict.evidence)}", cycle.effects.now(), cycle.paths):
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.finished(verdict.evidence) if finished else outcomes.unfinished(verdict.evidence), cycle.effects.now()):
return Stop(1, failure.reason)
if finished and (failure := delete_session(cycle.session)):
return Stop(1, failure.reason)
return Archived()
# Blockquoted, so evidence holding blank lines still ends visibly where the text after it starts.
def quoted(passage: str) -> str:
return "\n".join(f"> {line}" for line in passage.splitlines())
+4 -2
View File
@@ -1,6 +1,6 @@
import dataclasses
from loop_claude.cycle.judge_reply import Finished, Interrupt, QuotedAnswer, Unfinished, Verdict
from loop_claude.cycle.judge import Finished, Interrupt, QuotedAnswer, Unfinished, Verdict
# A backstop for what the judge misses, such as a reworded repeat, so the worker cannot loop unattended on answers quoted from README.md or the records.
QUOTED_ANSWERS_PER_CYCLE = 3
@@ -72,7 +72,9 @@ def transition(state: State, event: Event) -> State:
return Judging(transcript_path, quoted_answers)
case Judging(_, quoted_answers), Ruled(QuotedAnswer() as answered) if quoted_answers < QUOTED_ANSWERS_PER_CYCLE:
return Working(answered, quoted_answers + 1)
case Judging(transcript_path, quoted_answers), Ruled(Interrupt() | QuotedAnswer() as ruling):
case Judging(transcript_path, quoted_answers), Ruled(QuotedAnswer() as ruling) if quoted_answers >= QUOTED_ANSWERS_PER_CYCLE:
return Interrupting(transcript_path, ruling, quoted_answers)
case Judging(transcript_path, quoted_answers), Ruled(Interrupt() as ruling):
return Interrupting(transcript_path, ruling, quoted_answers)
case Judging(transcript_path, _), Ruled(Finished() | Unfinished() as verdict):
return Closing(transcript_path, verdict)
+7 -13
View File
@@ -1,10 +1,9 @@
from loop_claude.cycle.judge_reply import ANSWERS_RECORDED, QUOTE_SOURCES, QuotedAnswer
from loop_claude.claude_code.entries import assistant_parts, ends_stream, hit_limit, limit_said, parse_entry, rejected_until
from claude_code.entries import assistant_parts, ends_stream, hit_limit, limit_said, parse_entry, rejected_until
from loop_claude.effects import Effects
from failure import Failure
from loop_claude.cycle.retry import Limited
from loop_claude.claude_code.transcript import part_line
from loop_claude.claude_code.session import Session
from claude_code.transcript import part_line
from claude_code.session import Session
PRINTED_TOOL_INPUT = 160
@@ -38,17 +37,12 @@ def worker_outcome(code: int, result: dict | None, resets: list[int]) -> Failure
return None if result is not None else Failure("claude exited 0 without a result event")
def worker_args(task: str, session: Session, model: str, resuming: bool, answered: QuotedAnswer | None = None, remaining: str = "") -> list[str]:
if answered is not None:
start = ["-p", f"Your question: {answered.question}\n\n{QUOTE_SOURCES} answers it:\n\n{answered.answer}\n\nContinue the task.", "--resume", str(session.id)]
elif resuming:
ruled = f" A judge ruled the chunk unfinished, quoting this passage of your transcript as the work remaining:\n\n{remaining}\n\n" if remaining else " "
start = ["-p", f"Continue the task; the previous run ended before the chunk was finished.{ruled}The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}.", "--resume", str(session.id)]
else:
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
start = ["-p", f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.", "--name", session.name, "--session-id", str(session.id)]
def worker_args(session: Session, model: str, prompt: str, resume: bool) -> list[str]:
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
start = ["--resume", str(session.id)] if resume else ["--name", session.name, "--session-id", str(session.id)]
# -p fails without --verbose under stream-json.
return [
"-p", prompt,
*start,
"--output-format", "stream-json",
"--permission-mode", "bypassPermissions",
+12 -8
View File
@@ -77,17 +77,19 @@ import pathlib
import sys
import typing
from loop_claude.project.checkout import checkout_root, project_key
from loop_claude.project.checkout import checkout_root
from loop_claude.cycle.run import Cycle, run_cycle
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
from loop_claude.effects import Effects
from loop_claude.project.model_choice import get_or_ask_models
from loop_claude.paths import Paths
from transcript_archive.location import project_archive
from project_key import project_key
from loop_claude.project.premises import read_premises
from loop_claude.project.records import read_records
from loop_claude.claude_code.session import session_for
from claude_code.session import session_named
from loop_claude.stop import Stop
from loop_claude.cycle.judge_reply import Unfinished
from loop_claude.cycle.judge import Unfinished
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
@@ -105,8 +107,10 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
if isinstance(loop_run, Stop):
return loop_run
baseline = fingerprint(launcher)
key = project_key(loop_run.root, paths.code)
session = session_for(key, paths)
key = project_key(loop_run.root, paths.home)
archive = project_archive(loop_run.root, paths.home, env)
# "A " is the human's cue in the session picker that the session is the loop's.
session = session_named(f"A {key} loop-claude", paths.claude_code_transcripts)
while True:
if stop := exec_if_edited(launcher, baseline, loop_run, effects):
return stop
@@ -121,10 +125,10 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
if isinstance(models, Stop):
return models
print(f"\n=== cycle {loop_run.cycle_number} ===", flush=True)
verdict = run_cycle(Cycle(effects=effects, models=models, paths=paths, premises=premises, project_key=key, records=records, remaining=loop_run.remaining, root=loop_run.root, session=session, task=loop_run.task))
verdict = run_cycle(Cycle(effects=effects, models=models, paths=paths, premises=premises, project_archive=archive, records=records, root=loop_run.root, session=session, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence))
if isinstance(verdict, Stop):
return verdict
loop_run = dataclasses.replace(loop_run, remaining=verdict.evidence if isinstance(verdict, Unfinished) else "")
loop_run = dataclasses.replace(loop_run, unfinished_evidence=verdict.evidence if isinstance(verdict, Unfinished) else "")
def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopRun | Stop:
@@ -133,7 +137,7 @@ def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopR
if task in ("", "-h", "--help"):
return Stop(0 if task else 1, __doc__.strip())
root = checkout_root(cwd, effects)
return root if isinstance(root, Stop) else LoopRun(0, "", root, task)
return root if isinstance(root, Stop) else LoopRun(0, root, task, "")
loop_run = decode_handed(argv[1], LoopRun) if len(argv) == 2 else None
if not (loop_run and loop_run.root and loop_run.task):
return Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}")
+1 -1
View File
@@ -17,9 +17,9 @@ T = typing.TypeVar("T")
class LoopRun:
"""What a reload hands the edited script, as JSON on argv."""
cycle_number: int
remaining: str
root: str
task: str
unfinished_evidence: str
def decode_handed(handed: str, kind: type[T]) -> T | None:
+3 -6
View File
@@ -1,6 +1,5 @@
import dataclasses
import pathlib
import typing
@dataclasses.dataclass(frozen=True)
@@ -8,20 +7,18 @@ class Paths:
claude_code_transcripts: pathlib.Path
claude_json: pathlib.Path
claude_md: pathlib.Path
code: pathlib.Path
home: pathlib.Path
model_catalog: pathlib.Path
models_file: pathlib.Path
transcript_archive: pathlib.Path
def paths_from(home: pathlib.Path, env: typing.Mapping[str, str]) -> Paths:
def paths_from(home: pathlib.Path) -> Paths:
claude = home / ".claude"
return Paths(
claude_code_transcripts=claude / "projects",
claude_json=home / ".claude.json",
claude_md=claude / "CLAUDE.md",
code=home / "code",
home=home,
model_catalog=claude / "cache" / "model-catalog",
models_file=claude / "loop-claude-models.json",
transcript_archive=pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else claude / "loop-claude-transcripts",
)
-14
View File
@@ -23,17 +23,3 @@ def worktrees(root: str, effects: Effects) -> list[str]:
if listing.returncode == 0:
found += [line.split(" ", 1)[1] for line in listing.stdout.splitlines() if line.startswith("worktree ")]
return sorted(set(found))
def project_key(root: str, code: pathlib.Path) -> str:
return root.removeprefix(f"{code}/").lstrip("/")
def read_utf8(folder: str, name: str, consequence: str) -> str | Stop:
try:
return pathlib.Path(folder, name).read_text(encoding="utf-8")
except OSError as failure:
return Stop(1, f"{name} in {folder}: {failure.strerror} - {consequence}.")
except UnicodeDecodeError as failure:
line = failure.object.count(b"\n", 0, failure.start) + 1
return Stop(1, f"{name} in {folder} is not UTF-8 on line {line} - {consequence}.")
+1 -1
View File
@@ -4,7 +4,7 @@ import pathlib
import sys
import typing
from loop_claude.claude_code.model_catalog import catalog_models
from claude_code.model_catalog import catalog_models
from loop_claude.effects import Effects
from failure import Failure
from loop_claude.stop import needs_human, needs_terminal, Stop
+2 -2
View File
@@ -1,5 +1,5 @@
from loop_claude.project.checkout import read_utf8
from markdown_sections import sections
from loop_claude.project.text_file import read_utf8
from markdown_text import sections
from loop_claude.stop import Stop
+1 -1
View File
@@ -1,6 +1,6 @@
import pathlib
from loop_claude.project.checkout import read_utf8
from loop_claude.project.text_file import read_utf8
from loop_claude.stop import Stop
# The records: these project files and HUMAN_RULES; the judge reads them beside the README's premises.
+13
View File
@@ -0,0 +1,13 @@
import pathlib
from loop_claude.stop import Stop
def read_utf8(folder: str, name: str, consequence: str) -> str | Stop:
try:
return pathlib.Path(folder, name).read_text(encoding="utf-8")
except OSError as failure:
return Stop(1, f"{name} in {folder}: {failure.strerror} - {consequence}.")
except UnicodeDecodeError as failure:
line = failure.object.count(b"\n", 0, failure.start) + 1
return Stop(1, f"{name} in {folder} is not UTF-8 on line {line} - {consequence}.")
@@ -15,14 +15,14 @@ import tempfile
import time
import unittest
BIN = pathlib.Path(__file__).resolve().parent
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from test_env import isolated_env
BIN = pathlib.Path(__file__).resolve().parents[1]
TASK = "Do the next chunk."
# A git hook's GIT_DIR, the host's LOOP_CLAUDE_* or PYTHONPATH would point the program under test at the host's own state.
def host_env() -> dict[str, str]:
return {name: value for name, value in os.environ.items() if not name.startswith(("GIT_", "LOOP_CLAUDE_", "PYTHON"))}
# Plays claude from FAKE_CLAUDE_SCENARIO, one step per call, logging each call; a step for another role,
# or none left, is Ctrl-C to loop-claude, which otherwise loops forever.
FAKE_CLAUDE = """#!{python}
@@ -160,7 +160,7 @@ class Checkout:
self.root.mkdir(parents=True)
(self.root / "README.md").write_text(PREMISES)
self.script = BIN / "loop-claude"
self.env = host_env()
self.env = isolated_env()
fake = self.temp / "fake-bin"
fake.mkdir()
(fake / "claude").write_text(FAKE_CLAUDE.format(python=sys.executable))
@@ -687,12 +687,12 @@ class Reload(unittest.TestCase):
self.assertEqual(run.roles(), ["worker", "judge"] * 5)
self.assertIn("keeping the running script", run.stderr)
reloads = [json.loads(argv[1]) for argv in run.starts if argv[:1] == ["--reloaded"]]
self.assertEqual(reloads, [{"cycle_number": cycle, "remaining": "working", "root": str(checkout.root), "task": TASK} for cycle in (2, 3, 4)])
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working"} for cycle in (2, 3, 4)])
self.assertIn("=== cycle 5 ===", run.stdout)
outside = checkout.temp / "outside"
outside.mkdir()
state = {"cycle_number": 3, "remaining": "", "root": str(checkout.root), "task": TASK}
state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK, "unfinished_evidence": ""}
run = checkout.run([worker()], argv=["--reloaded", json.dumps(state)], cwd=outside)
self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"]))
self.assertEqual(run.calls[0]["cwd"], str(checkout.root))
@@ -1,7 +1,6 @@
import ast
import contextlib
import dataclasses
import hashlib
import io
import json
import os
@@ -14,18 +13,18 @@ import time
import unittest
import uuid
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from loop_claude import transcript_archive as archive
from loop_claude.project import checkout, model_choice, records
from loop_claude.effects import Effects
from loop_claude.claude_code import entries, session, transcript, trust
from claude_code import entries, session
from failure import Failure
from loop_claude.cycle.judge_reply import Finished, Interrupt, QuotedAnswer, Unfinished, verdict_from
from loop_claude.cycle import interrupt, judge, retry, run, states, worker
from loop_claude.cycle.judge import Finished, Interrupt, QuotedAnswer, Unfinished, verdict_from
from loop_claude.cycle import interrupt, judge, prompts, retry, run, states, worker
from loop_claude.cycle.retry import Limited
from loop_claude.loop import main, reload
from loop_claude import paths
from transcript_archive.location import project_archive
from loop_claude.stop import Stop
CWD = pathlib.Path("/c")
@@ -40,10 +39,6 @@ def human(content):
return {"type": "user", "message": {"content": content}}
def queued(prompt, kind):
return {"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "prompt", "origin": {"kind": kind}, "prompt": prompt}}
class FakeEffects:
"""Effects that record each call and answer from a script, one answer per call."""
@@ -164,24 +159,20 @@ class QuestionFor(unittest.TestCase):
self.assertEqual(run.question_for(QuotedAnswer("e", "Which one?", "A.")), "Which one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md or ~/.claude/CLAUDE.md says:\nA.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.")
class Quoted(unittest.TestCase):
def test_quotes_every_line_of_a_passage_blank_ones_included(self):
self.assertEqual(run.quoted("merged\n\nPR #1"), "> merged\n> \n> PR #1")
class Cycles(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.home = pathlib.Path(temp.name)
self.paths = paths.paths_from(self.home, {})
self.session = session.session_for("proj", self.paths)
self.paths = paths.paths_from(self.home)
self.session = session.session_named("A proj loop-claude", self.paths.claude_code_transcripts)
self.archive = project_archive(str(self.home / "code" / "proj"), self.home, {})
def cycle(self, fake, records="", remaining=""):
return run.Cycle(effects=fake.effects(), models=model_choice.Models("judge-model", "worker-model"), paths=self.paths, premises="## Goals", project_key="proj", records=records, remaining=remaining, root=str(self.home), session=self.session, task="Do it.")
def cycle(self, fake, records="", unfinished_evidence=""):
return run.Cycle(effects=fake.effects(), models=model_choice.Models("judge-model", "worker-model"), paths=self.paths, premises="## Goals", project_archive=self.archive, records=records, root=str(self.home), session=self.session, task="Do it.", unfinished_evidence=unfinished_evidence)
def archived_outcomes(self):
return sorted(path.read_text() for path in (self.paths.transcript_archive / "proj").glob("*/*.outcomes.md"))
return sorted(path.read_text() for path in self.archive.glob("*/*.outcomes.md"))
def test_a_finished_chunk_is_archived_and_its_session_deleted(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
@@ -193,7 +184,7 @@ class Cycles(unittest.TestCase):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[judged("unfinished")] * 2)
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
self.assertTrue(session.transcript_file(self.session))
self.assertEqual(run.run_cycle(self.cycle(fake, remaining="working")), Unfinished("working"))
self.assertEqual(run.run_cycle(self.cycle(fake, unfinished_evidence="working")), Unfinished("working"))
workers = fake.made("stream_claude")
self.assertIn("--session-id", workers[0])
self.assertIn("--resume", workers[1])
@@ -216,7 +207,7 @@ class Cycles(unittest.TestCase):
(folder.parent / f"{self.session.id}.orphaned-1.jsonl").write_text("o\n")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 3, claude=[judged("unfinished"), judged("finished"), judged("unfinished")])
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
[chunk] = (self.paths.transcript_archive / "proj").iterdir()
[chunk] = self.archive.iterdir()
kept = chunk / "session" / str(self.session.id)
self.assertEqual((chunk / "session" / f"{self.session.id}.orphaned-1.jsonl").read_text(), "o\n")
os.utime(kept / "subagents" / "workflows" / "journal.jsonl", (0, 0))
@@ -232,7 +223,7 @@ class Cycles(unittest.TestCase):
folder.mkdir()
(folder / "custom-title.json").write_text("third")
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
[next_chunk] = set((self.paths.transcript_archive / "proj").iterdir()) - {chunk}
[next_chunk] = set(self.archive.iterdir()) - {chunk}
self.assertEqual(sorted(path.read_text() for path in (next_chunk / "session" / str(self.session.id)).glob("custom-title.json*")), ["third"])
@unittest.skipIf(os.geteuid() == 0, "root reads an unreadable folder")
@@ -253,10 +244,10 @@ class Cycles(unittest.TestCase):
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nEvidence:\n> working\n", "Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n"])
self.assertIn(str(self.session.id), fake.made("claude_on_terminal")[0])
prompts = [stdin for args, stdin in fake.made("claude")]
self.assertEqual(len(prompts), 2)
self.assertNotIn("the second", prompts[0])
self.assertIn("[human] the second", prompts[1])
judge_prompts = [stdin for args, stdin in fake.made("claude")]
self.assertEqual(len(judge_prompts), 2)
self.assertNotIn("the second", judge_prompts[0])
self.assertIn("[human] the second", judge_prompts[1])
def test_a_question_the_records_answer_resumes_the_worker_with_the_answer_three_times_a_cycle(self):
answered = judged("quoted", answer="Use *A*.")
@@ -272,11 +263,12 @@ class Cycles(unittest.TestCase):
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md or ~/.claude/CLAUDE.md says:\nUse *A*.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.\n\nEvidence:\n> working\n\n2026-09-21T14:13:20Z Stopped: the human is needed to answer the question above, but the interrupt ended with nothing typed\n", *[f"Task: Do it.\n\n{quoted}"] * 3])
def test_stops_when_a_quoted_answer_cannot_be_archived(self):
self.paths.transcript_archive.parent.mkdir(parents=True)
self.paths.transcript_archive.write_text("")
self.archive.parent.mkdir(parents=True)
self.archive.write_text("")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("quoted", answer="Use A.")])
with contextlib.redirect_stdout(io.StringIO()):
self.assertEqual(run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse A.")), Stop(1, f"could not archive the transcript: {self.paths.transcript_archive / 'proj'}: Not a directory"))
stop = run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse A."))
self.assertRegex(stop.reason, f"^could not archive the transcript: {self.archive}/[0-9a-f]{{64}}: Not a directory$")
self.assertEqual(len(fake.made("stream_claude")), 1)
def test_an_answer_the_records_do_not_hold_is_dropped_and_the_question_goes_to_the_human(self):
@@ -357,7 +349,7 @@ class Limits(unittest.TestCase):
class Interrupts(unittest.TestCase):
def test_stops_off_a_terminal_before_starting_claude(self):
fake = FakeEffects(on_terminal=False)
stop = interrupt.interrupt("/r", uuid7(), "m", "/t", "Which one?", paths.paths_from(pathlib.Path("/h"), {}), fake.effects())
stop = interrupt.interrupt("/r", uuid7(), "m", "/t", "Which one?", paths.paths_from(pathlib.Path("/h")), fake.effects())
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"))
self.assertEqual(fake.calls, [])
@@ -366,7 +358,7 @@ class Interrupts(unittest.TestCase):
path = pathlib.Path(home, "t.jsonl")
path.write_text(json.dumps(human("a")) + "\n")
fake = FakeEffects(on_terminal_claude=[0], git=[ran()])
stop = interrupt.interrupt(home, uuid7(), "m", str(path), "Which one?", paths.paths_from(pathlib.Path(home), {}), fake.effects())
stop = interrupt.interrupt(home, uuid7(), "m", str(path), "Which one?", paths.paths_from(pathlib.Path(home)), fake.effects())
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"))
@@ -391,13 +383,13 @@ class ModelPrompt(unittest.TestCase):
class WorkerArgs(unittest.TestCase):
def test_opens_a_fresh_session_by_name_and_resumes_an_open_one(self):
opened = session.session_for("proj", paths.paths_from(pathlib.Path("/h"), {}))
fresh = worker.worker_args("Do it.", opened, "m", False)
opened = session.session_named("A proj loop-claude", pathlib.Path("/h/.claude/projects"))
fresh = worker.worker_args(opened, "m", prompts.fresh("Do it."), resume=False)
self.assertEqual(fresh[fresh.index("--session-id") + 1], str(opened.id))
self.assertEqual(fresh[fresh.index("--name") + 1], "A proj loop-claude")
resumed = worker.worker_args("Do it.", opened, "m", True)
resumed = worker.worker_args(opened, "m", prompts.resumed(""), resume=True)
self.assertEqual(resumed[resumed.index("--resume") + 1], str(opened.id))
self.assertNotIn("Do it.", " ".join(resumed))
self.assertNotIn("--session-id", resumed)
self.assertTrue(resumed[resumed.index("-p") + 1].startswith("Continue the task; the previous run ended before the chunk was finished. The chunk is finished once"))
@@ -425,101 +417,6 @@ class Records(unittest.TestCase):
self.assertEqual(records.read_records(root, claude_md), Stop(1, f"AGENTS.md in {root} is not UTF-8 on line 2 - the judge cannot read the records."))
class Render(unittest.TestCase):
def test_joins_assistant_text_tool_calls_human_turns_and_loop_prompts_and_skips_the_rest(self):
said = transcript.render([
human("do it"),
assistant({"type": "text", "text": "Starting."}, {"type": "tool_use", "name": "Bash", "input": {"command": "ls"}}),
assistant({"type": "text", "text": " "}),
human([{"type": "tool_result", "content": "not typed"}]),
human("<command-name>/exit</command-name>"),
human([{"type": "text", "text": "yes, "}, {"type": "text", "text": "merge it"}]),
queued("B, while you work", "human"),
queued("origin/main moved", "coordinator"),
{**human("Continue the task; the previous run stopped before it was finished."), "promptSource": "sdk"},
])
self.assertEqual(said, '[human] do it\n\nStarting.\n\n[tool Bash] {"command": "ls"}\n\n[human] yes, merge it\n\n[human] B, while you work\n\n[loop] Continue the task; the previous run stopped before it was finished.')
class Archive(unittest.TestCase):
def test_writes_afresh_a_copy_whose_outcomes_file_is_missing(self):
with tempfile.TemporaryDirectory() as root:
kept = paths.paths_from(pathlib.Path(root), {})
transcript = pathlib.Path(root, "t.jsonl")
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(archive.archive("proj", str(transcript), "Do it.", "Verdict: unfinished", NOW, kept))
(outcomes,) = kept.transcript_archive.glob("proj/*/*.outcomes.md")
outcomes.unlink()
self.assertIsNone(archive.archive("proj", str(transcript), "Do it.", "Stopped: Ctrl-C", NOW, kept))
self.assertEqual(outcomes.read_text(), "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n")
class CopyFor(unittest.TestCase):
def test_skips_a_version_already_kept_beside_its_copy(self):
with tempfile.TemporaryDirectory() as temp:
copy = pathlib.Path(temp, "custom-title.json")
copy.write_bytes(b"first")
beside = archive.copy_for(copy, b"second")
self.assertEqual(beside, copy.with_name(f"custom-title.json.{hashlib.sha256(b'second').hexdigest()}"))
beside.write_bytes(b"second")
self.assertIsNone(archive.copy_for(copy, b"second"))
self.assertEqual(archive.copy_for(copy, b"first, grown"), copy)
class FirstTimestampedLine(unittest.TestCase):
def test_skips_the_lines_every_chunk_shares_and_else_takes_the_first(self):
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\nnot JSON\n{"timestamp": "t"}\n{"timestamp": "u"}\n'), b'{"timestamp": "t"}')
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\n[1]\n'), b'{"type": "custom-title"}')
self.assertEqual(transcript.first_timestamped_line(b""), b"")
class HumanTurns(unittest.TestCase):
def test_counts_typed_turns_and_not_tool_results(self):
self.assertEqual(transcript.human_turns([
human("typed"),
human([{"type": "text", "text": "typed as a block"}]),
human([{"type": "tool_result", "content": "not typed"}]),
human("<command-name>/model</command-name>"),
{**human("<3 merge it"), "promptSource": "typed"},
{**human("<task-notification>done</task-notification>"), "promptSource": "system"},
{**human([{"type": "text", "text": "<task-notification>done</task-notification>"}]), "promptSource": "system"},
{**human("peer message"), "isMeta": True},
{**human("the loop's own prompt"), "promptSource": "sdk"},
human([{"type": "text", "text": "[Request interrupted by user]"}]),
human([]),
queued("typed while claude worked", "human"),
queued("<task-notification>done</task-notification>", "task-notification"),
{"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "task-notification", "prompt": "<task-notification>done</task-notification>"}},
assistant({"type": "text", "text": "said"}),
]), 4)
class PartLine(unittest.TestCase):
def test_renders_text_and_tool_calls_and_nothing_else(self):
self.assertEqual(transcript.part_line({"type": "text", "text": "hi"}, 20), "hi")
self.assertEqual(transcript.part_line({"type": "text", "text": " \n"}, 20), "")
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Bash", "input": {"command": "x" * 500}}, 20), '[tool Bash] {"command": "xxxxxxx')
self.assertEqual(transcript.part_line("not a part", 20), "")
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Write", "input": {"content": "Hälsa på"}}, 40), '[tool Write] {"content": "Hälsa på"}')
class Parse(unittest.TestCase):
def test_yields_only_json_objects(self):
self.assertEqual(entries.parse_entry('{"type": "user"}'), {"type": "user"})
self.assertIsNone(entries.parse_entry("not json"))
self.assertIsNone(entries.parse_entry("[1, 2]"))
class MarkTrusted(unittest.TestCase):
def test_returns_the_updated_config_and_leaves_the_one_it_was_handed(self):
config = {"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"kept": True}}}
handed = json.dumps(config)
self.assertEqual(trust.mark_trusted(config, ["/a", "/b", "/c"]), ({"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"hasTrustDialogAccepted": True, "kept": True}, "/c": {"hasTrustDialogAccepted": True}}}, ["/b", "/c"]))
self.assertEqual(json.dumps(config), handed)
self.assertEqual(trust.mark_trusted({}, ["/a"]), ({"projects": {"/a": {"hasTrustDialogAccepted": True}}}, ["/a"]))
class JudgePrompt(unittest.TestCase):
def test_shows_the_transcripts_tail_and_returns_what_it_showed(self):
prompt, shown = judge.judge_prompt("Do it.", "## Goals", "", "old " + "x" * judge.JUDGE_PROMPT_CHARS + " new")
@@ -569,7 +466,7 @@ class Verdict(unittest.TestCase):
class LoopRuns(unittest.TestCase):
def test_stops_naming_the_reload_when_it_hands_over_anything_but_the_state_with_a_task(self):
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": ""})], [json.dumps({"cycle_number": "3", "remaining": "", "root": "/r", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": "do it", "turns": 1})], [json.dumps({"cycle_number": 3, "failures": 0, "remaining": "", "root": "/r", "task": "do it"})]):
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "root": "/r", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "", "unfinished_evidence": ""})], [json.dumps({"cycle_number": "3", "root": "/r", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "do it", "turns": 1, "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "failures": 0, "root": "/r", "task": "do it", "unfinished_evidence": ""})]):
with self.subTest(handed=handed):
self.assertEqual(main.loop_run_from([main.RELOAD_FLAG, *handed], CWD, FakeEffects().effects()), Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(handed)}"))
@@ -586,7 +483,7 @@ class LoopRuns(unittest.TestCase):
launcher = pathlib.Path(root, "loop-claude")
launcher.write_text("")
fake = FakeEffects(python=[ran()])
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(1, "", "/r", "do it"), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence=""), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
class HelpLimits(unittest.TestCase):
@@ -622,18 +519,18 @@ class ExitContract(unittest.TestCase):
("loop/reload.py", "exec_if_edited", "1", "f'could not reload loop-claude:", "python cannot start for a reload"),
("project/checkout.py", "checkout_root", "1", "f'could not start git:", git),
("project/checkout.py", "checkout_root", "1", "found.stderr.strip()", git),
("project/checkout.py", "read_utf8", "1", "f'{name} in {folder}:", unreadable),
("project/checkout.py", "read_utf8", "1", "f'{name} in {folder} is not UTF-8", unreadable),
("project/model_choice.py", "get_or_ask_model", "1", "f'could not save the model in", models_),
("project/model_choice.py", "read_models_file", "1", 'f"could not read {saved}: {getattr(', models_),
("project/model_choice.py", "read_models_file", "1", "f'could not read {saved}: not an object", models_),
("project/premises.py", "read_premises", "1", "f'README.md has no ##", premises_),
("project/premises.py", "read_premises", "1", "f'README.md has ## {name} more than once", premises_),
("project/premises.py", "read_premises", "1", "f'README.md has nothing under", premises_),
("project/text_file.py", "read_utf8", "1", "f'{name} in {folder}:", unreadable),
("project/text_file.py", "read_utf8", "1", "f'{name} in {folder} is not UTF-8", unreadable),
]
root = pathlib.Path(main.__file__).parents[1]
found = []
for path in sorted(root.rglob("*.py")):
for path in sorted(path for path in root.rglob("*.py") if not path.name.startswith("test_")):
functions = [node for node in ast.walk(ast.parse(path.read_text())) if isinstance(node, ast.FunctionDef)]
stops = [(call.lineno, function.name, ast.unparse(call.args[0]), ast.unparse(call.args[1])) for function in functions for call in ast.walk(function) if isinstance(call, ast.Call) and getattr(call.func, "id", None) == "Stop" and not (isinstance(call.args[0], ast.Constant) and call.args[0].value in (0, 2, 130))]
found += [(str(path.relative_to(root)), function, code, reason) for _, function, code, reason in sorted(stops)]
@@ -650,25 +547,21 @@ class JudgeFailures(unittest.TestCase):
started, field, unasked = "claude cannot start or exits non-zero", "its reply misses or mistypes a field", "interrupts or quotes an answer without a question"
# Every Failure the judge's run returns, in source order: path, function, start of reason, its --help clause.
clauses = [
("cycle/judge.py", "run_judge", "f'could not start claude:", started),
("cycle/judge.py", "run_judge", "proc.stderr.strip() or f'the judge exited", started),
("cycle/judge.py", "run_judge", 'f"{verdict.reason}; it replied:', field),
("cycle/judge.py", "run_judge", "f'the judge quoted {verdict.evidence!r}, which is not in the transcript", "quotes a passage the transcript does not hold"),
("cycle/judge_reply.py", "verdict_from", "'the judge returned no reply'", field),
("cycle/judge_reply.py", "verdict_from", 'f"the judge\'s reply has no {name}', field),
("cycle/judge_reply.py", "verdict_from", "'the judge interrupted without a question'", unasked),
("cycle/judge_reply.py", "verdict_from", "'the judge quoted an answer without a question'", unasked),
("verdict_from", "'the judge returned no reply'", field),
("verdict_from", 'f"the judge\'s reply has no {name}', field),
("verdict_from", "'the judge interrupted without a question'", unasked),
("verdict_from", "'the judge quoted an answer without a question'", unasked),
("run_judge", "f'could not start claude:", started),
("run_judge", "proc.stderr.strip() or f'the judge exited", started),
("run_judge", 'f"{verdict.reason}; it replied:', field),
("run_judge", "f'the judge quoted {verdict.evidence!r}, which is not in the transcript", "quotes a passage the transcript does not hold"),
]
root = pathlib.Path(main.__file__).parents[1]
found = []
for path in ("cycle/judge.py", "cycle/judge_reply.py"):
functions = [node for node in ast.walk(ast.parse((root / path).read_text())) if isinstance(node, ast.FunctionDef)]
failures = sorted((call.lineno, function.name, ast.unparse(call.args[0])) for function in functions for call in ast.walk(function) if isinstance(call, ast.Call) and getattr(call.func, "id", None) == "Failure")
found += [(path, function, reason) for _, function, reason in failures]
functions = [node for node in ast.walk(ast.parse(pathlib.Path(judge.__file__).read_text())) if isinstance(node, ast.FunctionDef)]
found = sorted((call.lineno, function.name, ast.unparse(call.args[0])) for function in functions for call in ast.walk(function) if isinstance(call, ast.Call) and getattr(call.func, "id", None) == "Failure")
self.assertEqual(len(found), len(clauses), found)
for (path, function, reason), (want_path, want_function, reason_start, _) in zip(found, clauses):
self.assertEqual((path, function), (want_path, want_function))
self.assertTrue(reason.startswith(reason_start), f"{path} {function}: {reason}")
for (_, function, reason), (want_function, reason_start, _) in zip(found, clauses):
self.assertEqual(function, want_function)
self.assertTrue(reason.startswith(reason_start), f"{function}: {reason}")
judge_clause = " ".join(main.__doc__[main.__doc__.index("a judge when"):main.__doc__.index("A usage limit")].split())
self.assertEqual([clause for *_, clause in clauses if clause not in judge_clause], [])
@@ -25,3 +25,8 @@ def sections(text: str) -> dict[str, list[str]]:
if name is not None:
found.setdefault(name, []).append("\n".join(body))
return found
# Blockquoted, so a passage holding blank lines still ends visibly where the text after it starts.
def quoted(passage: str) -> str:
return "\n".join(f"> {line}" for line in passage.splitlines())
+1 -1
View File
@@ -84,7 +84,7 @@ def medians(runs: list[RunScores]) -> dict[str, Score]:
return {metric: Score(statistics.median(run.scores[metric].auc for run in runs), statistics.median(run.scores[metric].top_k for run in runs), all(run.scores[metric].constant for run in runs)) for metric in METRICS}
def resolved(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
def tracked_places(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
"""Entries written before paths were written from the repository's root shorten a path once they have named its folder: `cycle/run.py` after `bin/loop_claude/cycle/states.py`. Each place resolves to the one tracked file it names, or, where several match, the one in the folder of the place resolved before it; a place naming no file, or several, is dropped. Returns the resolved places and the dropped ones."""
found = []
dropped = []
+2 -2
View File
@@ -35,7 +35,7 @@ import statistics
import subprocess
import sys
from metric_bench.bench import bench, co_changed, medians, METRICS, resolved, RunScores, Score
from metric_bench.bench import bench, co_changed, medians, METRICS, RunScores, Score, tracked_places
from comprehension_history import entries, Entry
from failure import Failure
from metric_bench.python_units import measured, python_units, Unit
@@ -128,7 +128,7 @@ def run_scores(run: Entry, top: str) -> tuple[RunScores | None, list[str]] | Fai
log = git(["-c", "core.quotePath=false", "log", "--no-merges", "--format=tformat:@@", "--name-only", run.commit], top)
if isinstance(log, Failure):
return log
places, dropped = resolved(run.scored_places, tracked)
places, dropped = tracked_places(run.scored_places, tracked)
return bench(units, places, len(set(dropped)), co_changed(log)), unparsed
@@ -7,16 +7,16 @@ import tempfile
import textwrap
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from metric_bench.bench import auc, bench, co_changed, resolved, top_k
from metric_bench.bench import auc, bench, co_changed, top_k, tracked_places
from metric_bench.python_units import cognitive, measured, python_units, Unit
from comprehension_history import entries, Place
from failure import Failure
from test_loop_claude_black_box import host_env
from test_env import isolated_env
from metric_bench.main import scored_runs
TOOL = pathlib.Path(__file__).resolve().parent / "metric-bench"
TOOL = pathlib.Path(__file__).resolve().parents[1] / "metric-bench"
def scored(source: str) -> int:
@@ -153,18 +153,18 @@ class Places(unittest.TestCase):
def test_resolves_a_shortened_path_to_the_file_nearest_the_one_before_it(self):
tracked = ["bin/tool/cycle/states.py", "bin/tool/cycle/run.py", "bin/other/cycle/run.py"]
places = [Place("bin/tool/cycle/states.py", "transition"), Place("cycle/run.py", "step"), Place("gone.py", "x")]
self.assertEqual(resolved(places, tracked), ([Place("bin/tool/cycle/states.py", "transition"), Place("bin/tool/cycle/run.py", "step")], [Place("gone.py", "x")]))
self.assertEqual(tracked_places(places, tracked), ([Place("bin/tool/cycle/states.py", "transition"), Place("bin/tool/cycle/run.py", "step")], [Place("gone.py", "x")]))
def test_a_full_path_wins_over_a_file_in_the_folder_before_it(self):
self.assertEqual(resolved([Place("pkg/x.py", "a"), Place("main.py", "f")], ["pkg/x.py", "main.py", "pkg/main.py"])[0][-1], Place("main.py", "f"))
self.assertEqual(tracked_places([Place("pkg/x.py", "a"), Place("main.py", "f")], ["pkg/x.py", "main.py", "pkg/main.py"])[0][-1], Place("main.py", "f"))
def test_resolves_a_shortened_path_after_a_repeated_place_by_the_repeat(self):
tracked = ["bin/a/main.py", "bin/b/main.py", "bin/b/bench.py"]
places = [Place("bin/a/main.py", "main"), Place("bin/b/bench.py", "bench"), Place("bin/a/main.py", "main"), Place("main.py", "figures")]
self.assertEqual(resolved(places, tracked)[0][-1], Place("bin/a/main.py", "figures"))
self.assertEqual(tracked_places(places, tracked)[0][-1], Place("bin/a/main.py", "figures"))
def test_drops_a_shortened_path_two_files_match_with_no_place_before_it(self):
self.assertEqual(resolved([Place("cycle/run.py", "step")], ["bin/tool/cycle/run.py", "bin/other/cycle/run.py"]), ([], [Place("cycle/run.py", "step")]))
self.assertEqual(tracked_places([Place("cycle/run.py", "step")], ["bin/tool/cycle/run.py", "bin/other/cycle/run.py"]), ([], [Place("cycle/run.py", "step")]))
class Bench(unittest.TestCase):
@@ -198,7 +198,7 @@ class BlackBox(unittest.TestCase):
directory = tempfile.TemporaryDirectory()
self.addCleanup(directory.cleanup)
top = directory.name
env = host_env()
env = isolated_env()
for args in (["init", "-q"], ["config", "user.email", "t@t"], ["config", "user.name", "t"]):
subprocess.run(["git", *args], cwd=top, check=True, env=env)
pathlib.Path(top, "a.py").write_text("def hard(a):\n\tif a:\n\t\tfor x in a:\n\t\t\tif x:\n\t\t\t\treturn x\n\n\ndef easy():\n\treturn 1\n")
@@ -211,7 +211,7 @@ class BlackBox(unittest.TestCase):
return top
def run_tool(self, top: str, *args: str) -> subprocess.CompletedProcess:
return subprocess.run([sys.executable, str(TOOL), *args], cwd=top, capture_output=True, text=True, env=host_env())
return subprocess.run([sys.executable, str(TOOL), *args], cwd=top, capture_output=True, text=True, env=isolated_env())
def test_prints_each_metrics_score_on_each_run_and_the_medians(self):
top = self.project("# Comprehension history\n\n## 2026-10-08T12:50:49Z, PR #1 at COMMIT\n\nJunior A, hardest first:\n\n- `a.py:1` `hard`\n")
+6
View File
@@ -0,0 +1,6 @@
import pathlib
def project_key(checkout_root: str, home: pathlib.Path) -> str:
"""The checkout's path relative to ~/code, or to / outside it: what names a project's loop-claude session, its entry in the models file and its archive folder."""
return checkout_root.removeprefix(f"{home / 'code'}/").lstrip("/")
+2 -61
View File
@@ -57,12 +57,11 @@ import pathlib
import re
import sys
from todo_table import cells, Item, ITEMS_HEADER, parse_row, table_rows
from failure import Failure
from markdown_sections import FENCE, sections
from markdown_text import FENCE, sections
from file_writes import replace_file
EXEMPTIONS = ("", "decision", "defect", "principle", "question", "security")
ITEMS_HEADER = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |"
LARGEST_FORMULA = 200
LONGEST_FORMULA = 4 * LARGEST_FORMULA
OPERATORS = (ast.Add, ast.BinOp, ast.Div, ast.Expression, ast.Load, ast.Mult, ast.Sub, ast.UAdd, ast.UnaryOp, ast.USub)
@@ -70,16 +69,6 @@ VARIABLES = ("A", "G", "R", "S", "W")
WEIGHTS_HEADER = "| Goal | W |"
@dataclasses.dataclass(frozen=True)
class Item:
id: int
release: tuple[int, int, int]
exempt: str
tagline: str
scores: dict[str, int]
goals: tuple[str, ...]
@dataclasses.dataclass(frozen=True)
class Scoring:
formula: ast.Expression
@@ -88,13 +77,6 @@ class Scoring:
weights: dict[str, float]
@dataclasses.dataclass(frozen=True)
class Table:
"""Body rows of a markdown table, and the index of the first row in the file's lines."""
start: int
rows: list[str]
@dataclasses.dataclass(frozen=True)
class Ranked:
text: str
@@ -357,23 +339,6 @@ def parse_weights(lines: list[str], goals: list[str]) -> dict[str, float] | Fail
return weights
def table_rows(lines: list[str], header: str) -> Table | Failure:
if lines.count(header) != 1:
return Failure(f"todo.md needs the table header {header} once")
start = lines.index(header) + 2
if start > len(lines) or not re.fullmatch(r"\|(-+\|)+", lines[start - 1].replace(" ", "")):
return Failure(f"the table under {header} has no |---| divider line")
end = start
while end < len(lines) and lines[end].startswith("|"):
end += 1
return Table(start, lines[start:end])
def cells(row: str, count: int) -> list[str] | Failure:
found = [cell.strip() for cell in re.split(r"(?<!\\)\|", row.strip())[1:-1]]
return found if len(found) == count else Failure(f"this row needs {count} cells: {row}")
def parse_item(row: str, weights: dict[str, float]) -> Item | Failure:
item = parse_row(row)
if isinstance(item, Failure):
@@ -383,30 +348,6 @@ def parse_item(row: str, weights: dict[str, float]) -> Item | Failure:
return item
def parse_row(row: str) -> Item | Failure:
"""An item row, every cell checked but the goals it cites."""
found = cells(row, 10)
if isinstance(found, Failure):
return found
id_, release, exempt, tagline, r, s, a, g, goals, _ = found
if not re.fullmatch(r"[0-9]+", id_) or int(id_) < 1:
return Failure(f"the ID {id_!r} is not a positive whole number: {row}")
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", release):
return Failure(f"item {id_}'s release {release!r} is not a semver version")
if exempt not in EXEMPTIONS:
return Failure(f"item {id_}'s Exempt {exempt!r} is not empty or one of {', '.join(EXEMPTIONS[1:])}")
if not re.fullmatch(r"\*\*[^*].*[^*]\*\*", tagline):
return Failure(f"item {id_}'s Item cell is not one bold tagline: {tagline}")
scores = {}
for name, value in zip("RSAG", (r, s, a, g)):
if not re.fullmatch(r"[0-9]+", value) or not 1 <= int(value) <= 10:
return Failure(f"item {id_}'s {name} {value!r} is not a whole number from 1 to 10")
scores[name] = int(value)
cited = tuple(goal.strip() for goal in goals.split(","))
major, minor, patch = (int(part) for part in release.split("."))
return Item(int(id_), (major, minor, patch), exempt, tagline, scores, cited)
def render_item(item: Item, value: float) -> str:
release = ".".join(str(part) for part in item.release)
scores = " | ".join(str(item.scores[name]) for name in "RSAG")
-20
View File
@@ -1,20 +0,0 @@
import pathlib
import typing
from failure import Failure
from loop_claude.transcript_archive import FinishedChunk
from loop_claude.paths import paths_from
from loop_claude.project.checkout import project_key
def project_archive(checkout_root: str, home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
paths = paths_from(home, env)
return paths.transcript_archive / project_key(checkout_root, paths.code)
def read_threads(chunk: FinishedChunk) -> list[bytes] | Failure:
"""The worker's transcript, then each subagent's."""
try:
return [path.read_bytes() for path in (chunk.transcript, *chunk.subagents)]
except OSError as failure:
return Failure(f"could not read {failure.filename}: {failure.strerror}")
+5 -4
View File
@@ -3,12 +3,13 @@ import re
import statistics
from failure import Failure
from loop_claude.transcript_archive import FinishedChunk, STAMPED
from transcript_archive.read import FinishedChunk
from reading_cost.measure import Reading
from markdown_sections import sections
from markdown_text import sections
from utc_stamp import STAMPED
FIGURE = re.compile(rf"^Reading cost, chunk (\w+), finished ({STAMPED})$")
PLACE = re.compile(r"^- `(.+)` (\d+)$")
FIGURE_PLACE = re.compile(r"^- `(.+)` (\d+)$")
PLACES_KEPT = 10
RESET = "Reading cost reset, "
RESET_AT = re.compile(rf"^{RESET}({STAMPED})$")
@@ -42,7 +43,7 @@ def readings_of(record: str) -> list[Reading] | Failure:
tokens = [int(match.group(1)) for line in lines if (match := TOKENS.match(line))]
if len(tokens) != 1:
return Failure(f"the figure under \"## {heading}\" holds no single tokens line; restore it from git history")
dated.append((named.group(2), heading, Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := PLACE.match(line))}))))
dated.append((named.group(2), heading, Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := FIGURE_PLACE.match(line))}))))
elif heading.startswith("Reading cost"):
return Failure(f"\"## {heading}\" names no chunk and UTC time, or no reset time; restore it from git history")
return [reading for finished, _, reading in sorted(dated, key=lambda item: item[:2]) if finished > max(resets)]
+7 -6
View File
@@ -32,9 +32,10 @@ import time
from reading_cost.measure import Checkout, measure, Reading
from failure import Failure
from reading_cost.figures import figure, merged, readings_of, RESET, trend, unheld
from loop_claude.transcript_archive import finished_chunks, utc
from reading_cost.archive import project_archive, read_threads
from transcript_archive.read import finished_chunks, read_threads
from transcript_archive.location import project_archive
from file_writes import replace_file
from utc_stamp import utc
@dataclasses.dataclass(frozen=True)
class Updated:
@@ -55,8 +56,8 @@ def main(argv: list[str]) -> int:
if isinstance(checkout, Failure):
return failed(checkout)
home = pathlib.Path.home()
archive = project_archive(checkout.root, home, os.environ)
record = pathlib.Path(given[0]) if given else pathlib.Path(checkout.top, "docs", "reading-cost.md")
archive = project_archive(checkout.main_root, home, os.environ)
record = pathlib.Path(given[0]) if given else pathlib.Path(checkout.current_root, "docs", "reading-cost.md")
updated = update(record, archive, checkout, str(home), utc(time.time()) if reset else None)
if isinstance(updated, Failure):
return failed(updated)
@@ -121,8 +122,8 @@ def located() -> Checkout | Failure:
return Failure(f"could not find the checkout: {failure}")
if listed.returncode != 0:
return Failure(listed.stderr.strip())
# The common dir is the checkout's .git, whichever worktree runs this.
return Checkout(str(pathlib.Path(common).parent), top, frozenset(filter(None, listed.stdout.split("\0"))))
# The common dir is the main checkout's .git, whichever worktree runs this.
return Checkout(top, str(pathlib.Path(common).parent), frozenset(filter(None, listed.stdout.split("\0"))))
def read_record(record: pathlib.Path) -> str | Failure:
+5 -5
View File
@@ -2,7 +2,7 @@ import collections
import dataclasses
import re
from loop_claude.claude_code.entries import assistant_parts, parse_entry, tool_call
from claude_code.entries import assistant_parts, parse_entry, tool_call
from reading_cost.shell_reads import Reads, resolved, shell_reads
COMMIT = re.compile(r"(^|[\s;&|(])git\s+(-C\s+\S+\s+)?commit\b")
@@ -16,11 +16,11 @@ class Reading:
places: collections.Counter[str]
@dataclasses.dataclass(frozen=True)
class Checkout:
root: str
top: str
"""main_root is the main checkout, whose path keys the archive; current_root is the checkout or worktree this runs in."""
current_root: str
main_root: str
tracked: frozenset[str]
@@ -31,7 +31,7 @@ def measure(threads: list[bytes], checkout: Checkout, home: str) -> Reading | No
return None
before = [[entry for entry in thread if entry.get("type") == "assistant" and str(entry.get("timestamp", "")) < commit] for thread in (worker, *subagents)]
reads = [reads_of(part, str(entry.get("cwd") or "/"), home) for thread in before for entry in thread for part in assistant_parts(entry)]
roots = [checkout.root, *(added for found in reads for added in found.added)]
roots = [checkout.main_root, *(added for found in reads for added in found.added)]
places = collections.Counter(place for found in reads for path in found.read if (place := relative(path, roots)) in checkout.tracked)
return Reading(sum(map(tokens_in, before)), places)
@@ -10,17 +10,19 @@ import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from loop_claude.transcript_archive import archive, FINISHED, FINISHED_AT, finished_chunks, FinishedChunk, UNFINISHED, utc
from transcript_archive.write import archive
from reading_cost.measure import Checkout, Reading
from failure import Failure
from reading_cost import figures, measure
from test_loop_claude_black_box import host_env
from loop_claude.paths import paths_from
from transcript_archive.read import finished_chunks, FinishedChunk
from test_env import isolated_env
from transcript_archive import outcomes
from transcript_archive.location import project_archive
from reading_cost.shell_reads import Reads, shell_reads
TOOL = pathlib.Path(__file__).resolve().parent / "reading-cost"
TOOL = pathlib.Path(__file__).resolve().parents[1] / "reading-cost"
def said(timestamp, message_id, command=None, tokens=0, cwd="/c/p", **call):
@@ -59,7 +61,7 @@ class Measure(unittest.TestCase):
said("6", "m5", tokens=100, name="Read", input={"file_path": "/c/p/late.py"}),
]
subagent = [said("2", "s1", tokens=1, name="Grep", input={"path": "/c/w/src/a.py"}), said("7", "s2", tokens=100, name="Read", input={"file_path": "/c/p/late.py"})]
checkout = Checkout("/c/p", "/c/w", frozenset({"README.md", "late.py", "src/a.py", "src/b.py", "x.py"}))
checkout = Checkout("/c/w", "/c/p", frozenset({"README.md", "late.py", "src/a.py", "src/b.py", "x.py"}))
self.assertEqual(measure.measure([jsonl(worker), jsonl(subagent)], checkout, "/h"), Reading(33, collections.Counter({"src/a.py": 2, "README.md": 1, "src/b.py": 1})))
def test_measures_nothing_for_a_worker_that_never_committed(self):
@@ -94,35 +96,32 @@ class Figures(unittest.TestCase):
history = "\n".join([written("a", "2026-10-01T00:00:00Z", 1), written("c", "2026-10-03T00:00:00Z", 3), "## Reading cost reset, 2026-10-02T00:00:00Z\n", written("b", "2026-10-04T00:00:00Z", 2)])
self.assertEqual(figures.readings_of(history), [Reading(3, collections.Counter()), Reading(2, collections.Counter())])
def test_the_archive_stamps_what_it_reads_back(self):
self.assertRegex(f"{utc(1790000000)} {FINISHED}", FINISHED_AT)
class ReadingCostTool(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.home = pathlib.Path(os.path.realpath(temp.name))
self.env = {**host_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.home)}
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.home)}
self.checkout = self.home / "code" / "proj"
self.checkout.mkdir(parents=True)
(self.checkout / "a.py").write_text("")
self.git("init", "-q")
self.git("add", "a.py")
self.git("-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "-m", "x")
self.paths = paths_from(self.home, {})
self.archive = project_archive(str(self.checkout), self.home, {})
def git(self, *args):
subprocess.run(["git", *args], cwd=self.checkout, check=True, env=self.env)
def archive_chunk(self, at, outcome=FINISHED):
def archive_chunk(self, at, outcome=outcomes.finished):
transcript = self.home / ".claude" / "projects" / "-proj" / f"{folder_name()}.jsonl"
(transcript.with_suffix("") / "subagents").mkdir(parents=True)
worker = [said("1", "m1", tokens=10, cwd=str(self.checkout), name="Read", input={"file_path": str(self.checkout / "a.py")}), said("2", "m2", command="git commit -m x")]
transcript.write_bytes(jsonl([{"timestamp": str(at)}, *worker]))
(transcript.with_suffix("") / "subagents" / "agent-a.jsonl").write_bytes(jsonl([said("1", "s1", tokens=1)]))
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(archive("proj", str(transcript), "Do it.", f"{outcome}\n\nEvidence:\n> merged", at, self.paths))
self.assertIsNone(archive(self.archive, str(transcript), "Do it.", outcome("merged"), at))
def run_tool(self, cwd, *args):
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=cwd, env=self.env, text=True)
@@ -130,8 +129,8 @@ class ReadingCostTool(unittest.TestCase):
def test_appends_each_finished_chunk_once_in_the_order_they_finished_from_a_worktree(self):
self.archive_chunk(1790000200)
self.archive_chunk(1790000100)
self.archive_chunk(1790000000, UNFINISHED)
chunks = finished_chunks(self.paths.transcript_archive / "proj")
self.archive_chunk(1790000000, outcomes.unfinished)
chunks = finished_chunks(self.archive)
self.assertEqual([chunk.finished for chunk in chunks], ["2026-09-21T14:15:00Z", "2026-09-21T14:16:40Z"])
worktree = self.home / "code" / "proj-wt"
self.git("worktree", "add", "-q", str(worktree))
View File
@@ -1,9 +1,13 @@
import pathlib
import re
import subprocess
import sys
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[1]
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import ROOT
CITATION = re.compile(r'`?CLAUDE\.md`? → ("[^"]+"(?:,? (?:and )?"[^"]+")*)')
HEADING = re.compile(r"^#+ (.+?)(?: \(.*\))?$", re.MULTILINE)
LEAD = re.compile(r"^(?:- )?\*\*(.+?)(?:\*\*|[:.](?=\*\*| )| —)", re.MULTILINE)
@@ -1,7 +1,11 @@
import pathlib
import subprocess
import sys
import unittest
from test_python_floor import ROOT
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import ROOT
PRIVATE = [
".claude/settings.local.json",
@@ -1,8 +1,12 @@
import ast
import pathlib
import subprocess
import sys
import unittest
from test_python_floor import ROOT
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import ROOT
def bound(node: ast.Import | ast.ImportFrom) -> list[str]:
@@ -1,7 +1,11 @@
import json
import pathlib
import sys
import unittest
from test_python_floor import ROOT
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import ROOT
class Install(unittest.TestCase):
@@ -5,7 +5,7 @@ import subprocess
import sys
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[1]
ROOT = pathlib.Path(__file__).resolve().parents[2]
FLOOR = re.compile(r"Python 3\.(\d+) or newer")
+19
View File
@@ -0,0 +1,19 @@
import pathlib
import shutil
import subprocess
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from repo_checks.test_python_floor import readme_floor, ROOT
class Types(unittest.TestCase):
def test_mypy_passes_on_the_tools_at_the_floor(self):
floor = readme_floor()
self.assertIsNotNone(floor, "README.md's Install names `Python 3.N or newer`")
if shutil.which("uv") is None:
self.fail("uv on PATH fetches mypy; see README.md's Test line")
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "--exclude", "/test_", "-p", "claude_code", "-m", "comprehension_history", "-p", "larv_review", "-p", "loop_claude", "-m", "markdown_text", "-p", "metric_bench", "-m", "project_key", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline", "-m", "todo_table", "-p", "transcript_archive", "-m", "utc_stamp"], capture_output=True, cwd=ROOT / "bin", text=True)
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
+6
View File
@@ -0,0 +1,6 @@
import os
# A git hook's GIT_DIR, the host's LOOP_CLAUDE_* or PYTHONPATH would point the program under test at the host's own state.
def isolated_env() -> dict[str, str]:
return {name: value for name, value in os.environ.items() if not name.startswith(("GIT_", "LOOP_CLAUDE_", "PYTHON"))}
@@ -4,18 +4,21 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
import markdown_sections
import markdown_text
class Sections(unittest.TestCase):
def test_keeps_every_headed_section_with_its_bodies(self):
text = "# Title\n\nintro\n\n## Goals\n\n1. one\n```sh\n# not a heading\n```\n### Later\n2. two\n#1 priority\n\n## Empty\n\n## Audience\nme\n## Audience\nagain\n# Appendix\nnot audience\n"
self.assertEqual(markdown_sections.sections(text), {"Goals": ["1. one\n```sh\n# not a heading\n```\n### Later\n2. two\n#1 priority"], "Empty": [""], "Audience": ["me", "again"]})
self.assertEqual(markdown_text.sections(text), {"Goals": ["1. one\n```sh\n# not a heading\n```\n### Later\n2. two\n#1 priority"], "Empty": [""], "Audience": ["me", "again"]})
def test_closes_a_fence_only_on_its_own_marker(self):
text = "## Goals\n````md\n~~~\n## Quoted\n```\n````\n ~~~\n## Also quoted\n ~~~\n1. one\n## Audience\nme\n"
self.assertEqual(list(markdown_sections.sections(text)), ["Goals", "Audience"])
self.assertEqual(list(markdown_sections.sections("```\nquoted:\n```python\n## Inside\n```\n## Goals\n1. one\n")), ["Goals"])
self.assertEqual(list(markdown_text.sections(text)), ["Goals", "Audience"])
self.assertEqual(list(markdown_text.sections("```\nquoted:\n```python\n## Inside\n```\n## Goals\n1. one\n")), ["Goals"])
def test_quotes_every_line_of_a_passage_blank_ones_included(self):
self.assertEqual(markdown_text.quoted("merged\n\nPR #1"), "> merged\n> \n> PR #1")
if __name__ == "__main__":
+1 -1
View File
@@ -9,7 +9,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from failure import Failure
import rank_todo
from test_python_floor import ROOT
from repo_checks.test_python_floor import ROOT
README = "# P\n\n## Goals\n\n1. Fast.\n2. Correct.\n 1. Tested.\n\n## Audience\n\nMe.\n"
-15
View File
@@ -1,15 +0,0 @@
import shutil
import subprocess
import unittest
from test_python_floor import readme_floor, ROOT
class Types(unittest.TestCase):
def test_mypy_passes_on_the_tools_at_the_floor(self):
floor = readme_floor()
self.assertIsNotNone(floor, "README.md's Install names `Python 3.N or newer`")
if shutil.which("uv") is None:
self.fail("uv on PATH fetches mypy; see README.md's Test line")
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "-m", "comprehension_history", "-p", "larv_review", "-p", "loop_claude", "-p", "metric_bench", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline"], capture_output=True, cwd=ROOT / "bin", text=True)
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
+79
View File
@@ -0,0 +1,79 @@
import dataclasses
import re
from failure import Failure
EXEMPTIONS = ("", "decision", "defect", "principle", "question", "security")
ITEMS_HEADER = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |"
@dataclasses.dataclass(frozen=True)
class Item:
id: int
release: tuple[int, int, int]
exempt: str
tagline: str
scores: dict[str, int]
goals: tuple[str, ...]
@dataclasses.dataclass(frozen=True)
class Table:
"""Body rows of a markdown table, and the index of the first row in the file's lines."""
start: int
rows: list[str]
def items(todo: str) -> list[Item] | Failure:
"""Each row of a todo.md's items table."""
table = table_rows(todo.splitlines(), ITEMS_HEADER)
if isinstance(table, Failure):
return table
items = []
for row in table.rows:
item = parse_row(row)
if isinstance(item, Failure):
return item
items.append(item)
return items
def table_rows(lines: list[str], header: str) -> Table | Failure:
if lines.count(header) != 1:
return Failure(f"todo.md needs the table header {header} once")
start = lines.index(header) + 2
if start > len(lines) or not re.fullmatch(r"\|(-+\|)+", lines[start - 1].replace(" ", "")):
return Failure(f"the table under {header} has no |---| divider line")
end = start
while end < len(lines) and lines[end].startswith("|"):
end += 1
return Table(start, lines[start:end])
def cells(row: str, count: int) -> list[str] | Failure:
found = [cell.strip() for cell in re.split(r"(?<!\\)\|", row.strip())[1:-1]]
return found if len(found) == count else Failure(f"this row needs {count} cells: {row}")
def parse_row(row: str) -> Item | Failure:
"""An item row, every cell checked but the goals it cites."""
found = cells(row, 10)
if isinstance(found, Failure):
return found
id_, release, exempt, tagline, r, s, a, g, goals, _ = found
if not re.fullmatch(r"[0-9]+", id_) or int(id_) < 1:
return Failure(f"the ID {id_!r} is not a positive whole number: {row}")
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", release):
return Failure(f"item {id_}'s release {release!r} is not a semver version")
if exempt not in EXEMPTIONS:
return Failure(f"item {id_}'s Exempt {exempt!r} is not empty or one of {', '.join(EXEMPTIONS[1:])}")
if not re.fullmatch(r"\*\*[^*].*[^*]\*\*", tagline):
return Failure(f"item {id_}'s Item cell is not one bold tagline: {tagline}")
scores = {}
for name, value in zip("RSAG", (r, s, a, g)):
if not re.fullmatch(r"[0-9]+", value) or not 1 <= int(value) <= 10:
return Failure(f"item {id_}'s {name} {value!r} is not a whole number from 1 to 10")
scores[name] = int(value)
cited = tuple(goal.strip() for goal in goals.split(","))
major, minor, patch = (int(part) for part in release.split("."))
return Item(int(id_), (major, minor, patch), exempt, tagline, scores, cited)
View File
+9
View File
@@ -0,0 +1,9 @@
import pathlib
import typing
from project_key import project_key
def project_archive(checkout_root: str, home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
archive = pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else home / ".claude" / "loop-claude-transcripts"
return archive / project_key(checkout_root, home)
+27
View File
@@ -0,0 +1,27 @@
import re
from markdown_text import quoted
from utc_stamp import STAMPED
FINISHED = "Verdict: finished"
FINISHED_AT = re.compile(rf"^({STAMPED}) {re.escape(FINISHED)}$", re.MULTILINE)
def finished(evidence: str) -> str:
return f"{FINISHED}\n\nEvidence:\n{quoted(evidence)}"
def unfinished(evidence: str) -> str:
return f"Verdict: unfinished\n\nEvidence:\n{quoted(evidence)}"
def interrupted(question: str, evidence: str) -> str:
return f"Verdict: interrupt\n\n{question}\n\nEvidence:\n{quoted(evidence)}"
def answered(question: str, sources: str, answer: str, evidence: str) -> str:
return f"Verdict: quoted answer\n\n{question}\n\n{sources} says:\n{quoted(answer)}\n\nEvidence:\n{quoted(evidence)}"
def stopped(why: str) -> str:
return f"Stopped: {why}"
+35
View File
@@ -0,0 +1,35 @@
import dataclasses
import pathlib
from failure import Failure
from transcript_archive.outcomes import FINISHED_AT
@dataclasses.dataclass(frozen=True)
class FinishedChunk:
folder: str
finished: str
transcript: pathlib.Path
subagents: list[pathlib.Path]
# One per chunk folder, its latest finished copy, in the order they finished.
def finished_chunks(project_archive: pathlib.Path) -> list[FinishedChunk] | Failure:
found = []
try:
for folder in sorted(path for path in project_archive.glob("*") if path.is_dir()):
copies = sorted((stamps[-1], outcomes) for outcomes in folder.glob("*.outcomes.md") if (stamps := FINISHED_AT.findall(outcomes.read_text(encoding="utf-8"))))
if copies:
finished, outcomes = copies[-1]
found.append(FinishedChunk(folder.name, finished, outcomes.with_name(outcomes.name.removesuffix(".outcomes.md") + ".jsonl"), sorted(folder.glob("session/*/subagents/*.jsonl"))))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read the archive {project_archive}: {failure}")
return sorted(found, key=lambda chunk: (chunk.finished, chunk.folder))
def read_threads(chunk: FinishedChunk) -> list[bytes] | Failure:
"""The worker's transcript, then each subagent's."""
try:
return [path.read_bytes() for path in (chunk.transcript, *chunk.subagents)]
except OSError as failure:
return Failure(f"could not read {failure.filename}: {failure.strerror}")
@@ -0,0 +1,61 @@
import contextlib
import hashlib
import io
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from transcript_archive import outcomes, write
from transcript_archive.location import project_archive
from project_key import project_key
from utc_stamp import utc
NOW = 1790000000
class Archive(unittest.TestCase):
def test_writes_afresh_a_copy_whose_outcomes_file_is_missing(self):
with tempfile.TemporaryDirectory() as root:
kept = pathlib.Path(root, "archive")
transcript = pathlib.Path(root, "t.jsonl")
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.unfinished("working"), NOW))
(kept_outcomes,) = kept.glob("*/*.outcomes.md")
kept_outcomes.unlink()
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.stopped("Ctrl-C"), NOW))
self.assertEqual(kept_outcomes.read_text(), "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n")
class CopyFor(unittest.TestCase):
def test_skips_a_version_already_kept_beside_its_copy(self):
with tempfile.TemporaryDirectory() as temp:
copy = pathlib.Path(temp, "custom-title.json")
copy.write_bytes(b"first")
beside = write.copy_for(copy, b"second")
self.assertEqual(beside, copy.with_name(f"custom-title.json.{hashlib.sha256(b'second').hexdigest()}"))
beside.write_bytes(b"second")
self.assertIsNone(write.copy_for(copy, b"second"))
self.assertEqual(write.copy_for(copy, b"first, grown"), copy)
class Outcomes(unittest.TestCase):
def test_the_archive_stamps_what_it_reads_back(self):
self.assertRegex(f"{utc(NOW)} {outcomes.finished('merged')}", outcomes.FINISHED_AT)
class Location(unittest.TestCase):
def test_keys_a_project_by_its_path_under_code_or_else_under_the_root(self):
home = pathlib.Path("/h")
self.assertEqual(project_key("/h/code/log", home), "log")
self.assertEqual(project_key("/h/code", home), "h/code")
self.assertEqual(project_key("/srv/log", home), "srv/log")
self.assertEqual(project_archive("/h/code/log", home, {}), pathlib.Path("/h/.claude/loop-claude-transcripts/log"))
self.assertEqual(project_archive("/h/code/log", home, {"LOOP_CLAUDE_TRANSCRIPTS": "/a"}), pathlib.Path("/a/log"))
if __name__ == "__main__":
unittest.main()
@@ -1,38 +1,21 @@
import dataclasses
import hashlib
import os
import pathlib
import re
import sys
import time
from failure import Failure
from loop_claude.claude_code.transcript import first_timestamped_line
from claude_code.transcript import first_timestamped_line
from file_writes import make_private, replace_file
from loop_claude.paths import Paths
from loop_claude.claude_code.session import session_files
FINISHED = "Verdict: finished"
UNFINISHED = "Verdict: unfinished"
STAMP = "%Y-%m-%dT%H:%M:%SZ"
STAMPED = r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ"
FINISHED_AT = re.compile(rf"^({STAMPED}) {re.escape(FINISHED)}$", re.MULTILINE)
from claude_code.session import session_files
from utc_stamp import utc
@dataclasses.dataclass(frozen=True)
class FinishedChunk:
folder: str
finished: str
transcript: pathlib.Path
subagents: list[pathlib.Path]
def archive(project_key: str, path: str, task: str, outcome: str, at: float, paths: Paths) -> Failure | None:
def archive(project_archive: pathlib.Path, path: str, task: str, outcome: str, at: float) -> Failure | None:
try:
with open(path, "rb") as handle:
raw = handle.read()
# Each chunk's first timestamped line is its own: a finished chunk's session is deleted, so the next chunk's transcript starts afresh.
folder = paths.transcript_archive / project_key / hashlib.sha256(first_timestamped_line(raw)).hexdigest()
folder = project_archive / hashlib.sha256(first_timestamped_line(raw)).hexdigest()
make_private(folder)
except OSError as failure:
return unarchived(failure.filename, failure.strerror)
@@ -101,21 +84,3 @@ def copy_for(copy: pathlib.Path, data: bytes) -> pathlib.Path | None:
return copy if data != kept else None
beside = copy.with_name(f"{copy.name}.{hashlib.sha256(data).hexdigest()}")
return None if beside.exists() else beside
def utc(at: float) -> str:
return time.strftime(STAMP, time.gmtime(at))
# One per chunk folder, its latest finished copy, in the order they finished.
def finished_chunks(project_archive: pathlib.Path) -> list[FinishedChunk] | Failure:
found = []
try:
for folder in sorted(path for path in project_archive.glob("*") if path.is_dir()):
copies = sorted((stamps[-1], outcomes) for outcomes in folder.glob("*.outcomes.md") if (stamps := FINISHED_AT.findall(outcomes.read_text(encoding="utf-8"))))
if copies:
finished, outcomes = copies[-1]
found.append(FinishedChunk(folder.name, finished, outcomes.with_name(outcomes.name.removesuffix(".outcomes.md") + ".jsonl"), sorted(folder.glob("session/*/subagents/*.jsonl"))))
except (OSError, UnicodeDecodeError) as failure:
return Failure(f"could not read the archive {project_archive}: {failure}")
return sorted(found, key=lambda chunk: (chunk.finished, chunk.folder))
+8
View File
@@ -0,0 +1,8 @@
import time
STAMP = "%Y-%m-%dT%H:%M:%SZ"
STAMPED = r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ"
def utc(at: float) -> str:
return time.strftime(STAMP, time.gmtime(at))
+396
View File
@@ -732,3 +732,399 @@ Inherited architect, would restructure first:
- `bin/larv_review/git.py:10` `git`, `bin/metric_bench/main.py:143` `git` and `bin/reading_cost/main.py:113` `located`: one git runner
- `bin/loop_claude/project/checkout.py:32` `read_utf8`: a generic reader under a lying name
- `bin/larv_review/main.py:101` `main`: an if/elif dispatch where the next command lands
## 2026-10-09T12:31:47Z, PR #234 at b1913ed, against 2157f59
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | better | same | better | better |
| Inherited architect | better | better | better | better | better |
Mid A, decided by:
- Locality, Navigation: `bin/transcript_archive/outcomes.py:4` `FINISHED_AT` and `finished`: one module now writes and parses the outcome lines, which `bin/loop_claude/cycle/run.py:83` built inline and another file parsed
- Locality, Navigation: `bin/reading_cost/main.py:35` imports: reading-cost depends only on the shared `transcript_archive`, no longer on loop-claude's internals
- Locality: `bin/loop_claude/paths.py:15` `paths_from`: no longer reads the environment; `LOOP_CLAUDE_TRANSCRIPTS` has one home in `bin/transcript_archive/location.py:5` `project_archive`
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the import root, the shared packages and where tests live
- Self-sufficiency: `bin/transcript_archive/write.py:13` `archive`: states the archive's layout at the write
- Shape: `bin/loop_claude/loop/main.py:86` `run_loop`: imports `project_key`, which names the session and the models entry, from the archive package; `bin/loop_claude/cycle/worker.py:7` takes `quoted` from the archive's outcomes
- Shape: `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES`: judge_reply merged in, 183 lines
Inherited architect, decided by:
- Navigation, Shape: `bin/reading_cost/main.py:35` `main`: imports one shared package named for what it holds, in place of loop-claude's private packages
- Locality, Self-sufficiency: `bin/transcript_archive/outcomes.py:10` `finished`: the outcome texts sit beside `FINISHED_AT`, the regex that reads them back
- Shape: `bin/claude_code/session.py:21` `session_named`: the Claude Code package depends on nothing in loop-claude
- Locality: `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES` and `:93` `verdict_from`: in one file with `run_judge`
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the cap no longer depends on case order
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the layout and puts each test beside its package
- Self-sufficiency: `bin/loop_claude/cycle/run.py:34` `Cycle.unfinished_evidence`: the name says what `remaining` needed a comment for
- Shape: `bin/loop_claude/loop/main.py:86` `run_loop`, `bin/loop_claude/cycle/worker.py:7` `worker_args` and `bin/host_env.py:5` `host_env`: small seams against the decoupling
## 2026-10-09T12:32:00Z, PR #234 at b1913ed
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Junior A | 7 | 6 | 6.5 | 6 | 6.5 |
| Mid A | 7 | 6 | 6.5 | 6 | 6.5 |
| Maintainability senior | 7 | 6.5 | 7 | 6.5 | 7 |
| Inherited architect | 7 | 6.5 | 6 | 6.5 | 6.5 |
| Mean | 7.00 | 6.25 | 6.50 | 6.25 | 6.63 |
Junior A, hardest first:
- `skills/larv-review/SKILL.md:55` "Round cap": the counting rule spread across Round cap, Re-entry, Fresh and Suspended
- `bin/larv_review/main.py:197` `record_ruling`: split, fell, refuse and stand branches; `ruling` returns None for a split and `fallen` reuses it
- `bin/larv_review/scoring.py:54` `scoring_run_due`: why the next-release fallback holds is stated nowhere
- `bin/larv_review/base.py:14` `measured_commit`: mechanics without purpose
- `bin/loop_claude/cycle/judge.py:126` `run_judge`: the prompt assembled from `records.py` constants; two texts, two matchers
- `bin/reading_cost/measure.py:21` `Checkout`: `root` and `top` read as synonyms
Mid A, hardest first:
- `bin/larv_review/main.py:197` `record_ruling`: said and counted, None as split, three fall steps
- `bin/comprehension_history.py:74` `entries`: the history format is defined only by its writer
- `bin/loop_claude/cycle/judge.py:126` `run_judge`: elided evidence fails, a misquoted answer becomes an interrupt
- `bin/larv_review/scoring.py:54` `scoring_run_due`: one dense sentence over two plans; the module also parses the plan
- `bin/larv_review/base.py:14` `measured_commit`: the why lives in the help and the skill
- `bin/loop_claude/cycle/states.py:69` `transition`: `quoted_answers` threaded through three states and two files
Maintainability senior, hardest first:
- `bin/larv_review/main.py:197` `record_ruling`: said and count, ruled reassigned, a silent fall-through; `ruling`'s None means two things
- `bin/larv_review/base.py:14` `measured_commit`: a three-step fallback across three files, ordered by an ISO string unsaid
- `bin/comprehension_history.py:74` `entries`: the format is never stated in one place
- `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES`: the prompt interpolates four constants from another module
- `bin/larv_review/scoring.py:54` `scoring_run_due`: five inputs; `scoring` also parses the plan
- `bin/rank_todo.py:416` `rank_details`: regex surgery in a script larv-review imports as a library
Inherited architect, would restructure first:
- `bin/loop_claude/project/records.py:6` `QUOTE_SOURCES`: prompt fragments spliced into the judge, worker and interrupt texts
- `bin/transcript_archive/location.py:10` `project_key`: the project's identity filed under the archive
- `bin/host_env.py:5` `host_env`: a test helper among the production modules
- `bin/larv_review/rounds.py:8` `round_counts`: only counts; the cap lives in the skill
- `bin/larv_review/base.py:1` the module: the ruling base, named like a base class
- `bin/larv_review/scoring.py:6` imports: one tool imports another tool's internals from `bin/rank_todo.py`
- `bin/loop_claude/loop/main.py:1` the docstring: restates constants it cannot keep true
## 2026-10-09T12:46:51Z, PR #234 at dde06fc, against 2157f59
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | better | better | better | better |
| Inherited architect | better | better | better | better | better |
Mid A, decided by:
- Navigation, Locality, Shape: `bin/transcript_archive/location.py:7` `project_archive`: the archive's location lives in one shared package, where reading-cost reached into loop-claude's internals
- Locality, Shape, Self-sufficiency: `bin/loop_claude/cycle/run.py:83` `worker_start`: `worker_args` builds only flags and each prompt is a named function in `bin/loop_claude/cycle/prompts.py:41`
- Locality, Self-sufficiency: `bin/transcript_archive/outcomes.py:10` `finished`: the outcomes format's writer and reader sit in one file
- Shape: `bin/larv_review/ruling.py:42` `split`: `ruling` no longer returns None for a split
- Shape, Locality: `bin/larv_review/main.py:204` `record_ruling`: two plain checks in place of `fall_step`'s strings; plan logic in `bin/larv_review/plan.py:5`
- Navigation: `README.md:77` "Tools": states the import root and the shared packages
- Self-sufficiency: `bin/comprehension_history.py:74` `entries`: a worked example of an entry above the parser
- Self-sufficiency, Shape: `bin/metric_bench/bench.py:88` `tracked_places`: the rename garbled the docstring
- Locality: `bin/loop_claude/cycle/prompts.py:60` `quoted`: duplicates `bin/transcript_archive/outcomes.py:31` `quoted`
Inherited architect, decided by:
- Navigation, Locality: `bin/transcript_archive/outcomes.py:10` `finished`: every outcome line and `FINISHED_AT` sit in one module
- Navigation, Locality: `bin/reading_cost/main.py:35` imports: reading-cost no longer reaches into loop-claude's internals
- Navigation, Self-sufficiency: `README.md:77` "Tools": the layout is stated
- Shape, Locality: `bin/todo_table.py:27` `items`: the table format is its own module, not a CLI script used as a library
- Shape: `bin/larv_review/plan.py:5` `scoring_run_due`: the plan trigger with its why, apart from the floor arithmetic
- Locality, Shape: `bin/larv_review/ruling.py:42` `split`: split, ruling and lead are separate
- Locality, Shape: `bin/loop_claude/cycle/prompts.py:41` `fresh`: every prompt text in one module
- Shape: `bin/comprehension_history.py:67` `ScoringRun`: one name no longer covers two concepts
- Self-sufficiency: `bin/larv_review/main.py:5` the help: defines the ratchet, the paired ruling and the scoring run first
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the past-the-cap case has its own guard
- Shape: `bin/metric_bench/bench.py:87` `tracked_places`: the docstring reads worse after the rename
## 2026-10-09T12:46:51Z, PR #234 at dde06fc
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Junior A | 7 | 6.5 | 6 | 6 | 6 |
| Mid A | 7.5 | 6.5 | 7 | 6.5 | 7 |
| Maintainability senior | 7 | 6.5 | 7 | 7 | 7 |
| Inherited architect | 7 | 6.5 | 6.5 | 7 | 7 |
| Mean | 7.13 | 6.50 | 6.63 | 6.63 | 6.75 |
Junior A, hardest first:
- `bin/larv_review/main.py:204` `record_ruling`: six outcomes, said against count, `[-1]` meaning Overall
- `bin/larv_review/ruling.py:8` `votes`: a parser that also enforces the seat-count rule
- `bin/larv_review/ruling_base.py:14` `measured_commit`: the docstring explains only after the code confused
- `bin/larv_review/plan.py:5` `scoring_run_due`: four ideas at once; `emptied` traced by hand
- `bin/metric_bench/bench.py:87` `tracked_places`: a garbled docstring
- `bin/loop_claude/cycle/states.py:69` `transition`: the quoted-answer cap across three files
Mid A, hardest first:
- `skills/larv-review/SKILL.md:55` "Round cap": four subsections pointing at each other
- `bin/larv_review/main.py:204` `record_ruling`: five outcomes; the duplicate check in two places
- `bin/loop_claude/cycle/judge.py:95` `run_judge`: many endings, two quote checks
- `bin/loop_claude/cycle/states.py:69` `transition`: the cap counter rides through three states
- `bin/larv_review/ruling_base.py:14` `measured_commit`: needs the history format and first-parent semantics
- `bin/reading_cost/measure.py:27` `measure`: string-compared timestamps and approximate shell parsing
Maintainability senior, hardest first:
- `bin/larv_review/main.py:204` `record_ruling`: five outcomes, competing local names
- `skills/larv-review/SKILL.md:55` "Round cap": a chain of cross-references
- `bin/comprehension_history.py:103` `seat_places`: depends unsaid on `sections` dropping blank lines
- `bin/larv_review/plan.py:5` `scoring_run_due`: four inputs; the rule restated in the help
- `bin/larv_review/ruling_base.py:14` `measured_commit`: why same is safe needs the skill
- `bin/metric_bench/bench.py:87` `tracked_places`: a garbled docstring
Inherited architect, would restructure first:
- `bin/larv_review/rounds.py:12` `round_counts`: the round cap lives in prose; a mis-cased title escapes the count
- `bin/loop_claude/cycle/judge.py:17` `Reply`: the judge contract spread over four files
- `bin/loop_claude/project/checkout.py:28` `read_utf8`: a generic reader under the checkout's name
- `bin/test_env.py:5` `isolated_env`: test support named like a test
- `bin/larv_review/git.py:6` imports: the git adapter depends on a domain type
- `bin/loop_claude/cycle/prompts.py:60` `quoted`: two copies, with `STAMP` and the git runner also duplicated
## 2026-10-09T12:53:41Z, PR #234 at b346a35, against 2157f59
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | better | better | better | better |
| Inherited architect | better | better | better | better | better |
Mid A, decided by:
- Navigation, Locality: `bin/reading_cost/main.py:35` `main`: the archive reader and location come from a shared package, not loop-claude's internals
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: picks one of three named prompts for a four-argument `worker_args`
- Navigation, Shape: `bin/loop_claude/cycle/prompts.py:1` the module: the texts the worker and the human see sit together
- Locality, Shape: `bin/larv_review/main.py:204` `record_ruling`: plain checks with `split`, `lead` and `overall`
- Navigation, Shape: `bin/larv_review/ruling_base.py:4` `measured_commit`: a truthful module name; `HistoryChange` beside the format
- Navigation, Locality: `bin/todo_table.py:27` `items`: the table parser is shared, not reached into a CLI script
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the layout
- Self-sufficiency: `bin/comprehension_history.py:82` `entries`: a worked example above the parser
- Shape: `bin/reading_cost/measure.py:20` `Checkout`: fields say which root keys the archive
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the past-the-cap case has its own guard
Inherited architect, decided by:
- Locality, Shape: `bin/transcript_archive/location.py:7` `project_archive`: both tools import one shared package
- Navigation, Locality: `bin/transcript_archive/outcomes.py:10` `finished`: the outcome line sits beside the regex that reads it back
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: `worker_args` takes the finished prompt
- Shape: `bin/larv_review/git.py:5` imports: the git helper no longer depends on the base search
- Shape, Self-sufficiency: `bin/larv_review/plan.py:5` `scoring_run_due`: its own module, saying why each trigger exists
- Self-sufficiency, Locality: `bin/larv_review/ruling.py:27` `overall` and `:47` `split`: no None meaning split, no bare index
- Navigation: `bin/reading_cost/measure.py:20` `Checkout`: the fields name which root
- Self-sufficiency: `bin/comprehension_history.py:82` `entries`: a worked example
- Navigation: `README.md:77` "Tools": the import root and the tests' place
- Shape: `bin/larv_review/main.py:234` `recorded_ruling`: eight positional parameters, a step back
- Self-sufficiency: `bin/loop_claude/cycle/prompts.py:29` `quoted`: a second copy
## 2026-10-09T12:53:41Z, PR #234 at b346a35
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Junior A | 7 | 6 | 7 | 6 | 6.5 |
| Mid A | 7.5 | 7 | 7 | 6.5 | 7 |
| Maintainability senior | 7.5 | 7 | 7 | 7 | 7 |
| Inherited architect | 7.5 | 7 | 6.5 | 7 | 7 |
| Mean | 7.38 | 6.75 | 6.88 | 6.63 | 6.88 |
Junior A, hardest first:
- `skills/larv-review/SKILL.md:55` "Round cap": rules pointing at each other; Fresh mixes five things
- `bin/larv_review/main.py:204` `record_ruling`: five exits; the counting rule in `bin/larv_review/ruling.py:32` `counted`
- `bin/larv_review/ruling_base.py:4` `measured_commit`: a set difference of headings per change
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: a prompt built from constants in three files; two quote checks
- `bin/comprehension_history.py:95` `entries`: two label regexes tied to the writer's wording
- `bin/reading_cost/shell_reads.py:52` `segments`: heuristic shell parsing
Mid A, hardest first:
- `bin/larv_review/main.py:204` `record_ruling`: five branches across four modules
- `bin/larv_review/ruling_base.py:4` `measured_commit`: reread three times to trust it
- `bin/loop_claude/cycle/judge.py:127` `run_judge`: two quote checks with opposite policies
- `bin/loop_claude/cycle/states.py:69` `transition`: the quoted-answer counter through three states
- `bin/comprehension_history.py:95` `entries`: Markdown parsed back into records
- `bin/metric_bench/bench.py:87` `tracked_places`: order-dependent legacy resolution
Maintainability senior, hardest first:
- `bin/larv_review/main.py:204` `record_ruling`: six exits; `fresh=False` hides a skipped check
- `bin/larv_review/ruling_base.py:4` `measured_commit`: why needs the skill
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: assembled across three files
- `bin/comprehension_history.py:95` `entries`: the format defined by example and regexes
- `bin/loop_claude/cycle/states.py:69` `transition`: the counter survives an interrupt
- `bin/reading_cost/shell_reads.py:52` `segments`: opaque on what it misses
Inherited architect, would restructure first:
- `bin/larv_review/rounds.py:8` `round_counts`: the cap lives in prose; the tool only reports
- `bin/larv_review/main.py:204` `record_ruling`: the record commands in a 326-line main
- `bin/larv_review/git.py:8` `git`: four git wrappers, one archive key computed two ways
- `bin/loop_claude/cycle/prompts.py:29` `quoted`: duplicated, as is `STAMP`
- `bin/larv_review/ruling_base.py:24` `base`: a name that reads as a base class
## 2026-10-09T13:04:47Z, PR #234 at b2a277b, against 2157f59
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | better | better | better | better |
| Inherited architect | better | better | better | better | better |
Mid A, decided by:
- Locality, Navigation: `bin/transcript_archive/location.py:7` `project_archive`: the archive's writer and readers share one package; outcome strings sit beside the regex that reads them
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: picks the prompt in three lines; `worker_args` only builds argv
- Shape, Navigation: `bin/larv_review/main.py:100` `main`: only dispatch; recording and plan reading have their own modules
- Locality, Navigation: `bin/todo_table.py:27` `items`: table parsing shared, not reached into a tool script
- Shape: `bin/larv_review/ruling.py:47` `split`: split, lead and overall each say what they compute
- Shape, Self-sufficiency: `bin/reading_cost/measure.py:20` `Checkout`: fields name which root
- Locality: `bin/claude_code/session.py:21` `session_named`: the adapter takes only a name and a folder
- Navigation: `README.md:77` "Tools": names the import root and where tests live
- Self-sufficiency: `bin/larv_review/main.py:5` the help: explains the ruling, ratchet, scoring run and floor first
Inherited architect, decided by:
- Locality, Navigation: `bin/transcript_archive/location.py:7` `project_archive`: location, writer and reader are peers both tools import
- Locality: `bin/transcript_archive/outcomes.py:6` `FINISHED`: the format and its parser in one module
- Shape, Navigation: `bin/larv_review/plan.py:8` `scoring_run_due`: the plan trigger has its own file; `scoring.py` holds only the floor
- Shape: `bin/larv_review/record.py:14` `record_ruling`: recording is one module, its guards inline
- Locality, Self-sufficiency: `bin/loop_claude/cycle/worker.py:40` `worker_args`: takes a ready prompt; the texts are named functions
- Shape: `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: the judge is one file
- Locality: `bin/loop_claude/paths.py:15` `paths_from`: the env override is read in one place
- Self-sufficiency: `bin/comprehension_history.py:81` `entries`: a worked example above the parser
- Navigation, Self-sufficiency: `README.md:77` "Tools": the layout is stated and checked
- Locality: `bin/larv_review/ruling.py:52` `ruling`: returns same for a split, so only `split` guards it
## 2026-10-09T13:04:48Z, PR #234 at b2a277b
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Junior A | 7 | 6 | 7 | 6 | 6.5 |
| Mid A | 7.5 | 6.5 | 7 | 6.5 | 6.5 |
| Maintainability senior | 7.5 | 6.5 | 7 | 6.5 | 7 |
| Inherited architect | 7.5 | 7 | 6.5 | 7 | 7 |
| Mean | 7.38 | 6.50 | 6.88 | 6.50 | 6.75 |
Junior A, hardest first:
- `bin/larv_review/record.py:14` `record_ruling`: seven exits; the terms only in the help and the skill
- `bin/larv_review/ruling_base.py:4` `measured_commit`: diffs heading sets per change
- `bin/comprehension_history.py:94` `entries`: regex groups used by position
- `bin/larv_review/ruling.py:32` `counted`: the counting rule stated in the help, not here
- `bin/loop_claude/cycle/states.py:69` `transition`: the cap counter across three files
- `bin/loop_claude/cycle/judge.py:127` `run_judge`: two quote checks with different rules
Mid A, hardest first:
- `skills/larv-review/SKILL.md:82` "Architecture": conditions with exceptions; Fresh took three rereads
- `bin/larv_review/record.py:14` `record_ruling`: six outcomes across five modules
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: one sentence takes three files
- `bin/larv_review/ruling_base.py:4` `measured_commit`: the fallback chain over two files
- `bin/larv_review/plan.py:8` `scoring_run_due`: release sets held at once
- `bin/comprehension_history.py:94` `entries`: the format pinned in three places
Maintainability senior, hardest first:
- `bin/larv_review/record.py:14` `record_ruling`: five exits; the rule spread over five modules
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: instructions stitched from three modules
- `bin/larv_review/plan.py:8` `scoring_run_due`: five inputs; the docstring restates the help
- `bin/larv_review/ruling_base.py:4` `measured_commit`: opaque without its docstring
- `bin/larv_review/seats.py:17` `SeatsFile`: `read` is a noun; locals rename one pair two ways
- `bin/comprehension_history.py:110` `seat_places`: a line-state parser two tools depend on
Inherited architect, would restructure first:
- `bin/larv_review/rounds.py:4` `PASSES`: the cap is the skill's; the pass list has two homes
- `bin/comprehension_history.py:149` `ruling_entry`: reader, writer and seat registry in one root module
- `bin/larv_review/record.py:67` `read_fall_scores`: a parameter named votes holds a heading
- `bin/larv_review/git.py:8` `git`: four ways to run git; the archive key's root differs between tools
- `bin/larv_review/record.py:14` `record_ruling`: branches on worse, fell, scores file and split at once
## 2026-10-09T13:23:40Z, PR #234 at 7e13778, against 2157f59
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | better | better | better | better |
| Inherited architect | better | better | better | better | better |
Mid A, decided by:
- Navigation, Shape: `bin/larv_review/record.py:14` `record_ruling`: recording, dispatch and plan reading each in a file named for it
- Shape, Navigation: `bin/larv_review/plan.py:8` `scoring_run_due`: file names tell the truth; todo parsing in `bin/todo_table.py:27` `items`
- Navigation, Shape: `bin/transcript_archive/location.py:7` `project_archive`: both tools share top-level packages, not another tool's internals
- Locality: `bin/loop_claude/cycle/run.py:83` `worker_start`: a prompt changes without reading the argument plumbing
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the past-the-cap case has its own guard
- Locality: `bin/transcript_archive/location.py:7` `project_archive`: the env override resolved in one place
- Self-sufficiency: `bin/larv_review/main.py:1` the help: defines seats, the ruling, the ratchet and the floor first
- Shape, Self-sufficiency: `bin/reading_cost/measure.py:20` `Checkout`: fields say which root keys the archive
Inherited architect, decided by:
- Locality, Navigation: `bin/transcript_archive/outcomes.py:6` `FINISHED`: outcome lines written and parsed in one file
- Locality, Shape: `bin/transcript_archive/location.py:7` `project_archive`: reading-cost no longer reaches into loop-claude
- Navigation, Locality: `bin/loop_claude/cycle/judge.py:94` `verdict_from`: the 3am question lands in one file
- Shape, Navigation: `bin/larv_review/main.py:102` `main`: dispatch only
- Locality, Shape: `bin/todo_table.py:27` `items`: larv-review imports no tool's internals
- Locality: `bin/larv_review/record.py:32` `record_ruling`: the fall handling reads in place
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: the start chosen at the call site
- Shape: `bin/reading_cost/measure.py:20` `Checkout`: names that say which root
- Self-sufficiency: `bin/comprehension_history.py:81` `entries`: a sample entry above the parser
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: no case-order dependence
## 2026-10-09T13:23:40Z, PR #234 at 7e13778
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Junior A | 7.5 | 7 | 7 | 6.5 | 7 |
| Mid A | 7 | 6.5 | 7 | 6.5 | 6.5 |
| Maintainability senior | 7 | 6.5 | 7 | 6 | 6.5 |
| Inherited architect | 7 | 6.5 | 6 | 7 | 6.5 |
| Mean | 7.13 | 6.63 | 6.75 | 6.50 | 6.63 |
Junior A, hardest first:
- `bin/larv_review/record.py:14` `record_ruling`: four outcomes, two parallel vote dicts
- `bin/larv_review/ruling.py:32` `counted`: an asymmetric rule; `PLACE` in another file
- `bin/metric_bench/python_units.py:86` `score`: needs a paper outside the repo
- `bin/comprehension_history.py:94` `entries`: leans on `sections` dropping blank lines
- `bin/loop_claude/cycle/judge.py:127` `run_judge`: two quote checks against two texts
- `skills/larv-review/SKILL.md:55` "Round cap": sections pointing at each other
Mid A, hardest first:
- `bin/larv_review/record.py:14` `record_ruling`: five outcomes; `read` as a noun
- `bin/comprehension_history.py:110` `seat_places`: depends unsaid on `sections` dropping blank lines
- `bin/larv_review/ruling_base.py:4` `measured_commit`: several ideas held at once
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: decision logic in English
- `bin/claude_code/entries.py:28` `written_by_human`: heuristics over the transcript shape
- `bin/loop_claude/loop/reload.py:50` `exec_if_edited`: any tool's edit reloads the loop
Maintainability senior, hardest first:
- `skills/larv-review/SKILL.md:82` "Architecture": six interlocking conditions
- `bin/larv_review/record.py:14` `record_ruling`: two checks run twice; the rule whole only in the help
- `bin/comprehension_history.py:94` `entries`: a format shared by three tools by string convention
- `bin/larv_review/plan.py:8` `scoring_run_due`: three things combined in "work has landed"
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: behaviour in prose, post-filtered three ways
- `bin/rank_todo.py:357` `rank_details`: rebuilds the file by slicing
Inherited architect, would restructure first:
- `bin/larv_review/rounds.py:8` `round_counts`: the cap is prose in the skill
- `bin/larv_review/record.py:14` `record_ruling`: an if-chain with prints interleaved
- `bin/larv_review/scoring.py:8` `fall_ruling`: a ruling rule in the scoring module
- `bin/claude_code/trust.py:10` `trust`: effects in a package documented as shapes
- `bin/larv_review/git.py:8` `git`: git run five ways
+2 -1
View File
@@ -52,7 +52,8 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
- 2026-10-08, PR #225: 7.25, 6.75, 7.00, 6.88, 6.88. Above PR #224's run on every score but Shape, which held.
- 2026-10-08, PR #228: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet: the seats blamed the new ratchet's own rules, since moved into the tested `bin/larv-review`.
- 2026-10-08, PR #229: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet for PR #228's reason: the seats' decisive places were still PR #228's rules, since moved into `bin/larv-review`.
- 2026-10-09, PR #232: 7.13, 6.50, 6.63, 6.63, 6.88, at 324d650, whose ruling the fall made worse. Its last paired ruling, at 53014f8 against 12d6d75, split: Mid A better, the inherited architect worse, both with Locality worse. lilleman shipped it below the ratchet: after a restructure the seats still weigh PRs #229 to #231's code against the base, and their Locality places are filed under items 62 and 83.
- 2026-10-09, PR #232: 7.13, 6.50, 6.63, 6.63, 6.88, at 324d650, whose ruling the fall made worse. Its last paired ruling, at 53014f8 against 12d6d75, split: Mid A better, the inherited architect worse, both with Locality worse. lilleman shipped it below the ratchet: after a restructure the seats still weigh PRs #229 to #231's code against the base, and their Locality places are filed under item 62; the `bin/` regrouping, item 83, shipped in PR #234.
- 2026-10-09, PR #234: 7.13, 6.63, 6.75, 6.50, 6.63, at 7e13778. Four architecture rounds and the rescore each ruled better, 2 votes to 0, while the means moved within the panel's noise between runs. lilleman shipped it below the floor: most of every seat's remaining places sit in larv-review's rules, which cannot be followed without `skills/larv-review/SKILL.md` §3 open beside the code. Fixing that is a chunk of its own, under item 62.
## loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
+1 -1
View File
@@ -6,7 +6,7 @@ A scoring-run seat's prompt is the opening, the seat's background, the shared br
- `{main}`: the project's main checkout.
- `{forbidden}`: `todo.md, docs/decisions.md, docs/comprehension-history.md or docs/reading-cost.md (the plan, the decision log, past panels' records and past chunks' reading cost)`.
- `{git}`: `git -C {checkout} ls-files`, and for a paired seat also `git -C {base checkout} ls-files`, each in backticks.
- `{files}`: every tracked non-test file but those `{forbidden}` names, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, such as `bin/loop_claude/**/*.py`.
- `{files}`: every tracked file in `git -C {checkout} ls-files`, except test files and those `{forbidden}` names. Give one recursive glob per top-level source directory, say which test files it leaves out, and list each file no glob matches. Example: `bin/**/*.py` except `test_*.py`, plus launchers such as `bin/loop-claude`.
- `{preloaded}`: the files the harness pre-loads into a seat, found per `SKILL.md` → "What makes the result trustworthy".
- `{anchors}` and `{dimensions}`: the six anchor bullets and the four dimension bullets in `SKILL.md` → "The score", verbatim, without bold and each on one line.
- `{language}`, `{kind}`, `{domain}`, `{domain docs}` and `{3am question}`: the project's `AGENTS.md` → "Scoring run".
+2 -11
View File
@@ -31,7 +31,6 @@
| 59 | 0.1.0 | | **Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.** | 3 | 3 | 6 | 6 | 1.4 | 19.8 |
| 12 | 0.1.0 | | **Run the test suite in CI on every PR, or record in `docs/decisions.md` why the local run is enough.** | 2 | 4 | 4 | 6 | 1.1 | 16.9 |
| 80 | 0.1.0 | | **State in README → Install that every machine taking this checkout must be a sandbox, as `settings.json`'s auto mode assumes.** | 1 | 1 | 5 | 5 | 4 | 16.8 |
| 83 | 0.1.0 | | **Group `bin/` by tool, so each tool's modules and tests sit together.** | 3 | 4 | 4 | 6 | 1.1 | 15.9 |
| 5 | 0.1.0 | decision | **Key a project on its checkout's full path, and refuse to start while another session uses that key.** | 5 | 5 | 5 | 6 | 1.1 | 15.6 |
| 84 | 0.1.0 | | **Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.** | 2 | 2 | 4 | 5 | 1.4 | 15.1 |
| 67 | 0.1.0 | principle | **Point stability-reviewer at the technical principles in place of its own restatement of them.** | 2 | 3 | 4 | 5 | 1.3 | 14.9 |
@@ -62,11 +61,9 @@ The README line covers the bench's transcripts in `workflow-bench/logs/` too (go
### 62. Lift Locality, Self-sufficiency and the overall to the comprehension floor, at the places the seats name.
Raised by the principles audit of PR #215 against KISS, in `principles/technical-principles.md`, and goal 1.1: the `comprehension floor:` entry in `docs/decisions.md` lists seven decisions to ship under the floor, and 0.1.0 waits on the floor. PR #215's last run put Locality at 6.63 and Self-sufficiency and the overall at 6.88. Its seats named, under Locality: the judge's contract spread over `cycle/judge_reply.py`, `cycle/worker.py` and `cycle/run.py`; `remaining`, which is `Unfinished.evidence` under another name, threaded through `loop/main.py`, `loop/reload.py`'s `LoopRun`, `cycle/run.py` and `worker_args`; the `quoted_answers` cap held by case order in `cycle/states.py`'s `transition` and restated in `run.py`'s `question_for`; the four outcome texts `run.py` builds inline for `archive`. Under Self-sufficiency: `skills/larv-review/SKILL.md`, which needs `CLAUDE.md` → "Vocabulary" open beside it; the archive layout, defined only in `bin/loop_claude/loop/main.py`'s docstring and not at `transcript_archive.py`; `JUDGE_RULES`' precedence in `cycle/judge_reply.py`, restated by `verdict_from`'s branch order and the `Reply` field descriptions; the `sys.path` insert in `bin/loop-claude` that the `from failure import` lines rely on, unstated at the import sites.
Raised by the principles audit of PR #215 against KISS, in `principles/technical-principles.md`, and goal 1.1: 0.1.0 waits on the floor, and the `comprehension floor:` entry in `docs/decisions.md` lists each decision to ship under it.
PR #225's last run, at 8c9bf09, put Locality at 6.75 and Self-sufficiency and the overall at 6.88. Its seats named: the judge's policy spread over `loop/main.py`'s docstring, `cycle/judge_reply.py`'s `JUDGE_RULES`, `cycle/judge.py`'s `judge_prompt` and `cycle/worker.py`'s `worker_args`; `loop_claude/claude_code/` and the read side of `transcript_archive.py`, which `bin/reading_cost/` also imports; two meanings of a checkout's root in `project/checkout.py`'s `checkout_root` and `reading_cost/main.py`'s `located`; and `skills/larv-review/SKILL.md` §3's round count, which only prose enforces.
2026-10-09, from two architecture studies on PR #232: give the judge's contract one home. `cycle/judge.py` takes `judge_reply.py`'s rules, schema and `verdict_from`; `QUOTE_SOURCES`, `ANSWER_DESTINATIONS` and `ANSWERS_RECORDED` move to `project/records.py`, so `worker.py` imports the records, not the judge; `states.py`'s two `Judging` cases get exclusive guards, so their order carries no meaning. PR #232's scoring run at 324d650 put Locality at 6.50.
PR #234's last scoring run, at b2a277b, put Locality and Self-sufficiency at 6.50 and the overall at 6.75. Every seat named larv-review's rules, which cannot be followed without `skills/larv-review/SKILL.md` §3 open beside the code: "Round cap", "Re-entry", "Fresh" and "Suspended" point at each other; `bin/larv_review/record.py` `record_ruling` branches on split, worse, fell and a scores file, with its terms defined only in `larv-review --help` and the skill; the pass list lives in both `bin/larv_review/rounds.py` `PASSES` and the skill's table, which also counts the auto-reviewer; `bin/comprehension_history.py` is reader, writer and seat registry in one, and parses the history by regex for two tools. Also named: the judge's prompt assembled from `cycle/judge.py`, `cycle/prompts.py` and `project/records.py`; git run four ways, in `bin/larv_review/git.py`, `bin/metric_bench/main.py`, `bin/reading_cost/main.py` and `bin/loop_claude/effects.py`; `SeatsFile.read`, a field named like a method.
### 21. Give `settings.json`'s model one writer, and sort its keys.
@@ -90,12 +87,6 @@ The repo has no CI, so larv-review's gate is the local run, and larv-review's Ga
Raised by the principles audit of PR #224. The `settings.json` decision is valid while every machine taking this checkout is a sandbox, but Install asks for a sandbox only for `loop-claude`, so a new machine is never told.
### 83. Group `bin/` by tool, so each tool's modules and tests sit together.
Raised by the architect on PR #225 and by the inherited architect of PR #224's scoring run: `bin/` is one flat folder of tools, the modules they share and the repo's checks, with `loop_claude`'s tests beside them instead of beside the package. `loop_claude/claude_code/` parses Claude Code's own formats and `bin/reading_cost/measure.py` imports it too, so it is a package of its own.
2026-10-09, from two architecture studies on PR #232: lift `loop_claude/claude_code/` and the archive's layout out of `loop_claude/` into packages of their own, `claude_code/` and `transcript_archive/`, so `reading_cost` imports no tool. `transcript_archive` takes the write side (`archive`), the read side (`finished_chunks`, `FinishedChunk`, the stamps) and one `project_archive(checkout_root, home, env)` that both tools call, which derives the archive key one way, as item 5 asks. Move each tool's tests beside its package, and `test_loop_claude_black_box.host_env`, which three other tools' tests import, into a test helper of its own.
### 5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
The key names the project's session, archive folder and models entry. `project_key` drops the leading `/`, so `~/code/a` and `/a` share all three, and one loop can resume or delete the other's session.