Merge pull request 'Group bin/ by tool, and lift its comprehension scores towards the floor' (#234) from floor-restructure into main
This commit was merged in pull request #234.
This commit is contained in:
+10
-1
@@ -17,6 +17,8 @@
|
||||
!agents/
|
||||
!agents/*.md
|
||||
!bin/
|
||||
!bin/claude_code/
|
||||
!bin/claude_code/*.py
|
||||
!bin/comprehension_history.py
|
||||
!bin/failure.py
|
||||
!bin/file_writes.py
|
||||
@@ -27,16 +29,23 @@
|
||||
!bin/loop_claude/
|
||||
!bin/loop_claude/**/
|
||||
!bin/loop_claude/**/*.py
|
||||
!bin/markdown_sections.py
|
||||
!bin/markdown_text.py
|
||||
!bin/metric-bench
|
||||
!bin/metric_bench/
|
||||
!bin/metric_bench/*.py
|
||||
!bin/project_key.py
|
||||
!bin/rank_todo.py
|
||||
!bin/reading-cost
|
||||
!bin/reading_cost/
|
||||
!bin/reading_cost/*.py
|
||||
!bin/repo_checks/
|
||||
!bin/repo_checks/*.py
|
||||
!bin/statusline.py
|
||||
!bin/test_*.py
|
||||
!bin/todo_table.py
|
||||
!bin/transcript_archive/
|
||||
!bin/transcript_archive/*.py
|
||||
!bin/utc_stamp.py
|
||||
!docs/
|
||||
!docs/*.md
|
||||
!principles/
|
||||
|
||||
+1
-1
@@ -3,7 +3,7 @@
|
||||
## Unreleased
|
||||
|
||||
- The stability reviewer keeps its scratch work in a directory it creates for the run, deletes that directory afterwards, and deletes nothing else.
|
||||
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling <votes file> [<scores file>]` and `record-scoring <scores file> [<item ID>...]`, which append to `docs/comprehension-history.md` and refuse a second entry on what a seats file's first line reads; `record-scoring` refuses a run the plan does not call, and prints `Floor:` naming each mean under the floor). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release whose last plan item moves to another release is due a scoring run too, once work has landed towards it. A seats file must name commits the checkout holds. When origin cannot be fetched, the tool reads it as last fetched and says so.
|
||||
- larv-review's mechanical rules run in the new, tested `bin/larv-review`: the paired ruling's base (`ruling-base`), each pass's round count (`round-count`, which ignores case in a round's commit title, counts a repeated round number as a round of its own, and says to ask you before another round once a pass is at or past the cap), the scoring run's trigger (`needs-scoring-run <item ID>...`), and the vote count, means and history format (`record-ruling <votes file> [<scores file>]` and `record-scoring <scores file> [<item ID>...]`, which append to `docs/comprehension-history.md` and refuse a second entry on what a seats file's first line reads; `record-scoring` refuses a run the plan does not call, and prints `Floor:` naming each mean under the floor). The skill says when to run each and what its answer means. A seats file line the tool cannot read fails the run, so a mistyped vote never counts as same. The base search compares the history before and after each merged change, so it no longer depends on how a line is worded, and orders a change's rulings by their headings' times. A release whose last plan item moves to another release is due a scoring run too, once work has landed towards it. A seats file must name commits the checkout holds. When origin cannot be fetched, the tool reads it as last fetched and says so.
|
||||
- The new `bin/metric-bench` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest: cognitive complexity, length, shared state and change spread, each scored by how well it ranks the named functions above the rest. No metric enters review yet.
|
||||
- larv-review blocks a merge when simulated readers rule that the PR makes the project read worse than the last measured commit: the default branch's newest merged change that added a scoring run to `docs/comprehension-history.md`, or whose last ruling there is better. A ruling of same leaves that commit in place, so a worse ruling may name places earlier chunks wrote. In each architecture pass Mid A and the inherited architect vote better, same or worse on each dimension and overall: `Navigation same, Locality worse, Shape same, Self-sufficiency same, Overall better`. When their overall votes differ, the maintainability senior and Junior A vote too. A ruling needs a lead of two or more votes, else it is same. Each place a seat names opens with the dimensions it moves, such as `Locality, Shape: `, or the run fails, and a vote that no place opens with counts as same. A dimension falls when its worse votes lead by two. When a dimension falls in a better or same ruling, `record-ruling` asks for the four scoring seats and, given their scores file, counts the ruling as worse when that dimension's mean is under the floor. A worse ruling never ships: after three rounds larv-review asks you for another round, a no leaves the PR unmerged, and so does an unattended run. The four-seat scoring run checks the comprehension floor only when a chunk ships or removes a release's last plan item, or ships an item naming the floor; between those runs, a score under the floor blocks a merge only through a fallen dimension, and that block counts as a worse ruling. On those runs, a score under the floor blocks the merge until fixed or until you decide to ship; a decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours.
|
||||
- larv-review reruns fewer passes after a fix. A fix reruns the pass whose 🔴 or 🟡 it answers, plus stability when the fix changes behaviour, security when it opens an untrusted-input boundary or touches authn/authz, secrets or crypto, and architecture when it carries an architectural decision. A rerun reviews only what changed since that pass last reviewed. A fix for a 🟢, a ❓ or any prose finding reruns no pass, and prose takes one round.
|
||||
|
||||
@@ -76,6 +76,8 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
|
||||
Test on the host, with `uv` on `PATH` to fetch mypy and the Python version Install names (`docs/decisions.md` → `bin/`): `python3 -m unittest discover -s bin -p 'test_*.py'`
|
||||
|
||||
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
|
||||
|
||||
## Tracking a new file
|
||||
|
||||
`.gitignore` is an allowlist. Track a file by negating it and its parent directories, then confirm with `git check-ignore -v <path>`:
|
||||
|
||||
@@ -6,7 +6,6 @@ import shutil
|
||||
import uuid
|
||||
|
||||
from failure import Failure
|
||||
from loop_claude.paths import Paths
|
||||
|
||||
NAMESPACE = uuid.UUID("64fb586b-d926-4bb3-a44c-c903e86d3eec")
|
||||
|
||||
@@ -19,10 +18,8 @@ class Session:
|
||||
name: str
|
||||
|
||||
|
||||
def session_for(project_key: str, paths: Paths) -> Session:
|
||||
# "A " is the human's cue in the session picker that the session is the loop's.
|
||||
name = f"A {project_key} loop-claude"
|
||||
return Session(id=uuid.uuid5(NAMESPACE, name), claude_code_transcripts=paths.claude_code_transcripts, name=name)
|
||||
def session_named(name: str, claude_code_transcripts: pathlib.Path) -> Session:
|
||||
return Session(id=uuid.uuid5(NAMESPACE, name), claude_code_transcripts=claude_code_transcripts, name=name)
|
||||
|
||||
|
||||
# <claude_code_transcripts>/-home-me-code-log/<session id>.jsonl
|
||||
@@ -0,0 +1,93 @@
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from claude_code import entries, transcript, trust
|
||||
|
||||
|
||||
def assistant(*parts):
|
||||
return {"type": "assistant", "message": {"content": list(parts)}}
|
||||
|
||||
|
||||
def human(content):
|
||||
return {"type": "user", "message": {"content": content}}
|
||||
|
||||
|
||||
def queued(prompt, kind):
|
||||
return {"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "prompt", "origin": {"kind": kind}, "prompt": prompt}}
|
||||
|
||||
|
||||
class Render(unittest.TestCase):
|
||||
def test_joins_assistant_text_tool_calls_human_turns_and_loop_prompts_and_skips_the_rest(self):
|
||||
said = transcript.render([
|
||||
human("do it"),
|
||||
assistant({"type": "text", "text": "Starting."}, {"type": "tool_use", "name": "Bash", "input": {"command": "ls"}}),
|
||||
assistant({"type": "text", "text": " "}),
|
||||
human([{"type": "tool_result", "content": "not typed"}]),
|
||||
human("<command-name>/exit</command-name>"),
|
||||
human([{"type": "text", "text": "yes, "}, {"type": "text", "text": "merge it"}]),
|
||||
queued("B, while you work", "human"),
|
||||
queued("origin/main moved", "coordinator"),
|
||||
{**human("Continue the task; the previous run stopped before it was finished."), "promptSource": "sdk"},
|
||||
])
|
||||
self.assertEqual(said, '[human] do it\n\nStarting.\n\n[tool Bash] {"command": "ls"}\n\n[human] yes, merge it\n\n[human] B, while you work\n\n[loop] Continue the task; the previous run stopped before it was finished.')
|
||||
|
||||
|
||||
class FirstTimestampedLine(unittest.TestCase):
|
||||
def test_skips_the_lines_every_chunk_shares_and_else_takes_the_first(self):
|
||||
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\nnot JSON\n{"timestamp": "t"}\n{"timestamp": "u"}\n'), b'{"timestamp": "t"}')
|
||||
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\n[1]\n'), b'{"type": "custom-title"}')
|
||||
self.assertEqual(transcript.first_timestamped_line(b""), b"")
|
||||
|
||||
|
||||
class HumanTurns(unittest.TestCase):
|
||||
def test_counts_typed_turns_and_not_tool_results(self):
|
||||
self.assertEqual(transcript.human_turns([
|
||||
human("typed"),
|
||||
human([{"type": "text", "text": "typed as a block"}]),
|
||||
human([{"type": "tool_result", "content": "not typed"}]),
|
||||
human("<command-name>/model</command-name>"),
|
||||
{**human("<3 merge it"), "promptSource": "typed"},
|
||||
{**human("<task-notification>done</task-notification>"), "promptSource": "system"},
|
||||
{**human([{"type": "text", "text": "<task-notification>done</task-notification>"}]), "promptSource": "system"},
|
||||
{**human("peer message"), "isMeta": True},
|
||||
{**human("the loop's own prompt"), "promptSource": "sdk"},
|
||||
human([{"type": "text", "text": "[Request interrupted by user]"}]),
|
||||
human([]),
|
||||
queued("typed while claude worked", "human"),
|
||||
queued("<task-notification>done</task-notification>", "task-notification"),
|
||||
{"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "task-notification", "prompt": "<task-notification>done</task-notification>"}},
|
||||
assistant({"type": "text", "text": "said"}),
|
||||
]), 4)
|
||||
|
||||
|
||||
class PartLine(unittest.TestCase):
|
||||
def test_renders_text_and_tool_calls_and_nothing_else(self):
|
||||
self.assertEqual(transcript.part_line({"type": "text", "text": "hi"}, 20), "hi")
|
||||
self.assertEqual(transcript.part_line({"type": "text", "text": " \n"}, 20), "")
|
||||
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Bash", "input": {"command": "x" * 500}}, 20), '[tool Bash] {"command": "xxxxxxx')
|
||||
self.assertEqual(transcript.part_line("not a part", 20), "")
|
||||
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Write", "input": {"content": "Hälsa på"}}, 40), '[tool Write] {"content": "Hälsa på"}')
|
||||
|
||||
|
||||
class Parse(unittest.TestCase):
|
||||
def test_yields_only_json_objects(self):
|
||||
self.assertEqual(entries.parse_entry('{"type": "user"}'), {"type": "user"})
|
||||
self.assertIsNone(entries.parse_entry("not json"))
|
||||
self.assertIsNone(entries.parse_entry("[1, 2]"))
|
||||
|
||||
|
||||
class MarkTrusted(unittest.TestCase):
|
||||
def test_returns_the_updated_config_and_leaves_the_one_it_was_handed(self):
|
||||
config = {"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"kept": True}}}
|
||||
handed = json.dumps(config)
|
||||
self.assertEqual(trust.mark_trusted(config, ["/a", "/b", "/c"]), ({"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"hasTrustDialogAccepted": True, "kept": True}, "/c": {"hasTrustDialogAccepted": True}}}, ["/b", "/c"]))
|
||||
self.assertEqual(json.dumps(config), handed)
|
||||
self.assertEqual(trust.mark_trusted({}, ["/a"]), ({"projects": {"/a": {"hasTrustDialogAccepted": True}}}, ["/a"]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,6 +1,6 @@
|
||||
import json
|
||||
|
||||
from loop_claude.claude_code.entries import assistant_parts, human_text, loop_text, parse_entry, part_text, tool_call
|
||||
from claude_code.entries import assistant_parts, human_text, loop_text, parse_entry, part_text, tool_call
|
||||
from failure import Failure
|
||||
|
||||
JUDGED_TOOL_INPUT = 400
|
||||
@@ -1,10 +1,10 @@
|
||||
import dataclasses
|
||||
import decimal
|
||||
import re
|
||||
import time
|
||||
|
||||
from failure import Failure
|
||||
from markdown_sections import sections
|
||||
from markdown_text import sections
|
||||
from utc_stamp import utc
|
||||
|
||||
ARCHITECT = "Inherited architect"
|
||||
DECIDED_BY = "decided by"
|
||||
@@ -22,7 +22,6 @@ RULING = re.compile(r"^Ruling: (better|same|worse)$", re.M)
|
||||
RULED_PLACES = re.compile(rf"^.+, {DECIDED_BY}:$")
|
||||
SCORED_PLACES = re.compile(rf"^.+, ({HARDEST_FIRST}|{RESTRUCTURE_FIRST}):$")
|
||||
SENIOR = "Maintainability senior"
|
||||
STAMP = "%Y-%m-%dT%H:%M:%SZ"
|
||||
TITLE = "# Comprehension history\n"
|
||||
|
||||
|
||||
@@ -55,6 +54,14 @@ class Entry:
|
||||
return self.against is not None
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class HistoryChange:
|
||||
"""A merged change on the default branch that touched the history, and the history before and after it."""
|
||||
commit: str
|
||||
before: str
|
||||
after: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Heading:
|
||||
title: str
|
||||
@@ -64,13 +71,26 @@ class Heading:
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Scoring:
|
||||
class ScoringRun:
|
||||
"""The four scoring seats' lines, their scores as written and the means, in DIMENSIONS' order."""
|
||||
seats: list[Seat]
|
||||
scores: dict[str, list[str]]
|
||||
means: list[decimal.Decimal]
|
||||
|
||||
|
||||
# A paired ruling as entry_text writes it. A scoring run's heading has no `, against`; it has no Ruling line, ends its table with a Mean row, and labels each seat's places `<seat>, hardest first:` or, for the architect, `<seat>, would restructure first:`.
|
||||
#
|
||||
# ## 2026-10-09T09:47:25Z, PR #232 at 324d650, against 12d6d75
|
||||
#
|
||||
# Ruling: same
|
||||
#
|
||||
# | Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
# |---|---|---|---|---|---|
|
||||
# | Mid A | better | worse | same | same | better |
|
||||
#
|
||||
# Mid A, decided by:
|
||||
#
|
||||
# - Locality: `bin/x.py:12` `run`: three branches to hold at once
|
||||
def entries(history: str) -> list[Entry]:
|
||||
"""Each paired ruling and scoring run in a comprehension history, in the order the file holds them; a section whose heading names no commit is left out."""
|
||||
found = []
|
||||
@@ -114,7 +134,7 @@ def bullet_places(bullet: str) -> list[Place]:
|
||||
|
||||
def heading(at: float, read: str, paired: bool) -> Heading | Failure:
|
||||
"""An entry's heading, matched by HEADING: when the seats finished, in UTC, and what they read."""
|
||||
title = f"{time.strftime(STAMP, time.gmtime(at))}, {read}"
|
||||
title = f"{utc(at)}, {read}"
|
||||
if (named := HEADING.match(title)) and bool(named.group(4)) == paired:
|
||||
return Heading(title, named.group(2), named.group(3), named.group(4))
|
||||
return Failure(f"the seats file's first line is not `<PR or branch> at <commit>{', against <base>' if paired else ''}`: {read}")
|
||||
@@ -126,7 +146,7 @@ def appended(history: str, entry: str) -> str:
|
||||
return f"{text}\n{entry}"
|
||||
|
||||
|
||||
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, notes: list[str], scoring: Scoring | None) -> str:
|
||||
def ruling_entry(title: str, seats: list[Seat], said: dict[str, list[str]], counted: dict[str, list[str]], ruled: str, notes: list[str], scoring: ScoringRun | None) -> str:
|
||||
"""A paired ruling's entry, with any notes, such as why a fall decided it, and the scores that decided a fall; a seat's vote that counted as same for want of a place says so."""
|
||||
votes = [[seat.name, *map(vote_text, DIMENSIONS, said[seat.name], counted[seat.name])] for seat in seats]
|
||||
tables = [[["Seat", *DIMENSIONS], *votes]] + ([scores_table(scoring)] if scoring else [])
|
||||
@@ -140,11 +160,11 @@ def vote_text(dimension: str, said: str, counted: str) -> str:
|
||||
return f"same (voted {said}, naming no place)" if dimension == DIMENSIONS[-1] else f"same (voted {said}, no place naming {dimension})"
|
||||
|
||||
|
||||
def scoring_entry(title: str, scoring: Scoring) -> str:
|
||||
def scoring_entry(title: str, scoring: ScoringRun) -> str:
|
||||
return entry_text(title, [], [scores_table(scoring)], scoring_labels(scoring.seats))
|
||||
|
||||
|
||||
def scores_table(scoring: Scoring) -> list[list[str]]:
|
||||
def scores_table(scoring: ScoringRun) -> list[list[str]]:
|
||||
return [["Seat", *DIMENSIONS], *([seat, *values] for seat, values in scoring.scores.items()), [MEAN, *map(str, scoring.means)]]
|
||||
|
||||
|
||||
|
||||
@@ -2,8 +2,7 @@ import re
|
||||
import subprocess
|
||||
|
||||
from failure import Failure
|
||||
from comprehension_history import HISTORY
|
||||
from larv_review.base import HistoryChange
|
||||
from comprehension_history import HISTORY, HistoryChange
|
||||
|
||||
|
||||
def git(*args: str) -> str | Failure:
|
||||
|
||||
+28
-165
@@ -2,6 +2,12 @@
|
||||
|
||||
usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]
|
||||
|
||||
The comprehension check keeps a project from reading worse, two ways. In a paired ruling, simulated
|
||||
readers, the seats, read
|
||||
the branch's head beside a base and vote better, same or worse, overall and on each dimension; a
|
||||
worse ruling blocks the merge, which is the ratchet. In a scoring run, four seats score the project
|
||||
from 1 to 10; a mean under the floor of {floor} blocks the merge, unless you decide to ship.
|
||||
|
||||
Run it inside a project checkout or one of its worktrees. Each command but record-ruling first
|
||||
fetches origin, when the checkout has that remote, and takes origin's default branch, or else the
|
||||
local main or master, as the default branch. When the fetch fails, round-count reads origin as
|
||||
@@ -14,9 +20,10 @@ history, or whose last added paired ruling, by heading time, is better; a merge
|
||||
one change against its first parent. With none, the base is the one the history's first paired
|
||||
ruling names; with no ruling either, it is the merge base of HEAD and the default branch.
|
||||
|
||||
round-count prints each pass's count: the highest <n> among the titles opening `Answer <pass> round
|
||||
<n>:` of commits the default branch does not hold. It marks each pass at the cap of {cap} rounds,
|
||||
and each name in such a title that is not a pass.
|
||||
round-count prints each pass's round count among the commits not yet on the default branch whose
|
||||
titles open `Answer <pass> round <n>:`, ignoring case. The count is the highest <n>, or the number of
|
||||
such commits when that is more. At or past the cap of {cap} rounds, it says to ask the human before
|
||||
another round. It also lists each name in such a title that is not a pass.
|
||||
|
||||
needs-scoring-run takes the IDs of the todo.md items the chunk ships, none when it ships none, and
|
||||
prints why the chunk is due a scoring run, or `No scoring run due.` It is due when the chunk ships
|
||||
@@ -71,25 +78,23 @@ time and the seats file's first line. Each commit that line names must be in the
|
||||
when the history already holds an entry on what that line reads.
|
||||
|
||||
Exit 1, saying why, on unknown arguments, an unknown item ID, a seats file it cannot read or that
|
||||
breaks the shape above, or when git cannot start, finds no checkout or no default branch, cannot
|
||||
fetch origin for ruling-base or needs-scoring-run, or cannot read or write the history or todo.md.
|
||||
breaks the shape above, a scores file record-ruling did not call for or one naming another commit,
|
||||
or when git cannot start, finds no checkout or no default branch, cannot
|
||||
fetch origin for ruling-base, needs-scoring-run or record-scoring, or cannot read or write the history or todo.md.
|
||||
"""
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import typing
|
||||
|
||||
from comprehension_history import appended, DIMENSIONS, heading, Heading, HISTORY, recorded, ruling_entry, Scoring, scoring_entry, vote_text
|
||||
from larv_review.base import base
|
||||
from larv_review.ruling_base import base
|
||||
from larv_review.rounds import CAP, PASSES, round_counts
|
||||
from larv_review.git import commit_exists, default_branch, file_at, git, history_changes, landed_since_release, top
|
||||
from larv_review.ruling import counted, fallen, ruling, votes
|
||||
from larv_review.git import default_branch, file_at, git, history_changes
|
||||
from failure import Failure
|
||||
from larv_review.scoring import fall_ruling, fall_step, FLOOR, means, plan_items, scoring_run_due, under_floor
|
||||
from rank_todo import ITEMS_HEADER
|
||||
from file_writes import replace_file
|
||||
from larv_review.seats import scores, seats_file, SeatsFile
|
||||
from larv_review.scoring import FLOOR
|
||||
from comprehension_history import HISTORY
|
||||
from larv_review.plan import plan_reasons
|
||||
from larv_review.record import record_ruling, record_scoring
|
||||
|
||||
USAGE = "usage: larv-review ruling-base | round-count | needs-scoring-run [<item ID>...] | record-ruling <votes file> [<scores file>] | record-scoring <scores file> [<item ID>...]"
|
||||
|
||||
@@ -100,7 +105,7 @@ def main(argv: list[str]) -> int:
|
||||
return 0
|
||||
command, *given = argv or [""]
|
||||
if command == "ruling-base" and not given:
|
||||
done = with_default_branch(ruling_base, stale_ok=False)
|
||||
done = with_default_branch(show_ruling_base, stale_ok=False)
|
||||
elif command == "round-count" and not given:
|
||||
done = with_default_branch(round_count, stale_ok=True)
|
||||
elif command == "needs-scoring-run" and all(re.fullmatch(r"[0-9]+", id_) for id_ in given):
|
||||
@@ -135,7 +140,7 @@ def with_default_branch(command: typing.Callable[[str], Failure | None], stale_o
|
||||
return command(branch)
|
||||
|
||||
|
||||
def ruling_base(default: str) -> Failure | None:
|
||||
def show_ruling_base(default: str) -> Failure | None:
|
||||
changes = history_changes(default)
|
||||
if isinstance(changes, Failure):
|
||||
return changes
|
||||
@@ -157,161 +162,19 @@ def round_count(default: str) -> Failure | None:
|
||||
if isinstance(titles, Failure):
|
||||
return titles
|
||||
for name, count in round_counts(titles.splitlines()).items():
|
||||
print(f"{name}: {count}{', at the cap' if count >= CAP else ''}{'' if name in PASSES else ', not a pass: check the commit title'}")
|
||||
print(f"{name}: {count}{cap_note(count) if name in PASSES else ', not a pass: check the commit title'}")
|
||||
return None
|
||||
|
||||
|
||||
def cap_note(count: int) -> str:
|
||||
if count < CAP:
|
||||
return ""
|
||||
return f", {'at' if count == CAP else 'past'} the cap: ask the human before another round"
|
||||
|
||||
|
||||
def needs_scoring_run(default: str, ids: list[int]) -> Failure | None:
|
||||
reasons = plan_reasons(default, ids)
|
||||
if isinstance(reasons, Failure):
|
||||
return reasons
|
||||
print(f"Scoring run due: the chunk {' and '.join(reasons)}." if reasons else "No scoring run due.")
|
||||
return None
|
||||
|
||||
|
||||
def plan_reasons(default: str, ids: list[int]) -> list[str] | Failure:
|
||||
"""Why the plan calls a scoring run, from the default branch's todo.md and the checkout's."""
|
||||
checkout = top()
|
||||
if isinstance(checkout, Failure):
|
||||
return checkout
|
||||
before = file_at(default, "todo.md")
|
||||
if isinstance(before, Failure):
|
||||
return before
|
||||
try:
|
||||
after = pathlib.Path(checkout, "todo.md").read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read todo.md: {failure}")
|
||||
# While a chunk reshapes todo.md into one table, the default branch's todo.md has no items table.
|
||||
planned = plan_items(before) if ITEMS_HEADER in before.splitlines() else []
|
||||
if isinstance(planned, Failure):
|
||||
return Failure(f"{default}'s todo.md: {planned.reason}")
|
||||
chunk = plan_items(after)
|
||||
if isinstance(chunk, Failure):
|
||||
return Failure(f"the checkout's todo.md: {chunk.reason}")
|
||||
landed = landed_since_release(default)
|
||||
if isinstance(landed, Failure):
|
||||
return landed
|
||||
return scoring_run_due(planned, chunk, ids, landed)
|
||||
|
||||
|
||||
def record_ruling(votes_file: pathlib.Path, scores_file: pathlib.Path | None = None) -> Failure | None:
|
||||
read = read_seats(votes_file, paired=True)
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
named, given = read
|
||||
said = votes(given.seats)
|
||||
if isinstance(said, Failure):
|
||||
return said
|
||||
count = counted(given.seats, said)
|
||||
tally = ", ".join(f"{seat.name} {vote_text(DIMENSIONS[-1], said[seat.name][-1], count[seat.name][-1])}" for seat in given.seats)
|
||||
ruled = ruling([each[-1] for each in count.values()])
|
||||
if ruled is None:
|
||||
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the votes file, and run this again.")
|
||||
return None
|
||||
fell = fallen(count) if ruled != "worse" else []
|
||||
scoring = None
|
||||
notes = []
|
||||
match fall_step(fell, scores_file is not None):
|
||||
case "ask":
|
||||
print(f"Fell: {', '.join(fell)}, in a {ruled} ruling. Launch the four scoring seats on {named.label} at {named.commit}, write their lines to a scores file, and run this again with both files.")
|
||||
return None
|
||||
case "refuse":
|
||||
return Failure(f"no dimension fell in this {ruled} ruling, so it takes no scores file")
|
||||
if scores_file:
|
||||
scored = read_scoring(scores_file, named)
|
||||
if isinstance(scored, Failure):
|
||||
return scored
|
||||
scoring = scored
|
||||
ruled, why = fall_ruling(ruled, fell, scoring.means)
|
||||
notes = [why]
|
||||
print(f"Ruling: {ruled}. {' '.join(notes + [f'Overall votes as counted: {tally}.'])}")
|
||||
return append(given.read, ruling_entry(named.title, given.seats, said, count, ruled, notes, scoring))
|
||||
|
||||
|
||||
def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> Failure | None:
|
||||
read = read_seats(scores_file, paired=False)
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
named, given = read
|
||||
scoring = scored_seats(given)
|
||||
if isinstance(scoring, Failure):
|
||||
return scoring
|
||||
reasons = plan_reasons(default, ids)
|
||||
if isinstance(reasons, Failure):
|
||||
return reasons
|
||||
if not reasons:
|
||||
return Failure("no plan item or release calls this scoring run; name the item IDs the chunk ships, as needs-scoring-run took them, or record a fall's scores with record-ruling")
|
||||
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, scoring.means))}.")
|
||||
if under := under_floor(scoring.means):
|
||||
print(f"Floor: {', '.join(under)} under the floor of {FLOOR} block the merge, as the chunk {' and '.join(reasons)}.")
|
||||
return append(given.read, scoring_entry(named.title, scoring))
|
||||
|
||||
|
||||
def read_scoring(scores_file: pathlib.Path, votes: Heading) -> Scoring | Failure:
|
||||
"""The scores a fall called, once their file names what the votes file read."""
|
||||
read = read_seats(scores_file, paired=False, fresh=False)
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
named, given = read
|
||||
if (named.label, named.commit) != (votes.label, votes.commit):
|
||||
return Failure(f"the scores file's first line is not `{votes.label} at {votes.commit}`, as the votes file reads: {given.read}")
|
||||
return scored_seats(given)
|
||||
|
||||
|
||||
def scored_seats(given: SeatsFile) -> Scoring | Failure:
|
||||
found = scores(given.seats)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
return Scoring(given.seats, found, means(list(found.values())))
|
||||
|
||||
|
||||
def read_seats(path: pathlib.Path, paired: bool, fresh: bool = True) -> tuple[Heading, SeatsFile] | Failure:
|
||||
"""The seats file, and the heading its record goes under, once each commit it names is in the checkout and, when fresh, before anything is printed, once the history holds no entry on what it reads."""
|
||||
try:
|
||||
given = seats_file(path.read_text(encoding="utf-8"))
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read {path}: {failure}")
|
||||
if isinstance(given, Failure):
|
||||
return given
|
||||
named = heading(time.time(), given.read, paired)
|
||||
if isinstance(named, Failure):
|
||||
return named
|
||||
if unknown := [commit for commit in (named.commit, named.against) if commit and not commit_exists(commit)]:
|
||||
return Failure(f"the seats file names {', '.join(unknown)}, which is no commit in this checkout")
|
||||
if fresh:
|
||||
history = history_file()
|
||||
if isinstance(history, Failure):
|
||||
return history
|
||||
if recorded(history[1], given.read):
|
||||
return Failure(f"{history[0]} already holds an entry on {given.read}; record each ruling or run once")
|
||||
return named, given
|
||||
|
||||
|
||||
def history_file() -> tuple[pathlib.Path, str] | Failure:
|
||||
"""The checkout's history and its text, empty when the file is missing."""
|
||||
checkout = top()
|
||||
if isinstance(checkout, Failure):
|
||||
return checkout
|
||||
history = pathlib.Path(checkout, HISTORY)
|
||||
try:
|
||||
return history, history.read_text(encoding="utf-8") if history.exists() else ""
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read {history}: {failure}")
|
||||
|
||||
|
||||
def append(read: str, entry: str) -> Failure | None:
|
||||
"""Append entry unless the history already holds one on what the seats read."""
|
||||
read_history = history_file()
|
||||
if isinstance(read_history, Failure):
|
||||
return read_history
|
||||
history, text = read_history
|
||||
if recorded(text, read):
|
||||
return Failure(f"{history} already holds an entry on {read}; record each ruling or run once")
|
||||
try:
|
||||
history.parent.mkdir(parents=True, exist_ok=True)
|
||||
except OSError as failure:
|
||||
return Failure(f"could not create {history.parent}: {failure}")
|
||||
if unwritten := replace_file(history, appended(text, entry).encode()):
|
||||
return Failure(f"could not write {history}: {unwritten.reason}")
|
||||
print(f"Appended to {history}.")
|
||||
return None
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import pathlib
|
||||
|
||||
from failure import Failure
|
||||
from larv_review.git import file_at, landed_since_release, top
|
||||
from todo_table import Item, items, ITEMS_HEADER
|
||||
|
||||
|
||||
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
|
||||
"""Why the plan calls a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag.
|
||||
|
||||
A chunk that ships an item naming the floor is due one, so it can prove the floor met. So is a chunk that empties a release after work has landed towards it, so no release ends unmeasured. Work has landed when the chunk ships one of the release's items, or when the release is the earliest one `before` names and `landed` holds. That covers a release whose last item moved to another release.
|
||||
"""
|
||||
plan = {item.id: item for item in before + after}
|
||||
if unknown := sorted(set(ships) - set(plan)):
|
||||
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
|
||||
shipped = [plan[id_] for id_ in ships]
|
||||
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
|
||||
next_release = min((item.release for item in before), default=None)
|
||||
for release in emptied(before, after, shipped):
|
||||
if any(item.release == release for item in shipped) or (release == next_release and landed):
|
||||
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
|
||||
return reasons
|
||||
|
||||
|
||||
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
|
||||
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
|
||||
named = {item.release for item in before + shipped}
|
||||
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
|
||||
return sorted(named - left)
|
||||
|
||||
|
||||
def plan_reasons(default: str, ids: list[int]) -> list[str] | Failure:
|
||||
"""Why the plan calls a scoring run, from the default branch's todo.md and the checkout's."""
|
||||
checkout = top()
|
||||
if isinstance(checkout, Failure):
|
||||
return checkout
|
||||
before = file_at(default, "todo.md")
|
||||
if isinstance(before, Failure):
|
||||
return before
|
||||
try:
|
||||
after = pathlib.Path(checkout, "todo.md").read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read todo.md: {failure}")
|
||||
# While a chunk reshapes todo.md into one table, the default branch's todo.md has no items table.
|
||||
planned = items(before) if ITEMS_HEADER in before.splitlines() else []
|
||||
if isinstance(planned, Failure):
|
||||
return Failure(f"{default}'s todo.md: {planned.reason}")
|
||||
chunk = items(after)
|
||||
if isinstance(chunk, Failure):
|
||||
return Failure(f"the checkout's todo.md: {chunk.reason}")
|
||||
landed = landed_since_release(default)
|
||||
if isinstance(landed, Failure):
|
||||
return landed
|
||||
return scoring_run_due(planned, chunk, ids, landed)
|
||||
@@ -0,0 +1,135 @@
|
||||
import pathlib
|
||||
import time
|
||||
|
||||
from comprehension_history import appended, DIMENSIONS, heading, Heading, HISTORY, recorded, ruling_entry, scoring_entry, ScoringRun, vote_text
|
||||
from larv_review.git import commit_exists, top
|
||||
from larv_review.ruling import counted, fallen, overall, ruling, split, votes
|
||||
from failure import Failure
|
||||
from larv_review.scoring import fall_ruling, FLOOR, means, under_floor
|
||||
from larv_review.plan import plan_reasons
|
||||
from file_writes import replace_file
|
||||
from larv_review.seats import scores, seats_file, SeatsFile
|
||||
|
||||
|
||||
def record_ruling(votes_file: pathlib.Path, scores_file: pathlib.Path | None = None) -> Failure | None:
|
||||
read = read_seats(votes_file, paired=True)
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
heading_read, seats_read = read
|
||||
written = votes(seats_read.seats)
|
||||
if isinstance(written, Failure):
|
||||
return written
|
||||
counts = counted(seats_read.seats, written)
|
||||
tally = ", ".join(f"{seat.name} {vote_text(DIMENSIONS[-1], overall(written[seat.name]), overall(counts[seat.name]))}" for seat in seats_read.seats)
|
||||
overall_votes = [overall(each) for each in counts.values()]
|
||||
if split(overall_votes):
|
||||
print(f"Split: {tally}. Launch the maintainability senior and Junior A together, add their lines to the votes file, and run this again.")
|
||||
return None
|
||||
ruled = ruling(overall_votes)
|
||||
fell = fallen(counts) if ruled != "worse" else []
|
||||
scoring = None
|
||||
notes = []
|
||||
if not fell and scores_file:
|
||||
why = "a worse ruling takes no scores file" if ruled == "worse" else f"no dimension fell in this {ruled} ruling, so it takes no scores file"
|
||||
return Failure(f"{why}; run record-ruling again with the votes file alone")
|
||||
if fell and not scores_file:
|
||||
print(f"Fell: {', '.join(fell)}, in a {ruled} ruling. Launch the four scoring seats on {heading_read.label} at {heading_read.commit}, write their lines to a scores file, and run this again with both files.")
|
||||
return None
|
||||
if fell and scores_file:
|
||||
scored = read_fall_scores(scores_file, heading_read)
|
||||
if isinstance(scored, Failure):
|
||||
return scored
|
||||
scoring = scored
|
||||
ruled, why = fall_ruling(ruled, fell, scoring.means)
|
||||
notes = [why]
|
||||
print(f"Ruling: {ruled}. {' '.join(notes + [f'Overall votes as counted: {tally}.'])}")
|
||||
return append(seats_read.read, ruling_entry(heading_read.title, seats_read.seats, written, counts, ruled, notes, scoring))
|
||||
|
||||
|
||||
def record_scoring(default: str, scores_file: pathlib.Path, ids: list[int]) -> Failure | None:
|
||||
read = read_seats(scores_file, paired=False)
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
named, given = read
|
||||
scoring = scored_seats(given)
|
||||
if isinstance(scoring, Failure):
|
||||
return scoring
|
||||
reasons = plan_reasons(default, ids)
|
||||
if isinstance(reasons, Failure):
|
||||
return reasons
|
||||
if not reasons:
|
||||
return Failure("no plan item or release calls this scoring run; name the item IDs the chunk ships, as needs-scoring-run took them, or record a fall's scores with record-ruling")
|
||||
print(f"Means: {', '.join(f'{name} {value}' for name, value in zip(DIMENSIONS, scoring.means))}.")
|
||||
if under := under_floor(scoring.means):
|
||||
print(f"Floor: {', '.join(under)} under the floor of {FLOOR} block the merge, as the chunk {' and '.join(reasons)}.")
|
||||
return append(given.read, scoring_entry(named.title, scoring))
|
||||
|
||||
|
||||
def read_fall_scores(scores_file: pathlib.Path, votes: Heading) -> ScoringRun | Failure:
|
||||
"""The scores a fall called, once their file names what the votes file read."""
|
||||
read = read_seats(scores_file, paired=False, refuse_recorded=False)
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
named, given = read
|
||||
if (named.label, named.commit) != (votes.label, votes.commit):
|
||||
return Failure(f"the scores file's first line is not `{votes.label} at {votes.commit}`, as the votes file reads: {given.read}")
|
||||
return scored_seats(given)
|
||||
|
||||
|
||||
def scored_seats(given: SeatsFile) -> ScoringRun | Failure:
|
||||
found = scores(given.seats)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
return ScoringRun(given.seats, found, means(list(found.values())))
|
||||
|
||||
|
||||
def read_seats(path: pathlib.Path, paired: bool, refuse_recorded: bool = True) -> tuple[Heading, SeatsFile] | Failure:
|
||||
"""The seats file and the heading its record goes under. Fails when the file names a commit the checkout lacks, or, when refuse_recorded, when the history already holds an entry on what it reads; it runs before anything is printed."""
|
||||
try:
|
||||
given = seats_file(path.read_text(encoding="utf-8"))
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read {path}: {failure}")
|
||||
if isinstance(given, Failure):
|
||||
return given
|
||||
named = heading(time.time(), given.read, paired)
|
||||
if isinstance(named, Failure):
|
||||
return named
|
||||
if unknown := [commit for commit in (named.commit, named.against) if commit and not commit_exists(commit)]:
|
||||
return Failure(f"the seats file names {', '.join(unknown)}, which is no commit in this checkout")
|
||||
if refuse_recorded:
|
||||
history = history_file()
|
||||
if isinstance(history, Failure):
|
||||
return history
|
||||
if recorded(history[1], given.read):
|
||||
return Failure(f"{history[0]} already holds an entry on {given.read}; record each ruling or run once")
|
||||
return named, given
|
||||
|
||||
|
||||
def history_file() -> tuple[pathlib.Path, str] | Failure:
|
||||
"""The checkout's history and its text, empty when the file is missing."""
|
||||
checkout = top()
|
||||
if isinstance(checkout, Failure):
|
||||
return checkout
|
||||
history = pathlib.Path(checkout, HISTORY)
|
||||
try:
|
||||
return history, history.read_text(encoding="utf-8") if history.exists() else ""
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read {history}: {failure}")
|
||||
|
||||
|
||||
def append(read: str, entry: str) -> Failure | None:
|
||||
"""Append entry unless the history already holds one on what the seats read."""
|
||||
read_history = history_file()
|
||||
if isinstance(read_history, Failure):
|
||||
return read_history
|
||||
history, text = read_history
|
||||
if recorded(text, read):
|
||||
return Failure(f"{history} already holds an entry on {read}; record each ruling or run once")
|
||||
try:
|
||||
history.parent.mkdir(parents=True, exist_ok=True)
|
||||
except OSError as failure:
|
||||
return Failure(f"could not create {history.parent}: {failure}")
|
||||
if unwritten := replace_file(history, appended(text, entry).encode()):
|
||||
return Failure(f"could not write {history}: {unwritten.reason}")
|
||||
print(f"Appended to {history}.")
|
||||
return None
|
||||
@@ -2,13 +2,16 @@ import re
|
||||
|
||||
CAP = 3
|
||||
PASSES = ("architecture", "full scan", "security", "stability", "product-owner", "prose")
|
||||
ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):")
|
||||
ROUND = re.compile(r"^Answer (.+?) round ([0-9]+):", re.IGNORECASE)
|
||||
|
||||
|
||||
def round_counts(titles: list[str]) -> dict[str, int]:
|
||||
"""Each pass's highest round among the branch's `Answer <pass> round <n>:` commit titles; a pass with none has 0, and a title naming no known pass adds its name."""
|
||||
counts = dict.fromkeys(PASSES, 0)
|
||||
"""Each pass's round count among the branch's `Answer <pass> round <n>:` commit titles: the highest <n>, or the number of such commits where that is more, so a repeated number still counts. A pass with none has 0, and a title naming no known pass adds its name. Case is ignored."""
|
||||
highest: dict[str, int] = dict.fromkeys(PASSES, 0)
|
||||
commits: dict[str, int] = dict.fromkeys(PASSES, 0)
|
||||
for title in titles:
|
||||
if named := ROUND.match(title):
|
||||
counts[named.group(1)] = max(counts.get(named.group(1), 0), int(named.group(2)))
|
||||
return counts
|
||||
name = named.group(1).lower()
|
||||
highest[name] = max(highest.get(name, 0), int(named.group(2)))
|
||||
commits[name] = commits.get(name, 0) + 1
|
||||
return {name: max(highest[name], commits[name]) for name in highest}
|
||||
|
||||
+19
-10
@@ -18,12 +18,17 @@ def votes(seats: list[Seat]) -> dict[str, list[str]] | Failure:
|
||||
if unnamed := [place for place in seat.places if not place_dimensions(place)]:
|
||||
return Failure(f"{seat.name}'s place does not open with the dimensions it moves, each one of {', '.join(DIMENSIONS)}, then `: `: {unnamed[0]}")
|
||||
said[seat.name] = line
|
||||
first, second, *more = (each[-1] for each in counted(seats, said).values())
|
||||
first, second, *more = (overall(each) for each in counted(seats, said).values())
|
||||
if more and first == second:
|
||||
return Failure(f"{' and '.join(FIRST_SEATS)} both count as {first} overall, which settles the ruling; drop the other seats' lines")
|
||||
return said
|
||||
|
||||
|
||||
def overall(votes: list[str]) -> str:
|
||||
"""The overall vote, which DIMENSIONS lists last."""
|
||||
return votes[-1]
|
||||
|
||||
|
||||
def counted(seats: list[Seat], said: dict[str, list[str]]) -> dict[str, list[str]]:
|
||||
"""Each seat's votes as they count: a dimension's vote counts as same unless one of the seat's `path:line` places opens with that dimension, and the overall vote unless the seat names any such place."""
|
||||
return {seat.name: [vote if names_a_place(seat, name) else "same" for name, vote in zip(DIMENSIONS, said[seat.name])] for seat in seats}
|
||||
@@ -39,16 +44,20 @@ def place_dimensions(place: str) -> list[str]:
|
||||
return named if all(name in DIMENSIONS for name in named) else []
|
||||
|
||||
|
||||
def ruling(counted: list[str]) -> str | None:
|
||||
"""The vote that leads each other vote by two or more, else same; None when only the first two seats voted and they differ, so two more must vote."""
|
||||
if len(counted) == 2 and counted[0] != counted[1]:
|
||||
return None
|
||||
for vote in VOTES:
|
||||
if all(counted.count(vote) >= counted.count(other) + 2 for other in VOTES if other != vote):
|
||||
return vote
|
||||
return "same"
|
||||
def split(overall: list[str]) -> bool:
|
||||
"""Whether only the first two seats voted overall and they differ, so two more must vote: two votes that differ cannot reach a lead of two."""
|
||||
return len(overall) == 2 and overall[0] != overall[1]
|
||||
|
||||
|
||||
def ruling(overall: list[str]) -> str:
|
||||
return lead(overall) or "same"
|
||||
|
||||
|
||||
def fallen(counted: dict[str, list[str]]) -> list[str]:
|
||||
"""Each dimension, Overall aside, that the seats rule worse by a lead of two."""
|
||||
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if ruling(list(column)) == "worse"]
|
||||
return [name for name, column in zip(DIMENSIONS[:-1], zip(*counted.values())) if lead(list(column)) == "worse"]
|
||||
|
||||
|
||||
def lead(votes: list[str]) -> str | None:
|
||||
"""The vote that leads each other vote by two or more, if one does."""
|
||||
return next((vote for vote in VOTES if all(votes.count(vote) >= votes.count(other) + 2 for other in VOTES if other != vote)), None)
|
||||
|
||||
@@ -1,18 +1,11 @@
|
||||
import dataclasses
|
||||
|
||||
from comprehension_history import entries, Entry
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class HistoryChange:
|
||||
"""A merged change on the default branch that touched the history, and the history before and after it."""
|
||||
commit: str
|
||||
before: str
|
||||
after: str
|
||||
from comprehension_history import entries, Entry, HistoryChange
|
||||
|
||||
|
||||
def measured_commit(changes: list[HistoryChange]) -> str | None:
|
||||
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better."""
|
||||
"""The newest change, of changes given newest first, that adds a scoring run to the history, or whose last added paired ruling is better.
|
||||
|
||||
It is the ratchet's base. A ruling of same leaves the base where it was, so a run of small falls each ruled same still adds up against one commit. Entry times are UTC stamps in ISO order, so they sort as text.
|
||||
"""
|
||||
for change in changes:
|
||||
held = {entry.heading for entry in entries(change.before)}
|
||||
added = [entry for entry in entries(change.after) if entry.heading not in held]
|
||||
@@ -1,24 +1,12 @@
|
||||
import decimal
|
||||
import typing
|
||||
|
||||
from comprehension_history import DIMENSIONS
|
||||
from failure import Failure
|
||||
from rank_todo import Item, ITEMS_HEADER, parse_row, table_rows
|
||||
|
||||
FLOOR = decimal.Decimal("7.0")
|
||||
|
||||
|
||||
def fall_step(fell: list[str], scored: bool) -> typing.Literal["ask", "refuse", "rule"]:
|
||||
"""What record-ruling does next: ask for the scoring seats when a dimension fell and no scores file came, refuse a scores file no fall calls, or rule."""
|
||||
if fell and not scored:
|
||||
return "ask"
|
||||
if scored and not fell:
|
||||
return "refuse"
|
||||
return "rule"
|
||||
|
||||
|
||||
def fall_ruling(ruled: str, fell: list[str], found: list[decimal.Decimal]) -> tuple[str, str]:
|
||||
"""The ruling once the scoring run has scored the dimensions that fell, and the line that says why: worse when one of their means is under the floor."""
|
||||
"""The ruling once the scoring run has scored the dimensions that fell, and the line that says why: worse when one of their means is under the floor, so a chunk cannot trade one dimension below the floor for gains in the others."""
|
||||
if under := [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if name in fell and value < FLOOR]:
|
||||
return "worse", f"Fell: {', '.join(fell)}. Under the floor of {FLOOR}: {', '.join(under)}, so the {ruled} ruling counts as worse."
|
||||
return ruled, f"Fell: {', '.join(fell)}, each at or above the floor of {FLOOR}, so the {ruled} ruling stands."
|
||||
@@ -35,38 +23,3 @@ def mean(values: list[decimal.Decimal]) -> decimal.Decimal:
|
||||
|
||||
def under_floor(found: list[decimal.Decimal]) -> list[str]:
|
||||
return [f"{name} {value}" for name, value in zip(DIMENSIONS, found) if value < FLOOR]
|
||||
|
||||
|
||||
def plan_items(todo: str) -> list[Item] | Failure:
|
||||
"""Each row of a todo.md's items table, read as `rank_todo.py` reads it."""
|
||||
table = table_rows(todo.splitlines(), ITEMS_HEADER)
|
||||
if isinstance(table, Failure):
|
||||
return table
|
||||
items = []
|
||||
for row in table.rows:
|
||||
item = parse_row(row)
|
||||
if isinstance(item, Failure):
|
||||
return item
|
||||
items.append(item)
|
||||
return items
|
||||
|
||||
|
||||
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: bool) -> list[str] | Failure:
|
||||
"""Why the plan calls a scoring run, empty when it does not. before is the default branch's plan, after the chunk's, ships the IDs it ships, and landed whether the default branch holds a commit after its newest release tag. Work has landed towards an emptied release when the chunk ships one of its items, or when it is the next release, the earliest before names, and landed holds."""
|
||||
plan = {item.id: item for item in before + after}
|
||||
if unknown := sorted(set(ships) - set(plan)):
|
||||
return Failure(f"neither plan holds item {', '.join(map(str, unknown))}")
|
||||
shipped = [plan[id_] for id_ in ships]
|
||||
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
|
||||
next_release = min((item.release for item in before), default=None)
|
||||
for release in emptied(before, after, shipped):
|
||||
if any(item.release == release for item in shipped) or (release == next_release and landed):
|
||||
reasons.append(f"leaves release {'.'.join(map(str, release))} with no plan item")
|
||||
return reasons
|
||||
|
||||
|
||||
def emptied(before: list[Item], after: list[Item], shipped: list[Item]) -> list[tuple[int, int, int]]:
|
||||
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer."""
|
||||
named = {item.release for item in before + shipped}
|
||||
left = {item.release for item in after if item.id not in {ship.id for ship in shipped}}
|
||||
return sorted(named - left)
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import decimal
|
||||
import itertools
|
||||
import os
|
||||
import pathlib
|
||||
import subprocess
|
||||
@@ -7,19 +8,21 @@ import tempfile
|
||||
import textwrap
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from comprehension_history import appended, entries, Place, recorded, ruling_entry, Scoring, Seat
|
||||
from larv_review.base import base, first_ruling, HistoryChange, measured_commit
|
||||
from larv_review.ruling import counted, fallen, ruling, votes
|
||||
from comprehension_history import appended, entries, HistoryChange, Place, recorded, ruling_entry, ScoringRun, Seat
|
||||
from larv_review.ruling_base import base, first_ruling, measured_commit
|
||||
import comprehension_history
|
||||
from larv_review.ruling import counted, fallen, ruling, split, votes
|
||||
from failure import Failure
|
||||
from larv_review.scoring import fall_ruling, fall_step, means, plan_items, scoring_run_due, under_floor
|
||||
from test_loop_claude_black_box import host_env
|
||||
from rank_todo import Item
|
||||
from larv_review.scoring import fall_ruling, means, under_floor
|
||||
from test_env import isolated_env
|
||||
from todo_table import Item, items
|
||||
from larv_review.rounds import round_counts
|
||||
from larv_review.seats import score_line, scores, seats_file
|
||||
from larv_review.plan import scoring_run_due
|
||||
|
||||
TOOL = pathlib.Path(__file__).resolve().parent / "larv-review"
|
||||
TOOL = pathlib.Path(__file__).resolve().parents[1] / "larv-review"
|
||||
SCORING = "## 2026-10-08T12:00:00Z, PR #1 at aaaaaaa\n\n| Seat | Overall |\n|---|---|\n| Mean | 7.00 |\n"
|
||||
|
||||
|
||||
@@ -93,7 +96,7 @@ class Seats(unittest.TestCase):
|
||||
]
|
||||
for counted, expected in cases:
|
||||
with self.subTest(counted):
|
||||
self.assertEqual(ruling(counted), expected)
|
||||
self.assertEqual(None if split(counted) else ruling(counted), expected)
|
||||
|
||||
def test_reads_four_seats_scores_and_refuses_an_overall_above_the_lowest_by_more_than_one(self):
|
||||
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
|
||||
@@ -112,10 +115,18 @@ class Seats(unittest.TestCase):
|
||||
|
||||
|
||||
class History(unittest.TestCase):
|
||||
def test_writes_the_sample_entry_its_parser_shows(self):
|
||||
source = pathlib.Path(comprehension_history.__file__).read_text(encoding="utf-8").splitlines()
|
||||
start = next(index for index, line in enumerate(source) if line.startswith("# ## "))
|
||||
sample = [line.removeprefix("#").removeprefix(" ") for line in itertools.takewhile(lambda line: line.startswith("#"), source[start:])]
|
||||
seat = Seat("Mid A", said("better", "worse"), ["Locality: `bin/x.py:12` `run`: three branches to hold at once"])
|
||||
votes = {"Mid A": ["better", "worse", "same", "same", "better"]}
|
||||
self.assertEqual(ruling_entry("2026-10-09T09:47:25Z, PR #232 at 324d650, against 12d6d75", [seat], votes, votes, "same", [], None), "\n".join(sample) + "\n")
|
||||
|
||||
def test_reads_back_what_it_writes(self):
|
||||
seats = [Seat("Mid A", said("better", "worse"), ["`bin/a.py:3` `f`: hard, with `:9` `g`"]), Seat("Inherited architect", said("worse", "worse"), [])]
|
||||
written = {"Mid A": ["same", "worse", "same", "same", "better"], "Inherited architect": ["same", "worse", "same", "same", "worse"]}
|
||||
scoring = Scoring([Seat("Junior A", "", ["`bin/b.py:4` `h`: long"])], {"Junior A": ["7", "6.5", "7", "7", "7"]}, [decimal.Decimal(mean) for mean in ("7.00", "6.50", "7.00", "7.00", "7.00")])
|
||||
scoring = ScoringRun([Seat("Junior A", "", ["`bin/b.py:4` `h`: long"])], {"Junior A": ["7", "6.5", "7", "7", "7"]}, [decimal.Decimal(mean) for mean in ("7.00", "6.50", "7.00", "7.00", "7.00")])
|
||||
ruled = ruling_entry("2026-10-08T18:37:21Z, PR #2 at bbbbbbb, against aaaaaaa", seats, written, {**written, "Inherited architect": ["same"] * 5}, "worse", ["Fell: Locality, under the floor."], scoring)
|
||||
history = appended(appended("", SCORING), ruled)
|
||||
self.assertTrue(history.startswith("# Comprehension history\n\n## 2026-10-08T12:00:00Z"))
|
||||
@@ -136,9 +147,6 @@ class FallRuling(unittest.TestCase):
|
||||
self.assertEqual(fall_ruling("better", ["Locality", "Shape"], found), ("worse", "Fell: Locality, Shape. Under the floor of 7.0: Locality 6.5, so the better ruling counts as worse."))
|
||||
self.assertEqual(fall_ruling("same", ["Shape"], found), ("same", "Fell: Shape, each at or above the floor of 7.0, so the same ruling stands."))
|
||||
|
||||
def test_asks_for_scores_on_a_fall_refuses_them_without_one_and_rules_otherwise(self):
|
||||
self.assertEqual([fall_step(["Locality"], False), fall_step([], True), fall_step(["Locality"], True), fall_step([], False)], ["ask", "refuse", "rule", "rule"])
|
||||
|
||||
|
||||
class Base(unittest.TestCase):
|
||||
def test_takes_the_newest_change_adding_a_scoring_run_or_ending_on_a_better_ruling(self):
|
||||
@@ -171,12 +179,18 @@ class Rounds(unittest.TestCase):
|
||||
counts = round_counts(["Answer prose round 1: x", "Answer full scan round 2: y", "Answer full scan round 1: z", "Answer stability: no round", "Answer stabilty round 1: typo", "Answer security round 1: rename round 2: x", "Rescore: w"])
|
||||
self.assertEqual((counts["prose"], counts["full scan"], counts["stability"], counts["architecture"], counts["stabilty"], counts["security"]), (1, 2, 0, 0, 1, 1))
|
||||
|
||||
def test_counts_a_mis_cased_title_under_its_pass(self):
|
||||
self.assertEqual(round_counts(["Answer Architecture round 4: x", "Answer architecture round 3: y"])["architecture"], 4)
|
||||
|
||||
def test_counts_a_repeated_round_number_as_a_round_of_its_own(self):
|
||||
self.assertEqual(round_counts(["Answer stability round 3: x", "Answer stability round 3: y", "Answer stability round 2: z", "Answer stability round 1: w"])["stability"], 4)
|
||||
|
||||
|
||||
class ScoringRunDue(unittest.TestCase):
|
||||
def test_reads_each_rows_id_release_and_tagline(self):
|
||||
self.assertEqual(plan_items(todo("| 3 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), [item(3, (0, 1, 0))])
|
||||
self.assertIsInstance(plan_items("# todo\n"), Failure)
|
||||
self.assertIsInstance(plan_items(todo("| 0 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), Failure)
|
||||
self.assertEqual(items(todo("| 3 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), [item(3, (0, 1, 0))])
|
||||
self.assertIsInstance(items("# todo\n"), Failure)
|
||||
self.assertIsInstance(items(todo("| 0 | 0.1.0 | | **Do it.** | 1 | 1 | 1 | 1 | 1 | 9.0 |")), Failure)
|
||||
|
||||
def test_is_due_for_an_item_naming_the_floor(self):
|
||||
plan = [item(1, (0, 1, 0), "**Lift the project to the Comprehension floor.**"), item(2, (0, 1, 0))]
|
||||
@@ -208,7 +222,7 @@ class LarvReviewTool(unittest.TestCase):
|
||||
temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temp.cleanup)
|
||||
root = pathlib.Path(os.path.realpath(temp.name))
|
||||
self.env = {**host_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(root)}
|
||||
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(root)}
|
||||
self.checkout = root / "proj"
|
||||
subprocess.run(["git", "init", "-q", "--bare", "-b", "main", str(root / "origin.git")], check=True, env=self.env)
|
||||
subprocess.run(["git", "clone", "-q", str(root / "origin.git"), str(self.checkout)], check=True, env=self.env, capture_output=True)
|
||||
@@ -254,7 +268,9 @@ class LarvReviewTool(unittest.TestCase):
|
||||
self.commit("Answer prose round 1: c")
|
||||
self.commit("Answer stability round 3: d")
|
||||
ran = self.run_tool("round-count")
|
||||
self.assertIn("stability: 3, at the cap\n", ran.stdout)
|
||||
self.assertIn("stability: 3, at the cap: ask the human before another round\n", ran.stdout)
|
||||
self.commit("Answer stability round 4: e")
|
||||
self.assertIn("stability: 4, past the cap: ask the human before another round\n", self.run_tool("round-count").stdout)
|
||||
self.assertIn("prose: 1\n", ran.stdout)
|
||||
self.assertIn("architecture: 0\n", ran.stdout)
|
||||
|
||||
@@ -324,7 +340,9 @@ class LarvReviewTool(unittest.TestCase):
|
||||
self.assertEqual(ran.returncode, 0, ran.stderr)
|
||||
self.assertNotIn("Fell:", ran.stdout + (self.checkout / "docs" / "comprehension-history.md").read_text())
|
||||
extra = self.run_tool("record-ruling", self.seats_file(f"PR #3 at {self.start}, against {self.start}\n\nMid A: {said('worse')}{place}Inherited architect: {said('worse')}{place}"), self.seats_file("PR #2", "scores.md"))
|
||||
self.assertIn("takes no scores file", extra.stderr)
|
||||
self.assertIn("a worse ruling takes no scores file; run record-ruling again with the votes file alone", extra.stderr)
|
||||
better = self.run_tool("record-ruling", self.seats_file(f"PR #5 at {self.start}, against {self.start}\n\nMid A: {said('better')}{place}Inherited architect: {said('better')}{place}"), self.seats_file("PR #5", "scores.md"))
|
||||
self.assertIn("no dimension fell in this better ruling, so it takes no scores file", better.stderr)
|
||||
|
||||
def test_records_a_scoring_run_with_its_means(self):
|
||||
line = "Navigation 7, Locality 6.5, Shape 7, Self-sufficiency 6.5, Overall 7"
|
||||
+1
-1
@@ -9,4 +9,4 @@ sys.path.insert(0, str(LAUNCHER.parent))
|
||||
from loop_claude import effects, paths
|
||||
from loop_claude.loop import main
|
||||
|
||||
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home(), os.environ), LAUNCHER, pathlib.Path.cwd(), effects.system_effects()))
|
||||
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home()), LAUNCHER, pathlib.Path.cwd(), effects.system_effects()))
|
||||
|
||||
@@ -3,10 +3,10 @@ import uuid
|
||||
|
||||
from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.claude_code.transcript import human_turns, read_transcript
|
||||
from claude_code.transcript import human_turns, read_transcript
|
||||
from loop_claude.stop import needs_human, needs_terminal, Stop
|
||||
from loop_claude.paths import Paths
|
||||
from loop_claude.claude_code.trust import trust
|
||||
from claude_code.trust import trust
|
||||
from loop_claude.project.checkout import worktrees
|
||||
|
||||
ANSWER_QUESTION = "answer the question above"
|
||||
|
||||
@@ -1,15 +1,111 @@
|
||||
import dataclasses
|
||||
import json
|
||||
import re
|
||||
import typing
|
||||
|
||||
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, ANSWERS_RECORDED
|
||||
from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.claude_code.entries import hit_limit, limit_said, parse_entry, structured_output
|
||||
from loop_claude.cycle.judge_reply import Interrupt, JUDGE_RULES, JUDGE_SCHEMA, QuotedAnswer, Verdict, verdict_from
|
||||
from claude_code.entries import hit_limit, limit_said, parse_entry, structured_output
|
||||
from loop_claude.project.records import HUMAN_RULES
|
||||
from loop_claude.cycle.retry import Limited, retried
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
# At 3 to 4 characters a token, 100,000 to 133,000 tokens, which leaves a 200,000-token context room for the reply; the largest session rendered here was 364,000 characters on 2026-10-03.
|
||||
JUDGE_PROMPT_CHARS = 400000
|
||||
JUDGE_RULES = (
|
||||
"Decide whether the human must now be interrupted. interrupt=true when any of these appear:\n"
|
||||
"- the agent asks the human anything, seeks approval, or says a decision is the human's, and neither "
|
||||
"README.md nor the records settle the matter\n"
|
||||
"- it asks again a question a [loop] line already answered\n"
|
||||
"- it carried on under an assumption that decides the deliverable, that the human could "
|
||||
"reasonably reject and that neither README.md nor the records settle; a caveat about "
|
||||
"transient state it then handled is not one\n"
|
||||
"- it is blocked on something only the human can supply: a credential, a version bump, a choice "
|
||||
"outside the repo, or a release ready to cut, which only the human may publish\n"
|
||||
"- it flags that the goals, the plan in todo.md or the task itself is unclear, missing or in need of change\n"
|
||||
"- it chose between alternatives that neither README.md nor the records discriminate "
|
||||
"between - neither rules the rejected one out - and the choice materially changes the deliverable\n"
|
||||
"interrupt=false otherwise. Bookkeeping the agent settled silently - what to document, wording, "
|
||||
"where a note goes - is not on its own a reason to interrupt. The human may have replied inside the transcript, on a [human] line: judge only what "
|
||||
"is still open, never a question they already answered there. A [loop] line is what loop-claude told "
|
||||
"the agent, such as an answer quoted from README.md or the records.\n"
|
||||
"Where the agent asked or flagged something itself, the question is that one, in the agent's own "
|
||||
"wording; never construct another from facts it stated.\n"
|
||||
"Where README.md or the records answer the agent's own question and no [loop] line has "
|
||||
"quoted that answer yet, set interrupt=false and quote the passage that answers it; the agent is "
|
||||
"then given that passage in place of the human's reply. An interrupt outranks a quoted answer, and either "
|
||||
"outranks finished.\n"
|
||||
"The chunk is the piece of work the agent is on, shipped as one pull request. "
|
||||
"finished=true when the chunk is complete: merged, or nothing remains that the agent could do on "
|
||||
f"its own. Even then, it is finished only once {ANSWERS_RECORDED}. An answer counts as recorded "
|
||||
f"where README.md or the records above hold it, {HUMAN_RULES} included. It also counts where "
|
||||
f"the transcript shows the agent writing it into {ANSWER_DESTINATIONS}; writing it into "
|
||||
f"{HUMAN_RULES} does not count. finished=false when work remains that a further run can "
|
||||
"continue: an open PR, a review not yet run, CI not yet reported, an answer not yet recorded."
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Reply:
|
||||
answer: str = dataclasses.field(metadata={"description": "With interrupt false, the passage of README.md or the records that answers the agent's question, quoted verbatim, without elisions. Empty otherwise."})
|
||||
evidence: str = dataclasses.field(metadata={"description": "A verbatim quote of the one transcript passage the verdict rests on, without elisions: what triggers the interrupt; for a quoted answer, the agent's question; else what shows the chunk finished or the work remaining."})
|
||||
finished: bool = dataclasses.field(metadata={"description": "Whether the chunk is finished."})
|
||||
interrupt: bool = dataclasses.field(metadata={"description": "Whether the human must now be interrupted."})
|
||||
question: str = dataclasses.field(metadata={"description": "With interrupt true, the single question to put to the human. With a quoted answer, the agent's question that the answer field answers. Empty otherwise."})
|
||||
|
||||
|
||||
REPLY_TYPES = typing.get_type_hints(Reply)
|
||||
JSON_TYPES = {bool: "boolean", str: "string"}
|
||||
JUDGE_SCHEMA = {
|
||||
"properties": {field.name: {**field.metadata, "type": JSON_TYPES[REPLY_TYPES[field.name]]} for field in dataclasses.fields(Reply)},
|
||||
"required": list(REPLY_TYPES),
|
||||
"type": "object",
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Interrupt:
|
||||
evidence: str
|
||||
question: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class QuotedAnswer:
|
||||
"""The agent asked a question that README.md or the records answer. The answer field quotes the passage."""
|
||||
evidence: str
|
||||
question: str
|
||||
answer: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Finished:
|
||||
evidence: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Unfinished:
|
||||
evidence: str
|
||||
|
||||
|
||||
Verdict = Interrupt | QuotedAnswer | Finished | Unfinished
|
||||
|
||||
|
||||
def verdict_from(output: object) -> Verdict | Failure:
|
||||
if not isinstance(output, dict):
|
||||
return Failure("the judge returned no reply")
|
||||
for name, kind in REPLY_TYPES.items():
|
||||
if not isinstance(output.get(name), kind):
|
||||
return Failure(f"the judge's reply has no {name} of type {JSON_TYPES[kind]}")
|
||||
if output["interrupt"]:
|
||||
if not output["question"].strip():
|
||||
return Failure("the judge interrupted without a question")
|
||||
return Interrupt(output["evidence"], output["question"])
|
||||
if output["answer"].strip():
|
||||
if not output["question"].strip():
|
||||
return Failure("the judge quoted an answer without a question")
|
||||
return QuotedAnswer(output["evidence"], output["question"], output["answer"])
|
||||
return Finished(output["evidence"]) if output["finished"] else Unfinished(output["evidence"])
|
||||
|
||||
|
||||
def judge(root: str, task: str, premises: str, records: str, rendered_transcript: str, model: str, effects: Effects) -> Verdict | Stop:
|
||||
|
||||
@@ -1,104 +0,0 @@
|
||||
import dataclasses
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
from loop_claude.project.records import HUMAN_RULES, RECORD_FILES
|
||||
|
||||
# Where the judge may quote an answer from, named as the human and the worker see them; of README.md the judge sees only the premises.
|
||||
QUOTE_SOURCES = "README.md's Goals and Audience, " + ", ".join(RECORD_FILES) + " or " + HUMAN_RULES
|
||||
# Where the worker may write an answer the human typed, so it counts as recorded.
|
||||
ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES) + ", todo.md or a comment"
|
||||
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
|
||||
JUDGE_RULES = (
|
||||
"Decide whether the human must now be interrupted. interrupt=true when any of these appear:\n"
|
||||
"- the agent asks the human anything, seeks approval, or says a decision is the human's, and neither "
|
||||
"README.md nor the records settle the matter\n"
|
||||
"- it asks again a question a [loop] line already answered\n"
|
||||
"- it carried on under an assumption that decides the deliverable, that the human could "
|
||||
"reasonably reject and that neither README.md nor the records settle; a caveat about "
|
||||
"transient state it then handled is not one\n"
|
||||
"- it is blocked on something only the human can supply: a credential, a version bump, a choice "
|
||||
"outside the repo, or a release ready to cut, which only the human may publish\n"
|
||||
"- it flags that the goals, the plan in todo.md or the task itself is unclear, missing or in need of change\n"
|
||||
"- it chose between alternatives that neither README.md nor the records discriminate "
|
||||
"between - neither rules the rejected one out - and the choice materially changes the deliverable\n"
|
||||
"interrupt=false otherwise. Bookkeeping the agent settled silently - what to document, wording, "
|
||||
"where a note goes - is not on its own a reason to interrupt. The human may have replied inside the transcript, on a [human] line: judge only what "
|
||||
"is still open, never a question they already answered there. A [loop] line is what loop-claude told "
|
||||
"the agent, such as an answer quoted from README.md or the records.\n"
|
||||
"Where the agent asked or flagged something itself, the question is that one, in the agent's own "
|
||||
"wording; never construct another from facts it stated.\n"
|
||||
"Where README.md or the records answer the agent's own question and no [loop] line has "
|
||||
"quoted that answer yet, set interrupt=false and quote the passage that answers it; the agent is "
|
||||
"then given that passage in place of the human's reply. An interrupt outranks a quoted answer, and either "
|
||||
"outranks finished.\n"
|
||||
"The chunk is the piece of work the agent is on, shipped as one pull request. "
|
||||
"finished=true when the chunk is complete: merged, or nothing remains that the agent could do on "
|
||||
f"its own. Even then, it is finished only once {ANSWERS_RECORDED}. An answer counts as recorded "
|
||||
f"where README.md or the records above hold it, {HUMAN_RULES} included. It also counts where "
|
||||
f"the transcript shows the agent writing it into {ANSWER_DESTINATIONS}; writing it into "
|
||||
f"{HUMAN_RULES} does not count. finished=false when work remains that a further run can "
|
||||
"continue: an open PR, a review not yet run, CI not yet reported, an answer not yet recorded."
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Reply:
|
||||
answer: str = dataclasses.field(metadata={"description": "With interrupt false, the passage of README.md or the records that answers the agent's question, quoted verbatim, without elisions. Empty otherwise."})
|
||||
evidence: str = dataclasses.field(metadata={"description": "A verbatim quote of the one transcript passage the verdict rests on, without elisions: what triggers the interrupt; for a quoted answer, the agent's question; else what shows the chunk finished or the work remaining."})
|
||||
finished: bool = dataclasses.field(metadata={"description": "Whether the chunk is finished."})
|
||||
interrupt: bool = dataclasses.field(metadata={"description": "Whether the human must now be interrupted."})
|
||||
question: str = dataclasses.field(metadata={"description": "With interrupt true, the single question to put to the human. With a quoted answer, the agent's question that the answer field answers. Empty otherwise."})
|
||||
|
||||
|
||||
REPLY_TYPES = typing.get_type_hints(Reply)
|
||||
JSON_TYPES = {bool: "boolean", str: "string"}
|
||||
JUDGE_SCHEMA = {
|
||||
"properties": {field.name: {**field.metadata, "type": JSON_TYPES[REPLY_TYPES[field.name]]} for field in dataclasses.fields(Reply)},
|
||||
"required": list(REPLY_TYPES),
|
||||
"type": "object",
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Interrupt:
|
||||
evidence: str
|
||||
question: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class QuotedAnswer:
|
||||
"""The agent asked a question that README.md or the records answer. The answer field quotes the passage."""
|
||||
evidence: str
|
||||
question: str
|
||||
answer: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Finished:
|
||||
evidence: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Unfinished:
|
||||
evidence: str
|
||||
|
||||
|
||||
Verdict = Interrupt | QuotedAnswer | Finished | Unfinished
|
||||
|
||||
|
||||
def verdict_from(output: object) -> Verdict | Failure:
|
||||
if not isinstance(output, dict):
|
||||
return Failure("the judge returned no reply")
|
||||
for name, kind in REPLY_TYPES.items():
|
||||
if not isinstance(output.get(name), kind):
|
||||
return Failure(f"the judge's reply has no {name} of type {JSON_TYPES[kind]}")
|
||||
if output["interrupt"]:
|
||||
if not output["question"].strip():
|
||||
return Failure("the judge interrupted without a question")
|
||||
return Interrupt(output["evidence"], output["question"])
|
||||
if output["answer"].strip():
|
||||
if not output["question"].strip():
|
||||
return Failure("the judge quoted an answer without a question")
|
||||
return QuotedAnswer(output["evidence"], output["question"], output["answer"])
|
||||
return Finished(output["evidence"]) if output["finished"] else Unfinished(output["evidence"])
|
||||
@@ -0,0 +1,26 @@
|
||||
from loop_claude.project.records import HUMAN_RULES, RECORD_FILES
|
||||
from markdown_text import quoted
|
||||
|
||||
# Where the judge may quote an answer from, named as the human and the worker see them; of README.md the judge sees only the premises.
|
||||
QUOTE_SOURCES = "README.md's Goals and Audience, " + ", ".join(RECORD_FILES) + " or " + HUMAN_RULES
|
||||
# Where the worker may write an answer the human typed, so it counts as recorded.
|
||||
ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES) + ", todo.md or a comment"
|
||||
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
|
||||
|
||||
|
||||
def fresh(task: str) -> str:
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one."
|
||||
|
||||
|
||||
def resumed(unfinished_evidence: str) -> str:
|
||||
ruled = f" A judge ruled the chunk unfinished, quoting this passage of your transcript as the work remaining:\n\n{quoted(unfinished_evidence)}\n\n" if unfinished_evidence else " "
|
||||
return f"Continue the task; the previous run ended before the chunk was finished.{ruled}The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}."
|
||||
|
||||
|
||||
def answered(question: str, answer: str) -> str:
|
||||
return f"Your question: {question}\n\n{QUOTE_SOURCES} answers it:\n\n{answer}\n\nContinue the task."
|
||||
|
||||
|
||||
def past_the_cap(question: str, answer: str, cap: int) -> str:
|
||||
"""The question a quoted answer puts to the human once the cycle's quoted answers reach the cap."""
|
||||
return f"{question}\n\n{QUOTE_SOURCES} says:\n{answer}\n\nThose files have answered {cap} questions this cycle, the most allowed, so this one comes to you."
|
||||
@@ -1,17 +1,19 @@
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from loop_claude.cycle.states import Active, Archived, Closing, Ended, Event, HumanAnswered, Interrupting, Judging, QUOTED_ANSWERS_PER_CYCLE, Ruled, State, transition, WorkerRan, Working
|
||||
from loop_claude.transcript_archive import archive, FINISHED, UNFINISHED
|
||||
from loop_claude.claude_code.session import delete_session, Session, session_open, transcript_file
|
||||
from transcript_archive.write import archive
|
||||
from claude_code.session import delete_session, Session, session_open, transcript_file
|
||||
from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.cycle.judge_reply import Finished, Interrupt, QUOTE_SOURCES, QuotedAnswer, Unfinished
|
||||
from loop_claude.cycle.judge import Finished, Interrupt, judge, QuotedAnswer, Unfinished
|
||||
from loop_claude.cycle.interrupt import interrupt
|
||||
from loop_claude.cycle.judge import judge
|
||||
from loop_claude.project.model_choice import Models
|
||||
from transcript_archive import outcomes
|
||||
from loop_claude.paths import Paths
|
||||
from loop_claude.claude_code.transcript import read_transcript, render
|
||||
from loop_claude.cycle import prompts
|
||||
from claude_code.transcript import read_transcript, render
|
||||
from loop_claude.cycle.retry import retried
|
||||
from loop_claude.cycle.worker import run_worker, worker_args
|
||||
from loop_claude.stop import Stop
|
||||
@@ -23,13 +25,13 @@ class Cycle:
|
||||
models: Models
|
||||
paths: Paths
|
||||
premises: str
|
||||
project_key: str
|
||||
project_archive: pathlib.Path
|
||||
records: str
|
||||
# The evidence of the previous cycle's unfinished verdict, empty after any other.
|
||||
remaining: str
|
||||
root: str
|
||||
session: Session
|
||||
task: str
|
||||
# Empty when the previous cycle's verdict was not Unfinished.
|
||||
unfinished_evidence: str
|
||||
|
||||
|
||||
def run_cycle(cycle: Cycle) -> Stop | Finished | Unfinished:
|
||||
@@ -47,7 +49,7 @@ def run_cycle(cycle: Cycle) -> Stop | Finished | Unfinished:
|
||||
|
||||
def stopped(cycle: Cycle, stop: Stop, why: str) -> Stop:
|
||||
path = transcript_file(cycle.session)
|
||||
failure = archive(cycle.project_key, path, cycle.task, f"Stopped: {why}", cycle.effects.now(), cycle.paths) if path else None
|
||||
failure = archive(cycle.project_archive, path, cycle.task, outcomes.stopped(why), cycle.effects.now()) if path else None
|
||||
# A failed archive exits 1 whatever the stop; when the stop was this archive failing, its reason is said once.
|
||||
return Stop(1, f"{failure.reason}\n{stop.reason}".rstrip()) if failure and failure.reason != stop.reason else stop
|
||||
|
||||
@@ -68,19 +70,27 @@ def step(cycle: Cycle, state: Active) -> Event | Stop:
|
||||
|
||||
def work(cycle: Cycle, answered: QuotedAnswer | None) -> WorkerRan | Stop:
|
||||
if answered is not None:
|
||||
print(f"\n=== {QUOTE_SOURCES} answered: {answered.question} ===\n{answered.answer}", flush=True)
|
||||
print(f"\n=== {prompts.QUOTE_SOURCES} answered: {answered.question} ===\n{answered.answer}", flush=True)
|
||||
if stop := keep_answer(cycle, answered):
|
||||
return stop
|
||||
# session_open runs inside the lambda: a failed attempt may have opened the session.
|
||||
if stop := retried("worker", lambda: run_worker(cycle.root, worker_args(cycle.task, cycle.session, cycle.models.worker, session_open(cycle.session), answered, quoted(cycle.remaining)), cycle.effects), cycle.effects):
|
||||
# worker_start runs inside the lambda: a failed attempt may have opened the session.
|
||||
if stop := retried("worker", lambda: run_worker(cycle.root, worker_start(cycle, answered), cycle.effects), cycle.effects):
|
||||
return stop
|
||||
path = transcript_file(cycle.session)
|
||||
return WorkerRan(path) if path is not None else Stop(1, f"claude left no transcript for session {cycle.session.id}")
|
||||
|
||||
|
||||
def worker_start(cycle: Cycle, answered: QuotedAnswer | None) -> list[str]:
|
||||
if answered is not None:
|
||||
return worker_args(cycle.session, cycle.models.worker, prompts.answered(answered.question, answered.answer), resume=True)
|
||||
if session_open(cycle.session):
|
||||
return worker_args(cycle.session, cycle.models.worker, prompts.resumed(cycle.unfinished_evidence), resume=True)
|
||||
return worker_args(cycle.session, cycle.models.worker, prompts.fresh(cycle.task), resume=False)
|
||||
|
||||
|
||||
def keep_answer(cycle: Cycle, answered: QuotedAnswer) -> Stop | None:
|
||||
path = transcript_file(cycle.session)
|
||||
if path and (failure := archive(cycle.project_key, path, cycle.task, f"Verdict: quoted answer\n\n{answered.question}\n\n{QUOTE_SOURCES} says:\n{quoted(answered.answer)}\n\nEvidence:\n{quoted(answered.evidence)}", cycle.effects.now(), cycle.paths)):
|
||||
if path and (failure := archive(cycle.project_archive, path, cycle.task, outcomes.answered(answered.question, prompts.QUOTE_SOURCES, answered.answer, answered.evidence), cycle.effects.now())):
|
||||
return Stop(1, failure.reason)
|
||||
return None
|
||||
|
||||
@@ -96,7 +106,7 @@ def rule(cycle: Cycle, transcript_path: str) -> Ruled | Stop:
|
||||
def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) -> HumanAnswered | Stop:
|
||||
question = question_for(ruling)
|
||||
# Archived now, so the transcript the question was asked on is kept with it; the human's answer then grows the transcript.
|
||||
if failure := archive(cycle.project_key, transcript_path, cycle.task, f"Verdict: interrupt\n\n{question}\n\nEvidence:\n{quoted(ruling.evidence)}", cycle.effects.now(), cycle.paths):
|
||||
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.interrupted(question, ruling.evidence), cycle.effects.now()):
|
||||
return Stop(1, failure.reason)
|
||||
return interrupt(cycle.root, cycle.session.id, cycle.models.worker, transcript_path, question, cycle.paths, cycle.effects) or HumanAnswered()
|
||||
|
||||
@@ -104,18 +114,13 @@ def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) ->
|
||||
def question_for(ruling: Interrupt | QuotedAnswer) -> str:
|
||||
if isinstance(ruling, Interrupt):
|
||||
return ruling.question
|
||||
return f"{ruling.question}\n\n{QUOTE_SOURCES} says:\n{ruling.answer}\n\nThose files have answered {QUOTED_ANSWERS_PER_CYCLE} questions this cycle, the most allowed, so this one comes to you."
|
||||
return prompts.past_the_cap(ruling.question, ruling.answer, QUOTED_ANSWERS_PER_CYCLE)
|
||||
|
||||
|
||||
def close(cycle: Cycle, transcript_path: str, verdict: Finished | Unfinished) -> Archived | Stop:
|
||||
finished = isinstance(verdict, Finished)
|
||||
if failure := archive(cycle.project_key, transcript_path, cycle.task, f"{FINISHED if finished else UNFINISHED}\n\nEvidence:\n{quoted(verdict.evidence)}", cycle.effects.now(), cycle.paths):
|
||||
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.finished(verdict.evidence) if finished else outcomes.unfinished(verdict.evidence), cycle.effects.now()):
|
||||
return Stop(1, failure.reason)
|
||||
if finished and (failure := delete_session(cycle.session)):
|
||||
return Stop(1, failure.reason)
|
||||
return Archived()
|
||||
|
||||
|
||||
# Blockquoted, so evidence holding blank lines still ends visibly where the text after it starts.
|
||||
def quoted(passage: str) -> str:
|
||||
return "\n".join(f"> {line}" for line in passage.splitlines())
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import dataclasses
|
||||
|
||||
from loop_claude.cycle.judge_reply import Finished, Interrupt, QuotedAnswer, Unfinished, Verdict
|
||||
from loop_claude.cycle.judge import Finished, Interrupt, QuotedAnswer, Unfinished, Verdict
|
||||
|
||||
# A backstop for what the judge misses, such as a reworded repeat, so the worker cannot loop unattended on answers quoted from README.md or the records.
|
||||
QUOTED_ANSWERS_PER_CYCLE = 3
|
||||
@@ -72,7 +72,9 @@ def transition(state: State, event: Event) -> State:
|
||||
return Judging(transcript_path, quoted_answers)
|
||||
case Judging(_, quoted_answers), Ruled(QuotedAnswer() as answered) if quoted_answers < QUOTED_ANSWERS_PER_CYCLE:
|
||||
return Working(answered, quoted_answers + 1)
|
||||
case Judging(transcript_path, quoted_answers), Ruled(Interrupt() | QuotedAnswer() as ruling):
|
||||
case Judging(transcript_path, quoted_answers), Ruled(QuotedAnswer() as ruling) if quoted_answers >= QUOTED_ANSWERS_PER_CYCLE:
|
||||
return Interrupting(transcript_path, ruling, quoted_answers)
|
||||
case Judging(transcript_path, quoted_answers), Ruled(Interrupt() as ruling):
|
||||
return Interrupting(transcript_path, ruling, quoted_answers)
|
||||
case Judging(transcript_path, _), Ruled(Finished() | Unfinished() as verdict):
|
||||
return Closing(transcript_path, verdict)
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
from loop_claude.cycle.judge_reply import ANSWERS_RECORDED, QUOTE_SOURCES, QuotedAnswer
|
||||
from loop_claude.claude_code.entries import assistant_parts, ends_stream, hit_limit, limit_said, parse_entry, rejected_until
|
||||
from claude_code.entries import assistant_parts, ends_stream, hit_limit, limit_said, parse_entry, rejected_until
|
||||
from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.cycle.retry import Limited
|
||||
from loop_claude.claude_code.transcript import part_line
|
||||
from loop_claude.claude_code.session import Session
|
||||
from claude_code.transcript import part_line
|
||||
from claude_code.session import Session
|
||||
|
||||
PRINTED_TOOL_INPUT = 160
|
||||
|
||||
@@ -38,17 +37,12 @@ def worker_outcome(code: int, result: dict | None, resets: list[int]) -> Failure
|
||||
return None if result is not None else Failure("claude exited 0 without a result event")
|
||||
|
||||
|
||||
def worker_args(task: str, session: Session, model: str, resuming: bool, answered: QuotedAnswer | None = None, remaining: str = "") -> list[str]:
|
||||
if answered is not None:
|
||||
start = ["-p", f"Your question: {answered.question}\n\n{QUOTE_SOURCES} answers it:\n\n{answered.answer}\n\nContinue the task.", "--resume", str(session.id)]
|
||||
elif resuming:
|
||||
ruled = f" A judge ruled the chunk unfinished, quoting this passage of your transcript as the work remaining:\n\n{remaining}\n\n" if remaining else " "
|
||||
start = ["-p", f"Continue the task; the previous run ended before the chunk was finished.{ruled}The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}.", "--resume", str(session.id)]
|
||||
else:
|
||||
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
|
||||
start = ["-p", f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.", "--name", session.name, "--session-id", str(session.id)]
|
||||
def worker_args(session: Session, model: str, prompt: str, resume: bool) -> list[str]:
|
||||
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
|
||||
start = ["--resume", str(session.id)] if resume else ["--name", session.name, "--session-id", str(session.id)]
|
||||
# -p fails without --verbose under stream-json.
|
||||
return [
|
||||
"-p", prompt,
|
||||
*start,
|
||||
"--output-format", "stream-json",
|
||||
"--permission-mode", "bypassPermissions",
|
||||
|
||||
@@ -77,17 +77,19 @@ import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from loop_claude.project.checkout import checkout_root, project_key
|
||||
from loop_claude.project.checkout import checkout_root
|
||||
from loop_claude.cycle.run import Cycle, run_cycle
|
||||
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
|
||||
from loop_claude.effects import Effects
|
||||
from loop_claude.project.model_choice import get_or_ask_models
|
||||
from loop_claude.paths import Paths
|
||||
from transcript_archive.location import project_archive
|
||||
from project_key import project_key
|
||||
from loop_claude.project.premises import read_premises
|
||||
from loop_claude.project.records import read_records
|
||||
from loop_claude.claude_code.session import session_for
|
||||
from claude_code.session import session_named
|
||||
from loop_claude.stop import Stop
|
||||
from loop_claude.cycle.judge_reply import Unfinished
|
||||
from loop_claude.cycle.judge import Unfinished
|
||||
|
||||
|
||||
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
|
||||
@@ -105,8 +107,10 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
if isinstance(loop_run, Stop):
|
||||
return loop_run
|
||||
baseline = fingerprint(launcher)
|
||||
key = project_key(loop_run.root, paths.code)
|
||||
session = session_for(key, paths)
|
||||
key = project_key(loop_run.root, paths.home)
|
||||
archive = project_archive(loop_run.root, paths.home, env)
|
||||
# "A " is the human's cue in the session picker that the session is the loop's.
|
||||
session = session_named(f"A {key} loop-claude", paths.claude_code_transcripts)
|
||||
while True:
|
||||
if stop := exec_if_edited(launcher, baseline, loop_run, effects):
|
||||
return stop
|
||||
@@ -121,10 +125,10 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
if isinstance(models, Stop):
|
||||
return models
|
||||
print(f"\n=== cycle {loop_run.cycle_number} ===", flush=True)
|
||||
verdict = run_cycle(Cycle(effects=effects, models=models, paths=paths, premises=premises, project_key=key, records=records, remaining=loop_run.remaining, root=loop_run.root, session=session, task=loop_run.task))
|
||||
verdict = run_cycle(Cycle(effects=effects, models=models, paths=paths, premises=premises, project_archive=archive, records=records, root=loop_run.root, session=session, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence))
|
||||
if isinstance(verdict, Stop):
|
||||
return verdict
|
||||
loop_run = dataclasses.replace(loop_run, remaining=verdict.evidence if isinstance(verdict, Unfinished) else "")
|
||||
loop_run = dataclasses.replace(loop_run, unfinished_evidence=verdict.evidence if isinstance(verdict, Unfinished) else "")
|
||||
|
||||
|
||||
def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopRun | Stop:
|
||||
@@ -133,7 +137,7 @@ def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopR
|
||||
if task in ("", "-h", "--help"):
|
||||
return Stop(0 if task else 1, __doc__.strip())
|
||||
root = checkout_root(cwd, effects)
|
||||
return root if isinstance(root, Stop) else LoopRun(0, "", root, task)
|
||||
return root if isinstance(root, Stop) else LoopRun(0, root, task, "")
|
||||
loop_run = decode_handed(argv[1], LoopRun) if len(argv) == 2 else None
|
||||
if not (loop_run and loop_run.root and loop_run.task):
|
||||
return Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}")
|
||||
|
||||
@@ -17,9 +17,9 @@ T = typing.TypeVar("T")
|
||||
class LoopRun:
|
||||
"""What a reload hands the edited script, as JSON on argv."""
|
||||
cycle_number: int
|
||||
remaining: str
|
||||
root: str
|
||||
task: str
|
||||
unfinished_evidence: str
|
||||
|
||||
|
||||
def decode_handed(handed: str, kind: type[T]) -> T | None:
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -8,20 +7,18 @@ class Paths:
|
||||
claude_code_transcripts: pathlib.Path
|
||||
claude_json: pathlib.Path
|
||||
claude_md: pathlib.Path
|
||||
code: pathlib.Path
|
||||
home: pathlib.Path
|
||||
model_catalog: pathlib.Path
|
||||
models_file: pathlib.Path
|
||||
transcript_archive: pathlib.Path
|
||||
|
||||
|
||||
def paths_from(home: pathlib.Path, env: typing.Mapping[str, str]) -> Paths:
|
||||
def paths_from(home: pathlib.Path) -> Paths:
|
||||
claude = home / ".claude"
|
||||
return Paths(
|
||||
claude_code_transcripts=claude / "projects",
|
||||
claude_json=home / ".claude.json",
|
||||
claude_md=claude / "CLAUDE.md",
|
||||
code=home / "code",
|
||||
home=home,
|
||||
model_catalog=claude / "cache" / "model-catalog",
|
||||
models_file=claude / "loop-claude-models.json",
|
||||
transcript_archive=pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else claude / "loop-claude-transcripts",
|
||||
)
|
||||
|
||||
@@ -23,17 +23,3 @@ def worktrees(root: str, effects: Effects) -> list[str]:
|
||||
if listing.returncode == 0:
|
||||
found += [line.split(" ", 1)[1] for line in listing.stdout.splitlines() if line.startswith("worktree ")]
|
||||
return sorted(set(found))
|
||||
|
||||
|
||||
def project_key(root: str, code: pathlib.Path) -> str:
|
||||
return root.removeprefix(f"{code}/").lstrip("/")
|
||||
|
||||
|
||||
def read_utf8(folder: str, name: str, consequence: str) -> str | Stop:
|
||||
try:
|
||||
return pathlib.Path(folder, name).read_text(encoding="utf-8")
|
||||
except OSError as failure:
|
||||
return Stop(1, f"{name} in {folder}: {failure.strerror} - {consequence}.")
|
||||
except UnicodeDecodeError as failure:
|
||||
line = failure.object.count(b"\n", 0, failure.start) + 1
|
||||
return Stop(1, f"{name} in {folder} is not UTF-8 on line {line} - {consequence}.")
|
||||
|
||||
@@ -4,7 +4,7 @@ import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from loop_claude.claude_code.model_catalog import catalog_models
|
||||
from claude_code.model_catalog import catalog_models
|
||||
from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.stop import needs_human, needs_terminal, Stop
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from loop_claude.project.checkout import read_utf8
|
||||
from markdown_sections import sections
|
||||
from loop_claude.project.text_file import read_utf8
|
||||
from markdown_text import sections
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import pathlib
|
||||
|
||||
from loop_claude.project.checkout import read_utf8
|
||||
from loop_claude.project.text_file import read_utf8
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
# The records: these project files and HUMAN_RULES; the judge reads them beside the README's premises.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
import pathlib
|
||||
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
|
||||
def read_utf8(folder: str, name: str, consequence: str) -> str | Stop:
|
||||
try:
|
||||
return pathlib.Path(folder, name).read_text(encoding="utf-8")
|
||||
except OSError as failure:
|
||||
return Stop(1, f"{name} in {folder}: {failure.strerror} - {consequence}.")
|
||||
except UnicodeDecodeError as failure:
|
||||
line = failure.object.count(b"\n", 0, failure.start) + 1
|
||||
return Stop(1, f"{name} in {folder} is not UTF-8 on line {line} - {consequence}.")
|
||||
@@ -15,14 +15,14 @@ import tempfile
|
||||
import time
|
||||
import unittest
|
||||
|
||||
BIN = pathlib.Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from test_env import isolated_env
|
||||
|
||||
BIN = pathlib.Path(__file__).resolve().parents[1]
|
||||
TASK = "Do the next chunk."
|
||||
|
||||
|
||||
# A git hook's GIT_DIR, the host's LOOP_CLAUDE_* or PYTHONPATH would point the program under test at the host's own state.
|
||||
def host_env() -> dict[str, str]:
|
||||
return {name: value for name, value in os.environ.items() if not name.startswith(("GIT_", "LOOP_CLAUDE_", "PYTHON"))}
|
||||
|
||||
# Plays claude from FAKE_CLAUDE_SCENARIO, one step per call, logging each call; a step for another role,
|
||||
# or none left, is Ctrl-C to loop-claude, which otherwise loops forever.
|
||||
FAKE_CLAUDE = """#!{python}
|
||||
@@ -160,7 +160,7 @@ class Checkout:
|
||||
self.root.mkdir(parents=True)
|
||||
(self.root / "README.md").write_text(PREMISES)
|
||||
self.script = BIN / "loop-claude"
|
||||
self.env = host_env()
|
||||
self.env = isolated_env()
|
||||
fake = self.temp / "fake-bin"
|
||||
fake.mkdir()
|
||||
(fake / "claude").write_text(FAKE_CLAUDE.format(python=sys.executable))
|
||||
@@ -687,12 +687,12 @@ class Reload(unittest.TestCase):
|
||||
self.assertEqual(run.roles(), ["worker", "judge"] * 5)
|
||||
self.assertIn("keeping the running script", run.stderr)
|
||||
reloads = [json.loads(argv[1]) for argv in run.starts if argv[:1] == ["--reloaded"]]
|
||||
self.assertEqual(reloads, [{"cycle_number": cycle, "remaining": "working", "root": str(checkout.root), "task": TASK} for cycle in (2, 3, 4)])
|
||||
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working"} for cycle in (2, 3, 4)])
|
||||
self.assertIn("=== cycle 5 ===", run.stdout)
|
||||
|
||||
outside = checkout.temp / "outside"
|
||||
outside.mkdir()
|
||||
state = {"cycle_number": 3, "remaining": "", "root": str(checkout.root), "task": TASK}
|
||||
state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK, "unfinished_evidence": ""}
|
||||
run = checkout.run([worker()], argv=["--reloaded", json.dumps(state)], cwd=outside)
|
||||
self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"]))
|
||||
self.assertEqual(run.calls[0]["cwd"], str(checkout.root))
|
||||
@@ -1,7 +1,6 @@
|
||||
import ast
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
@@ -14,18 +13,18 @@ import time
|
||||
import unittest
|
||||
import uuid
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from loop_claude import transcript_archive as archive
|
||||
from loop_claude.project import checkout, model_choice, records
|
||||
from loop_claude.effects import Effects
|
||||
from loop_claude.claude_code import entries, session, transcript, trust
|
||||
from claude_code import entries, session
|
||||
from failure import Failure
|
||||
from loop_claude.cycle.judge_reply import Finished, Interrupt, QuotedAnswer, Unfinished, verdict_from
|
||||
from loop_claude.cycle import interrupt, judge, retry, run, states, worker
|
||||
from loop_claude.cycle.judge import Finished, Interrupt, QuotedAnswer, Unfinished, verdict_from
|
||||
from loop_claude.cycle import interrupt, judge, prompts, retry, run, states, worker
|
||||
from loop_claude.cycle.retry import Limited
|
||||
from loop_claude.loop import main, reload
|
||||
from loop_claude import paths
|
||||
from transcript_archive.location import project_archive
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
CWD = pathlib.Path("/c")
|
||||
@@ -40,10 +39,6 @@ def human(content):
|
||||
return {"type": "user", "message": {"content": content}}
|
||||
|
||||
|
||||
def queued(prompt, kind):
|
||||
return {"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "prompt", "origin": {"kind": kind}, "prompt": prompt}}
|
||||
|
||||
|
||||
class FakeEffects:
|
||||
"""Effects that record each call and answer from a script, one answer per call."""
|
||||
|
||||
@@ -164,24 +159,20 @@ class QuestionFor(unittest.TestCase):
|
||||
self.assertEqual(run.question_for(QuotedAnswer("e", "Which one?", "A.")), "Which one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md or ~/.claude/CLAUDE.md says:\nA.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.")
|
||||
|
||||
|
||||
class Quoted(unittest.TestCase):
|
||||
def test_quotes_every_line_of_a_passage_blank_ones_included(self):
|
||||
self.assertEqual(run.quoted("merged\n\nPR #1"), "> merged\n> \n> PR #1")
|
||||
|
||||
|
||||
class Cycles(unittest.TestCase):
|
||||
def setUp(self):
|
||||
temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temp.cleanup)
|
||||
self.home = pathlib.Path(temp.name)
|
||||
self.paths = paths.paths_from(self.home, {})
|
||||
self.session = session.session_for("proj", self.paths)
|
||||
self.paths = paths.paths_from(self.home)
|
||||
self.session = session.session_named("A proj loop-claude", self.paths.claude_code_transcripts)
|
||||
self.archive = project_archive(str(self.home / "code" / "proj"), self.home, {})
|
||||
|
||||
def cycle(self, fake, records="", remaining=""):
|
||||
return run.Cycle(effects=fake.effects(), models=model_choice.Models("judge-model", "worker-model"), paths=self.paths, premises="## Goals", project_key="proj", records=records, remaining=remaining, root=str(self.home), session=self.session, task="Do it.")
|
||||
def cycle(self, fake, records="", unfinished_evidence=""):
|
||||
return run.Cycle(effects=fake.effects(), models=model_choice.Models("judge-model", "worker-model"), paths=self.paths, premises="## Goals", project_archive=self.archive, records=records, root=str(self.home), session=self.session, task="Do it.", unfinished_evidence=unfinished_evidence)
|
||||
|
||||
def archived_outcomes(self):
|
||||
return sorted(path.read_text() for path in (self.paths.transcript_archive / "proj").glob("*/*.outcomes.md"))
|
||||
return sorted(path.read_text() for path in self.archive.glob("*/*.outcomes.md"))
|
||||
|
||||
def test_a_finished_chunk_is_archived_and_its_session_deleted(self):
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
|
||||
@@ -193,7 +184,7 @@ class Cycles(unittest.TestCase):
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[judged("unfinished")] * 2)
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
|
||||
self.assertTrue(session.transcript_file(self.session))
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake, remaining="working")), Unfinished("working"))
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake, unfinished_evidence="working")), Unfinished("working"))
|
||||
workers = fake.made("stream_claude")
|
||||
self.assertIn("--session-id", workers[0])
|
||||
self.assertIn("--resume", workers[1])
|
||||
@@ -216,7 +207,7 @@ class Cycles(unittest.TestCase):
|
||||
(folder.parent / f"{self.session.id}.orphaned-1.jsonl").write_text("o\n")
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 3, claude=[judged("unfinished"), judged("finished"), judged("unfinished")])
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
|
||||
[chunk] = (self.paths.transcript_archive / "proj").iterdir()
|
||||
[chunk] = self.archive.iterdir()
|
||||
kept = chunk / "session" / str(self.session.id)
|
||||
self.assertEqual((chunk / "session" / f"{self.session.id}.orphaned-1.jsonl").read_text(), "o\n")
|
||||
os.utime(kept / "subagents" / "workflows" / "journal.jsonl", (0, 0))
|
||||
@@ -232,7 +223,7 @@ class Cycles(unittest.TestCase):
|
||||
folder.mkdir()
|
||||
(folder / "custom-title.json").write_text("third")
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
|
||||
[next_chunk] = set((self.paths.transcript_archive / "proj").iterdir()) - {chunk}
|
||||
[next_chunk] = set(self.archive.iterdir()) - {chunk}
|
||||
self.assertEqual(sorted(path.read_text() for path in (next_chunk / "session" / str(self.session.id)).glob("custom-title.json*")), ["third"])
|
||||
|
||||
@unittest.skipIf(os.geteuid() == 0, "root reads an unreadable folder")
|
||||
@@ -253,10 +244,10 @@ class Cycles(unittest.TestCase):
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nEvidence:\n> working\n", "Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n"])
|
||||
self.assertIn(str(self.session.id), fake.made("claude_on_terminal")[0])
|
||||
prompts = [stdin for args, stdin in fake.made("claude")]
|
||||
self.assertEqual(len(prompts), 2)
|
||||
self.assertNotIn("the second", prompts[0])
|
||||
self.assertIn("[human] the second", prompts[1])
|
||||
judge_prompts = [stdin for args, stdin in fake.made("claude")]
|
||||
self.assertEqual(len(judge_prompts), 2)
|
||||
self.assertNotIn("the second", judge_prompts[0])
|
||||
self.assertIn("[human] the second", judge_prompts[1])
|
||||
|
||||
def test_a_question_the_records_answer_resumes_the_worker_with_the_answer_three_times_a_cycle(self):
|
||||
answered = judged("quoted", answer="Use *A*.")
|
||||
@@ -272,11 +263,12 @@ class Cycles(unittest.TestCase):
|
||||
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md or ~/.claude/CLAUDE.md says:\nUse *A*.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.\n\nEvidence:\n> working\n\n2026-09-21T14:13:20Z Stopped: the human is needed to answer the question above, but the interrupt ended with nothing typed\n", *[f"Task: Do it.\n\n{quoted}"] * 3])
|
||||
|
||||
def test_stops_when_a_quoted_answer_cannot_be_archived(self):
|
||||
self.paths.transcript_archive.parent.mkdir(parents=True)
|
||||
self.paths.transcript_archive.write_text("")
|
||||
self.archive.parent.mkdir(parents=True)
|
||||
self.archive.write_text("")
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("quoted", answer="Use A.")])
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse A.")), Stop(1, f"could not archive the transcript: {self.paths.transcript_archive / 'proj'}: Not a directory"))
|
||||
stop = run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse A."))
|
||||
self.assertRegex(stop.reason, f"^could not archive the transcript: {self.archive}/[0-9a-f]{{64}}: Not a directory$")
|
||||
self.assertEqual(len(fake.made("stream_claude")), 1)
|
||||
|
||||
def test_an_answer_the_records_do_not_hold_is_dropped_and_the_question_goes_to_the_human(self):
|
||||
@@ -357,7 +349,7 @@ class Limits(unittest.TestCase):
|
||||
class Interrupts(unittest.TestCase):
|
||||
def test_stops_off_a_terminal_before_starting_claude(self):
|
||||
fake = FakeEffects(on_terminal=False)
|
||||
stop = interrupt.interrupt("/r", uuid7(), "m", "/t", "Which one?", paths.paths_from(pathlib.Path("/h"), {}), fake.effects())
|
||||
stop = interrupt.interrupt("/r", uuid7(), "m", "/t", "Which one?", paths.paths_from(pathlib.Path("/h")), fake.effects())
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"))
|
||||
self.assertEqual(fake.calls, [])
|
||||
|
||||
@@ -366,7 +358,7 @@ class Interrupts(unittest.TestCase):
|
||||
path = pathlib.Path(home, "t.jsonl")
|
||||
path.write_text(json.dumps(human("a")) + "\n")
|
||||
fake = FakeEffects(on_terminal_claude=[0], git=[ran()])
|
||||
stop = interrupt.interrupt(home, uuid7(), "m", str(path), "Which one?", paths.paths_from(pathlib.Path(home), {}), fake.effects())
|
||||
stop = interrupt.interrupt(home, uuid7(), "m", str(path), "Which one?", paths.paths_from(pathlib.Path(home)), fake.effects())
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"))
|
||||
|
||||
|
||||
@@ -391,13 +383,13 @@ class ModelPrompt(unittest.TestCase):
|
||||
|
||||
class WorkerArgs(unittest.TestCase):
|
||||
def test_opens_a_fresh_session_by_name_and_resumes_an_open_one(self):
|
||||
opened = session.session_for("proj", paths.paths_from(pathlib.Path("/h"), {}))
|
||||
fresh = worker.worker_args("Do it.", opened, "m", False)
|
||||
opened = session.session_named("A proj loop-claude", pathlib.Path("/h/.claude/projects"))
|
||||
fresh = worker.worker_args(opened, "m", prompts.fresh("Do it."), resume=False)
|
||||
self.assertEqual(fresh[fresh.index("--session-id") + 1], str(opened.id))
|
||||
self.assertEqual(fresh[fresh.index("--name") + 1], "A proj loop-claude")
|
||||
resumed = worker.worker_args("Do it.", opened, "m", True)
|
||||
resumed = worker.worker_args(opened, "m", prompts.resumed(""), resume=True)
|
||||
self.assertEqual(resumed[resumed.index("--resume") + 1], str(opened.id))
|
||||
self.assertNotIn("Do it.", " ".join(resumed))
|
||||
self.assertNotIn("--session-id", resumed)
|
||||
self.assertTrue(resumed[resumed.index("-p") + 1].startswith("Continue the task; the previous run ended before the chunk was finished. The chunk is finished once"))
|
||||
|
||||
|
||||
@@ -425,101 +417,6 @@ class Records(unittest.TestCase):
|
||||
self.assertEqual(records.read_records(root, claude_md), Stop(1, f"AGENTS.md in {root} is not UTF-8 on line 2 - the judge cannot read the records."))
|
||||
|
||||
|
||||
class Render(unittest.TestCase):
|
||||
def test_joins_assistant_text_tool_calls_human_turns_and_loop_prompts_and_skips_the_rest(self):
|
||||
said = transcript.render([
|
||||
human("do it"),
|
||||
assistant({"type": "text", "text": "Starting."}, {"type": "tool_use", "name": "Bash", "input": {"command": "ls"}}),
|
||||
assistant({"type": "text", "text": " "}),
|
||||
human([{"type": "tool_result", "content": "not typed"}]),
|
||||
human("<command-name>/exit</command-name>"),
|
||||
human([{"type": "text", "text": "yes, "}, {"type": "text", "text": "merge it"}]),
|
||||
queued("B, while you work", "human"),
|
||||
queued("origin/main moved", "coordinator"),
|
||||
{**human("Continue the task; the previous run stopped before it was finished."), "promptSource": "sdk"},
|
||||
])
|
||||
self.assertEqual(said, '[human] do it\n\nStarting.\n\n[tool Bash] {"command": "ls"}\n\n[human] yes, merge it\n\n[human] B, while you work\n\n[loop] Continue the task; the previous run stopped before it was finished.')
|
||||
|
||||
|
||||
class Archive(unittest.TestCase):
|
||||
def test_writes_afresh_a_copy_whose_outcomes_file_is_missing(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
kept = paths.paths_from(pathlib.Path(root), {})
|
||||
transcript = pathlib.Path(root, "t.jsonl")
|
||||
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(archive.archive("proj", str(transcript), "Do it.", "Verdict: unfinished", NOW, kept))
|
||||
(outcomes,) = kept.transcript_archive.glob("proj/*/*.outcomes.md")
|
||||
outcomes.unlink()
|
||||
self.assertIsNone(archive.archive("proj", str(transcript), "Do it.", "Stopped: Ctrl-C", NOW, kept))
|
||||
self.assertEqual(outcomes.read_text(), "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n")
|
||||
|
||||
|
||||
class CopyFor(unittest.TestCase):
|
||||
def test_skips_a_version_already_kept_beside_its_copy(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
copy = pathlib.Path(temp, "custom-title.json")
|
||||
copy.write_bytes(b"first")
|
||||
beside = archive.copy_for(copy, b"second")
|
||||
self.assertEqual(beside, copy.with_name(f"custom-title.json.{hashlib.sha256(b'second').hexdigest()}"))
|
||||
beside.write_bytes(b"second")
|
||||
self.assertIsNone(archive.copy_for(copy, b"second"))
|
||||
self.assertEqual(archive.copy_for(copy, b"first, grown"), copy)
|
||||
|
||||
|
||||
class FirstTimestampedLine(unittest.TestCase):
|
||||
def test_skips_the_lines_every_chunk_shares_and_else_takes_the_first(self):
|
||||
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\nnot JSON\n{"timestamp": "t"}\n{"timestamp": "u"}\n'), b'{"timestamp": "t"}')
|
||||
self.assertEqual(transcript.first_timestamped_line(b'{"type": "custom-title"}\n[1]\n'), b'{"type": "custom-title"}')
|
||||
self.assertEqual(transcript.first_timestamped_line(b""), b"")
|
||||
|
||||
|
||||
class HumanTurns(unittest.TestCase):
|
||||
def test_counts_typed_turns_and_not_tool_results(self):
|
||||
self.assertEqual(transcript.human_turns([
|
||||
human("typed"),
|
||||
human([{"type": "text", "text": "typed as a block"}]),
|
||||
human([{"type": "tool_result", "content": "not typed"}]),
|
||||
human("<command-name>/model</command-name>"),
|
||||
{**human("<3 merge it"), "promptSource": "typed"},
|
||||
{**human("<task-notification>done</task-notification>"), "promptSource": "system"},
|
||||
{**human([{"type": "text", "text": "<task-notification>done</task-notification>"}]), "promptSource": "system"},
|
||||
{**human("peer message"), "isMeta": True},
|
||||
{**human("the loop's own prompt"), "promptSource": "sdk"},
|
||||
human([{"type": "text", "text": "[Request interrupted by user]"}]),
|
||||
human([]),
|
||||
queued("typed while claude worked", "human"),
|
||||
queued("<task-notification>done</task-notification>", "task-notification"),
|
||||
{"type": "attachment", "attachment": {"type": "queued_command", "commandMode": "task-notification", "prompt": "<task-notification>done</task-notification>"}},
|
||||
assistant({"type": "text", "text": "said"}),
|
||||
]), 4)
|
||||
|
||||
|
||||
class PartLine(unittest.TestCase):
|
||||
def test_renders_text_and_tool_calls_and_nothing_else(self):
|
||||
self.assertEqual(transcript.part_line({"type": "text", "text": "hi"}, 20), "hi")
|
||||
self.assertEqual(transcript.part_line({"type": "text", "text": " \n"}, 20), "")
|
||||
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Bash", "input": {"command": "x" * 500}}, 20), '[tool Bash] {"command": "xxxxxxx')
|
||||
self.assertEqual(transcript.part_line("not a part", 20), "")
|
||||
self.assertEqual(transcript.part_line({"type": "tool_use", "name": "Write", "input": {"content": "Hälsa på"}}, 40), '[tool Write] {"content": "Hälsa på"}')
|
||||
|
||||
|
||||
class Parse(unittest.TestCase):
|
||||
def test_yields_only_json_objects(self):
|
||||
self.assertEqual(entries.parse_entry('{"type": "user"}'), {"type": "user"})
|
||||
self.assertIsNone(entries.parse_entry("not json"))
|
||||
self.assertIsNone(entries.parse_entry("[1, 2]"))
|
||||
|
||||
|
||||
class MarkTrusted(unittest.TestCase):
|
||||
def test_returns_the_updated_config_and_leaves_the_one_it_was_handed(self):
|
||||
config = {"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"kept": True}}}
|
||||
handed = json.dumps(config)
|
||||
self.assertEqual(trust.mark_trusted(config, ["/a", "/b", "/c"]), ({"other": 1, "projects": {"/a": {"hasTrustDialogAccepted": True}, "/b": {"hasTrustDialogAccepted": True, "kept": True}, "/c": {"hasTrustDialogAccepted": True}}}, ["/b", "/c"]))
|
||||
self.assertEqual(json.dumps(config), handed)
|
||||
self.assertEqual(trust.mark_trusted({}, ["/a"]), ({"projects": {"/a": {"hasTrustDialogAccepted": True}}}, ["/a"]))
|
||||
|
||||
|
||||
class JudgePrompt(unittest.TestCase):
|
||||
def test_shows_the_transcripts_tail_and_returns_what_it_showed(self):
|
||||
prompt, shown = judge.judge_prompt("Do it.", "## Goals", "", "old " + "x" * judge.JUDGE_PROMPT_CHARS + " new")
|
||||
@@ -569,7 +466,7 @@ class Verdict(unittest.TestCase):
|
||||
|
||||
class LoopRuns(unittest.TestCase):
|
||||
def test_stops_naming_the_reload_when_it_hands_over_anything_but_the_state_with_a_task(self):
|
||||
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": ""})], [json.dumps({"cycle_number": "3", "remaining": "", "root": "/r", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": "do it", "turns": 1})], [json.dumps({"cycle_number": 3, "failures": 0, "remaining": "", "root": "/r", "task": "do it"})]):
|
||||
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "root": "/r", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "", "unfinished_evidence": ""})], [json.dumps({"cycle_number": "3", "root": "/r", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "do it", "turns": 1, "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "failures": 0, "root": "/r", "task": "do it", "unfinished_evidence": ""})]):
|
||||
with self.subTest(handed=handed):
|
||||
self.assertEqual(main.loop_run_from([main.RELOAD_FLAG, *handed], CWD, FakeEffects().effects()), Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(handed)}"))
|
||||
|
||||
@@ -586,7 +483,7 @@ class LoopRuns(unittest.TestCase):
|
||||
launcher = pathlib.Path(root, "loop-claude")
|
||||
launcher.write_text("")
|
||||
fake = FakeEffects(python=[ran()])
|
||||
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(1, "", "/r", "do it"), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
|
||||
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence=""), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
|
||||
|
||||
|
||||
class HelpLimits(unittest.TestCase):
|
||||
@@ -622,18 +519,18 @@ class ExitContract(unittest.TestCase):
|
||||
("loop/reload.py", "exec_if_edited", "1", "f'could not reload loop-claude:", "python cannot start for a reload"),
|
||||
("project/checkout.py", "checkout_root", "1", "f'could not start git:", git),
|
||||
("project/checkout.py", "checkout_root", "1", "found.stderr.strip()", git),
|
||||
("project/checkout.py", "read_utf8", "1", "f'{name} in {folder}:", unreadable),
|
||||
("project/checkout.py", "read_utf8", "1", "f'{name} in {folder} is not UTF-8", unreadable),
|
||||
("project/model_choice.py", "get_or_ask_model", "1", "f'could not save the model in", models_),
|
||||
("project/model_choice.py", "read_models_file", "1", 'f"could not read {saved}: {getattr(', models_),
|
||||
("project/model_choice.py", "read_models_file", "1", "f'could not read {saved}: not an object", models_),
|
||||
("project/premises.py", "read_premises", "1", "f'README.md has no ##", premises_),
|
||||
("project/premises.py", "read_premises", "1", "f'README.md has ## {name} more than once", premises_),
|
||||
("project/premises.py", "read_premises", "1", "f'README.md has nothing under", premises_),
|
||||
("project/text_file.py", "read_utf8", "1", "f'{name} in {folder}:", unreadable),
|
||||
("project/text_file.py", "read_utf8", "1", "f'{name} in {folder} is not UTF-8", unreadable),
|
||||
]
|
||||
root = pathlib.Path(main.__file__).parents[1]
|
||||
found = []
|
||||
for path in sorted(root.rglob("*.py")):
|
||||
for path in sorted(path for path in root.rglob("*.py") if not path.name.startswith("test_")):
|
||||
functions = [node for node in ast.walk(ast.parse(path.read_text())) if isinstance(node, ast.FunctionDef)]
|
||||
stops = [(call.lineno, function.name, ast.unparse(call.args[0]), ast.unparse(call.args[1])) for function in functions for call in ast.walk(function) if isinstance(call, ast.Call) and getattr(call.func, "id", None) == "Stop" and not (isinstance(call.args[0], ast.Constant) and call.args[0].value in (0, 2, 130))]
|
||||
found += [(str(path.relative_to(root)), function, code, reason) for _, function, code, reason in sorted(stops)]
|
||||
@@ -650,25 +547,21 @@ class JudgeFailures(unittest.TestCase):
|
||||
started, field, unasked = "claude cannot start or exits non-zero", "its reply misses or mistypes a field", "interrupts or quotes an answer without a question"
|
||||
# Every Failure the judge's run returns, in source order: path, function, start of reason, its --help clause.
|
||||
clauses = [
|
||||
("cycle/judge.py", "run_judge", "f'could not start claude:", started),
|
||||
("cycle/judge.py", "run_judge", "proc.stderr.strip() or f'the judge exited", started),
|
||||
("cycle/judge.py", "run_judge", 'f"{verdict.reason}; it replied:', field),
|
||||
("cycle/judge.py", "run_judge", "f'the judge quoted {verdict.evidence!r}, which is not in the transcript", "quotes a passage the transcript does not hold"),
|
||||
("cycle/judge_reply.py", "verdict_from", "'the judge returned no reply'", field),
|
||||
("cycle/judge_reply.py", "verdict_from", 'f"the judge\'s reply has no {name}', field),
|
||||
("cycle/judge_reply.py", "verdict_from", "'the judge interrupted without a question'", unasked),
|
||||
("cycle/judge_reply.py", "verdict_from", "'the judge quoted an answer without a question'", unasked),
|
||||
("verdict_from", "'the judge returned no reply'", field),
|
||||
("verdict_from", 'f"the judge\'s reply has no {name}', field),
|
||||
("verdict_from", "'the judge interrupted without a question'", unasked),
|
||||
("verdict_from", "'the judge quoted an answer without a question'", unasked),
|
||||
("run_judge", "f'could not start claude:", started),
|
||||
("run_judge", "proc.stderr.strip() or f'the judge exited", started),
|
||||
("run_judge", 'f"{verdict.reason}; it replied:', field),
|
||||
("run_judge", "f'the judge quoted {verdict.evidence!r}, which is not in the transcript", "quotes a passage the transcript does not hold"),
|
||||
]
|
||||
root = pathlib.Path(main.__file__).parents[1]
|
||||
found = []
|
||||
for path in ("cycle/judge.py", "cycle/judge_reply.py"):
|
||||
functions = [node for node in ast.walk(ast.parse((root / path).read_text())) if isinstance(node, ast.FunctionDef)]
|
||||
failures = sorted((call.lineno, function.name, ast.unparse(call.args[0])) for function in functions for call in ast.walk(function) if isinstance(call, ast.Call) and getattr(call.func, "id", None) == "Failure")
|
||||
found += [(path, function, reason) for _, function, reason in failures]
|
||||
functions = [node for node in ast.walk(ast.parse(pathlib.Path(judge.__file__).read_text())) if isinstance(node, ast.FunctionDef)]
|
||||
found = sorted((call.lineno, function.name, ast.unparse(call.args[0])) for function in functions for call in ast.walk(function) if isinstance(call, ast.Call) and getattr(call.func, "id", None) == "Failure")
|
||||
self.assertEqual(len(found), len(clauses), found)
|
||||
for (path, function, reason), (want_path, want_function, reason_start, _) in zip(found, clauses):
|
||||
self.assertEqual((path, function), (want_path, want_function))
|
||||
self.assertTrue(reason.startswith(reason_start), f"{path} {function}: {reason}")
|
||||
for (_, function, reason), (want_function, reason_start, _) in zip(found, clauses):
|
||||
self.assertEqual(function, want_function)
|
||||
self.assertTrue(reason.startswith(reason_start), f"{function}: {reason}")
|
||||
judge_clause = " ".join(main.__doc__[main.__doc__.index("a judge when"):main.__doc__.index("A usage limit")].split())
|
||||
self.assertEqual([clause for *_, clause in clauses if clause not in judge_clause], [])
|
||||
|
||||
@@ -25,3 +25,8 @@ def sections(text: str) -> dict[str, list[str]]:
|
||||
if name is not None:
|
||||
found.setdefault(name, []).append("\n".join(body))
|
||||
return found
|
||||
|
||||
|
||||
# Blockquoted, so a passage holding blank lines still ends visibly where the text after it starts.
|
||||
def quoted(passage: str) -> str:
|
||||
return "\n".join(f"> {line}" for line in passage.splitlines())
|
||||
@@ -84,7 +84,7 @@ def medians(runs: list[RunScores]) -> dict[str, Score]:
|
||||
return {metric: Score(statistics.median(run.scores[metric].auc for run in runs), statistics.median(run.scores[metric].top_k for run in runs), all(run.scores[metric].constant for run in runs)) for metric in METRICS}
|
||||
|
||||
|
||||
def resolved(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
|
||||
def tracked_places(places: list[Place], tracked: list[str]) -> tuple[list[Place], list[Place]]:
|
||||
"""Entries written before paths were written from the repository's root shorten a path once they have named its folder: `cycle/run.py` after `bin/loop_claude/cycle/states.py`. Each place resolves to the one tracked file it names, or, where several match, the one in the folder of the place resolved before it; a place naming no file, or several, is dropped. Returns the resolved places and the dropped ones."""
|
||||
found = []
|
||||
dropped = []
|
||||
|
||||
@@ -35,7 +35,7 @@ import statistics
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from metric_bench.bench import bench, co_changed, medians, METRICS, resolved, RunScores, Score
|
||||
from metric_bench.bench import bench, co_changed, medians, METRICS, RunScores, Score, tracked_places
|
||||
from comprehension_history import entries, Entry
|
||||
from failure import Failure
|
||||
from metric_bench.python_units import measured, python_units, Unit
|
||||
@@ -128,7 +128,7 @@ def run_scores(run: Entry, top: str) -> tuple[RunScores | None, list[str]] | Fai
|
||||
log = git(["-c", "core.quotePath=false", "log", "--no-merges", "--format=tformat:@@", "--name-only", run.commit], top)
|
||||
if isinstance(log, Failure):
|
||||
return log
|
||||
places, dropped = resolved(run.scored_places, tracked)
|
||||
places, dropped = tracked_places(run.scored_places, tracked)
|
||||
return bench(units, places, len(set(dropped)), co_changed(log)), unparsed
|
||||
|
||||
|
||||
|
||||
@@ -7,16 +7,16 @@ import tempfile
|
||||
import textwrap
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from metric_bench.bench import auc, bench, co_changed, resolved, top_k
|
||||
from metric_bench.bench import auc, bench, co_changed, top_k, tracked_places
|
||||
from metric_bench.python_units import cognitive, measured, python_units, Unit
|
||||
from comprehension_history import entries, Place
|
||||
from failure import Failure
|
||||
from test_loop_claude_black_box import host_env
|
||||
from test_env import isolated_env
|
||||
from metric_bench.main import scored_runs
|
||||
|
||||
TOOL = pathlib.Path(__file__).resolve().parent / "metric-bench"
|
||||
TOOL = pathlib.Path(__file__).resolve().parents[1] / "metric-bench"
|
||||
|
||||
|
||||
def scored(source: str) -> int:
|
||||
@@ -153,18 +153,18 @@ class Places(unittest.TestCase):
|
||||
def test_resolves_a_shortened_path_to_the_file_nearest_the_one_before_it(self):
|
||||
tracked = ["bin/tool/cycle/states.py", "bin/tool/cycle/run.py", "bin/other/cycle/run.py"]
|
||||
places = [Place("bin/tool/cycle/states.py", "transition"), Place("cycle/run.py", "step"), Place("gone.py", "x")]
|
||||
self.assertEqual(resolved(places, tracked), ([Place("bin/tool/cycle/states.py", "transition"), Place("bin/tool/cycle/run.py", "step")], [Place("gone.py", "x")]))
|
||||
self.assertEqual(tracked_places(places, tracked), ([Place("bin/tool/cycle/states.py", "transition"), Place("bin/tool/cycle/run.py", "step")], [Place("gone.py", "x")]))
|
||||
|
||||
def test_a_full_path_wins_over_a_file_in_the_folder_before_it(self):
|
||||
self.assertEqual(resolved([Place("pkg/x.py", "a"), Place("main.py", "f")], ["pkg/x.py", "main.py", "pkg/main.py"])[0][-1], Place("main.py", "f"))
|
||||
self.assertEqual(tracked_places([Place("pkg/x.py", "a"), Place("main.py", "f")], ["pkg/x.py", "main.py", "pkg/main.py"])[0][-1], Place("main.py", "f"))
|
||||
|
||||
def test_resolves_a_shortened_path_after_a_repeated_place_by_the_repeat(self):
|
||||
tracked = ["bin/a/main.py", "bin/b/main.py", "bin/b/bench.py"]
|
||||
places = [Place("bin/a/main.py", "main"), Place("bin/b/bench.py", "bench"), Place("bin/a/main.py", "main"), Place("main.py", "figures")]
|
||||
self.assertEqual(resolved(places, tracked)[0][-1], Place("bin/a/main.py", "figures"))
|
||||
self.assertEqual(tracked_places(places, tracked)[0][-1], Place("bin/a/main.py", "figures"))
|
||||
|
||||
def test_drops_a_shortened_path_two_files_match_with_no_place_before_it(self):
|
||||
self.assertEqual(resolved([Place("cycle/run.py", "step")], ["bin/tool/cycle/run.py", "bin/other/cycle/run.py"]), ([], [Place("cycle/run.py", "step")]))
|
||||
self.assertEqual(tracked_places([Place("cycle/run.py", "step")], ["bin/tool/cycle/run.py", "bin/other/cycle/run.py"]), ([], [Place("cycle/run.py", "step")]))
|
||||
|
||||
|
||||
class Bench(unittest.TestCase):
|
||||
@@ -198,7 +198,7 @@ class BlackBox(unittest.TestCase):
|
||||
directory = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(directory.cleanup)
|
||||
top = directory.name
|
||||
env = host_env()
|
||||
env = isolated_env()
|
||||
for args in (["init", "-q"], ["config", "user.email", "t@t"], ["config", "user.name", "t"]):
|
||||
subprocess.run(["git", *args], cwd=top, check=True, env=env)
|
||||
pathlib.Path(top, "a.py").write_text("def hard(a):\n\tif a:\n\t\tfor x in a:\n\t\t\tif x:\n\t\t\t\treturn x\n\n\ndef easy():\n\treturn 1\n")
|
||||
@@ -211,7 +211,7 @@ class BlackBox(unittest.TestCase):
|
||||
return top
|
||||
|
||||
def run_tool(self, top: str, *args: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run([sys.executable, str(TOOL), *args], cwd=top, capture_output=True, text=True, env=host_env())
|
||||
return subprocess.run([sys.executable, str(TOOL), *args], cwd=top, capture_output=True, text=True, env=isolated_env())
|
||||
|
||||
def test_prints_each_metrics_score_on_each_run_and_the_medians(self):
|
||||
top = self.project("# Comprehension history\n\n## 2026-10-08T12:50:49Z, PR #1 at COMMIT\n\nJunior A, hardest first:\n\n- `a.py:1` `hard`\n")
|
||||
@@ -0,0 +1,6 @@
|
||||
import pathlib
|
||||
|
||||
|
||||
def project_key(checkout_root: str, home: pathlib.Path) -> str:
|
||||
"""The checkout's path relative to ~/code, or to / outside it: what names a project's loop-claude session, its entry in the models file and its archive folder."""
|
||||
return checkout_root.removeprefix(f"{home / 'code'}/").lstrip("/")
|
||||
+2
-61
@@ -57,12 +57,11 @@ import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
from todo_table import cells, Item, ITEMS_HEADER, parse_row, table_rows
|
||||
from failure import Failure
|
||||
from markdown_sections import FENCE, sections
|
||||
from markdown_text import FENCE, sections
|
||||
from file_writes import replace_file
|
||||
|
||||
EXEMPTIONS = ("", "decision", "defect", "principle", "question", "security")
|
||||
ITEMS_HEADER = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |"
|
||||
LARGEST_FORMULA = 200
|
||||
LONGEST_FORMULA = 4 * LARGEST_FORMULA
|
||||
OPERATORS = (ast.Add, ast.BinOp, ast.Div, ast.Expression, ast.Load, ast.Mult, ast.Sub, ast.UAdd, ast.UnaryOp, ast.USub)
|
||||
@@ -70,16 +69,6 @@ VARIABLES = ("A", "G", "R", "S", "W")
|
||||
WEIGHTS_HEADER = "| Goal | W |"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Item:
|
||||
id: int
|
||||
release: tuple[int, int, int]
|
||||
exempt: str
|
||||
tagline: str
|
||||
scores: dict[str, int]
|
||||
goals: tuple[str, ...]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Scoring:
|
||||
formula: ast.Expression
|
||||
@@ -88,13 +77,6 @@ class Scoring:
|
||||
weights: dict[str, float]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Table:
|
||||
"""Body rows of a markdown table, and the index of the first row in the file's lines."""
|
||||
start: int
|
||||
rows: list[str]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Ranked:
|
||||
text: str
|
||||
@@ -357,23 +339,6 @@ def parse_weights(lines: list[str], goals: list[str]) -> dict[str, float] | Fail
|
||||
return weights
|
||||
|
||||
|
||||
def table_rows(lines: list[str], header: str) -> Table | Failure:
|
||||
if lines.count(header) != 1:
|
||||
return Failure(f"todo.md needs the table header {header} once")
|
||||
start = lines.index(header) + 2
|
||||
if start > len(lines) or not re.fullmatch(r"\|(-+\|)+", lines[start - 1].replace(" ", "")):
|
||||
return Failure(f"the table under {header} has no |---| divider line")
|
||||
end = start
|
||||
while end < len(lines) and lines[end].startswith("|"):
|
||||
end += 1
|
||||
return Table(start, lines[start:end])
|
||||
|
||||
|
||||
def cells(row: str, count: int) -> list[str] | Failure:
|
||||
found = [cell.strip() for cell in re.split(r"(?<!\\)\|", row.strip())[1:-1]]
|
||||
return found if len(found) == count else Failure(f"this row needs {count} cells: {row}")
|
||||
|
||||
|
||||
def parse_item(row: str, weights: dict[str, float]) -> Item | Failure:
|
||||
item = parse_row(row)
|
||||
if isinstance(item, Failure):
|
||||
@@ -383,30 +348,6 @@ def parse_item(row: str, weights: dict[str, float]) -> Item | Failure:
|
||||
return item
|
||||
|
||||
|
||||
def parse_row(row: str) -> Item | Failure:
|
||||
"""An item row, every cell checked but the goals it cites."""
|
||||
found = cells(row, 10)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
id_, release, exempt, tagline, r, s, a, g, goals, _ = found
|
||||
if not re.fullmatch(r"[0-9]+", id_) or int(id_) < 1:
|
||||
return Failure(f"the ID {id_!r} is not a positive whole number: {row}")
|
||||
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", release):
|
||||
return Failure(f"item {id_}'s release {release!r} is not a semver version")
|
||||
if exempt not in EXEMPTIONS:
|
||||
return Failure(f"item {id_}'s Exempt {exempt!r} is not empty or one of {', '.join(EXEMPTIONS[1:])}")
|
||||
if not re.fullmatch(r"\*\*[^*].*[^*]\*\*", tagline):
|
||||
return Failure(f"item {id_}'s Item cell is not one bold tagline: {tagline}")
|
||||
scores = {}
|
||||
for name, value in zip("RSAG", (r, s, a, g)):
|
||||
if not re.fullmatch(r"[0-9]+", value) or not 1 <= int(value) <= 10:
|
||||
return Failure(f"item {id_}'s {name} {value!r} is not a whole number from 1 to 10")
|
||||
scores[name] = int(value)
|
||||
cited = tuple(goal.strip() for goal in goals.split(","))
|
||||
major, minor, patch = (int(part) for part in release.split("."))
|
||||
return Item(int(id_), (major, minor, patch), exempt, tagline, scores, cited)
|
||||
|
||||
|
||||
def render_item(item: Item, value: float) -> str:
|
||||
release = ".".join(str(part) for part in item.release)
|
||||
scores = " | ".join(str(item.scores[name]) for name in "RSAG")
|
||||
|
||||
@@ -1,20 +0,0 @@
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
from loop_claude.transcript_archive import FinishedChunk
|
||||
from loop_claude.paths import paths_from
|
||||
from loop_claude.project.checkout import project_key
|
||||
|
||||
|
||||
def project_archive(checkout_root: str, home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
|
||||
paths = paths_from(home, env)
|
||||
return paths.transcript_archive / project_key(checkout_root, paths.code)
|
||||
|
||||
|
||||
def read_threads(chunk: FinishedChunk) -> list[bytes] | Failure:
|
||||
"""The worker's transcript, then each subagent's."""
|
||||
try:
|
||||
return [path.read_bytes() for path in (chunk.transcript, *chunk.subagents)]
|
||||
except OSError as failure:
|
||||
return Failure(f"could not read {failure.filename}: {failure.strerror}")
|
||||
@@ -3,12 +3,13 @@ import re
|
||||
import statistics
|
||||
|
||||
from failure import Failure
|
||||
from loop_claude.transcript_archive import FinishedChunk, STAMPED
|
||||
from transcript_archive.read import FinishedChunk
|
||||
from reading_cost.measure import Reading
|
||||
from markdown_sections import sections
|
||||
from markdown_text import sections
|
||||
from utc_stamp import STAMPED
|
||||
|
||||
FIGURE = re.compile(rf"^Reading cost, chunk (\w+), finished ({STAMPED})$")
|
||||
PLACE = re.compile(r"^- `(.+)` (\d+)$")
|
||||
FIGURE_PLACE = re.compile(r"^- `(.+)` (\d+)$")
|
||||
PLACES_KEPT = 10
|
||||
RESET = "Reading cost reset, "
|
||||
RESET_AT = re.compile(rf"^{RESET}({STAMPED})$")
|
||||
@@ -42,7 +43,7 @@ def readings_of(record: str) -> list[Reading] | Failure:
|
||||
tokens = [int(match.group(1)) for line in lines if (match := TOKENS.match(line))]
|
||||
if len(tokens) != 1:
|
||||
return Failure(f"the figure under \"## {heading}\" holds no single tokens line; restore it from git history")
|
||||
dated.append((named.group(2), heading, Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := PLACE.match(line))}))))
|
||||
dated.append((named.group(2), heading, Reading(tokens[0], collections.Counter({match.group(1): int(match.group(2)) for line in lines if (match := FIGURE_PLACE.match(line))}))))
|
||||
elif heading.startswith("Reading cost"):
|
||||
return Failure(f"\"## {heading}\" names no chunk and UTC time, or no reset time; restore it from git history")
|
||||
return [reading for finished, _, reading in sorted(dated, key=lambda item: item[:2]) if finished > max(resets)]
|
||||
|
||||
@@ -32,9 +32,10 @@ import time
|
||||
from reading_cost.measure import Checkout, measure, Reading
|
||||
from failure import Failure
|
||||
from reading_cost.figures import figure, merged, readings_of, RESET, trend, unheld
|
||||
from loop_claude.transcript_archive import finished_chunks, utc
|
||||
from reading_cost.archive import project_archive, read_threads
|
||||
from transcript_archive.read import finished_chunks, read_threads
|
||||
from transcript_archive.location import project_archive
|
||||
from file_writes import replace_file
|
||||
from utc_stamp import utc
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Updated:
|
||||
@@ -55,8 +56,8 @@ def main(argv: list[str]) -> int:
|
||||
if isinstance(checkout, Failure):
|
||||
return failed(checkout)
|
||||
home = pathlib.Path.home()
|
||||
archive = project_archive(checkout.root, home, os.environ)
|
||||
record = pathlib.Path(given[0]) if given else pathlib.Path(checkout.top, "docs", "reading-cost.md")
|
||||
archive = project_archive(checkout.main_root, home, os.environ)
|
||||
record = pathlib.Path(given[0]) if given else pathlib.Path(checkout.current_root, "docs", "reading-cost.md")
|
||||
updated = update(record, archive, checkout, str(home), utc(time.time()) if reset else None)
|
||||
if isinstance(updated, Failure):
|
||||
return failed(updated)
|
||||
@@ -121,8 +122,8 @@ def located() -> Checkout | Failure:
|
||||
return Failure(f"could not find the checkout: {failure}")
|
||||
if listed.returncode != 0:
|
||||
return Failure(listed.stderr.strip())
|
||||
# The common dir is the checkout's .git, whichever worktree runs this.
|
||||
return Checkout(str(pathlib.Path(common).parent), top, frozenset(filter(None, listed.stdout.split("\0"))))
|
||||
# The common dir is the main checkout's .git, whichever worktree runs this.
|
||||
return Checkout(top, str(pathlib.Path(common).parent), frozenset(filter(None, listed.stdout.split("\0"))))
|
||||
|
||||
|
||||
def read_record(record: pathlib.Path) -> str | Failure:
|
||||
|
||||
@@ -2,7 +2,7 @@ import collections
|
||||
import dataclasses
|
||||
import re
|
||||
|
||||
from loop_claude.claude_code.entries import assistant_parts, parse_entry, tool_call
|
||||
from claude_code.entries import assistant_parts, parse_entry, tool_call
|
||||
from reading_cost.shell_reads import Reads, resolved, shell_reads
|
||||
|
||||
COMMIT = re.compile(r"(^|[\s;&|(])git\s+(-C\s+\S+\s+)?commit\b")
|
||||
@@ -16,11 +16,11 @@ class Reading:
|
||||
places: collections.Counter[str]
|
||||
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Checkout:
|
||||
root: str
|
||||
top: str
|
||||
"""main_root is the main checkout, whose path keys the archive; current_root is the checkout or worktree this runs in."""
|
||||
current_root: str
|
||||
main_root: str
|
||||
tracked: frozenset[str]
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ def measure(threads: list[bytes], checkout: Checkout, home: str) -> Reading | No
|
||||
return None
|
||||
before = [[entry for entry in thread if entry.get("type") == "assistant" and str(entry.get("timestamp", "")) < commit] for thread in (worker, *subagents)]
|
||||
reads = [reads_of(part, str(entry.get("cwd") or "/"), home) for thread in before for entry in thread for part in assistant_parts(entry)]
|
||||
roots = [checkout.root, *(added for found in reads for added in found.added)]
|
||||
roots = [checkout.main_root, *(added for found in reads for added in found.added)]
|
||||
places = collections.Counter(place for found in reads for path in found.read if (place := relative(path, roots)) in checkout.tracked)
|
||||
return Reading(sum(map(tokens_in, before)), places)
|
||||
|
||||
|
||||
@@ -10,17 +10,19 @@ import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from loop_claude.transcript_archive import archive, FINISHED, FINISHED_AT, finished_chunks, FinishedChunk, UNFINISHED, utc
|
||||
from transcript_archive.write import archive
|
||||
from reading_cost.measure import Checkout, Reading
|
||||
from failure import Failure
|
||||
from reading_cost import figures, measure
|
||||
from test_loop_claude_black_box import host_env
|
||||
from loop_claude.paths import paths_from
|
||||
from transcript_archive.read import finished_chunks, FinishedChunk
|
||||
from test_env import isolated_env
|
||||
from transcript_archive import outcomes
|
||||
from transcript_archive.location import project_archive
|
||||
from reading_cost.shell_reads import Reads, shell_reads
|
||||
|
||||
TOOL = pathlib.Path(__file__).resolve().parent / "reading-cost"
|
||||
TOOL = pathlib.Path(__file__).resolve().parents[1] / "reading-cost"
|
||||
|
||||
|
||||
def said(timestamp, message_id, command=None, tokens=0, cwd="/c/p", **call):
|
||||
@@ -59,7 +61,7 @@ class Measure(unittest.TestCase):
|
||||
said("6", "m5", tokens=100, name="Read", input={"file_path": "/c/p/late.py"}),
|
||||
]
|
||||
subagent = [said("2", "s1", tokens=1, name="Grep", input={"path": "/c/w/src/a.py"}), said("7", "s2", tokens=100, name="Read", input={"file_path": "/c/p/late.py"})]
|
||||
checkout = Checkout("/c/p", "/c/w", frozenset({"README.md", "late.py", "src/a.py", "src/b.py", "x.py"}))
|
||||
checkout = Checkout("/c/w", "/c/p", frozenset({"README.md", "late.py", "src/a.py", "src/b.py", "x.py"}))
|
||||
self.assertEqual(measure.measure([jsonl(worker), jsonl(subagent)], checkout, "/h"), Reading(33, collections.Counter({"src/a.py": 2, "README.md": 1, "src/b.py": 1})))
|
||||
|
||||
def test_measures_nothing_for_a_worker_that_never_committed(self):
|
||||
@@ -94,35 +96,32 @@ class Figures(unittest.TestCase):
|
||||
history = "\n".join([written("a", "2026-10-01T00:00:00Z", 1), written("c", "2026-10-03T00:00:00Z", 3), "## Reading cost reset, 2026-10-02T00:00:00Z\n", written("b", "2026-10-04T00:00:00Z", 2)])
|
||||
self.assertEqual(figures.readings_of(history), [Reading(3, collections.Counter()), Reading(2, collections.Counter())])
|
||||
|
||||
def test_the_archive_stamps_what_it_reads_back(self):
|
||||
self.assertRegex(f"{utc(1790000000)} {FINISHED}", FINISHED_AT)
|
||||
|
||||
|
||||
class ReadingCostTool(unittest.TestCase):
|
||||
def setUp(self):
|
||||
temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temp.cleanup)
|
||||
self.home = pathlib.Path(os.path.realpath(temp.name))
|
||||
self.env = {**host_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.home)}
|
||||
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.home)}
|
||||
self.checkout = self.home / "code" / "proj"
|
||||
self.checkout.mkdir(parents=True)
|
||||
(self.checkout / "a.py").write_text("")
|
||||
self.git("init", "-q")
|
||||
self.git("add", "a.py")
|
||||
self.git("-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "-m", "x")
|
||||
self.paths = paths_from(self.home, {})
|
||||
self.archive = project_archive(str(self.checkout), self.home, {})
|
||||
|
||||
def git(self, *args):
|
||||
subprocess.run(["git", *args], cwd=self.checkout, check=True, env=self.env)
|
||||
|
||||
def archive_chunk(self, at, outcome=FINISHED):
|
||||
def archive_chunk(self, at, outcome=outcomes.finished):
|
||||
transcript = self.home / ".claude" / "projects" / "-proj" / f"{folder_name()}.jsonl"
|
||||
(transcript.with_suffix("") / "subagents").mkdir(parents=True)
|
||||
worker = [said("1", "m1", tokens=10, cwd=str(self.checkout), name="Read", input={"file_path": str(self.checkout / "a.py")}), said("2", "m2", command="git commit -m x")]
|
||||
transcript.write_bytes(jsonl([{"timestamp": str(at)}, *worker]))
|
||||
(transcript.with_suffix("") / "subagents" / "agent-a.jsonl").write_bytes(jsonl([said("1", "s1", tokens=1)]))
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(archive("proj", str(transcript), "Do it.", f"{outcome}\n\nEvidence:\n> merged", at, self.paths))
|
||||
self.assertIsNone(archive(self.archive, str(transcript), "Do it.", outcome("merged"), at))
|
||||
|
||||
def run_tool(self, cwd, *args):
|
||||
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=cwd, env=self.env, text=True)
|
||||
@@ -130,8 +129,8 @@ class ReadingCostTool(unittest.TestCase):
|
||||
def test_appends_each_finished_chunk_once_in_the_order_they_finished_from_a_worktree(self):
|
||||
self.archive_chunk(1790000200)
|
||||
self.archive_chunk(1790000100)
|
||||
self.archive_chunk(1790000000, UNFINISHED)
|
||||
chunks = finished_chunks(self.paths.transcript_archive / "proj")
|
||||
self.archive_chunk(1790000000, outcomes.unfinished)
|
||||
chunks = finished_chunks(self.archive)
|
||||
self.assertEqual([chunk.finished for chunk in chunks], ["2026-09-21T14:15:00Z", "2026-09-21T14:16:40Z"])
|
||||
worktree = self.home / "code" / "proj-wt"
|
||||
self.git("worktree", "add", "-q", str(worktree))
|
||||
@@ -1,9 +1,13 @@
|
||||
import pathlib
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from repo_checks.test_python_floor import ROOT
|
||||
|
||||
CITATION = re.compile(r'`?CLAUDE\.md`? → ("[^"]+"(?:,? (?:and )?"[^"]+")*)')
|
||||
HEADING = re.compile(r"^#+ (.+?)(?: \(.*\))?$", re.MULTILINE)
|
||||
LEAD = re.compile(r"^(?:- )?\*\*(.+?)(?:\*\*|[:.](?=\*\*| )| —)", re.MULTILINE)
|
||||
@@ -1,7 +1,11 @@
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from test_python_floor import ROOT
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from repo_checks.test_python_floor import ROOT
|
||||
|
||||
PRIVATE = [
|
||||
".claude/settings.local.json",
|
||||
@@ -1,8 +1,12 @@
|
||||
import ast
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from test_python_floor import ROOT
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from repo_checks.test_python_floor import ROOT
|
||||
|
||||
|
||||
def bound(node: ast.Import | ast.ImportFrom) -> list[str]:
|
||||
@@ -1,7 +1,11 @@
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from test_python_floor import ROOT
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from repo_checks.test_python_floor import ROOT
|
||||
|
||||
|
||||
class Install(unittest.TestCase):
|
||||
@@ -5,7 +5,7 @@ import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[2]
|
||||
FLOOR = re.compile(r"Python 3\.(\d+) or newer")
|
||||
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
import pathlib
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from repo_checks.test_python_floor import readme_floor, ROOT
|
||||
|
||||
|
||||
class Types(unittest.TestCase):
|
||||
def test_mypy_passes_on_the_tools_at_the_floor(self):
|
||||
floor = readme_floor()
|
||||
self.assertIsNotNone(floor, "README.md's Install names `Python 3.N or newer`")
|
||||
if shutil.which("uv") is None:
|
||||
self.fail("uv on PATH fetches mypy; see README.md's Test line")
|
||||
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "--exclude", "/test_", "-p", "claude_code", "-m", "comprehension_history", "-p", "larv_review", "-p", "loop_claude", "-m", "markdown_text", "-p", "metric_bench", "-m", "project_key", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline", "-m", "todo_table", "-p", "transcript_archive", "-m", "utc_stamp"], capture_output=True, cwd=ROOT / "bin", text=True)
|
||||
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
|
||||
@@ -0,0 +1,6 @@
|
||||
import os
|
||||
|
||||
|
||||
# A git hook's GIT_DIR, the host's LOOP_CLAUDE_* or PYTHONPATH would point the program under test at the host's own state.
|
||||
def isolated_env() -> dict[str, str]:
|
||||
return {name: value for name, value in os.environ.items() if not name.startswith(("GIT_", "LOOP_CLAUDE_", "PYTHON"))}
|
||||
@@ -4,18 +4,21 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
|
||||
|
||||
import markdown_sections
|
||||
import markdown_text
|
||||
|
||||
|
||||
class Sections(unittest.TestCase):
|
||||
def test_keeps_every_headed_section_with_its_bodies(self):
|
||||
text = "# Title\n\nintro\n\n## Goals\n\n1. one\n```sh\n# not a heading\n```\n### Later\n2. two\n#1 priority\n\n## Empty\n\n## Audience\nme\n## Audience\nagain\n# Appendix\nnot audience\n"
|
||||
self.assertEqual(markdown_sections.sections(text), {"Goals": ["1. one\n```sh\n# not a heading\n```\n### Later\n2. two\n#1 priority"], "Empty": [""], "Audience": ["me", "again"]})
|
||||
self.assertEqual(markdown_text.sections(text), {"Goals": ["1. one\n```sh\n# not a heading\n```\n### Later\n2. two\n#1 priority"], "Empty": [""], "Audience": ["me", "again"]})
|
||||
|
||||
def test_closes_a_fence_only_on_its_own_marker(self):
|
||||
text = "## Goals\n````md\n~~~\n## Quoted\n```\n````\n ~~~\n## Also quoted\n ~~~\n1. one\n## Audience\nme\n"
|
||||
self.assertEqual(list(markdown_sections.sections(text)), ["Goals", "Audience"])
|
||||
self.assertEqual(list(markdown_sections.sections("```\nquoted:\n```python\n## Inside\n```\n## Goals\n1. one\n")), ["Goals"])
|
||||
self.assertEqual(list(markdown_text.sections(text)), ["Goals", "Audience"])
|
||||
self.assertEqual(list(markdown_text.sections("```\nquoted:\n```python\n## Inside\n```\n## Goals\n1. one\n")), ["Goals"])
|
||||
|
||||
def test_quotes_every_line_of_a_passage_blank_ones_included(self):
|
||||
self.assertEqual(markdown_text.quoted("merged\n\nPR #1"), "> merged\n> \n> PR #1")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
@@ -9,7 +9,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
|
||||
|
||||
from failure import Failure
|
||||
import rank_todo
|
||||
from test_python_floor import ROOT
|
||||
from repo_checks.test_python_floor import ROOT
|
||||
|
||||
README = "# P\n\n## Goals\n\n1. Fast.\n2. Correct.\n 1. Tested.\n\n## Audience\n\nMe.\n"
|
||||
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
import shutil
|
||||
import subprocess
|
||||
import unittest
|
||||
|
||||
from test_python_floor import readme_floor, ROOT
|
||||
|
||||
|
||||
class Types(unittest.TestCase):
|
||||
def test_mypy_passes_on_the_tools_at_the_floor(self):
|
||||
floor = readme_floor()
|
||||
self.assertIsNotNone(floor, "README.md's Install names `Python 3.N or newer`")
|
||||
if shutil.which("uv") is None:
|
||||
self.fail("uv on PATH fetches mypy; see README.md's Test line")
|
||||
ran = subprocess.run(["uv", "tool", "run", "--python", "%d.%d" % floor, "mypy==2.3.1", "--cache-dir=/dev/null", "--check-untyped-defs", "--disallow-untyped-defs", "-m", "comprehension_history", "-p", "larv_review", "-p", "loop_claude", "-p", "metric_bench", "-m", "rank_todo", "-p", "reading_cost", "-m", "statusline"], capture_output=True, cwd=ROOT / "bin", text=True)
|
||||
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
|
||||
@@ -0,0 +1,79 @@
|
||||
import dataclasses
|
||||
import re
|
||||
|
||||
from failure import Failure
|
||||
|
||||
EXEMPTIONS = ("", "decision", "defect", "principle", "question", "security")
|
||||
ITEMS_HEADER = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Item:
|
||||
id: int
|
||||
release: tuple[int, int, int]
|
||||
exempt: str
|
||||
tagline: str
|
||||
scores: dict[str, int]
|
||||
goals: tuple[str, ...]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Table:
|
||||
"""Body rows of a markdown table, and the index of the first row in the file's lines."""
|
||||
start: int
|
||||
rows: list[str]
|
||||
|
||||
|
||||
def items(todo: str) -> list[Item] | Failure:
|
||||
"""Each row of a todo.md's items table."""
|
||||
table = table_rows(todo.splitlines(), ITEMS_HEADER)
|
||||
if isinstance(table, Failure):
|
||||
return table
|
||||
items = []
|
||||
for row in table.rows:
|
||||
item = parse_row(row)
|
||||
if isinstance(item, Failure):
|
||||
return item
|
||||
items.append(item)
|
||||
return items
|
||||
|
||||
|
||||
def table_rows(lines: list[str], header: str) -> Table | Failure:
|
||||
if lines.count(header) != 1:
|
||||
return Failure(f"todo.md needs the table header {header} once")
|
||||
start = lines.index(header) + 2
|
||||
if start > len(lines) or not re.fullmatch(r"\|(-+\|)+", lines[start - 1].replace(" ", "")):
|
||||
return Failure(f"the table under {header} has no |---| divider line")
|
||||
end = start
|
||||
while end < len(lines) and lines[end].startswith("|"):
|
||||
end += 1
|
||||
return Table(start, lines[start:end])
|
||||
|
||||
|
||||
def cells(row: str, count: int) -> list[str] | Failure:
|
||||
found = [cell.strip() for cell in re.split(r"(?<!\\)\|", row.strip())[1:-1]]
|
||||
return found if len(found) == count else Failure(f"this row needs {count} cells: {row}")
|
||||
|
||||
|
||||
def parse_row(row: str) -> Item | Failure:
|
||||
"""An item row, every cell checked but the goals it cites."""
|
||||
found = cells(row, 10)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
id_, release, exempt, tagline, r, s, a, g, goals, _ = found
|
||||
if not re.fullmatch(r"[0-9]+", id_) or int(id_) < 1:
|
||||
return Failure(f"the ID {id_!r} is not a positive whole number: {row}")
|
||||
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", release):
|
||||
return Failure(f"item {id_}'s release {release!r} is not a semver version")
|
||||
if exempt not in EXEMPTIONS:
|
||||
return Failure(f"item {id_}'s Exempt {exempt!r} is not empty or one of {', '.join(EXEMPTIONS[1:])}")
|
||||
if not re.fullmatch(r"\*\*[^*].*[^*]\*\*", tagline):
|
||||
return Failure(f"item {id_}'s Item cell is not one bold tagline: {tagline}")
|
||||
scores = {}
|
||||
for name, value in zip("RSAG", (r, s, a, g)):
|
||||
if not re.fullmatch(r"[0-9]+", value) or not 1 <= int(value) <= 10:
|
||||
return Failure(f"item {id_}'s {name} {value!r} is not a whole number from 1 to 10")
|
||||
scores[name] = int(value)
|
||||
cited = tuple(goal.strip() for goal in goals.split(","))
|
||||
major, minor, patch = (int(part) for part in release.split("."))
|
||||
return Item(int(id_), (major, minor, patch), exempt, tagline, scores, cited)
|
||||
@@ -0,0 +1,9 @@
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from project_key import project_key
|
||||
|
||||
|
||||
def project_archive(checkout_root: str, home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
|
||||
archive = pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else home / ".claude" / "loop-claude-transcripts"
|
||||
return archive / project_key(checkout_root, home)
|
||||
@@ -0,0 +1,27 @@
|
||||
import re
|
||||
|
||||
from markdown_text import quoted
|
||||
from utc_stamp import STAMPED
|
||||
|
||||
FINISHED = "Verdict: finished"
|
||||
FINISHED_AT = re.compile(rf"^({STAMPED}) {re.escape(FINISHED)}$", re.MULTILINE)
|
||||
|
||||
|
||||
def finished(evidence: str) -> str:
|
||||
return f"{FINISHED}\n\nEvidence:\n{quoted(evidence)}"
|
||||
|
||||
|
||||
def unfinished(evidence: str) -> str:
|
||||
return f"Verdict: unfinished\n\nEvidence:\n{quoted(evidence)}"
|
||||
|
||||
|
||||
def interrupted(question: str, evidence: str) -> str:
|
||||
return f"Verdict: interrupt\n\n{question}\n\nEvidence:\n{quoted(evidence)}"
|
||||
|
||||
|
||||
def answered(question: str, sources: str, answer: str, evidence: str) -> str:
|
||||
return f"Verdict: quoted answer\n\n{question}\n\n{sources} says:\n{quoted(answer)}\n\nEvidence:\n{quoted(evidence)}"
|
||||
|
||||
|
||||
def stopped(why: str) -> str:
|
||||
return f"Stopped: {why}"
|
||||
@@ -0,0 +1,35 @@
|
||||
import dataclasses
|
||||
import pathlib
|
||||
|
||||
from failure import Failure
|
||||
from transcript_archive.outcomes import FINISHED_AT
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class FinishedChunk:
|
||||
folder: str
|
||||
finished: str
|
||||
transcript: pathlib.Path
|
||||
subagents: list[pathlib.Path]
|
||||
|
||||
|
||||
# One per chunk folder, its latest finished copy, in the order they finished.
|
||||
def finished_chunks(project_archive: pathlib.Path) -> list[FinishedChunk] | Failure:
|
||||
found = []
|
||||
try:
|
||||
for folder in sorted(path for path in project_archive.glob("*") if path.is_dir()):
|
||||
copies = sorted((stamps[-1], outcomes) for outcomes in folder.glob("*.outcomes.md") if (stamps := FINISHED_AT.findall(outcomes.read_text(encoding="utf-8"))))
|
||||
if copies:
|
||||
finished, outcomes = copies[-1]
|
||||
found.append(FinishedChunk(folder.name, finished, outcomes.with_name(outcomes.name.removesuffix(".outcomes.md") + ".jsonl"), sorted(folder.glob("session/*/subagents/*.jsonl"))))
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read the archive {project_archive}: {failure}")
|
||||
return sorted(found, key=lambda chunk: (chunk.finished, chunk.folder))
|
||||
|
||||
|
||||
def read_threads(chunk: FinishedChunk) -> list[bytes] | Failure:
|
||||
"""The worker's transcript, then each subagent's."""
|
||||
try:
|
||||
return [path.read_bytes() for path in (chunk.transcript, *chunk.subagents)]
|
||||
except OSError as failure:
|
||||
return Failure(f"could not read {failure.filename}: {failure.strerror}")
|
||||
@@ -0,0 +1,61 @@
|
||||
import contextlib
|
||||
import hashlib
|
||||
import io
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from transcript_archive import outcomes, write
|
||||
from transcript_archive.location import project_archive
|
||||
from project_key import project_key
|
||||
from utc_stamp import utc
|
||||
|
||||
NOW = 1790000000
|
||||
|
||||
|
||||
class Archive(unittest.TestCase):
|
||||
def test_writes_afresh_a_copy_whose_outcomes_file_is_missing(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
kept = pathlib.Path(root, "archive")
|
||||
transcript = pathlib.Path(root, "t.jsonl")
|
||||
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.unfinished("working"), NOW))
|
||||
(kept_outcomes,) = kept.glob("*/*.outcomes.md")
|
||||
kept_outcomes.unlink()
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.stopped("Ctrl-C"), NOW))
|
||||
self.assertEqual(kept_outcomes.read_text(), "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n")
|
||||
|
||||
|
||||
class CopyFor(unittest.TestCase):
|
||||
def test_skips_a_version_already_kept_beside_its_copy(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
copy = pathlib.Path(temp, "custom-title.json")
|
||||
copy.write_bytes(b"first")
|
||||
beside = write.copy_for(copy, b"second")
|
||||
self.assertEqual(beside, copy.with_name(f"custom-title.json.{hashlib.sha256(b'second').hexdigest()}"))
|
||||
beside.write_bytes(b"second")
|
||||
self.assertIsNone(write.copy_for(copy, b"second"))
|
||||
self.assertEqual(write.copy_for(copy, b"first, grown"), copy)
|
||||
|
||||
|
||||
class Outcomes(unittest.TestCase):
|
||||
def test_the_archive_stamps_what_it_reads_back(self):
|
||||
self.assertRegex(f"{utc(NOW)} {outcomes.finished('merged')}", outcomes.FINISHED_AT)
|
||||
|
||||
|
||||
class Location(unittest.TestCase):
|
||||
def test_keys_a_project_by_its_path_under_code_or_else_under_the_root(self):
|
||||
home = pathlib.Path("/h")
|
||||
self.assertEqual(project_key("/h/code/log", home), "log")
|
||||
self.assertEqual(project_key("/h/code", home), "h/code")
|
||||
self.assertEqual(project_key("/srv/log", home), "srv/log")
|
||||
self.assertEqual(project_archive("/h/code/log", home, {}), pathlib.Path("/h/.claude/loop-claude-transcripts/log"))
|
||||
self.assertEqual(project_archive("/h/code/log", home, {"LOOP_CLAUDE_TRANSCRIPTS": "/a"}), pathlib.Path("/a/log"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,38 +1,21 @@
|
||||
import dataclasses
|
||||
import hashlib
|
||||
import os
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
|
||||
from failure import Failure
|
||||
from loop_claude.claude_code.transcript import first_timestamped_line
|
||||
from claude_code.transcript import first_timestamped_line
|
||||
from file_writes import make_private, replace_file
|
||||
from loop_claude.paths import Paths
|
||||
from loop_claude.claude_code.session import session_files
|
||||
|
||||
FINISHED = "Verdict: finished"
|
||||
UNFINISHED = "Verdict: unfinished"
|
||||
STAMP = "%Y-%m-%dT%H:%M:%SZ"
|
||||
STAMPED = r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ"
|
||||
FINISHED_AT = re.compile(rf"^({STAMPED}) {re.escape(FINISHED)}$", re.MULTILINE)
|
||||
from claude_code.session import session_files
|
||||
from utc_stamp import utc
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class FinishedChunk:
|
||||
folder: str
|
||||
finished: str
|
||||
transcript: pathlib.Path
|
||||
subagents: list[pathlib.Path]
|
||||
|
||||
|
||||
def archive(project_key: str, path: str, task: str, outcome: str, at: float, paths: Paths) -> Failure | None:
|
||||
def archive(project_archive: pathlib.Path, path: str, task: str, outcome: str, at: float) -> Failure | None:
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
raw = handle.read()
|
||||
# Each chunk's first timestamped line is its own: a finished chunk's session is deleted, so the next chunk's transcript starts afresh.
|
||||
folder = paths.transcript_archive / project_key / hashlib.sha256(first_timestamped_line(raw)).hexdigest()
|
||||
folder = project_archive / hashlib.sha256(first_timestamped_line(raw)).hexdigest()
|
||||
make_private(folder)
|
||||
except OSError as failure:
|
||||
return unarchived(failure.filename, failure.strerror)
|
||||
@@ -101,21 +84,3 @@ def copy_for(copy: pathlib.Path, data: bytes) -> pathlib.Path | None:
|
||||
return copy if data != kept else None
|
||||
beside = copy.with_name(f"{copy.name}.{hashlib.sha256(data).hexdigest()}")
|
||||
return None if beside.exists() else beside
|
||||
|
||||
|
||||
def utc(at: float) -> str:
|
||||
return time.strftime(STAMP, time.gmtime(at))
|
||||
|
||||
|
||||
# One per chunk folder, its latest finished copy, in the order they finished.
|
||||
def finished_chunks(project_archive: pathlib.Path) -> list[FinishedChunk] | Failure:
|
||||
found = []
|
||||
try:
|
||||
for folder in sorted(path for path in project_archive.glob("*") if path.is_dir()):
|
||||
copies = sorted((stamps[-1], outcomes) for outcomes in folder.glob("*.outcomes.md") if (stamps := FINISHED_AT.findall(outcomes.read_text(encoding="utf-8"))))
|
||||
if copies:
|
||||
finished, outcomes = copies[-1]
|
||||
found.append(FinishedChunk(folder.name, finished, outcomes.with_name(outcomes.name.removesuffix(".outcomes.md") + ".jsonl"), sorted(folder.glob("session/*/subagents/*.jsonl"))))
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read the archive {project_archive}: {failure}")
|
||||
return sorted(found, key=lambda chunk: (chunk.finished, chunk.folder))
|
||||
@@ -0,0 +1,8 @@
|
||||
import time
|
||||
|
||||
STAMP = "%Y-%m-%dT%H:%M:%SZ"
|
||||
STAMPED = r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ"
|
||||
|
||||
|
||||
def utc(at: float) -> str:
|
||||
return time.strftime(STAMP, time.gmtime(at))
|
||||
@@ -732,3 +732,399 @@ Inherited architect, would restructure first:
|
||||
- `bin/larv_review/git.py:10` `git`, `bin/metric_bench/main.py:143` `git` and `bin/reading_cost/main.py:113` `located`: one git runner
|
||||
- `bin/loop_claude/project/checkout.py:32` `read_utf8`: a generic reader under a lying name
|
||||
- `bin/larv_review/main.py:101` `main`: an if/elif dispatch where the next command lands
|
||||
|
||||
## 2026-10-09T12:31:47Z, PR #234 at b1913ed, against 2157f59
|
||||
|
||||
Ruling: better
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | better | better | same | better | better |
|
||||
| Inherited architect | better | better | better | better | better |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Locality, Navigation: `bin/transcript_archive/outcomes.py:4` `FINISHED_AT` and `finished`: one module now writes and parses the outcome lines, which `bin/loop_claude/cycle/run.py:83` built inline and another file parsed
|
||||
- Locality, Navigation: `bin/reading_cost/main.py:35` imports: reading-cost depends only on the shared `transcript_archive`, no longer on loop-claude's internals
|
||||
- Locality: `bin/loop_claude/paths.py:15` `paths_from`: no longer reads the environment; `LOOP_CLAUDE_TRANSCRIPTS` has one home in `bin/transcript_archive/location.py:5` `project_archive`
|
||||
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the import root, the shared packages and where tests live
|
||||
- Self-sufficiency: `bin/transcript_archive/write.py:13` `archive`: states the archive's layout at the write
|
||||
- Shape: `bin/loop_claude/loop/main.py:86` `run_loop`: imports `project_key`, which names the session and the models entry, from the archive package; `bin/loop_claude/cycle/worker.py:7` takes `quoted` from the archive's outcomes
|
||||
- Shape: `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES`: judge_reply merged in, 183 lines
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Navigation, Shape: `bin/reading_cost/main.py:35` `main`: imports one shared package named for what it holds, in place of loop-claude's private packages
|
||||
- Locality, Self-sufficiency: `bin/transcript_archive/outcomes.py:10` `finished`: the outcome texts sit beside `FINISHED_AT`, the regex that reads them back
|
||||
- Shape: `bin/claude_code/session.py:21` `session_named`: the Claude Code package depends on nothing in loop-claude
|
||||
- Locality: `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES` and `:93` `verdict_from`: in one file with `run_judge`
|
||||
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the cap no longer depends on case order
|
||||
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the layout and puts each test beside its package
|
||||
- Self-sufficiency: `bin/loop_claude/cycle/run.py:34` `Cycle.unfinished_evidence`: the name says what `remaining` needed a comment for
|
||||
- Shape: `bin/loop_claude/loop/main.py:86` `run_loop`, `bin/loop_claude/cycle/worker.py:7` `worker_args` and `bin/host_env.py:5` `host_env`: small seams against the decoupling
|
||||
|
||||
## 2026-10-09T12:32:00Z, PR #234 at b1913ed
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Junior A | 7 | 6 | 6.5 | 6 | 6.5 |
|
||||
| Mid A | 7 | 6 | 6.5 | 6 | 6.5 |
|
||||
| Maintainability senior | 7 | 6.5 | 7 | 6.5 | 7 |
|
||||
| Inherited architect | 7 | 6.5 | 6 | 6.5 | 6.5 |
|
||||
| Mean | 7.00 | 6.25 | 6.50 | 6.25 | 6.63 |
|
||||
|
||||
Junior A, hardest first:
|
||||
|
||||
- `skills/larv-review/SKILL.md:55` "Round cap": the counting rule spread across Round cap, Re-entry, Fresh and Suspended
|
||||
- `bin/larv_review/main.py:197` `record_ruling`: split, fell, refuse and stand branches; `ruling` returns None for a split and `fallen` reuses it
|
||||
- `bin/larv_review/scoring.py:54` `scoring_run_due`: why the next-release fallback holds is stated nowhere
|
||||
- `bin/larv_review/base.py:14` `measured_commit`: mechanics without purpose
|
||||
- `bin/loop_claude/cycle/judge.py:126` `run_judge`: the prompt assembled from `records.py` constants; two texts, two matchers
|
||||
- `bin/reading_cost/measure.py:21` `Checkout`: `root` and `top` read as synonyms
|
||||
|
||||
Mid A, hardest first:
|
||||
|
||||
- `bin/larv_review/main.py:197` `record_ruling`: said and counted, None as split, three fall steps
|
||||
- `bin/comprehension_history.py:74` `entries`: the history format is defined only by its writer
|
||||
- `bin/loop_claude/cycle/judge.py:126` `run_judge`: elided evidence fails, a misquoted answer becomes an interrupt
|
||||
- `bin/larv_review/scoring.py:54` `scoring_run_due`: one dense sentence over two plans; the module also parses the plan
|
||||
- `bin/larv_review/base.py:14` `measured_commit`: the why lives in the help and the skill
|
||||
- `bin/loop_claude/cycle/states.py:69` `transition`: `quoted_answers` threaded through three states and two files
|
||||
|
||||
Maintainability senior, hardest first:
|
||||
|
||||
- `bin/larv_review/main.py:197` `record_ruling`: said and count, ruled reassigned, a silent fall-through; `ruling`'s None means two things
|
||||
- `bin/larv_review/base.py:14` `measured_commit`: a three-step fallback across three files, ordered by an ISO string unsaid
|
||||
- `bin/comprehension_history.py:74` `entries`: the format is never stated in one place
|
||||
- `bin/loop_claude/cycle/judge.py:15` `JUDGE_RULES`: the prompt interpolates four constants from another module
|
||||
- `bin/larv_review/scoring.py:54` `scoring_run_due`: five inputs; `scoring` also parses the plan
|
||||
- `bin/rank_todo.py:416` `rank_details`: regex surgery in a script larv-review imports as a library
|
||||
|
||||
Inherited architect, would restructure first:
|
||||
|
||||
- `bin/loop_claude/project/records.py:6` `QUOTE_SOURCES`: prompt fragments spliced into the judge, worker and interrupt texts
|
||||
- `bin/transcript_archive/location.py:10` `project_key`: the project's identity filed under the archive
|
||||
- `bin/host_env.py:5` `host_env`: a test helper among the production modules
|
||||
- `bin/larv_review/rounds.py:8` `round_counts`: only counts; the cap lives in the skill
|
||||
- `bin/larv_review/base.py:1` the module: the ruling base, named like a base class
|
||||
- `bin/larv_review/scoring.py:6` imports: one tool imports another tool's internals from `bin/rank_todo.py`
|
||||
- `bin/loop_claude/loop/main.py:1` the docstring: restates constants it cannot keep true
|
||||
|
||||
## 2026-10-09T12:46:51Z, PR #234 at dde06fc, against 2157f59
|
||||
|
||||
Ruling: better
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | better | better | better | better | better |
|
||||
| Inherited architect | better | better | better | better | better |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Navigation, Locality, Shape: `bin/transcript_archive/location.py:7` `project_archive`: the archive's location lives in one shared package, where reading-cost reached into loop-claude's internals
|
||||
- Locality, Shape, Self-sufficiency: `bin/loop_claude/cycle/run.py:83` `worker_start`: `worker_args` builds only flags and each prompt is a named function in `bin/loop_claude/cycle/prompts.py:41`
|
||||
- Locality, Self-sufficiency: `bin/transcript_archive/outcomes.py:10` `finished`: the outcomes format's writer and reader sit in one file
|
||||
- Shape: `bin/larv_review/ruling.py:42` `split`: `ruling` no longer returns None for a split
|
||||
- Shape, Locality: `bin/larv_review/main.py:204` `record_ruling`: two plain checks in place of `fall_step`'s strings; plan logic in `bin/larv_review/plan.py:5`
|
||||
- Navigation: `README.md:77` "Tools": states the import root and the shared packages
|
||||
- Self-sufficiency: `bin/comprehension_history.py:74` `entries`: a worked example of an entry above the parser
|
||||
- Self-sufficiency, Shape: `bin/metric_bench/bench.py:88` `tracked_places`: the rename garbled the docstring
|
||||
- Locality: `bin/loop_claude/cycle/prompts.py:60` `quoted`: duplicates `bin/transcript_archive/outcomes.py:31` `quoted`
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Navigation, Locality: `bin/transcript_archive/outcomes.py:10` `finished`: every outcome line and `FINISHED_AT` sit in one module
|
||||
- Navigation, Locality: `bin/reading_cost/main.py:35` imports: reading-cost no longer reaches into loop-claude's internals
|
||||
- Navigation, Self-sufficiency: `README.md:77` "Tools": the layout is stated
|
||||
- Shape, Locality: `bin/todo_table.py:27` `items`: the table format is its own module, not a CLI script used as a library
|
||||
- Shape: `bin/larv_review/plan.py:5` `scoring_run_due`: the plan trigger with its why, apart from the floor arithmetic
|
||||
- Locality, Shape: `bin/larv_review/ruling.py:42` `split`: split, ruling and lead are separate
|
||||
- Locality, Shape: `bin/loop_claude/cycle/prompts.py:41` `fresh`: every prompt text in one module
|
||||
- Shape: `bin/comprehension_history.py:67` `ScoringRun`: one name no longer covers two concepts
|
||||
- Self-sufficiency: `bin/larv_review/main.py:5` the help: defines the ratchet, the paired ruling and the scoring run first
|
||||
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the past-the-cap case has its own guard
|
||||
- Shape: `bin/metric_bench/bench.py:87` `tracked_places`: the docstring reads worse after the rename
|
||||
|
||||
## 2026-10-09T12:46:51Z, PR #234 at dde06fc
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Junior A | 7 | 6.5 | 6 | 6 | 6 |
|
||||
| Mid A | 7.5 | 6.5 | 7 | 6.5 | 7 |
|
||||
| Maintainability senior | 7 | 6.5 | 7 | 7 | 7 |
|
||||
| Inherited architect | 7 | 6.5 | 6.5 | 7 | 7 |
|
||||
| Mean | 7.13 | 6.50 | 6.63 | 6.63 | 6.75 |
|
||||
|
||||
Junior A, hardest first:
|
||||
|
||||
- `bin/larv_review/main.py:204` `record_ruling`: six outcomes, said against count, `[-1]` meaning Overall
|
||||
- `bin/larv_review/ruling.py:8` `votes`: a parser that also enforces the seat-count rule
|
||||
- `bin/larv_review/ruling_base.py:14` `measured_commit`: the docstring explains only after the code confused
|
||||
- `bin/larv_review/plan.py:5` `scoring_run_due`: four ideas at once; `emptied` traced by hand
|
||||
- `bin/metric_bench/bench.py:87` `tracked_places`: a garbled docstring
|
||||
- `bin/loop_claude/cycle/states.py:69` `transition`: the quoted-answer cap across three files
|
||||
|
||||
Mid A, hardest first:
|
||||
|
||||
- `skills/larv-review/SKILL.md:55` "Round cap": four subsections pointing at each other
|
||||
- `bin/larv_review/main.py:204` `record_ruling`: five outcomes; the duplicate check in two places
|
||||
- `bin/loop_claude/cycle/judge.py:95` `run_judge`: many endings, two quote checks
|
||||
- `bin/loop_claude/cycle/states.py:69` `transition`: the cap counter rides through three states
|
||||
- `bin/larv_review/ruling_base.py:14` `measured_commit`: needs the history format and first-parent semantics
|
||||
- `bin/reading_cost/measure.py:27` `measure`: string-compared timestamps and approximate shell parsing
|
||||
|
||||
Maintainability senior, hardest first:
|
||||
|
||||
- `bin/larv_review/main.py:204` `record_ruling`: five outcomes, competing local names
|
||||
- `skills/larv-review/SKILL.md:55` "Round cap": a chain of cross-references
|
||||
- `bin/comprehension_history.py:103` `seat_places`: depends unsaid on `sections` dropping blank lines
|
||||
- `bin/larv_review/plan.py:5` `scoring_run_due`: four inputs; the rule restated in the help
|
||||
- `bin/larv_review/ruling_base.py:14` `measured_commit`: why same is safe needs the skill
|
||||
- `bin/metric_bench/bench.py:87` `tracked_places`: a garbled docstring
|
||||
|
||||
Inherited architect, would restructure first:
|
||||
|
||||
- `bin/larv_review/rounds.py:12` `round_counts`: the round cap lives in prose; a mis-cased title escapes the count
|
||||
- `bin/loop_claude/cycle/judge.py:17` `Reply`: the judge contract spread over four files
|
||||
- `bin/loop_claude/project/checkout.py:28` `read_utf8`: a generic reader under the checkout's name
|
||||
- `bin/test_env.py:5` `isolated_env`: test support named like a test
|
||||
- `bin/larv_review/git.py:6` imports: the git adapter depends on a domain type
|
||||
- `bin/loop_claude/cycle/prompts.py:60` `quoted`: two copies, with `STAMP` and the git runner also duplicated
|
||||
|
||||
## 2026-10-09T12:53:41Z, PR #234 at b346a35, against 2157f59
|
||||
|
||||
Ruling: better
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | better | better | better | better | better |
|
||||
| Inherited architect | better | better | better | better | better |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Navigation, Locality: `bin/reading_cost/main.py:35` `main`: the archive reader and location come from a shared package, not loop-claude's internals
|
||||
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: picks one of three named prompts for a four-argument `worker_args`
|
||||
- Navigation, Shape: `bin/loop_claude/cycle/prompts.py:1` the module: the texts the worker and the human see sit together
|
||||
- Locality, Shape: `bin/larv_review/main.py:204` `record_ruling`: plain checks with `split`, `lead` and `overall`
|
||||
- Navigation, Shape: `bin/larv_review/ruling_base.py:4` `measured_commit`: a truthful module name; `HistoryChange` beside the format
|
||||
- Navigation, Locality: `bin/todo_table.py:27` `items`: the table parser is shared, not reached into a CLI script
|
||||
- Navigation, Self-sufficiency: `README.md:77` "Tools": states the layout
|
||||
- Self-sufficiency: `bin/comprehension_history.py:82` `entries`: a worked example above the parser
|
||||
- Shape: `bin/reading_cost/measure.py:20` `Checkout`: fields say which root keys the archive
|
||||
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the past-the-cap case has its own guard
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Locality, Shape: `bin/transcript_archive/location.py:7` `project_archive`: both tools import one shared package
|
||||
- Navigation, Locality: `bin/transcript_archive/outcomes.py:10` `finished`: the outcome line sits beside the regex that reads it back
|
||||
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: `worker_args` takes the finished prompt
|
||||
- Shape: `bin/larv_review/git.py:5` imports: the git helper no longer depends on the base search
|
||||
- Shape, Self-sufficiency: `bin/larv_review/plan.py:5` `scoring_run_due`: its own module, saying why each trigger exists
|
||||
- Self-sufficiency, Locality: `bin/larv_review/ruling.py:27` `overall` and `:47` `split`: no None meaning split, no bare index
|
||||
- Navigation: `bin/reading_cost/measure.py:20` `Checkout`: the fields name which root
|
||||
- Self-sufficiency: `bin/comprehension_history.py:82` `entries`: a worked example
|
||||
- Navigation: `README.md:77` "Tools": the import root and the tests' place
|
||||
- Shape: `bin/larv_review/main.py:234` `recorded_ruling`: eight positional parameters, a step back
|
||||
- Self-sufficiency: `bin/loop_claude/cycle/prompts.py:29` `quoted`: a second copy
|
||||
|
||||
## 2026-10-09T12:53:41Z, PR #234 at b346a35
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Junior A | 7 | 6 | 7 | 6 | 6.5 |
|
||||
| Mid A | 7.5 | 7 | 7 | 6.5 | 7 |
|
||||
| Maintainability senior | 7.5 | 7 | 7 | 7 | 7 |
|
||||
| Inherited architect | 7.5 | 7 | 6.5 | 7 | 7 |
|
||||
| Mean | 7.38 | 6.75 | 6.88 | 6.63 | 6.88 |
|
||||
|
||||
Junior A, hardest first:
|
||||
|
||||
- `skills/larv-review/SKILL.md:55` "Round cap": rules pointing at each other; Fresh mixes five things
|
||||
- `bin/larv_review/main.py:204` `record_ruling`: five exits; the counting rule in `bin/larv_review/ruling.py:32` `counted`
|
||||
- `bin/larv_review/ruling_base.py:4` `measured_commit`: a set difference of headings per change
|
||||
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: a prompt built from constants in three files; two quote checks
|
||||
- `bin/comprehension_history.py:95` `entries`: two label regexes tied to the writer's wording
|
||||
- `bin/reading_cost/shell_reads.py:52` `segments`: heuristic shell parsing
|
||||
|
||||
Mid A, hardest first:
|
||||
|
||||
- `bin/larv_review/main.py:204` `record_ruling`: five branches across four modules
|
||||
- `bin/larv_review/ruling_base.py:4` `measured_commit`: reread three times to trust it
|
||||
- `bin/loop_claude/cycle/judge.py:127` `run_judge`: two quote checks with opposite policies
|
||||
- `bin/loop_claude/cycle/states.py:69` `transition`: the quoted-answer counter through three states
|
||||
- `bin/comprehension_history.py:95` `entries`: Markdown parsed back into records
|
||||
- `bin/metric_bench/bench.py:87` `tracked_places`: order-dependent legacy resolution
|
||||
|
||||
Maintainability senior, hardest first:
|
||||
|
||||
- `bin/larv_review/main.py:204` `record_ruling`: six exits; `fresh=False` hides a skipped check
|
||||
- `bin/larv_review/ruling_base.py:4` `measured_commit`: why needs the skill
|
||||
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: assembled across three files
|
||||
- `bin/comprehension_history.py:95` `entries`: the format defined by example and regexes
|
||||
- `bin/loop_claude/cycle/states.py:69` `transition`: the counter survives an interrupt
|
||||
- `bin/reading_cost/shell_reads.py:52` `segments`: opaque on what it misses
|
||||
|
||||
Inherited architect, would restructure first:
|
||||
|
||||
- `bin/larv_review/rounds.py:8` `round_counts`: the cap lives in prose; the tool only reports
|
||||
- `bin/larv_review/main.py:204` `record_ruling`: the record commands in a 326-line main
|
||||
- `bin/larv_review/git.py:8` `git`: four git wrappers, one archive key computed two ways
|
||||
- `bin/loop_claude/cycle/prompts.py:29` `quoted`: duplicated, as is `STAMP`
|
||||
- `bin/larv_review/ruling_base.py:24` `base`: a name that reads as a base class
|
||||
|
||||
## 2026-10-09T13:04:47Z, PR #234 at b2a277b, against 2157f59
|
||||
|
||||
Ruling: better
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | better | better | better | better | better |
|
||||
| Inherited architect | better | better | better | better | better |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Locality, Navigation: `bin/transcript_archive/location.py:7` `project_archive`: the archive's writer and readers share one package; outcome strings sit beside the regex that reads them
|
||||
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: picks the prompt in three lines; `worker_args` only builds argv
|
||||
- Shape, Navigation: `bin/larv_review/main.py:100` `main`: only dispatch; recording and plan reading have their own modules
|
||||
- Locality, Navigation: `bin/todo_table.py:27` `items`: table parsing shared, not reached into a tool script
|
||||
- Shape: `bin/larv_review/ruling.py:47` `split`: split, lead and overall each say what they compute
|
||||
- Shape, Self-sufficiency: `bin/reading_cost/measure.py:20` `Checkout`: fields name which root
|
||||
- Locality: `bin/claude_code/session.py:21` `session_named`: the adapter takes only a name and a folder
|
||||
- Navigation: `README.md:77` "Tools": names the import root and where tests live
|
||||
- Self-sufficiency: `bin/larv_review/main.py:5` the help: explains the ruling, ratchet, scoring run and floor first
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Locality, Navigation: `bin/transcript_archive/location.py:7` `project_archive`: location, writer and reader are peers both tools import
|
||||
- Locality: `bin/transcript_archive/outcomes.py:6` `FINISHED`: the format and its parser in one module
|
||||
- Shape, Navigation: `bin/larv_review/plan.py:8` `scoring_run_due`: the plan trigger has its own file; `scoring.py` holds only the floor
|
||||
- Shape: `bin/larv_review/record.py:14` `record_ruling`: recording is one module, its guards inline
|
||||
- Locality, Self-sufficiency: `bin/loop_claude/cycle/worker.py:40` `worker_args`: takes a ready prompt; the texts are named functions
|
||||
- Shape: `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: the judge is one file
|
||||
- Locality: `bin/loop_claude/paths.py:15` `paths_from`: the env override is read in one place
|
||||
- Self-sufficiency: `bin/comprehension_history.py:81` `entries`: a worked example above the parser
|
||||
- Navigation, Self-sufficiency: `README.md:77` "Tools": the layout is stated and checked
|
||||
- Locality: `bin/larv_review/ruling.py:52` `ruling`: returns same for a split, so only `split` guards it
|
||||
|
||||
## 2026-10-09T13:04:48Z, PR #234 at b2a277b
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Junior A | 7 | 6 | 7 | 6 | 6.5 |
|
||||
| Mid A | 7.5 | 6.5 | 7 | 6.5 | 6.5 |
|
||||
| Maintainability senior | 7.5 | 6.5 | 7 | 6.5 | 7 |
|
||||
| Inherited architect | 7.5 | 7 | 6.5 | 7 | 7 |
|
||||
| Mean | 7.38 | 6.50 | 6.88 | 6.50 | 6.75 |
|
||||
|
||||
Junior A, hardest first:
|
||||
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: seven exits; the terms only in the help and the skill
|
||||
- `bin/larv_review/ruling_base.py:4` `measured_commit`: diffs heading sets per change
|
||||
- `bin/comprehension_history.py:94` `entries`: regex groups used by position
|
||||
- `bin/larv_review/ruling.py:32` `counted`: the counting rule stated in the help, not here
|
||||
- `bin/loop_claude/cycle/states.py:69` `transition`: the cap counter across three files
|
||||
- `bin/loop_claude/cycle/judge.py:127` `run_judge`: two quote checks with different rules
|
||||
|
||||
Mid A, hardest first:
|
||||
|
||||
- `skills/larv-review/SKILL.md:82` "Architecture": conditions with exceptions; Fresh took three rereads
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: six outcomes across five modules
|
||||
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: one sentence takes three files
|
||||
- `bin/larv_review/ruling_base.py:4` `measured_commit`: the fallback chain over two files
|
||||
- `bin/larv_review/plan.py:8` `scoring_run_due`: release sets held at once
|
||||
- `bin/comprehension_history.py:94` `entries`: the format pinned in three places
|
||||
|
||||
Maintainability senior, hardest first:
|
||||
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: five exits; the rule spread over five modules
|
||||
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: instructions stitched from three modules
|
||||
- `bin/larv_review/plan.py:8` `scoring_run_due`: five inputs; the docstring restates the help
|
||||
- `bin/larv_review/ruling_base.py:4` `measured_commit`: opaque without its docstring
|
||||
- `bin/larv_review/seats.py:17` `SeatsFile`: `read` is a noun; locals rename one pair two ways
|
||||
- `bin/comprehension_history.py:110` `seat_places`: a line-state parser two tools depend on
|
||||
|
||||
Inherited architect, would restructure first:
|
||||
|
||||
- `bin/larv_review/rounds.py:4` `PASSES`: the cap is the skill's; the pass list has two homes
|
||||
- `bin/comprehension_history.py:149` `ruling_entry`: reader, writer and seat registry in one root module
|
||||
- `bin/larv_review/record.py:67` `read_fall_scores`: a parameter named votes holds a heading
|
||||
- `bin/larv_review/git.py:8` `git`: four ways to run git; the archive key's root differs between tools
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: branches on worse, fell, scores file and split at once
|
||||
|
||||
## 2026-10-09T13:23:40Z, PR #234 at 7e13778, against 2157f59
|
||||
|
||||
Ruling: better
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | better | better | better | better | better |
|
||||
| Inherited architect | better | better | better | better | better |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Navigation, Shape: `bin/larv_review/record.py:14` `record_ruling`: recording, dispatch and plan reading each in a file named for it
|
||||
- Shape, Navigation: `bin/larv_review/plan.py:8` `scoring_run_due`: file names tell the truth; todo parsing in `bin/todo_table.py:27` `items`
|
||||
- Navigation, Shape: `bin/transcript_archive/location.py:7` `project_archive`: both tools share top-level packages, not another tool's internals
|
||||
- Locality: `bin/loop_claude/cycle/run.py:83` `worker_start`: a prompt changes without reading the argument plumbing
|
||||
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: the past-the-cap case has its own guard
|
||||
- Locality: `bin/transcript_archive/location.py:7` `project_archive`: the env override resolved in one place
|
||||
- Self-sufficiency: `bin/larv_review/main.py:1` the help: defines seats, the ruling, the ratchet and the floor first
|
||||
- Shape, Self-sufficiency: `bin/reading_cost/measure.py:20` `Checkout`: fields say which root keys the archive
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Locality, Navigation: `bin/transcript_archive/outcomes.py:6` `FINISHED`: outcome lines written and parsed in one file
|
||||
- Locality, Shape: `bin/transcript_archive/location.py:7` `project_archive`: reading-cost no longer reaches into loop-claude
|
||||
- Navigation, Locality: `bin/loop_claude/cycle/judge.py:94` `verdict_from`: the 3am question lands in one file
|
||||
- Shape, Navigation: `bin/larv_review/main.py:102` `main`: dispatch only
|
||||
- Locality, Shape: `bin/todo_table.py:27` `items`: larv-review imports no tool's internals
|
||||
- Locality: `bin/larv_review/record.py:32` `record_ruling`: the fall handling reads in place
|
||||
- Locality, Shape: `bin/loop_claude/cycle/run.py:83` `worker_start`: the start chosen at the call site
|
||||
- Shape: `bin/reading_cost/measure.py:20` `Checkout`: names that say which root
|
||||
- Self-sufficiency: `bin/comprehension_history.py:81` `entries`: a sample entry above the parser
|
||||
- Locality: `bin/loop_claude/cycle/states.py:75` `transition`: no case-order dependence
|
||||
|
||||
## 2026-10-09T13:23:40Z, PR #234 at 7e13778
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Junior A | 7.5 | 7 | 7 | 6.5 | 7 |
|
||||
| Mid A | 7 | 6.5 | 7 | 6.5 | 6.5 |
|
||||
| Maintainability senior | 7 | 6.5 | 7 | 6 | 6.5 |
|
||||
| Inherited architect | 7 | 6.5 | 6 | 7 | 6.5 |
|
||||
| Mean | 7.13 | 6.63 | 6.75 | 6.50 | 6.63 |
|
||||
|
||||
Junior A, hardest first:
|
||||
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: four outcomes, two parallel vote dicts
|
||||
- `bin/larv_review/ruling.py:32` `counted`: an asymmetric rule; `PLACE` in another file
|
||||
- `bin/metric_bench/python_units.py:86` `score`: needs a paper outside the repo
|
||||
- `bin/comprehension_history.py:94` `entries`: leans on `sections` dropping blank lines
|
||||
- `bin/loop_claude/cycle/judge.py:127` `run_judge`: two quote checks against two texts
|
||||
- `skills/larv-review/SKILL.md:55` "Round cap": sections pointing at each other
|
||||
|
||||
Mid A, hardest first:
|
||||
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: five outcomes; `read` as a noun
|
||||
- `bin/comprehension_history.py:110` `seat_places`: depends unsaid on `sections` dropping blank lines
|
||||
- `bin/larv_review/ruling_base.py:4` `measured_commit`: several ideas held at once
|
||||
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: decision logic in English
|
||||
- `bin/claude_code/entries.py:28` `written_by_human`: heuristics over the transcript shape
|
||||
- `bin/loop_claude/loop/reload.py:50` `exec_if_edited`: any tool's edit reloads the loop
|
||||
|
||||
Maintainability senior, hardest first:
|
||||
|
||||
- `skills/larv-review/SKILL.md:82` "Architecture": six interlocking conditions
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: two checks run twice; the rule whole only in the help
|
||||
- `bin/comprehension_history.py:94` `entries`: a format shared by three tools by string convention
|
||||
- `bin/larv_review/plan.py:8` `scoring_run_due`: three things combined in "work has landed"
|
||||
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: behaviour in prose, post-filtered three ways
|
||||
- `bin/rank_todo.py:357` `rank_details`: rebuilds the file by slicing
|
||||
|
||||
Inherited architect, would restructure first:
|
||||
|
||||
- `bin/larv_review/rounds.py:8` `round_counts`: the cap is prose in the skill
|
||||
- `bin/larv_review/record.py:14` `record_ruling`: an if-chain with prints interleaved
|
||||
- `bin/larv_review/scoring.py:8` `fall_ruling`: a ruling rule in the scoring module
|
||||
- `bin/claude_code/trust.py:10` `trust`: effects in a package documented as shapes
|
||||
- `bin/larv_review/git.py:8` `git`: git run five ways
|
||||
|
||||
+2
-1
@@ -52,7 +52,8 @@ lilleman decided to ship each of these under the floor on the day listed. Each l
|
||||
- 2026-10-08, PR #225: 7.25, 6.75, 7.00, 6.88, 6.88. Above PR #224's run on every score but Shape, which held.
|
||||
- 2026-10-08, PR #228: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet: the seats blamed the new ratchet's own rules, since moved into the tested `bin/larv-review`.
|
||||
- 2026-10-08, PR #229: no scoring run; its last paired ruling, against 12d6d75, was worse, 2 votes to 0. lilleman shipped it below the ratchet for PR #228's reason: the seats' decisive places were still PR #228's rules, since moved into `bin/larv-review`.
|
||||
- 2026-10-09, PR #232: 7.13, 6.50, 6.63, 6.63, 6.88, at 324d650, whose ruling the fall made worse. Its last paired ruling, at 53014f8 against 12d6d75, split: Mid A better, the inherited architect worse, both with Locality worse. lilleman shipped it below the ratchet: after a restructure the seats still weigh PRs #229 to #231's code against the base, and their Locality places are filed under items 62 and 83.
|
||||
- 2026-10-09, PR #232: 7.13, 6.50, 6.63, 6.63, 6.88, at 324d650, whose ruling the fall made worse. Its last paired ruling, at 53014f8 against 12d6d75, split: Mid A better, the inherited architect worse, both with Locality worse. lilleman shipped it below the ratchet: after a restructure the seats still weigh PRs #229 to #231's code against the base, and their Locality places are filed under item 62; the `bin/` regrouping, item 83, shipped in PR #234.
|
||||
- 2026-10-09, PR #234: 7.13, 6.63, 6.75, 6.50, 6.63, at 7e13778. Four architecture rounds and the rescore each ruled better, 2 votes to 0, while the means moved within the panel's noise between runs. lilleman shipped it below the floor: most of every seat's remaining places sit in larv-review's rules, which cannot be followed without `skills/larv-review/SKILL.md` §3 open beside the code. Fixing that is a chunk of its own, under item 62.
|
||||
|
||||
## loop-claude: each project's model lives in `~/.claude/loop-claude-models.json`, which only loop-claude writes
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ A scoring-run seat's prompt is the opening, the seat's background, the shared br
|
||||
- `{main}`: the project's main checkout.
|
||||
- `{forbidden}`: `todo.md, docs/decisions.md, docs/comprehension-history.md or docs/reading-cost.md (the plan, the decision log, past panels' records and past chunks' reading cost)`.
|
||||
- `{git}`: `git -C {checkout} ls-files`, and for a paired seat also `git -C {base checkout} ls-files`, each in backticks.
|
||||
- `{files}`: every tracked non-test file but those `{forbidden}` names, from `git -C {checkout} ls-files`, listed as one recursive glob per top-level source directory, such as `bin/loop_claude/**/*.py`.
|
||||
- `{files}`: every tracked file in `git -C {checkout} ls-files`, except test files and those `{forbidden}` names. Give one recursive glob per top-level source directory, say which test files it leaves out, and list each file no glob matches. Example: `bin/**/*.py` except `test_*.py`, plus launchers such as `bin/loop-claude`.
|
||||
- `{preloaded}`: the files the harness pre-loads into a seat, found per `SKILL.md` → "What makes the result trustworthy".
|
||||
- `{anchors}` and `{dimensions}`: the six anchor bullets and the four dimension bullets in `SKILL.md` → "The score", verbatim, without bold and each on one line.
|
||||
- `{language}`, `{kind}`, `{domain}`, `{domain docs}` and `{3am question}`: the project's `AGENTS.md` → "Scoring run".
|
||||
|
||||
@@ -31,7 +31,6 @@
|
||||
| 59 | 0.1.0 | | **Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.** | 3 | 3 | 6 | 6 | 1.4 | 19.8 |
|
||||
| 12 | 0.1.0 | | **Run the test suite in CI on every PR, or record in `docs/decisions.md` why the local run is enough.** | 2 | 4 | 4 | 6 | 1.1 | 16.9 |
|
||||
| 80 | 0.1.0 | | **State in README → Install that every machine taking this checkout must be a sandbox, as `settings.json`'s auto mode assumes.** | 1 | 1 | 5 | 5 | 4 | 16.8 |
|
||||
| 83 | 0.1.0 | | **Group `bin/` by tool, so each tool's modules and tests sit together.** | 3 | 4 | 4 | 6 | 1.1 | 15.9 |
|
||||
| 5 | 0.1.0 | decision | **Key a project on its checkout's full path, and refuse to start while another session uses that key.** | 5 | 5 | 5 | 6 | 1.1 | 15.6 |
|
||||
| 84 | 0.1.0 | | **Check, at each release's scoring run, that seats in a paired ruling agree more often than two scoring runs on near-identical code do, as the paired-ruling decision assumes.** | 2 | 2 | 4 | 5 | 1.4 | 15.1 |
|
||||
| 67 | 0.1.0 | principle | **Point stability-reviewer at the technical principles in place of its own restatement of them.** | 2 | 3 | 4 | 5 | 1.3 | 14.9 |
|
||||
@@ -62,11 +61,9 @@ The README line covers the bench's transcripts in `workflow-bench/logs/` too (go
|
||||
|
||||
### 62. Lift Locality, Self-sufficiency and the overall to the comprehension floor, at the places the seats name.
|
||||
|
||||
Raised by the principles audit of PR #215 against KISS, in `principles/technical-principles.md`, and goal 1.1: the `comprehension floor:` entry in `docs/decisions.md` lists seven decisions to ship under the floor, and 0.1.0 waits on the floor. PR #215's last run put Locality at 6.63 and Self-sufficiency and the overall at 6.88. Its seats named, under Locality: the judge's contract spread over `cycle/judge_reply.py`, `cycle/worker.py` and `cycle/run.py`; `remaining`, which is `Unfinished.evidence` under another name, threaded through `loop/main.py`, `loop/reload.py`'s `LoopRun`, `cycle/run.py` and `worker_args`; the `quoted_answers` cap held by case order in `cycle/states.py`'s `transition` and restated in `run.py`'s `question_for`; the four outcome texts `run.py` builds inline for `archive`. Under Self-sufficiency: `skills/larv-review/SKILL.md`, which needs `CLAUDE.md` → "Vocabulary" open beside it; the archive layout, defined only in `bin/loop_claude/loop/main.py`'s docstring and not at `transcript_archive.py`; `JUDGE_RULES`' precedence in `cycle/judge_reply.py`, restated by `verdict_from`'s branch order and the `Reply` field descriptions; the `sys.path` insert in `bin/loop-claude` that the `from failure import` lines rely on, unstated at the import sites.
|
||||
Raised by the principles audit of PR #215 against KISS, in `principles/technical-principles.md`, and goal 1.1: 0.1.0 waits on the floor, and the `comprehension floor:` entry in `docs/decisions.md` lists each decision to ship under it.
|
||||
|
||||
PR #225's last run, at 8c9bf09, put Locality at 6.75 and Self-sufficiency and the overall at 6.88. Its seats named: the judge's policy spread over `loop/main.py`'s docstring, `cycle/judge_reply.py`'s `JUDGE_RULES`, `cycle/judge.py`'s `judge_prompt` and `cycle/worker.py`'s `worker_args`; `loop_claude/claude_code/` and the read side of `transcript_archive.py`, which `bin/reading_cost/` also imports; two meanings of a checkout's root in `project/checkout.py`'s `checkout_root` and `reading_cost/main.py`'s `located`; and `skills/larv-review/SKILL.md` §3's round count, which only prose enforces.
|
||||
|
||||
2026-10-09, from two architecture studies on PR #232: give the judge's contract one home. `cycle/judge.py` takes `judge_reply.py`'s rules, schema and `verdict_from`; `QUOTE_SOURCES`, `ANSWER_DESTINATIONS` and `ANSWERS_RECORDED` move to `project/records.py`, so `worker.py` imports the records, not the judge; `states.py`'s two `Judging` cases get exclusive guards, so their order carries no meaning. PR #232's scoring run at 324d650 put Locality at 6.50.
|
||||
PR #234's last scoring run, at b2a277b, put Locality and Self-sufficiency at 6.50 and the overall at 6.75. Every seat named larv-review's rules, which cannot be followed without `skills/larv-review/SKILL.md` §3 open beside the code: "Round cap", "Re-entry", "Fresh" and "Suspended" point at each other; `bin/larv_review/record.py` `record_ruling` branches on split, worse, fell and a scores file, with its terms defined only in `larv-review --help` and the skill; the pass list lives in both `bin/larv_review/rounds.py` `PASSES` and the skill's table, which also counts the auto-reviewer; `bin/comprehension_history.py` is reader, writer and seat registry in one, and parses the history by regex for two tools. Also named: the judge's prompt assembled from `cycle/judge.py`, `cycle/prompts.py` and `project/records.py`; git run four ways, in `bin/larv_review/git.py`, `bin/metric_bench/main.py`, `bin/reading_cost/main.py` and `bin/loop_claude/effects.py`; `SeatsFile.read`, a field named like a method.
|
||||
|
||||
### 21. Give `settings.json`'s model one writer, and sort its keys.
|
||||
|
||||
@@ -90,12 +87,6 @@ The repo has no CI, so larv-review's gate is the local run, and larv-review's Ga
|
||||
|
||||
Raised by the principles audit of PR #224. The `settings.json` decision is valid while every machine taking this checkout is a sandbox, but Install asks for a sandbox only for `loop-claude`, so a new machine is never told.
|
||||
|
||||
### 83. Group `bin/` by tool, so each tool's modules and tests sit together.
|
||||
|
||||
Raised by the architect on PR #225 and by the inherited architect of PR #224's scoring run: `bin/` is one flat folder of tools, the modules they share and the repo's checks, with `loop_claude`'s tests beside them instead of beside the package. `loop_claude/claude_code/` parses Claude Code's own formats and `bin/reading_cost/measure.py` imports it too, so it is a package of its own.
|
||||
|
||||
2026-10-09, from two architecture studies on PR #232: lift `loop_claude/claude_code/` and the archive's layout out of `loop_claude/` into packages of their own, `claude_code/` and `transcript_archive/`, so `reading_cost` imports no tool. `transcript_archive` takes the write side (`archive`), the read side (`finished_chunks`, `FinishedChunk`, the stamps) and one `project_archive(checkout_root, home, env)` that both tools call, which derives the archive key one way, as item 5 asks. Move each tool's tests beside its package, and `test_loop_claude_black_box.host_env`, which three other tools' tests import, into a test helper of its own.
|
||||
|
||||
### 5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
|
||||
|
||||
The key names the project's session, archive folder and models entry. `project_key` drops the leading `/`, so `~/code/a` and `/a` share all three, and one loop can resume or delete the other's session.
|
||||
|
||||
Reference in New Issue
Block a user