Release review of 0.6.0

This commit was merged in pull request #415.
This commit is contained in:
2026-11-04 23:52:56 +01:00
73 changed files with 1057 additions and 908 deletions
+2 -1
View File
@@ -21,7 +21,6 @@
!bin/branch_chunks/
!bin/branch_chunks/*.py
!bin/chunk-preflight
!bin/chunk_pick.py
!bin/chunk_preflight/
!bin/chunk_preflight/*.py
!bin/claude_code/
@@ -61,6 +60,8 @@
!bin/reading-cost
!bin/reading_cost/
!bin/reading_cost/*.py
!bin/release.py
!bin/release_standing.py
!bin/repo_checks/
!bin/repo_checks/*.py
!bin/review-replay
+5
View File
@@ -7,6 +7,11 @@
- `rule-hook` no longer hangs, or reads without end, on an Edit, MultiEdit or Write whose `file_path` is a FIFO or a device: it reads only a regular file.
- A `loop-inbox` ask file now stores when the loop asked as an ISO UTC datetime, such as `2026-11-04T09:30:00Z`, in place of seconds since the epoch. Restart any `loop-inbox` running when you deploy: one started before doesn't list the new asks. A `loop-claude` running then reloads itself at its next cycle, and `loop-inbox` doesn't list a question it asks before that; answer such a question on the loop's terminal.
- Each chunk's first worker prompt now holds a reading rule: never read more than 80 lines of a file at once, nor take a tool output over 3,000 characters into your context. Where the 20 chunks after it take more review rounds, unfinished verdicts, findings, misread findings, dollars or hours than the 20 before, a later release drops it.
- `rule-hook`'s `round-cap` now refuses a round commit whose title names no larv-review pass, such as `Answer architectur round 1:`. Before, a misspelled pass got a count of its own, so its rounds never reached the cap.
- A routing call from `loop-claude`'s figures now opens claude in the setup's development clone, `~/code/claude-config`, not in `~/.claude`. Recording the answer there no longer leaves `~/.claude` with changes that stop `deploy-release` from moving it.
- Where the default branch's tags cannot be read, `chunk-preflight` now says so, with git's reason, and that it cannot tell which release is next, and `loop-claude` routes the chunk on the project's rung. Before, both read such a branch as one with nothing cut, so `chunk-preflight` could report an empty plan where a release review was due.
- A branch that changes `workflow-bench/run-prompt.md` or `compare-prompt.md` now counts as changing config, as a skill does: its reviewers run on the rung its items pick, not on the project's rung.
- `deploy-release` now stops with git's own error where git cannot refresh the checkout's index. Before, it went on and could refuse with a wrong report of uncommitted changes.
## 0.5.0
+1 -1
View File
@@ -122,7 +122,7 @@ Claude Code reads this section by its heading when it compacts a conversation. K
- **chunk** — the unit of work, shipped as one PR; "Working loop" says what it holds.
- **cycle** — in `loop-claude`, worker runs and the judge's verdict on each, until the chunk closes or is left for the next cycle. In between, the human answers any interrupt, or the worker resumes with an answer the judge quoted from `README.md` or the judge's records.
- **gate** — the merge gate: the checks a PR passes before it merges.
- **interrupt** — `loop-claude` asking the human, on its terminal and in `loop-inbox`, for a decision that is theirs. They type a one-line answer in either, or `/talk` to open the worker's session with `claude --resume` where they typed it; a release ready to cut opens that session at once, and a routing call opens claude in `~/.claude`, both on loop-claude's terminal.
- **interrupt** — `loop-claude` asking the human, on its terminal and in `loop-inbox`, for a decision that is theirs. They type a one-line answer in either, or `/talk` to open the worker's session with `claude --resume` where they typed it. A release ready to cut opens that session at once, on loop-claude's terminal. A routing call opens claude on that terminal too, in `~/code/claude-config`, the checkout where this setup is developed.
- **judge** — the headless claude that reads a worker's transcript each cycle and decides whether to interrupt and whether the chunk is finished.
- **judge's records** — the files the judge reads beside the README's premises: the project's `AGENTS.md` and `docs/decisions.md`, then the human's `~/.claude/CLAUDE.md` and the `.md` files in `~/.claude/rules/` and `~/.claude/skills/larv-review/standards/`.
- **paired ruling** — the seats vote whether the project reads better, the same or worse at the branch's head than at the base, the commit `larv-review ruling-base` prints; `larv-review --help` says how the votes make the ruling.
+1 -1
View File
@@ -125,7 +125,7 @@ Then, in the copy: `uvx --with pytest==9.2.0 'mutmut>=3,<4' run && uvx --with py
- `judge_replay` imports `loop_claude`'s judge.
- `larv_review` imports `rank_todo.py`'s ranking.
- `loop_claude` runs `chunk-preflight` as a process of its own.
- `loop_inbox` imports `loop_claude`'s question files (`waiting/`), its effects and paths, and `hand_over.py`, which opens claude for `/talk`.
- `loop_inbox` imports `loop_claude`'s question files (`waiting/`), the files a running loop lists itself in (`running_loops.py`, `session_lock.py`), its effects and paths, and `hand_over.py`, which opens claude for `/talk`. A change to any of them changes loop-inbox too.
- `machine_load/` takes the machine's heavy slots, its shared lock files, for `run_heavy`, and reads the memory share that loop-claude waits for.
- `model_routing/` holds the routing ladder, which loop-claude, `rule_hook`, both replay tools and `subagent-rung` read.
- `rank_todo.py`, `settings_filter.py` and `statusline.py` are tools of one file each, tested beside them.
+3 -3
View File
@@ -32,11 +32,11 @@ class Commit:
@dataclasses.dataclass(frozen=True)
class Span:
"""A chunk on a default branch, as cut_branch cuts it. before is the commit its change starts from, None for the branch's oldest chunk; tip is its newest commit on the first-parent line. label names it as the reports do, landed says how it reached the branch, and merged is tip's commit time. titles are those of every commit it brought."""
"""A chunk on a default branch, as cut_branch cuts it. before is the commit its change starts from, None for the branch's oldest chunk; tip is its newest commit on the first-parent line. label names it as the reports do, landing says how it reached the branch, and merged is tip's commit time. titles are those of every commit it brought."""
before: str | None
tip: str
label: str
landed: Landing
landing: Landing
merged: str
titles: tuple[str, ...]
@@ -78,7 +78,7 @@ def cut_branch(log: str) -> Branch | Failure:
before=run[-1].parents[0] if run[-1].parents else None,
tip=run[0].hash,
label=label(run[-1].short, run[0].short),
landed=landing(run),
landing=landing(run),
merged=utc(run[0].committed),
titles=tuple(titles[index]),
))
+2 -2
View File
@@ -127,7 +127,7 @@ class WithFixes(unittest.TestCase):
def test_keeps_each_chunk_as_cut_with_the_items_it_shipped(self):
(found,) = with_fixes(Branch({}, [span("a")], {}), [(item(4),)], [])
self.assertEqual((found.before, found.tip, found.label, found.landed, found.merged, found.titles), ("a-before", "a-tip", "a", "merge", "2026-10-13T10:00:00Z", ()))
self.assertEqual((found.before, found.tip, found.label, found.landing, found.merged, found.titles), ("a-before", "a-tip", "a", "merge", "2026-10-13T10:00:00Z", ()))
self.assertEqual([shipped.id for shipped in found.shipped], [4])
@@ -273,7 +273,7 @@ class BranchChunks(unittest.TestCase):
fixed = self.merge("six", "Fix six", {"b.py": "x\nY\n", "todo.md": todo(SECOND)}, "2026-10-14T10:00:00Z")
found = self.read()
self.assertNotIsInstance(found, Failure)
self.assertEqual([(found.label, found.landed, found.merged, [item.id for item in found.shipped], found.fixed_by) for found in found[:2]], [(fixed, "merge", "2026-10-14T10:00:00Z", [6], ()), (shipped, "merge", "2026-10-13T10:00:00Z", [4], (fixed,))])
self.assertEqual([(found.label, found.landing, found.merged, [item.id for item in found.shipped], found.fixed_by) for found in found[:2]], [(fixed, "merge", "2026-10-14T10:00:00Z", [6], ()), (shipped, "merge", "2026-10-13T10:00:00Z", [4], (fixed,))])
self.assertEqual(sorted(found[1].titles), ["Do four", "Merge four"])
self.assertEqual((found[1].tip, found[2].tip), (self.git("rev-parse", "HEAD^").strip(), self.git("rev-parse", "HEAD^^").strip()))
self.assertEqual((found[1].before, found[2].before), (found[2].tip, None))
+6 -3
View File
@@ -2,10 +2,11 @@
import pathlib
import re
from chunk_pick import chunk_item, Cut, dated_cut, LastCut, ReadyToCut, ReviewDue, Unknown, Unplanned
from release_standing import chunk_item, Cut, dated_cut, LastCut, ReadyToCut, ReviewDue, Unknown, Unplanned
from failure import Failure
from chunk_preflight.gathered import Forge, NoItem, Plan
from todo_table import item_details, items, NoItemsTable, version
from todo_table import item_details, items, NoItemsTable
from release import version
# The command each forge's CLI lists pull requests with; pulls_command adds the flags that pick their rows and fields.
PULLS_LIST: dict[Forge, tuple[str, ...]] = {"gh": ("gh", "pr", "list"), "tea": ("tea", "pulls", "list")}
@@ -20,7 +21,7 @@ def cut_name(cut: LastCut) -> str | None:
def planned(todo: str | Failure, cut: LastCut, default: str) -> Plan | NoItem:
"""The plan in todo, the default branch's todo.md, with the item the next chunk takes first, chunk_pick.chunk_item's pick; else why there is none."""
"""The plan in todo, the default branch's todo.md, with the item the next chunk takes first, release_standing.chunk_item's pick; else why there is none."""
where = f"{default}'s todo.md"
if isinstance(todo, Failure):
return NoItem(f"{where} cannot be read: {todo.reason}.")
@@ -29,6 +30,8 @@ def planned(todo: str | Failure, cut: LastCut, default: str) -> Plan | NoItem:
return NoItem(f"{where} holds no items table.")
if isinstance(found, Failure):
return NoItem(f"{where} cannot be read: {found.reason}.")
if isinstance(cut.tag, Failure):
return NoItem(f"{default}'s tags cannot be read: {cut.tag.reason}. So which release is next cannot be told. Find the next release yourself.")
named = cut_name(cut)
last_cut = f"the last cut, {named}" if named is not None else "the start, since no release is cut yet"
pick = chunk_item(found, cut)
+3 -2
View File
@@ -1,10 +1,11 @@
"""The lines the preflight prints, stated from what it gathered."""
from todo_table import bundle_candidates, BUNDLE_MAX_R, Item, ships_alone, version
from todo_table import bundle_candidates, BUNDLE_MAX_R, Item, ships_alone
from chunk_preflight.choices import cut_name, PULLS_LIST
from chunk_pick import dated_cut, LastCut, unreleased_entries
from release_standing import dated_cut, LastCut, unreleased_entries
from failure import Failure
from chunk_preflight.gathered import Forge, Gathered, ItemFacts, NamedPath, NoItem, Plan, Ran
from markdown_text import quoted
from release import version
def render(found: Gathered) -> list[str]:
+1 -1
View File
@@ -5,7 +5,7 @@ import typing
from failure import Failure
from todo_table import Item
from chunk_pick import LastCut
from release_standing import LastCut
# The CLI that lists a forge's pull requests: gh for a github.com origin, tea for any other.
Forge = typing.Literal["gh", "tea"]
+1 -1
View File
@@ -58,7 +58,7 @@ import sys
import typing
from git_repo import commit_exists, default_branch, file_at, git, top
from chunk_pick import default_plan
from release_standing import default_plan
from failure import Failure
from chunk_preflight.gathered import Forge, Gathered, ItemFacts, NamedPath, NoItem, Plan, Ran, Worktree
from chunk_preflight.choices import forge_of, named_paths, planned, pulls_command, worktree_add, worktree_on
+7 -1
View File
@@ -12,7 +12,7 @@ from failure import Failure
from chunk_preflight.choices import forge_of, named_paths, planned, worktree_add
from chunk_preflight.gathered import Gathered, ItemFacts, NamedPath, NoItem, Ran, Worktree
from test_env import isolated_env
from chunk_pick import LastCut
from release_standing import LastCut
from chunk_preflight.main import line_count, ran, seconds, TIMEOUT
from chunk_preflight.facts import open_pulls, render
@@ -454,6 +454,12 @@ class Plans(unittest.TestCase):
said = planned(EMPTY, LastCut(None, "## 0.0.1 - 2026-10-03\n", 0, []), "origin/main").why
self.assertIn("the next release after the last cut, 0.0.1 (dated in CHANGELOG.md)", said)
def test_says_the_tags_cannot_be_read_with_gits_reason_and_names_no_cut(self):
said = planned(EMPTY, LastCut(Failure("git exited 128: not a git repository"), "", 4, []), "origin/main").why
self.assertIn("origin/main's tags cannot be read: git exited 128: not a git repository.", said)
self.assertIn("Find the next release yourself.", said)
self.assertNotIn("no release is cut yet", said)
def test_names_the_start_as_the_last_cut_where_nothing_is_cut(self):
said = planned(EMPTY, LastCut(None, "", 4, [TODO]), "origin/main").why
self.assertIn("release 0.2.0, the next after the start, since no release is cut yet,", said)
+11 -11
View File
@@ -17,12 +17,11 @@ also refuses where the newest tag doesn't hold the checkout's commit, since the
work no release holds. Untracked files, such as Claude Code's sessions and memory, stay as they are.
"""
import pathlib
import re
import sys
from failure import Failure
from git_repo import git
from todo_table import release_of
from release import newest_tag, parsed, tagged
from markdown_text import sections
@@ -47,14 +46,14 @@ def deploy(checkout: str) -> str | Failure:
fetched = git("fetch", "--quiet", "--tags", "origin", cwd=checkout)
if isinstance(fetched, Failure):
return fetched
newest = git("-c", "versionsort.suffix=-", "tag", "--list", "v*", "--sort=-v:refname", cwd=checkout)
if isinstance(newest, Failure):
return newest
if not newest.split():
tag = newest_tag(checkout)
if isinstance(tag, Failure):
return tag
if tag is None:
return Failure("origin has no v* tag to move to")
tag = newest.split()[0]
# Re-reads each file through the clean filter, so a model pick alone leaves settings.json unchanged.
git("update-index", "-q", "--refresh", cwd=checkout)
if isinstance(refreshed := git("update-index", "-q", "--refresh", cwd=checkout), Failure):
return refreshed
changed = git("diff", "--name-only", "HEAD", cwd=checkout)
if isinstance(changed, Failure):
return changed
@@ -90,11 +89,12 @@ def deploy(checkout: str) -> str | Failure:
def newer_sections(changelog: str, old_tag: str | None) -> list[str]:
"""Each `## <version>` section of changelog newer than old_tag, newest first, as it reads."""
since = release_of(old_tag.removeprefix("v")) if old_tag and re.fullmatch(r"v[0-9]+\.[0-9]+\.[0-9]+", old_tag) else None
since = tagged(old_tag)
found = []
for heading, bodies in sections(changelog).items():
if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", heading):
named = parsed(heading)
if named is None:
continue
if since is None or release_of(heading) > since:
if since is None or named > since:
found.append(f"## {heading}\n\n" + "\n".join(bodies))
return found
@@ -60,6 +60,15 @@ class DeployRelease(unittest.TestCase):
self.assertNotIn("Later.", ran.stdout)
self.assertNotIn("/model", ran.stdout)
def test_stops_with_gits_error_where_the_index_cannot_be_refreshed(self):
self.release("0.2.0", "- Second.\n", '{"theme": "dark"}\n')
(self.target / ".git" / "index").write_bytes(b"DIRC")
ran = self.deploy()
self.assertNotEqual(ran.returncode, 0)
self.assertIn("index", ran.stderr)
self.assertNotIn("uncommitted", ran.stderr)
self.assertEqual(self.git(self.target, "rev-parse", "HEAD"), self.git(self.source, "rev-parse", "v0.1.0^{commit}"))
def test_says_so_where_the_checkout_is_on_the_newest_tag_already(self):
ran = self.deploy()
self.assertEqual((ran.returncode, ran.stdout), (0, "Already on v0.1.0, the newest release.\n"))
+4 -3
View File
@@ -7,19 +7,20 @@ GIT_VALUED = {"-C", "-c", "--exec-path", "--git-dir", "--namespace", "--work-tre
@dataclasses.dataclass(frozen=True)
class Commit:
class CommitCall:
"""A `git commit` a shell command runs: the directory it runs in, its messages, whether it reads its message from a file, and whether it amends."""
directory: str
messages: list[str]
from_file: bool
amends: bool
def commits(simples: list[Simple], home: str) -> list[Commit]:
def commits(simples: list[Simple], home: str) -> list[CommitCall]:
found = []
for simple in simples:
invoked = git_subcommand(simple.words, simple.directory, home)
if invoked and invoked[0] == "commit":
found.append(Commit(invoked[2], *commit_message(invoked[1])))
found.append(CommitCall(invoked[2], *commit_message(invoked[1])))
return found
-7
View File
@@ -109,13 +109,6 @@ def origin_head(cwd: str | None, timeout: float | None, run: Git) -> str | Failu
return Failure("origin names no default branch; set it with `git remote set-head origin --auto`")
def newest_release_tag(branch: str, cwd: str | None = None) -> str | None | Failure:
"""The newest `v*` tag merged into branch, read in cwd, else in the current directory; None when it has none."""
# Sorts v1.0.0-rc.1 below v1.0.0.
tags = git("-c", "versionsort.suffix=-", "tag", "--list", "v*", "--merged", branch, "--sort=-v:refname", cwd=cwd)
return tags if isinstance(tags, Failure) else next(iter(tags.split()), None)
def commits_since(tag: str | None, branch: str, cwd: str | None = None) -> int | Failure:
"""How many commits branch holds after tag, or holds in all where tag is None; read in cwd, else in the current directory."""
count = git("rev-list", "--count", f"{tag}..{branch}" if tag else branch, cwd=cwd)
+4 -5
View File
@@ -40,7 +40,6 @@ from model_routing.ladder import lowest_worker_rung, project_rung, Rung
from loop_claude.project.premises import read_premises
from loop_claude.project.records import read_records
from claude_code.transcript import read_transcript
from loop_claude.stop import Stop
from loop_claude.verdicts import Verdict
TOOL = "judge-replay"
@@ -93,10 +92,10 @@ def archived_cases(model: str, root: pathlib.Path, home: pathlib.Path) -> list[C
def rule(case: Case, rung: Rung, claude_dir: pathlib.Path, effects: Effects) -> Kind | Limited:
premises, records, entries = read_premises(case.root), read_records(case.root, claude_dir), read_transcript(str(case.transcript))
if isinstance(premises, Stop):
return verdict_kind(Failure(premises.reason))
if isinstance(records, Stop):
return verdict_kind(Failure(records.reason))
if isinstance(premises, Failure):
return verdict_kind(premises)
if isinstance(records, Failure):
return verdict_kind(records)
if isinstance(entries, Failure):
return verdict_kind(entries)
# 0 for the route's mark, as before any step-up: the replay compares verdict kinds, and a mark changes only which escalation quotes count.
+21 -18
View File
@@ -1,17 +1,18 @@
import pathlib
from chunk_pick import Cut, default_plan, LastCut, next_release, Unknown
from release_standing import Cut, default_plan, LastCut, next_release, Unknown
from failure import Failure
from todo_table import Item, items, NoItemsTable, renames, version
from todo_table import Item, items, NoItemsTable, renames
from release import Release, version
from git_repo import top
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed: tuple[int, int, int] | None, renamed: set[tuple[int, int, int]]) -> list[str] | Failure:
def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], landed_release: Release | None, renamed: set[Release]) -> list[str] | Failure:
"""Why the plan calls a scoring run, empty when it does not.
A chunk that ships an item naming the floor is due one, so it can prove the floor met. So is a chunk that empties a release after work has landed towards it, so no release ends unmeasured. A release the chunk's plan marks renamed is due none.
before is the default branch's plan and after the chunk's. ships holds the IDs of the items the chunk ships. landed is the next release where work has landed towards it, else None (`landed_towards`). renamed holds the releases the chunk's plan marks renamed.
before is the default branch's plan and after the chunk's. ships holds the IDs of the items the chunk ships. landed_release is the next release where work has landed towards it, else None (`landed_towards`). renamed holds the releases the chunk's plan marks renamed.
"""
plan = {item.id: item for item in before + after}
if unknown := sorted(set(ships) - set(plan)):
@@ -19,32 +20,34 @@ def scoring_run_due(before: list[Item], after: list[Item], ships: list[int], lan
shipped = [plan[id_] for id_ in ships]
reasons = [f"ships item {item.id}, which names the comprehension floor" for item in shipped if "comprehension floor" in item.tagline.lower()]
for release in emptied(before, after, shipped, renamed):
if work_landed(release, shipped, landed):
if work_landed(release, shipped, landed_release):
# SKILL.md → "Second run" keys on this wording.
reasons.append(f"leaves release {version(release)} with no plan item")
return reasons
# larv-review --help points here rather than restating the rule, so the rule is written once.
def work_landed(release: tuple[int, int, int], shipped: list[Item], landed: tuple[int, int, int] | None) -> bool:
"""Whether work has landed towards release, for the scoring-run trigger. It has when the chunk ships one of the release's items. It has too when release equals landed: the default branch's next release, once work has landed towards it."""
return any(item.release == release for item in shipped) or release == landed
def work_landed(release: Release, shipped: list[Item], landed_release: Release | None) -> bool:
"""Whether work has landed towards release, for the scoring-run trigger. It has when the chunk ships one of the release's items. It has too when release equals landed_release: the default branch's next release, once work has landed towards it."""
return any(item.release == release for item in shipped) or release == landed_release
def landed_towards(found: list[Item], cut: LastCut) -> tuple[int, int, int] | None | Failure:
"""The next release on the default branch, once work has landed towards it: once one or more changes are unreleased, as `chunk_pick.unreleased_changes` counts them. None where there is no next release, or no change is unreleased. found is the default branch's plan."""
def landed_towards(found: list[Item], cut: LastCut) -> Release | None | Failure:
"""The next release on the default branch, once work has landed towards it: once one or more changes are unreleased, as `release_standing.unreleased_changes` counts them. None where there is no next release, or no change is unreleased. found is the default branch's plan."""
if isinstance(cut.tag, Failure):
return Failure(f"could not tell the next release: the default branch's tags cannot be read: {cut.tag.reason}")
upcoming = next_release(found, cut)
if isinstance(upcoming, Unknown):
return Failure("could not tell the next release: the default branch's todo.md history cannot be read")
# Cut's release is the last cut's version or earlier, so no work lands towards it.
if isinstance(upcoming, Cut):
return None
if isinstance(upcoming.landed, Failure):
return Failure(f"could not tell whether work landed towards the next release: {upcoming.landed.reason}")
return upcoming.release if upcoming.landed else None
if isinstance(upcoming.unreleased, Failure):
return Failure(f"could not tell whether work landed towards the next release: {upcoming.unreleased.reason}")
return upcoming.release if upcoming.unreleased else None
def emptied(before: list[Item], after: list[Item], shipped: list[Item], renamed: set[tuple[int, int, int]]) -> list[tuple[int, int, int]]:
def emptied(before: list[Item], after: list[Item], shipped: list[Item], renamed: set[Release]) -> list[Release]:
"""Each release the default branch's plan or a shipped item names that the chunk's plan, less the shipped items, names no longer, unless the chunk's plan marks it renamed."""
named = {item.release for item in before + shipped}
kept = [item for item in after if item.id not in {ship.id for ship in shipped}]
@@ -78,7 +81,7 @@ def plan_reasons(default: str, ids: list[int]) -> list[str] | Failure:
renamed = renames(after)
if isinstance(renamed, Failure):
return Failure(f"the checkout's todo.md: {renamed.reason}")
landed = landed_towards(planned, held.cut)
if isinstance(landed, Failure):
return landed
return scoring_run_due(planned, chunk, ids, landed, renamed)
landed_release = landed_towards(planned, held.cut)
if isinstance(landed_release, Failure):
return landed_release
return scoring_run_due(planned, chunk, ids, landed_release, renamed)
+2 -1
View File
@@ -18,7 +18,7 @@ from failure import Failure
from test_env import isolated_env
from todo_table import Item, items, NoItemsTable
from larv_review.plan import landed_towards, plan_reasons, scoring_run_due
from chunk_pick import LastCut
from release_standing import LastCut
from larv_review.scoring import means, under_floor
from larv_review.ruling_base import measured_commit
from larv_review.seats import score_line, scores, seats_file
@@ -337,6 +337,7 @@ class LandedTowards(unittest.TestCase):
plan = [item(1, (0, 2, 0))]
self.assertIsInstance(landed_towards(plan, LastCut("v0.1.0", Failure("unreadable"), Failure("count unreadable"), [])), Failure)
self.assertIsInstance(landed_towards([], LastCut("v0.1.0", "", 3, Failure("log unreadable"))), Failure)
self.assertEqual(landed_towards(plan, LastCut(Failure("fatal: bad object"), "", 3, [])), Failure("could not tell the next release: the default branch's tags cannot be read: fatal: bad object"))
class LarvReviewTool(unittest.TestCase):
+4
View File
@@ -0,0 +1,4 @@
"""loop-claude: headless claude cycles, judged, that interrupt the human only for a decision that is theirs (loop/main.py's help).
Its packages hold one concern each: cycle/ runs a cycle, loop/ runs the cycles, project/ reads a checkout's files, and waiting/ shows a question to loop-inbox. Each module at this level is imported by more than one package, and loop-inbox imports several. A reader finds a module by its name, so the modules stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows that.
"""
+11 -11
View File
@@ -1,4 +1,5 @@
import dataclasses
import pathlib
import sys
import typing
import uuid
@@ -12,7 +13,6 @@ from claude_code.transcript import human_turns, read_transcript
from loop_claude.waiting.listing import listed, shown
from loop_claude.waiting.talk import make_way, TALK
from loop_claude.stop import needs_human, needs_terminal, Stop
from loop_claude.paths import Paths
from model_routing.ladder import Rung
ANSWER_QUESTION = "answer the question above"
@@ -39,7 +39,7 @@ class Talked:
"""The human typed in the worker's claude session."""
def interrupt(question: str, answer_here: bool, session: WorkerSession, paths: Paths, effects: Effects) -> str | Talked | Stop:
def interrupt(question: str, answer_here: bool, session: WorkerSession, questions: pathlib.Path, effects: Effects) -> str | Talked | Stop:
"""Prints question for the human and returns their answer: where answer_here, the first line typed at loop-claude's terminal or in loop-inbox, stripped, or Talked once they typed in the worker's claude session, which TALK opens; else Talked once they typed in that session, which opens at once on the terminal. loop-inbox lists the question meanwhile. A Stop where loop-claude is off a terminal, or they type nothing, or the input ends."""
say(f"\n{question}", effects)
if stop := needs_terminal(ANSWER_QUESTION, effects.on_terminal()):
@@ -47,18 +47,18 @@ def interrupt(question: str, answer_here: bool, session: WorkerSession, paths: P
# Keys typed while the worker ran wait on the terminal, and would answer a question no one has read, here or in claude's session, which reads them too. Of a pasted answer, the first line answers; the rest wait, and the next question drops them. A longer answer goes through TALK.
effects.discard_typed()
ask = Ask(effects.new_id(), str(session.id), session.root, question, effects.now(), Asking(session.claude_command()) if answer_here else OpenOnTerminal())
with listed(paths.waiting, ask, effects):
answer = first_answer(ask, paths, effects) if answer_here else OpenOnTerminal()
with listed(questions, ask, effects):
answer = first_answer(ask, questions, effects) if answer_here else OpenOnTerminal()
if isinstance(answer, OpenOnTerminal | OpenInInbox):
return human_typed(session.transcript_path, lambda: claude_open(ask, answer, session, paths, effects)) or Talked()
return human_typed(session.transcript_path, lambda: claude_open(ask, answer, session, questions, effects)) or Talked()
return answer
def first_answer(ask: Ask, paths: Paths, effects: Effects) -> str | OpenOnTerminal | OpenInInbox | Stop:
def first_answer(ask: Ask, questions: pathlib.Path, effects: Effects) -> str | OpenOnTerminal | OpenInInbox | Stop:
"""The first answer to ask, from loop-claude's terminal or loop-inbox: the line typed, stripped, or where claude is to open for it. A Stop where they type nothing, or the input ends."""
try:
# Annotated since mypy infers input_or's T as object from the lambda.
answer: str | Answer = effects.input_or(ASK_ANSWER, lambda: answered(paths.waiting, ask))
answer: str | Answer = effects.input_or(ASK_ANSWER, lambda: answered(questions, ask))
except EOFError:
print(file=sys.stderr) # Ends the prompt's line. It isn't a message, so it gets no time.
return needs_human(ANSWER_QUESTION, "the input ended")
@@ -77,17 +77,17 @@ def first_answer(ask: Ask, paths: Paths, effects: Effects) -> str | OpenOnTermin
return OpenOnTerminal() if text == TALK else text
def claude_open(ask: Ask, where: OpenOnTerminal | OpenInInbox, session: WorkerSession, paths: Paths, effects: Effects) -> Stop | None:
def claude_open(ask: Ask, where: OpenOnTerminal | OpenInInbox, session: WorkerSession, questions: pathlib.Path, effects: Effects) -> Stop | None:
"""Lists ask in loop-inbox as having claude open where, then returns once that claude has ended: the one it opens on the terminal, or the one loop-inbox opened."""
match where:
case OpenOnTerminal():
shown(paths.waiting, dataclasses.replace(ask, state=where), effects)
shown(questions, dataclasses.replace(ask, state=where), effects)
if unstarted := hand_over(session.claude_command(), effects):
return Stop(1, f"could not start claude for the interrupt: {unstarted.reason}")
case OpenInInbox(pid):
make_way(paths.waiting, ask, pid, effects)
make_way(questions, ask, pid, effects)
say("claude is open for this question in loop-inbox; waiting for it to end", effects)
while not claude_ended(paths.waiting, ask) and effects.alive(pid):
while not claude_ended(questions, ask) and effects.alive(pid):
effects.sleep(POLL_SECONDS)
case _:
typing.assert_never(where)
+22 -10
View File
@@ -75,7 +75,7 @@ class Reply:
finished: bool = dataclasses.field(metadata={"description": "Whether the chunk is finished."})
interrupt: bool = dataclasses.field(metadata={"description": "Whether the human must now be interrupted."})
question: str = dataclasses.field(metadata={"description": "With interrupt true, the single question to put to the human. With a quoted answer, the worker's question that the answer field answers. Empty otherwise."})
# True opens claude's session at once, in place of the one-line answer (run.ask). The judge decides it, since loop-claude computes the plan's standing (chunk_pick.chunk_item) only when it routes a chunk on the ladder afresh: it would miss a release ready to cut in a resumed chunk or off the ladder. A wrong ruling costs the human one typed /talk, or answering in claude, which stops the loop if they leave it with nothing typed.
# True opens claude's session at once, in place of the one-line answer (run.ask). The judge decides it, since loop-claude computes the plan's standing (release_standing.chunk_item) only when it routes a chunk on the ladder afresh: it would miss a release ready to cut in a resumed chunk or off the ladder. A wrong ruling costs the human one typed /talk, or answering in claude, which stops the loop if they leave it with nothing typed.
release_ready: bool = dataclasses.field(metadata={"description": "With interrupt true, whether the question is about a release ready to cut, which only the human may publish. False otherwise."})
@@ -97,12 +97,12 @@ class Rejected(Failure):
class Grounds(typing.NamedTuple):
"""What run_judge checks the judge's quotes against, word for word, since stitched fragments can say what their source does not."""
# The transcript as the judge saw it. The evidence must quote it, its fragments in order: a misquote is Rejected, which judge retries.
"""The texts quotes_checked checks the judge's quotes against. A quote must appear in its text as written, since a quote reworded or stitched together can say what its source does not; only the evidence may join passages with an ellipsis, each passage in order."""
# The transcript as the judge saw it.
shown: str
# escalation_mark.since_step's lines. An escalation must quote both these and shown: a misquote drops the escalation, as escalation_mark.checked says.
# The worker's own lines after the escalation mark, as escalation_mark.since_step renders them.
since_step: str
# README.md's premises and the records. A quoted answer must quote them: a misquote puts the question to the human.
# README.md's premises and the records.
quotable: str
@@ -179,9 +179,10 @@ def run_judge(root: str, args: list[str], prompt: str, grounds: Grounds, effects
cost = cost_of(reply)
if isinstance(verdict, Failure):
return Rejected(verdict.reason, replied, cost)
if not grounded(verdict.evidence, grounds.shown):
return Rejected(f"the judge quoted {verdict.evidence!r}, which is not in the transcript" + (f", for the question: {verdict.question}" if isinstance(verdict, Interrupt | QuotedAnswer) else ""), replied, cost)
return dataclasses.replace(quotes_checked(verdict, grounds), judge_cost=cost)
checked = quotes_checked(verdict, grounds)
if isinstance(checked, Failure):
return Rejected(checked.reason, replied, cost)
return dataclasses.replace(checked, judge_cost=cost)
def cost_of(reply: dict) -> float:
@@ -190,8 +191,19 @@ def cost_of(reply: dict) -> float:
return float(cost) if isinstance(cost, int | float) and not isinstance(cost, bool) and cost > 0 else 0.0
def quotes_checked(verdict: Verdict, grounds: Grounds) -> Verdict:
"""The verdict with its quoted answer and its escalation checked. A quoted answer that neither README.md's premises nor the records hold goes to the human as an interrupt, which keeps the escalation the answer carried. Neither misquote is a Failure, which would otherwise exit an unattended loop."""
def quotes_checked(verdict: Verdict, grounds: Grounds) -> Verdict | Failure:
"""The verdict with each of its quotes checked against the text it must appear in, as written:
| Quote | Must be in | Where it isn't |
|---|---|---|
| the evidence | grounds.shown, the evidence's fragments in order | a Failure: run_judge rejects the reply, and judge retries it |
| a quoted answer | grounds.quotable | the question goes to the human as an interrupt, which keeps the escalation the answer carried |
| the escalation | grounds.shown and grounds.since_step | the escalation is dropped, so the worker's rung holds, as escalation_mark.checked says |
Only a misquoted evidence fails. A Failure for either of the other two would exit an unattended loop, so each changes the verdict instead.
"""
if not grounded(verdict.evidence, grounds.shown):
return Failure(f"the judge quoted {verdict.evidence!r}, which is not in the transcript" + (f", for the question: {verdict.question}" if isinstance(verdict, Interrupt | QuotedAnswer) else ""))
if isinstance(verdict, QuotedAnswer) and not verbatim(verdict.answer, grounds.quotable):
verdict = Interrupt(verdict.evidence, verdict.question, escalation=verdict.escalation, taken_in=verdict.taken_in)
return escalation_mark.checked(verdict, since=grounds.since_step, shown=grounds.shown)
+29 -26
View File
@@ -4,8 +4,8 @@ from model_routing.ladder import climbed, LADDER, position, Rung
from model_routing.route import ESCALATION_MARKER, Route
from loop_claude.routing import escalation_rungs
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES
from review_passes import INSTRUCTION_FILES, INSTRUCTION_FOLDERS
from markdown_text import listed, quoted
from review_passes import REVIEWERS
from claude_code.transcript import Typed
# Where the judge may quote an answer from, named as the human and the worker see them; of README.md the judge sees only the premises.
@@ -13,7 +13,7 @@ QUOTE_SOURCES = listed(("README.md's Goals and Audience", *RECORD_FILES, *HUMAN_
# Where the worker may write an answer the human typed, so it counts as recorded.
ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES) + ", todo.md or a comment"
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
# How review_launches names the reviews: "each reviewer" covers every pass with an agent that no replay lowered.
# How review_launches names the reviews: "each reviewer" covers every pass with an agent.
EACH_REVIEWER = "each reviewer"
EACH_SEAT = "each comprehension seat"
THE_SECURITY_PASS = "the security pass's subagent"
@@ -57,28 +57,37 @@ def preflight_failed(reason: str) -> str:
def review_launches(route: Route, project: Rung | None) -> list[str]:
"""The sentences naming where to launch reviews, in this order; none off the ladder:
- one for each rung, other than the worker's, that a reviewer, a seat or the security pass runs on;
- the rung of each pass a replay lowered;
- where the security pass runs on another rung than the worker's, that a full scan runs only from its subagent;
- where the worker runs below the project's rung, where to launch a subagent that files or scores todo.md items;
- the reviewers' rung each higher E + I picks;
- where the reviewers run above the project's rung, that a branch changing only Markdown no agent follows reviews on it, as rule-hook's reviewer-rung allows.
For a worker on Sonnet 5.5 at high, on an Opus 5.5 project, routed by item 3 of E + I 5, they open: `Launch each reviewer with the Agent tool set to model opus, effort high, the rung item 3's E + I of 5 picks.` and `Launch each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.` Where a replay lowered the prose pass to Sonnet 5.5 at high, and the E + I steps no review up, a sentence follows: `Launch each prose-reviewer instead with the Agent tool set to model sonnet, effort high, the rung the replay of its pass chose.`
"""
"""The sentences naming where to launch reviews, in this order, each from the function that says when it applies; none off the ladder."""
if project is None:
return []
choice = chosen(project, route)
lowered = [f"Launch each {REVIEWERS[kind]} instead with the Agent tool set to {rung.agent_args()}, the rung the replay of its pass chose." for kind, rung in choice.lowered.items()]
scan = ["Run a full scan only from such a subagent, since the scan's agents run on the model that starts them."] if choice.security != route.rung else []
planning = [f"Your rung is below the project's: launch each subagent that files or scores todo.md items on the rung `~/.claude/bin/subagent-rung --plan {project.model} <L>` prints."] if position(route.rung) < position(project) else []
markdown = [f"Where your branch changes only Markdown, none of it `CLAUDE.md`, `AGENTS.md` or under `agents/`, `principles/`, `rules/` or `skills/`, launch {EACH_REVIEWER} on the project's rung instead, with the Agent tool set to {project.agent_args()}."] if position(choice.reviewers) > position(project) else []
return [*shared_launches(choice, route, project), *lowered, *scan, *planning, *taken_in_launches(choice), *markdown]
return [*shared_launches(choice, route, project), *scan_launch(choice, route), *planning_launch(route, project), *taken_in_launches(choice), *markdown_launch(choice, project)]
def scan_launch(choice: ReviewChoice, route: Route) -> list[str]:
"""Where the security pass runs on another rung than the worker's, the sentence that a full scan runs only from the security pass's subagent, which shared_launches names just before: `Run a full scan only from such a subagent, since the scan's agents run on the model that starts them.`"""
return ["Run a full scan only from such a subagent, since the scan's agents run on the model that starts them."] if choice.security != route.rung else []
def planning_launch(route: Route, project: Rung) -> list[str]:
"""Where the worker runs below the project's rung, the sentence naming where to launch a subagent that files or scores todo.md items. On an Opus 5.5 project: `Your rung is below the project's: launch each subagent that files or scores todo.md items on the rung `~/.claude/bin/subagent-rung --plan claude-opus-5-5 <L>` prints.`"""
return [f"Your rung is below the project's: launch each subagent that files or scores todo.md items on the rung `~/.claude/bin/subagent-rung --plan {project.model} <L>` prints."] if position(route.rung) < position(project) else []
def markdown_launch(choice: ReviewChoice, project: Rung) -> list[str]:
"""Where the reviewers run above the project's rung, the sentence that a branch changing only Markdown no agent follows reviews on the project's rung, as rule-hook's reviewer-rung allows. On an Opus 5.5 project it ends: `launch each reviewer on the project's rung instead, with the Agent tool set to model opus, effort medium.`"""
if position(choice.reviewers) <= position(project):
return []
files = ", ".join(f"`{name}`" for name in INSTRUCTION_FILES)
folders = listed([f"`{folder}/`" for folder in INSTRUCTION_FOLDERS], "or")
return [f"Where your branch changes only Markdown, none of it {files} or under {folders}, launch {EACH_REVIEWER} on the project's rung instead, with the Agent tool set to {project.agent_args()}."]
def shared_launches(choice: ReviewChoice, route: Route, project: Rung) -> list[str]:
"""A sentence for each rung, other than the worker's, that the reviewers no replay lowered, the seats or the security pass run on."""
"""A sentence for each rung, other than the worker's, that the reviewers, the seats or the security pass run on.
For a worker on Sonnet 5.5 at high, on an Opus 5.5 project, routed by item 3 of E + I 5, they read: `Launch each reviewer with the Agent tool set to model opus, effort high, the rung item 3's E + I of 5 picks.` and `Launch each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.`
"""
together = {EACH_REVIEWER: choice.reviewers, EACH_SEAT: choice.seat, THE_SECURITY_PASS: choice.security}
launched: dict[Rung, list[str]] = {}
for reviews, rung in together.items():
@@ -131,19 +140,13 @@ def escalation(route: Route, project: Rung | None) -> list[str]:
def reruns(route: Route, project: Rung | None) -> list[str]:
"""The sentences naming the rung one above a review's, on which it reruns when it asks: shared_reruns' for a reviewer and a seat, then one for each pass a replay lowered. None off the ladder.
"""The sentence naming the rung one above a reviewer's and the one above a seat's, on which each reruns when it asks, leaving out either that is already on the top rung. None off the ladder.
For a worker on Sonnet 5.5 at high, on an Opus 5.5 project whose prose pass a replay lowered to Sonnet 5.5 at high, routed by an item of E + I 4, they read: `A reviewer or comprehension seat that asks for a rerun one rung up reruns with the Agent tool set to `model opus, effort high`.` and `Rerun each prose-reviewer that asks with the Agent tool set to `model opus, effort medium` instead.`
For a worker on Sonnet 5.5 at high, on an Opus 5.5 project, routed by an item of E + I 4, it reads: `A reviewer or comprehension seat that asks for a rerun one rung up reruns with the Agent tool set to `model opus, effort high`.`
"""
if project is None or route.rung not in LADDER:
return []
choice = chosen(project, route)
lowered = [f"Rerun each {REVIEWERS[kind]} that asks with the Agent tool set to `{climbed(rung, 1).agent_args()}` instead." for kind, rung in choice.lowered.items()]
return [*shared_reruns(choice), *lowered]
def shared_reruns(choice: ReviewChoice) -> list[str]:
"""The sentence naming the rung one above choice.reviewers and the one above choice.seat, leaving out either that is already on the top rung."""
launched = (("reviewer", choice.reviewers), ("comprehension seat", choice.seat))
rerun = {name: climbed(rung, 1) for name, rung in launched if rung != LADDER[-1]}
if not rerun:
+62 -64
View File
@@ -2,8 +2,7 @@ import dataclasses
import pathlib
import typing
from loop_claude.cycle.states import Active, Archived, Closing, Ended, Event, HumanTalked, HumanTyped, Interrupting, Judging, QUOTED_ANSWERS_PER_CYCLE, Ruled, State, transition, TypedAnswer, WorkerRan, Working
from loop_claude.routing import after_ruling, judge_rung
from loop_claude.cycle.states import Active, Archived, Closing, Ended, Event, HumanTalked, HumanTyped, Interrupting, Judging, QUOTED_ANSWERS_PER_CYCLE, Ruled, State, to_the_human, transition, TypedAnswer, WorkerRan, Working
from transcript_archive.write import archive
from loop_claude.cycle.worker import AUTO_COMPACT_WINDOW, COMPACT_WINDOW, run_worker, worker_args
from claude_code.session import delete_session, Session, session_open, transcript_file
@@ -12,38 +11,42 @@ from failure import Failure
from loop_claude.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished, Verdict
from loop_claude.cycle.interrupt import interrupt, Talked, WorkerSession
from loop_claude.cycle.judge import judge, Rejected
from loop_claude.routing import judge_rung, next_route, step_up, StepUp
from transcript_archive import outcomes
from loop_claude.paths import Paths
from loop_claude.cycle import prompts
from claude_code.transcript import read_transcript
from loop_claude.cycle.retry import retried
from model_routing.route import Route
from model_routing.ladder import Rung
from loop_claude.escalation_mark import StepUp
from loop_claude.stop import Stop
from model_routing import worker_rungs
@dataclasses.dataclass(frozen=True)
class Cycle:
"""What a cycle runs on. run_cycle replaces the route on a ruling's step-up; nothing changes the other fields.
Each step reads the few fields it needs by name, so the fields stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows more than seven where a reader finds them by name.
"""
effects: Effects
# What chunk-preflight found, or why it could not run, for a fresh worker's prompt; empty when the session was open as the cycle began.
facts: str
judge_model: str
paths: Paths
premises: str
project_archive: pathlib.Path
records: str
root: str
session: Session
task: str
route: Route
# The project's rung; None for a model off the ladder.
project: Rung | None
# Empty when the previous cycle's verdict was not Unfinished.
unfinished_evidence: str
project_archive: pathlib.Path
# Where the loop keeps its questions for loop-inbox: running_loops.Listed.folder.
questions: pathlib.Path
records: str
root: str
route: Route
session: Session
# Lists in loop-inbox the items the chunk takes, given the IDs an unfinished ruling found taken in.
show_taken_in: typing.Callable[[tuple[int, ...]], None]
task: str
# Empty when the previous cycle's verdict was not Unfinished.
unfinished_evidence: str
@dataclasses.dataclass(frozen=True)
@@ -60,9 +63,9 @@ def run_cycle(cycle: Cycle) -> Stop | Concluded:
event = step(cycle, state)
if isinstance(event, Stop):
return stopped(cycle, event, event.reason)
# Only a ruling's step-up changes the route within a cycle. No transition reads the current route, so Cycle holds it, not the states.
if isinstance(event, Ruled) and event.step_up:
cycle = dataclasses.replace(cycle, route=event.step_up.after)
# Only a ruling's step-up changes the route within a cycle, here, once rule has archived the ruling on the route before it. No transition reads the current route, so Cycle holds it, not the states.
if isinstance(event, Ruled) and event.stepped_to:
cycle = dataclasses.replace(cycle, route=event.stepped_to)
if isinstance(event, Ruled) and not isinstance(event.verdict, Finished):
cycle.show_taken_in(event.verdict.taken_in.items)
state = transition(state, event)
@@ -72,11 +75,8 @@ def run_cycle(cycle: Cycle) -> Stop | Concluded:
def ruling_step_up(cycle: Cycle, verdict: Verdict) -> StepUp | None:
"""The step-up an interrupt's or a quoted answer's escalation makes, so the work after the answer runs on the stepped route; None where the route holds."""
if not (isinstance(verdict, Interrupt | QuotedAnswer) and cycle.project):
return None
after = after_ruling(cycle.project, cycle.route, verdict)
return StepUp(cycle.route, after) if after != cycle.route else None
"""The step-up an interrupt's or a quoted answer's escalation makes, so the work after the answer runs on the stepped route; None where the route holds. A finished or unfinished verdict steps the route only as the loop starts the next cycle."""
return step_up(cycle.route, next_route(cycle.project, cycle.route, verdict)) if isinstance(verdict, Interrupt | QuotedAnswer) else None
def stopped(cycle: Cycle, stop: Stop, why: str) -> Stop:
@@ -97,25 +97,23 @@ def judge_of(cycle: Cycle) -> Rung:
def step(cycle: Cycle, state: Active) -> Event | Stop:
match state:
case Working(answered, _, step_up):
return work(cycle, answered, step_up)
case Judging(transcript_path, _):
return rule(cycle, transcript_path)
case Interrupting(transcript_path, ruling, _, step_up):
return ask(cycle, transcript_path, ruling, step_up)
case Working(answered, _, stepped):
return work(cycle, answered, stepped)
case Judging(transcript_path, quoted_answers):
return rule(cycle, transcript_path, quoted_answers)
case Interrupting(transcript_path, ruling, _, _):
return ask(cycle, transcript_path, ruling)
case Closing(transcript_path, verdict):
return close(cycle, transcript_path, verdict)
case _:
typing.assert_never(state)
def work(cycle: Cycle, answered: QuotedAnswer | TypedAnswer | None, step_up: StepUp | None) -> WorkerRan | Stop:
def work(cycle: Cycle, answered: QuotedAnswer | TypedAnswer | None, stepped: bool) -> WorkerRan | Stop:
if isinstance(answered, QuotedAnswer):
say(f"\n=== {prompts.QUOTE_SOURCES} answered: {answered.question} ===\n{answered.answer}", cycle.effects, stdout=True)
if stop := keep_answer(cycle, answered, step_up):
return stop
# worker_start runs inside the lambda: a failed attempt may have opened the session.
if stop := retried("worker", lambda: run_worker(cycle.root, worker_start(cycle, answered, step_up), worker_environment(cycle), cycle.effects), cycle.effects):
if stop := retried("worker", lambda: run_worker(cycle.root, worker_start(cycle, answered, stepped), worker_environment(cycle), cycle.effects), cycle.effects):
return stop
path = transcript_file(cycle.session)
return WorkerRan(path) if path is not None else Stop(1, f"claude left no transcript for session {cycle.session.id}")
@@ -130,33 +128,37 @@ def worker_session(cycle: Cycle, transcript_path: str) -> WorkerSession:
return WorkerSession(cycle.root, cycle.session.id, cycle.route.rung, transcript_path, worker_environment(cycle))
def worker_start(cycle: Cycle, answered: QuotedAnswer | TypedAnswer | None, step_up: StepUp | None) -> list[str]:
def worker_start(cycle: Cycle, answered: QuotedAnswer | TypedAnswer | None, stepped: bool) -> list[str]:
if isinstance(answered, QuotedAnswer):
return worker_args(cycle.session, cycle.route.rung, prompts.answered(answered.question, answered.answer, cycle.route, cycle.project, step_up is not None), resume=True)
return worker_args(cycle.session, cycle.route.rung, prompts.answered(answered.question, answered.answer, cycle.route, cycle.project, stepped), resume=True)
if isinstance(answered, TypedAnswer):
return worker_args(cycle.session, cycle.route.rung, prompts.typed(answered.question, answered.answer, cycle.route, cycle.project, step_up is not None), resume=True)
return worker_args(cycle.session, cycle.route.rung, prompts.typed(answered.question, answered.answer, cycle.route, cycle.project, stepped), resume=True)
if session_open(cycle.session):
return worker_args(cycle.session, cycle.route.rung, prompts.resumed(cycle.unfinished_evidence, cycle.route, cycle.project), resume=True)
return worker_args(cycle.session, cycle.route.rung, prompts.fresh(cycle.task, cycle.premises, cycle.route, cycle.project, cycle.facts), resume=False)
def keep_answer(cycle: Cycle, answered: QuotedAnswer, step_up: StepUp | None) -> Stop | None:
path = transcript_file(cycle.session)
if not path:
return None
# As in ask: the answer names the rung the worker ran on until then, and a step-up archived after it names the rung it runs on from then.
quoted = outcomes.answered(answered.question, prompts.QUOTE_SOURCES, answered.answer, answered.evidence, answered.escalation, str(judge_of(cycle)), ran_on(cycle, step_up).worker_line(), answered.judge_cost)
if failure := archive_stepped(cycle, path, quoted, step_up):
return Stop(1, failure.reason)
return None
def rule(cycle: Cycle, transcript_path: str) -> Ruled | Stop:
def rule(cycle: Cycle, transcript_path: str, quoted_answers: int) -> Ruled | Stop:
"""The judge's ruling. An interrupt or a quoted answer is archived as it is ruled, on the route the worker ran on until then, and then any step-up it makes, so the figures charge each stretch of work to its own rung."""
entries = read_transcript(transcript_path)
if isinstance(entries, Failure):
return Stop(1, entries.reason)
verdict = judge(cycle.root, cycle.task, cycle.premises, cycle.records, entries, cycle.route.escalated_at, judge_of(cycle), cycle.effects, lambda rejected: keep_rejection(cycle, transcript_path, rejected))
return verdict if isinstance(verdict, Stop) else Ruled(verdict, ruling_step_up(cycle, verdict))
if isinstance(verdict, Stop):
return verdict
if not isinstance(verdict, Interrupt | QuotedAnswer):
return Ruled(verdict)
stepped = ruling_step_up(cycle, verdict)
if failure := archive_stepped(cycle, transcript_path, ruling_outcome(cycle, verdict, quoted_answers), stepped):
return Stop(1, failure.reason)
return Ruled(verdict, stepped.after if stepped else None)
def ruling_outcome(cycle: Cycle, ruling: Interrupt | QuotedAnswer, quoted_answers: int) -> str:
"""The archive's outcome for a ruling: interrupted where it goes to the human, with the question put to them, and else answered from README.md or the records."""
if isinstance(ruling, QuotedAnswer) and not to_the_human(ruling, quoted_answers):
return outcomes.answered(ruling.question, prompts.QUOTE_SOURCES, ruling.answer, ruling.evidence, ruling.escalation, str(judge_of(cycle)), cycle.route.worker_line(), ruling.judge_cost)
return outcomes.interrupted(question_for(ruling), ruling.evidence, str(judge_of(cycle)), ruling.escalation, cycle.route.worker_line(), ruling.judge_cost)
def keep_rejection(cycle: Cycle, transcript_path: str, rejected: Rejected) -> Stop | None:
@@ -166,15 +168,10 @@ def keep_rejection(cycle: Cycle, transcript_path: str, rejected: Rejected) -> St
return None
def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer, step_up: StepUp | None) -> HumanTyped | HumanTalked | Stop:
question = question_for(ruling)
# Archived now, so the transcript the question was asked on is kept with it; the human's answer then grows the transcript.
# The interrupt names the rung the worker ran on before it; a step-up archived after it names the stepped rung, so the figures charge each stretch to its own rung.
interrupted = outcomes.interrupted(question, ruling.evidence, str(judge_of(cycle)), ruling.escalation, ran_on(cycle, step_up).worker_line(), ruling.judge_cost)
if failure := archive_stepped(cycle, transcript_path, interrupted, step_up):
return Stop(1, failure.reason)
def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) -> HumanTyped | HumanTalked | Stop:
"""The human's answer to a ruling that rule archived as it ruled; the human's answer then grows the transcript."""
# Only the human may cut a release, so a release ready to cut opens claude's session at once.
answer = interrupt(question, not (isinstance(ruling, Interrupt) and ruling.release_ready), worker_session(cycle, transcript_path), cycle.paths, cycle.effects)
answer = interrupt(question_for(ruling), not (isinstance(ruling, Interrupt) and ruling.release_ready), worker_session(cycle, transcript_path), cycle.questions, cycle.effects)
match answer:
case Stop():
return answer
@@ -184,17 +181,18 @@ def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer, st
return HumanTyped(TypedAnswer(ruling.question, answer))
def ran_on(cycle: Cycle, step_up: StepUp | None) -> Route:
"""The route the worker ran on until the ruling: the one before its step-up, if it made one."""
return step_up.before if step_up else cycle.route
def archive_stepped(cycle: Cycle, transcript_path: str, outcome: str, step_up: StepUp | None) -> Failure | None:
"""Archives the outcome of a ruling, then the step-up it made, if it made one. The first failure is returned."""
for kept in (outcome, outcomes.stepped_up(step_up.after.worker_line())) if step_up else (outcome,):
if failure := archive(cycle.project_archive, transcript_path, cycle.task, kept, cycle.effects.now(), lambda text: say(text, cycle.effects)):
return failure
return None
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcome, cycle.effects.now(), lambda text: say(text, cycle.effects)):
return failure
return archived_step(cycle.project_archive, transcript_path, cycle.task, step_up, cycle.effects)
def archived_step(project_archive: pathlib.Path, transcript_path: str, task: str, step_up: StepUp | None, effects: Effects) -> Failure | None:
"""Archives a step-up, if there is one, as the worker line it steps to. The outcome archived before it names the route the worker ran on until then, so the figures charge each stretch of work to its own rung. Both a ruling within a cycle and an unfinished verdict between cycles archive their step-up here."""
if step_up is None:
return None
return archive(project_archive, transcript_path, task, outcomes.stepped_up(step_up.after.worker_line()), effects.now(), lambda text: say(text, effects))
def question_for(ruling: Interrupt | QuotedAnswer) -> str:
+18 -15
View File
@@ -1,7 +1,7 @@
import dataclasses
from loop_claude.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished, Verdict
from loop_claude.escalation_mark import StepUp
from model_routing.route import Route
# A backstop for what the judge misses, such as a reworded repeat, so the worker cannot loop unattended on answers quoted from README.md or the records.
QUOTED_ANSWERS_PER_CYCLE = 3
@@ -19,8 +19,8 @@ class Working:
"""The worker runs, resumed with an answer quoted from README.md or the records, or typed by the human, when there is one."""
answered: QuotedAnswer | TypedAnswer | None = None
quoted_answers: int = 0
# The step-up made by the ruling that brought the answer, so the worker is told of it; None for none.
step_up: StepUp | None = None
# Whether the ruling that brought the answer stepped the worker up a rung, so the worker is told of it.
stepped: bool = False
@dataclasses.dataclass(frozen=True)
@@ -36,8 +36,8 @@ class Interrupting:
transcript_path: str
ruling: Interrupt | QuotedAnswer
quoted_answers: int = 0
# The step-up the ruling made. It holds through the answer the human types, and drops on /talk.
step_up: StepUp | None = None
# Whether the ruling stepped the worker up a rung. It holds through the answer the human types, and drops on /talk.
stepped: bool = False
@dataclasses.dataclass(frozen=True)
@@ -63,8 +63,8 @@ class WorkerRan:
@dataclasses.dataclass(frozen=True)
class Ruled:
verdict: Verdict
# None unless an interrupt's or a quoted answer's escalation stepped the route; always None for a finished or unfinished verdict.
step_up: StepUp | None = None
# The route an interrupt's or a quoted answer's escalation stepped the worker up to, which the cycle runs on from here; None where the route holds, and always for a finished or unfinished verdict.
stepped_to: Route | None = None
@dataclasses.dataclass(frozen=True)
@@ -85,20 +85,23 @@ class Archived:
Event = WorkerRan | Ruled | HumanTyped | HumanTalked | Archived
def to_the_human(ruling: Interrupt | QuotedAnswer, quoted_answers: int) -> bool:
"""Whether a ruling goes to the human: an interrupt always, a quoted answer once the cycle has used up its QUOTED_ANSWERS_PER_CYCLE."""
return isinstance(ruling, Interrupt) or quoted_answers >= QUOTED_ANSWERS_PER_CYCLE
def transition(state: State, event: Event) -> State:
match state, event:
case Working(_, quoted_answers), WorkerRan(transcript_path):
return Judging(transcript_path, quoted_answers)
case Judging(_, quoted_answers), Ruled(QuotedAnswer() as answered, step_up) if quoted_answers < QUOTED_ANSWERS_PER_CYCLE:
return Working(answered, quoted_answers + 1, step_up)
case Judging(transcript_path, quoted_answers), Ruled(QuotedAnswer() as ruling, step_up) if quoted_answers >= QUOTED_ANSWERS_PER_CYCLE:
return Interrupting(transcript_path, ruling, quoted_answers, step_up)
case Judging(transcript_path, quoted_answers), Ruled(Interrupt() as ruling, step_up):
return Interrupting(transcript_path, ruling, quoted_answers, step_up)
case Judging(transcript_path, quoted_answers), Ruled(Interrupt() | QuotedAnswer() as ruling, stepped_to) if to_the_human(ruling, quoted_answers):
return Interrupting(transcript_path, ruling, quoted_answers, stepped_to is not None)
case Judging(_, quoted_answers), Ruled(QuotedAnswer() as answered, stepped_to):
return Working(answered, quoted_answers + 1, stepped_to is not None)
case Judging(transcript_path, _), Ruled(Finished() | Unfinished() as verdict):
return Closing(transcript_path, verdict)
case Interrupting(_, _, quoted_answers, step_up), HumanTyped(typed):
return Working(typed, quoted_answers, step_up)
case Interrupting(_, _, quoted_answers, stepped), HumanTyped(typed):
return Working(typed, quoted_answers, stepped)
# The human worked in the worker's claude session, so the judge rules again on the transcript it leaves.
case Interrupting(transcript_path, _, quoted_answers), HumanTalked():
return Judging(transcript_path, quoted_answers)
+1
View File
@@ -1,3 +1,4 @@
"""The effects loop-claude reaches the world through, such as its terminal, clock, processes, git and the machine's memory. loop-inbox runs on them too, so a change here, such as to how input reads a line, changes both tools."""
import dataclasses
import os
import select
+1 -9
View File
@@ -3,7 +3,7 @@
A worker whose run ends asking loop-claude for a higher rung cites the trigger that fired, and the judge quotes the passage where it fired as the verdict's escalation. The mark, Route.escalated_at, is how many transcript entries the session held when an escalation last stepped the worker up, 0 before any. The session's transcript only grows, so the mark holds across cycles, and the archive's Worker: lines keep it for a rerun.
- Stamp. stamped() writes on the judge's verdict how many entries the judge read.
- Step. routing.after_unfinished() and routing.after_ruling() step the worker up on an escalation and move the mark to moved_to(verdict). A ruling's StepUp travels through the cycle's states, so the archive and the worker's next prompt name it.
- Step. routing.next_route() steps the worker up on an escalation and moves the mark to moved_to(verdict).
- Show. transcript() puts STEP_LINE after the mark, telling the judge that the triggers before that line are spent. QUOTE_AFTER_STEP, in the judge's schema, tells the judge to quote only the worker's words after that line.
- Check. checked() keeps an escalation only where it quotes both the transcript the judge was shown and the worker's own lines after the mark, which since_step() renders, so a passage that bought one step cannot buy another.
"""
@@ -13,7 +13,6 @@ import typing
from loop_claude.verdicts import Escalating, Verdict
from claude_code.entries import human_text, loop_text
from claude_code.transcript import render, Typed, verbatim
from model_routing.route import Route
# Only the judge reads the step line, so it is not a [loop] line.
STEP_TAG = "[step]"
@@ -22,13 +21,6 @@ STEP_LINE = f"{STEP_TAG} loop-claude stepped the worker up a rung here, so the e
QUOTE_AFTER_STEP = f"Where a {STEP_TAG} line says loop-claude stepped the worker up a rung, quote only the worker's own words after it, and leave this empty where the trigger fired only before it."
@dataclasses.dataclass(frozen=True)
class StepUp:
"""An interrupt's or a quoted answer's escalation stepping the route within a cycle: the route the worker ran on until the ruling, and the one it runs on after."""
before: Route
after: Route
def stamped(verdict: Verdict, entries_read: int) -> Verdict:
"""The verdict with how many transcript entries the judge read, where it may carry an escalation."""
return dataclasses.replace(verdict, entries_read=entries_read) if isinstance(verdict, Escalating) else verdict
+90
View File
@@ -0,0 +1,90 @@
"""How a cycle starts: the route it runs on, from the archive or the plan, the facts chunk-preflight finds for a fresh chunk, and the items loop-inbox lists."""
import pathlib
import typing
from release_standing import default_plan, DefaultPlan
from git_repo import default_ref, file_at
from loop_claude.effects import Effects, say
from failure import Failure
from loop_claude.routing import first_route, kept, routed_item
from todo_table import Item
from transcript_archive.read import latest_worker
from loop_claude.cycle import prompts
from model_routing.route import Route, route_from
from model_routing.ladder import Rung
from claude_code.session import Session, session_open, transcript_file
from loop_claude.running_loops import Show, taken_items
# Seconds routing waits for origin to name its default branch, asked only where the checkout records none; past them, the chunk runs on the project's rung.
ORIGIN_TIMEOUT = 60
def shown_items(show: Show, root: str, routed: int) -> typing.Callable[[tuple[int, ...]], None]:
"""A function that lists in loop-inbox the items a chunk takes, given the IDs the judge found taken in. routed is the item the chunk was routed on, 0 for none. shown_items reads todo.md once, as the cycle starts, and the function shows those taglines."""
todo = fetched_todo(root)
return lambda taken_in: show(taken_items(todo, routed, taken_in))
def fetched_todo(root: str) -> str | Failure:
"""todo.md on the default branch, as last fetched. fetched_plan reads it too, but also the release tag, CHANGELOG.md and each todo.md since the tag, which the listing doesn't need."""
ref = default_ref(root, ORIGIN_TIMEOUT)
return ref if isinstance(ref, Failure) else file_at(ref, "todo.md", root)
def routed(route: Route | None, project: Rung | None, model: str, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Item | None]) -> Route:
"""The route the cycle runs on. project is the project's rung, and model the project's model.
Where project is None, the model is off the ladder, and the worker runs on it. Otherwise the chunk keeps route, or on a rerun the route its archive last names, as routing.kept allows. Else the plan item routes the chunk afresh.
"""
if project is None:
return Route(Rung(model))
route = route or archived()
return route if route and kept(route, project) else first_route(project, item())
def archived_route(archive_folder: pathlib.Path, session: Session, effects: Effects) -> Route | None:
"""The route the latest verdict archived for the session's open chunk names; None for a chunk not yet judged."""
transcript = transcript_file(session)
worker = latest_worker(archive_folder, transcript) if transcript else None
if isinstance(worker, Failure):
say(f"routing the chunk afresh: {worker.reason}", effects)
return None
route = route_from(worker) if worker else None
if worker and route is None:
say(f"routing the chunk afresh: the archive's latest Worker: line names no route loop-claude can read: {worker}", effects)
return route
def chunk_start(session: Session, preflight: pathlib.Path, root: str, effects: Effects) -> tuple[str, typing.Callable[[], Item | None]]:
"""The facts for a fresh worker's prompt, empty where the session is open, and the reader of the item that routes the chunk. The reader reads the default branch as last fetched: for a fresh chunk, by the preflight that runs first; for an open session, which runs none, by whatever fetched last."""
facts = "" if session_open(session) else chunk_facts(preflight, root, effects)
return facts, lambda: plan_item(fetched_plan(root), effects)
def chunk_facts(preflight: pathlib.Path, root: str, effects: Effects) -> str:
"""What `chunk-preflight --worktree` printed at the checkout root, for a fresh worker's prompt, or why it could not run."""
done = effects.python([str(preflight), "--worktree"], root)
if done.returncode:
return prompts.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}")
return prompts.preflight_facts(done.stdout.strip())
def fetched_plan(root: str) -> DefaultPlan | Failure:
"""What the default branch, as last fetched, holds for routing."""
ref = default_ref(root, ORIGIN_TIMEOUT)
return ref if isinstance(ref, Failure) else default_plan(root, ref)
def plan_item(plan: DefaultPlan | Failure, effects: Effects) -> Item | None:
"""The item that routes a chunk, release_standing.chunk_item's pick from the default branch; None without one, so the chunk runs on the project's rung."""
# It prints the reason itself, through effects, like the other loop-claude functions that print. todo.md item 51 moves them all to one injected effect, so changing this one alone would be undone.
if isinstance(plan, Failure):
item: Item | Failure | None = plan
elif isinstance(plan.todo, Failure):
item = plan.todo
else:
item = routed_item(plan.todo, plan.cut)
if isinstance(item, Failure):
say(f"routing on the project's rung: {item.reason}", effects)
return None
return item
+47
View File
@@ -0,0 +1,47 @@
"""How a cycle closes: the loop run the next cycle starts from, the step-up an unfinished verdict makes archived, and the stop once a chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
import dataclasses
import pathlib
from loop_claude.cycle.run import archived_step, archived_stop, Concluded
from loop_claude.effects import Effects
from loop_claude.verdicts import Finished
from loop_claude.loop.reload import LoopRun
from loop_claude.stop import needs_human, Stop
from loop_claude.routing import next_route, step_up
from model_routing.ladder import Rung
from claude_code.session import Session, transcript_file
# How many unfinished verdicts running end the loop, so a worker and a judge that disagree about a chunk cannot cycle unattended without end. A policy number, not derived from the routing steps: well above the two unfinished verdicts running that archived chunks have reached at most, and low enough that a stuck pair is stopped after six cycles. It counts every unfinished verdict, including those of cycles where the human answered an interrupt: a rerun is cheap, and a chunk that needs that many answers is worth a look. The human confirmed both the number and what it counts.
UNFINISHED_LIMIT = 6
def closed_cycle(loop_run: LoopRun, project: Rung | None, concluded: Concluded, archive_folder: pathlib.Path, session: Session, effects: Effects) -> LoopRun | Stop:
"""The loop run after a cycle concluded, with the step-up archived; the stop that needs the human, archived, once the chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
after = after_verdict(loop_run, project, concluded)
# A finished chunk drops its route, so it has no step-up, and its run of unfinished verdicts restarts.
if after.route is None:
return after
# The unfinished verdict, archived as the cycle closed, names the route its cycle ended on; the step-up names the one the next cycle runs on.
path = transcript_file(session)
if path and (failure := archived_step(archive_folder, path, after.task, step_up(concluded.route, after.route), effects)):
return Stop(1, failure.reason)
if limit := reached_limit(after):
# Archived with the route the next cycle would run on, so a rerun resumes there.
return archived_stop(archive_folder, session, after.task, after.route, limit, limit.reason, effects)
return after
def reached_limit(loop_run: LoopRun) -> Stop | None:
"""The stop that needs the human once the loop run holds UNFINISHED_LIMIT unfinished verdicts running."""
if loop_run.unfinished_run < UNFINISHED_LIMIT:
return None
return needs_human("go on with the chunk", f"the judge ruled it unfinished {loop_run.unfinished_run} cycles running; finish the chunk yourself, or rerun loop-claude to give it {UNFINISHED_LIMIT} more cycles")
def after_verdict(loop_run: LoopRun, project: Rung | None, concluded: Concluded) -> LoopRun:
"""The loop run the next cycle starts from. A finished chunk drops its route, its run of unfinished verdicts and the items it took in. An unfinished one takes the route routing.next_route makes of the route its cycle ended on, which an interrupt's or a quoted answer's escalation may have raised; the evidence the judge gave for the verdict and the items it found taken in; and counts the verdict."""
verdict = concluded.verdict
if isinstance(verdict, Finished):
return dataclasses.replace(loop_run, route=None, unfinished_evidence="", unfinished_run=0, items_taken_in=())
route = next_route(project, concluded.route, verdict)
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence, unfinished_run=loop_run.unfinished_run + 1, items_taken_in=verdict.taken_in.items)
+38 -155
View File
@@ -79,9 +79,9 @@ Once 20 cycles have closed across all projects in the archive since the last mea
loop to close one measures every finished chunk there, with no model run, and writes
routing-figures.md at the archive's root (bin/routing_figures/measure.py). Each call the figures
make for a routing change, unless raised before, is an interrupt: claude, on the project's model,
on the terminal in ~/.claude, asked to put the calls to the human; loop-inbox lists it too. Once
that claude ends, the calls are listed in routing-calls-raised.md beside the figures, and not
raised again. Off a terminal, the calls are said on stderr, the loop goes on, and they are raised
on the terminal in the setup's development clone, ~/code/claude-config, asked to put the calls to
the human; loop-inbox lists it too. Once that claude ends, the calls are listed in
routing-calls-raised.md beside the figures, and not raised again. Off a terminal, the calls are said on stderr, the loop goes on, and they are raised
again at the next measurement.
A change to bin/loop-claude or any bin/**/*.py takes effect at the next cycle boundary, once
@@ -125,37 +125,25 @@ import dataclasses
import pathlib
import typing
from loop_claude.routing import after_unfinished, first_route, kept, routed_item
from transcript_archive.write import archive
from transcript_archive.location import archive_root, project_archive
from loop_claude.cycle.run import archived_stop, Concluded, Cycle, run_cycle
from loop_claude.loop.chunk_start import archived_route, chunk_start, routed, shown_items
from loop_claude.loop.routing_check import check_routing
from loop_claude.project.checkout import checkout_root
from loop_claude.loop.close import closed_cycle
from loop_claude.cycle.run import Cycle, run_cycle
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
from chunk_pick import default_plan, DefaultPlan
from git_repo import default_ref, file_at
from loop_claude.effects import Effects, say
from failure import Failure
from loop_claude.verdicts import Finished, Unfinished
from loop_claude.project.model_choice import get_or_ask_models
from todo_table import Item
from transcript_archive.read import latest_worker
from loop_claude.stop import needs_human, Stop
from transcript_archive import outcomes
from loop_claude.running_loops import Listed, shown_running
from loop_claude.paths import Paths
from project_key import project_key
from model_routing.ladder import project_rung, Rung
from loop_claude.cycle import prompts
from model_routing.ladder import project_rung
from loop_claude.project.premises import read_premises
from loop_claude.project.records import read_records
from model_routing.route import Route, route_from
from claude_code.session import Session, session_named, session_open, transcript_file
from loop_claude.running_loops import Show, shown_running, taken_items
from claude_code.session import Session, session_named
from loop_claude.stop import Stop
# Seconds routing waits for origin to name its default branch, asked only where the checkout records none; past them, the chunk runs on the project's rung.
ORIGIN_TIMEOUT = 60
# How many unfinished verdicts running end the loop, so a worker and a judge that disagree about a chunk cannot cycle unattended without end. A policy number, not derived from the routing steps: well above the two unfinished verdicts running that archived chunks have reached at most, and low enough that a stuck pair is stopped after six cycles. It counts every unfinished verdict, including those of cycles where the human answered an interrupt: a rerun is cheap, and a chunk that needs that many answers is worth a look. The human confirmed both the number and what it counts.
UNFINISHED_LIMIT = 6
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
@@ -173,153 +161,48 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
if isinstance(loop_run, Stop):
return loop_run
baseline = fingerprint(launcher)
key = project_key(loop_run.root, paths.home)
archive_folder = project_archive(loop_run.root, paths.home, env)
# "A " is the human's cue in the session picker that the session is the loop's.
session = session_named(f"A {key} loop-claude", paths.claude_code_transcripts)
session = session_named(f"A {project_key(loop_run.root, paths.home)} loop-claude", paths.claude_code_transcripts)
with shown_running(paths.waiting, str(session.id), loop_run.root, effects) as listed:
paths = dataclasses.replace(paths, waiting=listed.folder)
while True:
if stop := exec_if_edited(launcher, baseline, loop_run, effects):
return stop
loop_run = dataclasses.replace(loop_run, cycle_number=loop_run.cycle_number + 1)
premises = read_premises(loop_run.root)
if isinstance(premises, Stop):
return premises
records = read_records(loop_run.root, paths.claude_dir)
if isinstance(records, Stop):
return records
models = get_or_ask_models(key, paths, env, effects)
if isinstance(models, Stop):
return models
# After the reads above, so a loop that stops on one makes no worktree.
facts, item = chunk_start(session, launcher.with_name("chunk-preflight"), loop_run.root, effects)
project = project_rung(models.worker)
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, session, effects), item)
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", effects, stdout=True)
show_taken_in = shown_items(listed.show, loop_run.root, route.item)
show_taken_in(loop_run.items_taken_in)
concluded = run_cycle(Cycle(effects=effects, facts=facts, judge_model=models.judge, paths=paths, premises=premises, project=project, project_archive=archive_folder, records=records, root=loop_run.root, route=route, session=session, show_taken_in=show_taken_in, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence))
cycle = next_cycle(loop_run, session, listed, paths, env, launcher, effects)
if isinstance(cycle, Stop):
return cycle
concluded = run_cycle(cycle)
if isinstance(concluded, Stop):
return concluded
closed = closed_cycle(loop_run, project, concluded, archive_folder, session, effects)
closed = closed_cycle(loop_run, cycle.project, concluded, cycle.project_archive, session, effects)
if isinstance(closed, Stop):
return closed
loop_run = closed
if stop := check_routing(archive_root(paths.home, env), loop_run.root, str(session.id), project or Rung(models.worker), paths, effects):
# Off the ladder the worker runs on the project's model itself, which its route names.
if stop := check_routing(archive_root(paths.home, env), loop_run.root, str(session.id), cycle.project or cycle.route.rung, paths, listed.folder, effects):
return stop
def shown_items(show: Show, root: str, routed: int) -> typing.Callable[[tuple[int, ...]], None]:
"""A function that lists in loop-inbox the items a chunk takes, given the IDs the judge found taken in. routed is the item the chunk was routed on, 0 for none. shown_items reads todo.md once, as the cycle starts, and the function shows those taglines."""
todo = fetched_todo(root)
return lambda taken_in: show(taken_items(todo, routed, taken_in))
def fetched_todo(root: str) -> str | Failure:
"""todo.md on the default branch, as last fetched. fetched_plan reads it too, but also the release tag, CHANGELOG.md and each todo.md since the tag, which the listing doesn't need."""
ref = default_ref(root, ORIGIN_TIMEOUT)
return ref if isinstance(ref, Failure) else file_at(ref, "todo.md", root)
def routed(route: Route | None, project: Rung | None, model: str, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Item | None]) -> Route:
"""The route the cycle runs on. project is the project's rung, and model the project's model.
Where project is None, the model is off the ladder, and the worker runs on it. Otherwise the chunk keeps route, or on a rerun the route its archive last names, as routing.kept allows. Else the plan item routes the chunk afresh.
"""
if project is None:
return Route(Rung(model))
route = route or archived()
return route if route and kept(route, project) else first_route(project, item())
def closed_cycle(loop_run: LoopRun, project: Rung | None, concluded: Concluded, archive_folder: pathlib.Path, session: Session, effects: Effects) -> LoopRun | Stop:
"""The loop run after a cycle concluded, with the step-up archived; the stop that needs the human, archived, once the chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
after = after_verdict(loop_run, project, concluded)
if stop := archive_step(archive_folder, session, after.task, concluded.route, after.route, effects):
return stop
if limit := reached_limit(after):
# An unfinished verdict always leaves a route; the `or` only satisfies LoopRun.route's optional type. The stop is archived with that route, the one the next cycle would run on, so a rerun resumes there.
return archived_stop(archive_folder, session, after.task, after.route or concluded.route, limit, limit.reason, effects)
return after
def reached_limit(loop_run: LoopRun) -> Stop | None:
"""The stop that needs the human once the loop run holds UNFINISHED_LIMIT unfinished verdicts running."""
if loop_run.unfinished_run < UNFINISHED_LIMIT:
return None
return needs_human("go on with the chunk", f"the judge ruled it unfinished {loop_run.unfinished_run} cycles running; finish the chunk yourself, or rerun loop-claude to give it {UNFINISHED_LIMIT} more cycles")
def after_verdict(loop_run: LoopRun, project: Rung | None, concluded: Concluded) -> LoopRun:
"""The loop run the next cycle starts from. A finished chunk drops its route, its run of unfinished verdicts and the items it took in. An unfinished one takes the route stepped_route makes, the evidence the judge gave for the verdict and the items it found taken in, and counts the verdict."""
verdict = concluded.verdict
if isinstance(verdict, Finished):
return dataclasses.replace(loop_run, route=None, unfinished_evidence="", unfinished_run=0, items_taken_in=())
route = stepped_route(project, concluded.route, verdict)
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence, unfinished_run=loop_run.unfinished_run + 1, items_taken_in=verdict.taken_in.items)
def stepped_route(project: Rung | None, ended_on: Route, verdict: Unfinished) -> Route:
"""The route an unfinished chunk's next cycle starts on: the route after_unfinished makes of the route its cycle ended on, which an interrupt's or a quoted answer's escalation may have raised. Off the ladder there is no rung to step, and it keeps that route."""
return after_unfinished(project, ended_on, verdict) if project else ended_on
def archive_step(archive_folder: pathlib.Path, session: Session, task: str, before: Route, after: Route | None, effects: Effects) -> Stop | None:
"""Archives the route after an unfinished verdict as a step-up, where its worker line differs from the one before it: a higher rung, or a higher E + I taken in. The verdict itself names the route before, which its cycle ran on."""
if after is None or after.worker_line() == before.worker_line():
return None
path = transcript_file(session)
failure = archive(archive_folder, path, task, outcomes.stepped_up(after.worker_line()), effects.now(), lambda text: say(text, effects)) if path else None
return Stop(1, failure.reason) if failure else None
def archived_route(archive_folder: pathlib.Path, session: Session, effects: Effects) -> Route | None:
"""The route the latest verdict archived for the session's open chunk names; None for a chunk not yet judged."""
transcript = transcript_file(session)
worker = latest_worker(archive_folder, transcript) if transcript else None
if isinstance(worker, Failure):
say(f"routing the chunk afresh: {worker.reason}", effects)
return None
route = route_from(worker) if worker else None
if worker and route is None:
say(f"routing the chunk afresh: the archive's latest Worker: line names no route loop-claude can read: {worker}", effects)
return route
def chunk_start(session: Session, preflight: pathlib.Path, root: str, effects: Effects) -> tuple[str, typing.Callable[[], Item | None]]:
"""The facts for a fresh worker's prompt, empty where the session is open, and the reader of the item that routes the chunk. The reader reads the default branch as last fetched: for a fresh chunk, by the preflight that runs first; for an open session, which runs none, by whatever fetched last."""
facts = "" if session_open(session) else chunk_facts(preflight, root, effects)
return facts, lambda: plan_item(fetched_plan(root), effects)
def chunk_facts(preflight: pathlib.Path, root: str, effects: Effects) -> str:
"""What `chunk-preflight --worktree` printed at the checkout root, for a fresh worker's prompt, or why it could not run."""
done = effects.python([str(preflight), "--worktree"], root)
if done.returncode:
return prompts.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}")
return prompts.preflight_facts(done.stdout.strip())
def fetched_plan(root: str) -> DefaultPlan | Failure:
"""What the default branch, as last fetched, holds for routing."""
ref = default_ref(root, ORIGIN_TIMEOUT)
return ref if isinstance(ref, Failure) else default_plan(root, ref)
def plan_item(plan: DefaultPlan | Failure, effects: Effects) -> Item | None:
"""The item that routes a chunk, chunk_pick.chunk_item's pick from the default branch; None without one, so the chunk runs on the project's rung."""
# It prints the reason itself, through effects, like the other loop-claude functions that print. todo.md item 51 moves them all to one injected effect, so changing this one alone would be undone.
if isinstance(plan, Failure):
item: Item | Failure | None = plan
elif isinstance(plan.todo, Failure):
item = plan.todo
else:
item = routed_item(plan.todo, plan.cut)
if isinstance(item, Failure):
say(f"routing on the project's rung: {item.reason}", effects)
return None
return item
def next_cycle(loop_run: LoopRun, session: Session, listed: Listed, paths: Paths, env: typing.Mapping[str, str], launcher: pathlib.Path, effects: Effects) -> Cycle | Stop:
"""The cycle loop_run runs next, from the premises, the records and the project's models read afresh, and the route the chunk runs on; the stop where one of them cannot be read."""
premises = read_premises(loop_run.root)
if isinstance(premises, Failure):
return Stop(1, premises.reason)
records = read_records(loop_run.root, paths.claude_dir)
if isinstance(records, Failure):
return Stop(1, records.reason)
models = get_or_ask_models(project_key(loop_run.root, paths.home), paths, env, effects)
if isinstance(models, Stop):
return models
archive_folder = project_archive(loop_run.root, paths.home, env)
# After the reads above, so a loop that stops on one makes no worktree.
facts, item = chunk_start(session, launcher.with_name("chunk-preflight"), loop_run.root, effects)
project = project_rung(models.worker)
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, session, effects), item)
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", effects, stdout=True)
show_taken_in = shown_items(listed.show, loop_run.root, route.item)
show_taken_in(loop_run.items_taken_in)
return Cycle(effects=effects, facts=facts, judge_model=models.judge, premises=premises, project=project, project_archive=archive_folder, questions=listed.folder, records=records, root=loop_run.root, route=route, session=session, show_taken_in=show_taken_in, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence)
def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopRun | Stop:
+8 -8
View File
@@ -14,14 +14,14 @@ from loop_claude.stop import Stop
from utc_stamp import utc
def check_routing(root: pathlib.Path, checkout: str, session: str, rung: Rung, paths: Paths, effects: Effects) -> Stop | None:
def check_routing(root: pathlib.Path, checkout: str, session: str, rung: Rung, paths: Paths, questions: pathlib.Path, effects: Effects) -> Stop | None:
"""Stops only when claude cannot start for the calls; a figure it cannot measure or keep is said on stderr, and the loop goes on. checkout is the loop's, session is its worker's session id, which loop-inbox lists the calls under, and rung is the rung of the claude that puts the calls to the human."""
with archive.locked(root) as held:
new = new_calls(root, paths, effects) if held else []
if isinstance(new, Failure):
say(f"routing figures: {new.reason}", effects)
return None
return raise_calls(root, checkout, session, new, rung, paths, effects) if new else None
return raise_calls(root, checkout, session, new, rung, paths, questions, effects) if new else None
def new_calls(root: pathlib.Path, paths: Paths, effects: Effects) -> list[str] | Failure:
@@ -44,20 +44,20 @@ def new_calls(root: pathlib.Path, paths: Paths, effects: Effects) -> list[str] |
return [call for call in found if call not in kept.calls]
def raise_calls(root: pathlib.Path, checkout: str, session: str, new: list[str], rung: Rung, paths: Paths, effects: Effects) -> Stop | None:
"""Puts the calls to the human through a claude in this setup's checkout, where the routing decision lives; loop-inbox lists them under the loop's checkout while that claude is open. Off a terminal they are only said, and stay in the figures, raised at the next measurement on a terminal."""
def raise_calls(root: pathlib.Path, checkout: str, session: str, new: list[str], rung: Rung, paths: Paths, questions: pathlib.Path, effects: Effects) -> Stop | None:
"""Puts the calls to the human through a claude in this setup's development clone, where the routing decision is changed; loop-inbox lists them under the loop's checkout while that claude is open. Off a terminal they are only said, and stay in the figures, raised at the next measurement on a terminal."""
bullets = "".join(f"- {call}\n" for call in new)
question = f"loop-claude's routing figures call for a routing change:\n{bullets}The figures: {root / archive.FIGURES}"
say(f"\n{question}", effects)
if not effects.on_terminal():
say("Not on a terminal: the calls are raised again at the next measurement.", effects)
return None
prompt = f"Put these calls for a routing change from loop-claude's figures to the human, and change nothing until they answer:\n{bullets}The figures are in {root / archive.FIGURES}. The ladder, its steps and the delegation rule are docs/decisions.md → `routing:`, and a change to them is the human's. Record what they decide as ~/.claude/CLAUDE.md says; the next chunk of this repo files the plan item for these calls."
prompt = f"Put these calls for a routing change from loop-claude's figures to the human, and change nothing until they answer:\n{bullets}The figures are in {root / archive.FIGURES}. The ladder, its steps and the delegation rule are docs/decisions.md → `routing:`, and a change to them is the human's. Record what they decide in this checkout, the development clone, as ~/.claude/CLAUDE.md says; the next chunk of this repo files the plan item for these calls."
ask = Ask(effects.new_id(), session, checkout, question, effects.now(), OpenOnTerminal())
with listed(paths.waiting, ask, effects):
unstarted = hand_over(ClaudeCommand(claude_args(rung, [prompt]), str(paths.claude_dir), {}), effects)
with listed(questions, ask, effects):
unstarted = hand_over(ClaudeCommand(claude_args(rung, [prompt]), str(paths.config_clone), {}), effects)
if unstarted:
return Stop(1, f"could not start claude for the routing calls: {unstarted.reason}")
return Stop(1, f"could not start claude for the routing calls in {paths.config_clone}, the setup's development clone: {unstarted.reason}")
# Raised once the human has seen them, answered or not: an unanswered call reaches the plan through AGENTS.md, so it is not asked twice.
if unmarked := archive.mark_raised(root, new, utc(effects.now())):
say(f"routing figures: {unmarked.reason}", effects)
+3
View File
@@ -6,6 +6,8 @@ import pathlib
class Paths:
claude_code_transcripts: pathlib.Path
claude_dir: pathlib.Path
# The development clone of this setup, where a change to it is made: claude_dir holds only a release, docs/decisions.md → `deploy:`.
config_clone: pathlib.Path
home: pathlib.Path
model_catalog: pathlib.Path
models_file: pathlib.Path
@@ -18,6 +20,7 @@ def paths_from(home: pathlib.Path) -> Paths:
return Paths(
claude_code_transcripts=claude / "projects",
claude_dir=claude,
config_clone=home / "code" / "claude-config",
home=home,
model_catalog=claude / "cache" / "model-catalog",
models_file=claude / "loop-claude-models.json",
+8 -7
View File
@@ -31,8 +31,8 @@ def get_or_ask_models(project_key: str, paths: Paths, env: typing.Mapping[str, s
def get_or_ask_model(project_key: str, paths: Paths, effects: Effects) -> str | Stop:
saved = paths.models_file
models = read_models_file(saved)
if isinstance(models, Stop):
return models
if isinstance(models, Failure):
return Stop(1, models.reason)
if kept := models.get(project_key):
return kept
chosen = ask_for_model(project_key, saved, paths.model_catalog, effects)
@@ -40,24 +40,25 @@ def get_or_ask_model(project_key: str, paths: Paths, effects: Effects) -> str |
return chosen
# Reread: another loop may have saved its project while this one waited on the human.
models = read_models_file(saved)
if isinstance(models, Stop):
return models
if isinstance(models, Failure):
return Stop(1, models.reason)
if failure := replace_json(saved, {**models, project_key: chosen}, indent="\t", sort_keys=True):
return Stop(1, f"could not save the model in {saved}: {failure.reason}")
say(f"{project_key} runs on {chosen}", effects)
return chosen
def read_models_file(saved: pathlib.Path) -> dict[str, str] | Stop:
def read_models_file(saved: pathlib.Path) -> dict[str, str] | Failure:
"""Each project's model, as the models file at saved maps them; none where there is no such file. loop-claude and the replay tools read it."""
try:
with open(saved, encoding="utf-8") as handle:
models = json.load(handle)
except FileNotFoundError:
return {}
except (OSError, ValueError) as failure:
return Stop(1, f"could not read {saved}: {getattr(failure, 'strerror', None) or failure}")
return Failure(f"could not read {saved}: {getattr(failure, 'strerror', None) or failure}")
if not isinstance(models, dict) or not all(isinstance(chosen, str) and chosen for chosen in models.values()):
return Stop(1, f"could not read {saved}: not an object mapping projects to model names")
return Failure(f"could not read {saved}: not an object mapping projects to model names")
return models
+7 -6
View File
@@ -1,19 +1,20 @@
from failure import Failure
from loop_claude.project.text_file import read_utf8
from markdown_text import sections
from loop_claude.stop import Stop
def read_premises(root: str) -> str | Stop:
def read_premises(root: str) -> str | Failure:
"""README.md's ## Goals and ## Audience in root; a Failure where README.md cannot be read or lacks either, or holds one twice or empty."""
text = read_utf8(root, "README.md", "no premises")
if isinstance(text, Stop):
if isinstance(text, Failure):
return text
found = sections(text)
for name in ("Goals", "Audience"):
bodies = found.get(name, [])
if not bodies:
return Stop(1, f"README.md has no ## {name} heading - no premises.")
return Failure(f"README.md has no ## {name} heading - no premises.")
if len(bodies) > 1:
return Stop(1, f"README.md has ## {name} more than once - no premises.")
return Failure(f"README.md has ## {name} more than once - no premises.")
if not bodies[0]:
return Stop(1, f"README.md has nothing under ## {name} - no premises.")
return Failure(f"README.md has nothing under ## {name} - no premises.")
return f"## Goals\n{found['Goals'][0]}\n\n## Audience\n{found['Audience'][0]}"
+3 -3
View File
@@ -1,7 +1,7 @@
import pathlib
from failure import Failure
from loop_claude.project.text_file import read_utf8
from loop_claude.stop import Stop
# The records the judge reads beside the README's premises: these project files, then the human's rules below.
RECORD_FILES = ("AGENTS.md", "docs/decisions.md")
@@ -11,7 +11,7 @@ HUMAN_RULE_DIRS = ("rules", "skills/larv-review/standards")
HUMAN_RULE_NAMES = (f"~/.claude/{HUMAN_RULES_FILE}", *(f"~/.claude/{folder}/" for folder in HUMAN_RULE_DIRS))
def read_records(root: str, claude_dir: pathlib.Path) -> str | Stop:
def read_records(root: str, claude_dir: pathlib.Path) -> str | Failure:
"""The project's record files, then the human's rules for every project, CLAUDE.md first, each headed by its name; a file that does not exist is left out."""
rules = [claude_dir / HUMAN_RULES_FILE] + [rule for folder in HUMAN_RULE_DIRS for rule in sorted((claude_dir / folder).glob("*.md"))]
sources = [(root, name, name) for name in RECORD_FILES]
@@ -21,7 +21,7 @@ def read_records(root: str, claude_dir: pathlib.Path) -> str | Stop:
if not pathlib.Path(folder, name).exists():
continue
text = read_utf8(folder, name, "the judge cannot read the records")
if isinstance(text, Stop):
if isinstance(text, Failure):
return text
records.append(f"From {heading}:\n{text.strip()}")
return "\n\n".join(records)
+5 -4
View File
@@ -1,13 +1,14 @@
import pathlib
from loop_claude.stop import Stop
from failure import Failure
def read_utf8(folder: str, name: str, consequence: str) -> str | Stop:
def read_utf8(folder: str, name: str, consequence: str) -> str | Failure:
"""The text of the file name in folder; a Failure naming consequence where it cannot be read or is not UTF-8."""
try:
return pathlib.Path(folder, name).read_text(encoding="utf-8")
except OSError as failure:
return Stop(1, f"{name} in {folder}: {failure.strerror} - {consequence}.")
return Failure(f"{name} in {folder}: {failure.strerror} - {consequence}.")
except UnicodeDecodeError as failure:
line = failure.object.count(b"\n", 0, failure.start) + 1
return Stop(1, f"{name} in {folder} is not UTF-8 on line {line} - {consequence}.")
return Failure(f"{name} in {folder} is not UTF-8 on line {line} - {consequence}.")
+37 -35
View File
@@ -4,12 +4,12 @@ The worker, and the session /talk opens, run on a Route (model_routing/route.py)
- Start. routed_item finds the plan item that routes a fresh chunk, if there is one with L, E and I scored; first_route picks the worker's rung from its L, or the project's rung without an item.
- Rerun. kept decides whether a rerun resumes the route its archive last names.
- Step-up. after_unfinished raises the route after an unfinished verdict, after_ruling after an interrupt or a quoted answer that carries an escalation; escalation_cap bounds an escalation, and loop_claude/escalation_mark.py states the escalation mark's rule.
- Step-up. next_route raises the route after an unfinished verdict, or an interrupt or a quoted answer that carries an escalation; escalation_cap bounds an escalation, and loop_claude/escalation_mark.py states the escalation mark's rule. step_up names the change where it changes the worker line, which the archive records.
- Judge. judge_rung picks the rung of the judge of a JUDGE_REPLAYED model.
"""
import dataclasses
from chunk_pick import chunk_item, LastCut, Next
from release_standing import chunk_item, LastCut, Next
from model_routing.ladder import climbed, highest, LADDER, position, project_rung, review_floor, Rung, stepped_up, worker_rung
from loop_claude import escalation_mark
from failure import Failure
@@ -25,7 +25,7 @@ JUDGE_REPLAYED = ("claude-opus-5-5",)
def routed_item(todo: str, cut: LastCut) -> Item | Failure | None:
"""The item chunk_pick.chunk_item picks from todo.md, weighed against the last cut; None when there is none, or it lacks L, E or I."""
"""The item release_standing.chunk_item picks from todo.md, weighed against the last cut; None when there is none, or it lacks L, E or I."""
found = items(todo)
if isinstance(found, NoItemsTable):
return None
@@ -45,12 +45,7 @@ def first_route(project: Rung, item: Item | None) -> Route:
def kept(route: Route, project: Rung) -> bool:
"""Whether a chunk keeps route: its rung is on the ladder and at or above item_rung."""
return kept_rung(project, route) == route.rung
def kept_rung(project: Rung, route: Route) -> Rung:
"""The route's rung, raised to item_rung where it is lower or off the ladder."""
return highest(route.rung, item_rung(project, route))
return floor_rung(project, route) == route.rung
def item_rung(project: Rung, route: Route) -> Rung:
@@ -58,6 +53,12 @@ def item_rung(project: Rung, route: Route) -> Rung:
return worker_rung(project, route.likelihood) if route.item else project
def floor_rung(project: Rung, route: Route, likelihood_taken_in: int = 0) -> Rung:
"""The lowest rung the worker may run on: the route's own, raised to item_rung where it is lower or off the ladder, and to the rung likelihood_taken_in picks from project. likelihood_taken_in is the highest L the worker took into the chunk, 0 for none."""
floor = highest(route.rung, item_rung(project, route))
return highest(floor, worker_rung(project, likelihood_taken_in)) if likelihood_taken_in else floor
def escalation_rungs(project: Rung, route: Route) -> tuple[Rung, ...]:
"""The rungs an escalation trigger opens to the worker, least competent first: from one above its own up to escalation_cap; none off the ladder."""
if route.rung not in LADDER:
@@ -73,27 +74,38 @@ def escalation_cap(project: Rung, route: Route) -> Rung:
return climbed(item_rung(project, route), ESCALATION_CAP)
def after_unfinished(project: Rung, route: Route, verdict: Unfinished) -> Route:
"""The route the next cycle runs on after an unfinished verdict.
@dataclasses.dataclass(frozen=True)
class StepUp:
"""A verdict changing the route's worker line: the route the worker ran on until the verdict, and the one it runs on after. The archive names both, so the figures charge each stretch of work to its own rung."""
before: Route
after: Route
Its rung never falls. It first rises to the rung that the verdict's highest L taken in picks, where that is higher. From there it steps one rung up on every second unfinished verdict, or on an escalation while below escalation_cap; where both fall on one verdict, it steps once. A step on an escalation moves the mark, as loop_claude/escalation_mark.py says; a step on the count of unfinished verdicts alone keeps it. Last, the route takes in the verdict's highest E + I, as Route.taking_in does.
def next_route(project: Rung | None, route: Route, verdict: Unfinished | Interrupt | QuotedAnswer) -> Route:
"""The route the worker runs on after the judge's verdict. Off the ladder, project is None, and the route holds.
The rung never falls. It first rises to floor_rung, which the verdict's highest L taken in may raise. From there it steps one rung up on an escalation while below escalation_cap, and moves the mark, as loop_claude/escalation_mark.py says.
An unfinished verdict also counts towards UNFINISHED_PER_STEP, and the second such verdict steps the rung once, keeping the mark; where both fall on one verdict, it steps once. Last, it takes in the verdict's highest E + I, as Route.taking_in does.
An interrupt or a quoted answer without an escalation leaves the route as it is, since neither is an unfinished verdict.
"""
rung = raised(project, route, verdict.taken_in.highest_likelihood)
mark = escalation_mark.moved_to(verdict)
if route.unfinished + 1 >= UNFINISHED_PER_STEP or (mark is not None and below_cap(project, route, rung)):
after = one_rung_up(route, rung, route.escalated_at if mark is None else mark)
else:
after = dataclasses.replace(route, rung=rung, unfinished=route.unfinished + 1)
return after.taking_in(verdict.taken_in.highest_escape_and_impact)
def after_ruling(project: Rung, route: Route, verdict: Interrupt | QuotedAnswer) -> Route:
"""The route after the judge's interrupt or quoted answer: unchanged without an escalation. With one, raised to the rung the highest L taken in picks, and one rung up from there while below escalation_cap, moving the mark as after_unfinished does. Neither is an unfinished verdict, so the count of those restarts only with a step."""
if project is None:
return route
mark = escalation_mark.moved_to(verdict)
floor = floor_rung(project, route, verdict.taken_in.highest_likelihood)
escalated_at = route.escalated_at if mark is None else mark
escalates = mark is not None and position(floor) < position(escalation_cap(project, route))
if isinstance(verdict, Unfinished):
counted = route.unfinished + 1
after = one_rung_up(route, floor, escalated_at) if counted >= UNFINISHED_PER_STEP or escalates else dataclasses.replace(route, rung=floor, unfinished=counted)
return after.taking_in(verdict.taken_in.highest_escape_and_impact)
if mark is None:
return route
rung = raised(project, route, verdict.taken_in.highest_likelihood)
return one_rung_up(route, rung, mark) if below_cap(project, route, rung) else dataclasses.replace(route, rung=rung)
return one_rung_up(route, floor, escalated_at) if escalates else dataclasses.replace(route, rung=floor)
def step_up(before: Route, after: Route) -> StepUp | None:
"""The step-up from before to after; None where the worker line holds, as when only the count of unfinished verdicts changed."""
return StepUp(before, after) if after.worker_line() != before.worker_line() else None
def one_rung_up(route: Route, rung: Rung, escalated_at: int) -> Route:
@@ -101,16 +113,6 @@ def one_rung_up(route: Route, rung: Rung, escalated_at: int) -> Route:
return dataclasses.replace(route, rung=stepped_up(rung), unfinished=0, escalated_at=escalated_at)
def below_cap(project: Rung, route: Route, rung: Rung) -> bool:
"""Whether an escalation trigger may take the worker one rung up from rung without the human's yes: rung is below escalation_cap, two rungs above item_rung."""
return position(rung) < position(escalation_cap(project, route))
def raised(project: Rung, route: Route, likelihood_taken_in: int) -> Rung:
"""kept_rung, raised to the rung that likelihood_taken_in picks from project. likelihood_taken_in is the highest L the worker took into the chunk, or 0 for none."""
return highest(kept_rung(project, route), worker_rung(project, likelihood_taken_in)) if likelihood_taken_in else kept_rung(project, route)
def judge_rung(judge_model: str, route: Route) -> Rung:
"""The rung the route's reviewed_on E + I picks for a review, counted from the judge model's cheapest rung and never below it, whatever rung the worker stepped to.
+2
View File
@@ -6,6 +6,8 @@ branch as last fetched. The items are the one the chunk was routed on, then thos
worker took in, each listed once. loop-inbox lists a loop, and its question (waiting/), only while the loop holds its
lock. So a killed loop drops off the list, though its files stay until loop-inbox removes them: <session>.running
through drop_ended_loops here, and the question through waiting.listing.drop_ended.
This module, waiting/ and session_lock.py hold loop-claude's state, which loop-inbox reads. They stay in loop_claude/ because they import its effects.py and hand_over.py: a package of their own would import loop_claude, and loop_claude would import it, a cycle.
"""
import contextlib
import dataclasses
+129 -120
View File
@@ -22,15 +22,15 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from routing_figures import archive, measure
from loop_claude.waiting.ask import Ask, Asking, OpenInInbox, OpenOnTerminal
from loop_claude.project import checkout, model_choice, records
from loop_claude.loop import chunk_start, close
from loop_claude.waiting.answers import ClaudeEnded, ClaudeOpening, send, Typed
from chunk_pick import DefaultPlan, LastCut
from release_standing import DefaultPlan, LastCut
from loop_claude.effects import Effects, POLL_SECONDS, PYTHON_TIMEOUT, run_python, say, system_effects
from claude_code import entries, reply, session
from loop_claude import escalation_mark, hand_over, paths, routing, running_loops, session_lock
from failure import Failure
from loop_claude.verdicts import Finished, Interrupt, QuotedAnswer, TakenIn, Unfinished
from loop_claude import hand_over, paths, routing, running_loops, session_lock
from loop_claude.cycle import interrupt, judge, prompts, retry, run, states, worker
from model_routing import ladder
from model_routing.ladder import LADDER, Rung
from claude_code.reply import Limited
from loop_claude.loop import main, reload, routing_check
@@ -228,15 +228,15 @@ class Transitions(unittest.TestCase):
states.transition(state, event)
def test_a_rulings_step_up_reaches_the_worker_it_resumes_through_the_answer_the_human_types_and_no_further(self):
step_up = escalation_mark.StepUp(Route(Rung("claude-opus-5-5", "medium")), Route(Rung("claude-opus-5-5", "high"), escalated_at=3))
stepped_to = Route(Rung("claude-opus-5-5", "high"), escalated_at=3)
interrupt, quoted = Interrupt("e", "q", "round 2: 🔴"), QuotedAnswer("e", "q", "A.", "round 2: 🔴")
for state, event, after in (
(states.Judging("/t"), states.Ruled(quoted, step_up), states.Working(quoted, 1, step_up)),
(states.Judging("/t", 3), states.Ruled(quoted, step_up), states.Interrupting("/t", quoted, 3, step_up)),
(states.Judging("/t"), states.Ruled(interrupt, step_up), states.Interrupting("/t", interrupt, 0, step_up)),
(states.Interrupting("/t", interrupt, 0, step_up), states.HumanTyped(TYPED), states.Working(TYPED, 0, step_up)),
(states.Interrupting("/t", interrupt, 0, step_up), states.HumanTalked(), states.Judging("/t")),
(states.Working(TYPED, 0, step_up), states.WorkerRan("/t"), states.Judging("/t")),
(states.Judging("/t"), states.Ruled(quoted, stepped_to), states.Working(quoted, 1, True)),
(states.Judging("/t", 3), states.Ruled(quoted, stepped_to), states.Interrupting("/t", quoted, 3, True)),
(states.Judging("/t"), states.Ruled(interrupt, stepped_to), states.Interrupting("/t", interrupt, 0, True)),
(states.Interrupting("/t", interrupt, 0, True), states.HumanTyped(TYPED), states.Working(TYPED, 0, True)),
(states.Interrupting("/t", interrupt, 0, True), states.HumanTalked(), states.Judging("/t")),
(states.Working(TYPED, 0, True), states.WorkerRan("/t"), states.Judging("/t")),
):
with self.subTest(state=state, event=event):
self.assertEqual(states.transition(state, event), after)
@@ -258,7 +258,7 @@ class Cycles(unittest.TestCase):
self.archive = project_archive(str(self.home / "code" / "proj"), self.home, {})
def cycle(self, fake, records="", unfinished_evidence="", facts="- A preflight fact."):
return run.Cycle(effects=fake.effects(), facts=facts, judge_model="judge-model", paths=self.paths, premises="## Goals", project=None, project_archive=self.archive, records=records, root=str(self.home), route=Route(Rung("worker-model")), session=self.session, show_taken_in=lambda items: None, task="Do it.", unfinished_evidence=unfinished_evidence)
return run.Cycle(effects=fake.effects(), facts=facts, judge_model="judge-model", premises="## Goals", project=None, project_archive=self.archive, questions=self.paths.waiting, records=records, root=str(self.home), route=Route(Rung("worker-model")), session=self.session, show_taken_in=lambda items: None, task="Do it.", unfinished_evidence=unfinished_evidence)
def archived_outcomes(self):
return sorted(path.read_text() for path in self.archive.glob("*/*.outcomes.md"))
@@ -345,6 +345,29 @@ class Cycles(unittest.TestCase):
f"{stamp} Judge reply rejected: the judge quoted 'never said', which is not in the transcript\n\nReply:\n> {misquoted.stdout}\n\nJudge: judge-model\n\n"
f"{stamp} Verdict: finished\n\nEvidence:\n> working\n\nJudge: judge-model\nWorker: {OFF_LADDER}\n"])
def test_archives_what_each_judge_run_cost_on_an_interrupt_a_quoted_answer_and_a_rejected_misquote(self):
for first, then, inputs in (
(judged("interrupt", cost=0.25), judged("finished"), ["Use B."]),
(judged("quoted", answer="Use A.", cost=0.25), judged("finished"), []),
(judged("finished", evidence="never said", cost=0.25), judged("finished"), []),
):
with self.subTest(first=first.stdout), tempfile.TemporaryDirectory() as archive:
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[first, then], inputs=inputs)
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
run.run_cycle(dataclasses.replace(self.cycle(fake, records="From AGENTS.md:\nUse A."), project_archive=pathlib.Path(archive)))
kept = "".join(path.read_text() for path in pathlib.Path(archive).glob("*/*.outcomes.md"))
self.assertEqual(kept.count("Judge cost: 0.2500\n"), 1, kept)
def test_stops_with_exit_1_asking_nothing_where_an_interrupt_cannot_be_archived(self):
self.archive.parent.mkdir(parents=True)
self.archive.write_text("")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt")], inputs=["Use B."])
with contextlib.redirect_stderr(io.StringIO()) as said:
stop = run.run_cycle(self.cycle(fake))
self.assertEqual((stop.code, stop.reason.split(":")[0]), (1, "could not archive the transcript"))
self.assertNotIn("Which one?", said.getvalue())
self.assertEqual(fake.made("claude_on_terminal"), [])
def test_stops_when_a_rejected_reply_cannot_be_archived(self):
self.archive.parent.mkdir(parents=True)
self.archive.write_text("")
@@ -460,7 +483,7 @@ class Cycles(unittest.TestCase):
for ruling in (Interrupt("e", "q", entries_read=3), QuotedAnswer("e", "q", "A.", entries_read=3)):
with self.subTest(ruling=ruling):
self.assertIsNone(run.ruling_step_up(cycle, ruling))
self.assertEqual(run.ruling_step_up(cycle, Interrupt("e", "q", "round 2: 🔴", TakenIn(highest_likelihood=4), entries_read=3)), escalation_mark.StepUp(cycle.route, Route(Routing.XHIGH, 3, 3, 4, escalated_at=3)))
self.assertEqual(run.ruling_step_up(cycle, Interrupt("e", "q", "round 2: 🔴", TakenIn(highest_likelihood=4), entries_read=3)), routing.StepUp(cycle.route, Route(Routing.XHIGH, 3, 3, 4, escalated_at=3)))
def test_an_escalation_after_an_interrupt_stepped_the_worker_steps_it_again_in_the_same_cycle(self):
streams = [Worker(self.paths.claude_code_transcripts.parent, text="round 2: 🔴"), Worker(self.paths.claude_code_transcripts.parent), Worker(self.paths.claude_code_transcripts.parent)]
@@ -696,7 +719,7 @@ class Interrupts(unittest.TestCase):
def put(self, fake, answer_here=True, effects=None, question="Which one?"):
"""interrupt's answer and what it said, run while the session's lock is held, as a running loop holds it."""
with contextlib.redirect_stderr(io.StringIO()) as said, session_lock.held(self.paths.waiting, str(self.session.id)):
answer = interrupt.interrupt(question, answer_here, self.session, self.paths, effects or fake.effects())
answer = interrupt.interrupt(question, answer_here, self.session, self.paths.waiting, effects or fake.effects())
return answer, said.getvalue()
def asked(self):
@@ -859,7 +882,7 @@ class RoutingCheck(unittest.TestCase):
"""What check_routing returns and says for a loop in /code/log."""
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()) as said:
self.enterContext(session_lock.held(self.paths.waiting, "s1"))
stop = routing_check.check_routing(self.root, "/code/log", "s1", Rung("m", "high"), self.paths, fake.effects())
stop = routing_check.check_routing(self.root, "/code/log", "s1", Rung("m", "high"), self.paths, self.paths.waiting, fake.effects())
return stop, said.getvalue()
def test_lists_the_calls_in_loop_inbox_while_their_claude_is_open(self):
@@ -898,7 +921,8 @@ class RoutingCheck(unittest.TestCase):
self.assertEqual(args[:6], ["--model", "m", "--effort", "high", "--permission-mode", "bypassPermissions"])
self.assertIn(f"change nothing until they answer:\n- {call}", args[6])
self.assertEqual(fake.terminal_environments, [{"CLAUDE_CODE_SANDBOXED": "1"}])
self.assertEqual(fake.terminal_folders, [str(self.home / ".claude")])
self.assertEqual(fake.terminal_folders, [str(self.home / "code" / "claude-config")], "the development clone, since ~/.claude holds only a release")
self.assertIn("Record what they decide in this checkout, the development clone", args[6])
(self.root / "routing-figures.md").write_text("Measured x, at cycle 0 across")
self.assertEqual(self.check(FakeEffects(git=[UNREAD]))[0], None)
self.assertNotIn(call, self.check(FakeEffects())[1])
@@ -933,7 +957,7 @@ class RoutingCheck(unittest.TestCase):
def test_stops_when_claude_cannot_start_for_the_calls(self):
self.chunks(5, 4)
stop, _ = self.check(FakeEffects(on_terminal_claude=[OSError(2, "No such file or directory")], git=[UNREAD]))
self.assertEqual(stop, Stop(1, "could not start claude for the routing calls: No such file or directory"))
self.assertEqual(stop, Stop(1, f"could not start claude for the routing calls in {self.home / 'code' / 'claude-config'}, the setup's development clone: No such file or directory"))
self.assertEqual(asks_and_answers(self.paths.waiting), [])
@unittest.skipIf(os.geteuid() == 0, "root writes into a read-only directory")
@@ -1028,47 +1052,47 @@ class Routing(unittest.TestCase):
self.assertEqual(routing.first_route(self.OPUS, None), Route(self.OPUS))
def test_steps_up_a_rung_after_every_second_unfinished_verdict(self):
once = routing.after_unfinished(self.OPUS, Route(self.OPUS, 3, 3, 4), unfinished())
once = routing.next_route(self.OPUS, Route(self.OPUS, 3, 3, 4), unfinished())
self.assertEqual(once, Route(self.OPUS, 3, 3, 4, unfinished=1))
self.assertEqual(routing.after_unfinished(self.OPUS, once, unfinished()), Route(self.HIGH, 3, 3, 4))
self.assertEqual(routing.after_unfinished(self.OPUS, Route(LADDER[-1], unfinished=1), unfinished()).rung, LADDER[-1])
self.assertEqual(routing.next_route(self.OPUS, once, unfinished()), Route(self.HIGH, 3, 3, 4))
self.assertEqual(routing.next_route(self.OPUS, Route(LADDER[-1], unfinished=1), unfinished()).rung, LADDER[-1])
def test_steps_up_to_the_rung_a_taken_in_items_l_picks_and_never_down(self):
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.OPUS), unfinished(4)).rung, self.HIGH)
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.XHIGH), unfinished(4)).rung, self.XHIGH)
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.OPUS, unfinished=1), unfinished(4)).rung, self.XHIGH)
self.assertEqual(routing.next_route(self.OPUS, Route(self.OPUS), unfinished(4)).rung, self.HIGH)
self.assertEqual(routing.next_route(self.OPUS, Route(self.XHIGH), unfinished(4)).rung, self.XHIGH)
self.assertEqual(routing.next_route(self.OPUS, Route(self.OPUS, unfinished=1), unfinished(4)).rung, self.XHIGH)
def test_steps_up_from_a_rung_below_the_projects_one_rung_at_a_time(self):
low = Route(self.SONNET, 3, 1, 3, unfinished=1)
self.assertEqual(routing.after_unfinished(self.OPUS, low, unfinished()).rung, self.OPUS)
self.assertEqual(routing.after_unfinished(self.OPUS, Route(low.rung, 3, 1, 3), unfinished()).rung, low.rung)
self.assertEqual(routing.after_unfinished(self.OPUS, Route(low.rung, 3, 1, 3), unfinished(3)).rung, self.OPUS)
self.assertEqual(routing.next_route(self.OPUS, low, unfinished()).rung, self.OPUS)
self.assertEqual(routing.next_route(self.OPUS, Route(low.rung, 3, 1, 3), unfinished()).rung, low.rung)
self.assertEqual(routing.next_route(self.OPUS, Route(low.rung, 3, 1, 3), unfinished(3)).rung, self.OPUS)
def test_steps_up_a_rung_on_an_escalation_the_judge_found_up_to_two_above_the_items_rung_marking_the_transcript_entries_it_read(self):
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.HIGH, 3, 4, 5), unfinished(escalated_at=7)), Route(self.XHIGH, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.XHIGH, 3, 4, 5, escalated_at=7), unfinished(escalated_at=9)), Route(self.MAX, 3, 4, 5, escalated_at=9))
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.MAX, 3, 4, 5, escalated_at=9), unfinished(escalated_at=12)), Route(self.MAX, 3, 4, 5, unfinished=1, escalated_at=9))
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.HIGH, 3, 4, 5, unfinished=1), unfinished(escalated_at=7)), Route(self.XHIGH, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.XHIGH), unfinished(escalated_at=7)), Route(self.XHIGH, unfinished=1))
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.SONNET, 3, 2, 4), unfinished(4, escalated_at=7)), Route(self.HIGH, 3, 2, 4, unfinished=1))
self.assertEqual(routing.next_route(self.OPUS, Route(self.HIGH, 3, 4, 5), unfinished(escalated_at=7)), Route(self.XHIGH, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(self.OPUS, Route(self.XHIGH, 3, 4, 5, escalated_at=7), unfinished(escalated_at=9)), Route(self.MAX, 3, 4, 5, escalated_at=9))
self.assertEqual(routing.next_route(self.OPUS, Route(self.MAX, 3, 4, 5, escalated_at=9), unfinished(escalated_at=12)), Route(self.MAX, 3, 4, 5, unfinished=1, escalated_at=9))
self.assertEqual(routing.next_route(self.OPUS, Route(self.HIGH, 3, 4, 5, unfinished=1), unfinished(escalated_at=7)), Route(self.XHIGH, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(self.OPUS, Route(self.XHIGH), unfinished(escalated_at=7)), Route(self.XHIGH, unfinished=1))
self.assertEqual(routing.next_route(self.OPUS, Route(self.SONNET, 3, 2, 4), unfinished(4, escalated_at=7)), Route(self.HIGH, 3, 2, 4, unfinished=1))
def test_keeps_the_mark_through_a_step_no_escalation_took(self):
self.assertEqual(routing.after_unfinished(self.OPUS, Route(self.OPUS, 3, 3, 4, unfinished=1, escalated_at=5), unfinished()), Route(self.HIGH, 3, 3, 4, escalated_at=5))
self.assertEqual(routing.next_route(self.OPUS, Route(self.OPUS, 3, 3, 4, unfinished=1, escalated_at=5), unfinished()), Route(self.HIGH, 3, 3, 4, escalated_at=5))
def test_steps_up_a_rung_as_the_judge_interrupts_up_to_two_above_the_items_rung_without_counting_an_unfinished_verdict(self):
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.HIGH, 3, 4, 5, unfinished=1), interrupted(escalated_at=7)), Route(self.XHIGH, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.XHIGH, 3, 4, 5, escalated_at=7), interrupted(escalated_at=9)), Route(self.MAX, 3, 4, 5, escalated_at=9))
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.MAX, 3, 4, 5, unfinished=1, escalated_at=9), interrupted(escalated_at=12)), Route(self.MAX, 3, 4, 5, unfinished=1, escalated_at=9))
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.SONNET, 3, 2, 4), interrupted(escalated_at=7)), Route(self.OPUS, 3, 2, 4, escalated_at=7))
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.HIGH, 3, 4, 5, unfinished=1, escalated_at=2), interrupted(4)), Route(self.HIGH, 3, 4, 5, unfinished=1, escalated_at=2))
self.assertEqual(routing.next_route(self.OPUS, Route(self.HIGH, 3, 4, 5, unfinished=1), interrupted(escalated_at=7)), Route(self.XHIGH, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(self.OPUS, Route(self.XHIGH, 3, 4, 5, escalated_at=7), interrupted(escalated_at=9)), Route(self.MAX, 3, 4, 5, escalated_at=9))
self.assertEqual(routing.next_route(self.OPUS, Route(self.MAX, 3, 4, 5, unfinished=1, escalated_at=9), interrupted(escalated_at=12)), Route(self.MAX, 3, 4, 5, unfinished=1, escalated_at=9))
self.assertEqual(routing.next_route(self.OPUS, Route(self.SONNET, 3, 2, 4), interrupted(escalated_at=7)), Route(self.OPUS, 3, 2, 4, escalated_at=7))
self.assertEqual(routing.next_route(self.OPUS, Route(self.HIGH, 3, 4, 5, unfinished=1, escalated_at=2), interrupted(4)), Route(self.HIGH, 3, 4, 5, unfinished=1, escalated_at=2))
def test_raises_the_worker_to_a_taken_in_items_rung_above_the_escalation_cap_without_a_step_as_the_judge_interrupts(self):
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.OPUS, 3, 1, 3, escalated_at=2), interrupted(4, escalated_at=7)), Route(self.HIGH, 3, 1, 3, escalated_at=2))
self.assertEqual(routing.next_route(self.OPUS, Route(self.OPUS, 3, 1, 3, escalated_at=2), interrupted(4, escalated_at=7)), Route(self.HIGH, 3, 1, 3, escalated_at=2))
def test_steps_up_from_the_rung_a_taken_in_item_picks_as_the_judge_interrupts(self):
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.OPUS, 3, 3, 4), interrupted(4, escalated_at=7)), Route(self.XHIGH, 3, 3, 4, escalated_at=7))
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.XHIGH, 3, 4, 5), interrupted(4, escalated_at=7)), Route(self.MAX, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.after_ruling(self.OPUS, Route(self.MAX, 3, 4, 5, unfinished=1), interrupted(4, escalated_at=7)), Route(self.MAX, 3, 4, 5, unfinished=1))
self.assertEqual(routing.next_route(self.OPUS, Route(self.OPUS, 3, 3, 4), interrupted(4, escalated_at=7)), Route(self.XHIGH, 3, 3, 4, escalated_at=7))
self.assertEqual(routing.next_route(self.OPUS, Route(self.XHIGH, 3, 4, 5), interrupted(4, escalated_at=7)), Route(self.MAX, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(self.OPUS, Route(self.MAX, 3, 4, 5, unfinished=1), interrupted(4, escalated_at=7)), Route(self.MAX, 3, 4, 5, unfinished=1))
def test_keeps_an_e_plus_i_taken_in_only_where_it_picks_a_higher_review_rung(self):
taken = Route(self.OPUS, 3, 3, 4).taking_in(6)
@@ -1083,8 +1107,8 @@ class Routing(unittest.TestCase):
self.assertEqual(Route(self.OPUS).taking_in(0), Route(self.OPUS))
def test_a_rung_an_edit_took_off_the_ladder_counts_as_the_projects(self):
self.assertEqual(routing.after_unfinished(self.OPUS, Route(Rung("claude-opus-5-5", "low")), unfinished()).rung, self.OPUS)
self.assertEqual(routing.after_unfinished(self.OPUS, Route(Rung("claude-opus-5-5", "low")), unfinished(2)).rung, self.OPUS)
self.assertEqual(routing.next_route(self.OPUS, Route(Rung("claude-opus-5-5", "low")), unfinished()).rung, self.OPUS)
self.assertEqual(routing.next_route(self.OPUS, Route(Rung("claude-opus-5-5", "low")), unfinished(2)).rung, self.OPUS)
def test_runs_the_judge_on_the_rung_its_chunks_items_e_plus_i_picks_never_below_the_projects(self):
self.assertEqual(routing.judge_rung("claude-opus-5-5", Route(self.OPUS, 3, 3, 5)), self.HIGH)
@@ -1151,26 +1175,26 @@ class Routing(unittest.TestCase):
archived = Route(self.XHIGH, 3, 4, 5)
def never():
raise AssertionError("not asked")
self.assertEqual(main.routed(archived, None, "worker-model", never, never), Route(Rung("worker-model")))
self.assertEqual(main.routed(archived, self.OPUS, "claude-opus-5-5", never, never), archived)
self.assertEqual(main.routed(None, self.OPUS, "claude-opus-5-5", lambda: archived, never), archived)
self.assertEqual(main.routed(None, self.OPUS, "claude-opus-5-5", lambda: None, lambda: item), Route(self.HIGH, 3, 4, 5))
self.assertEqual(main.routed(Route(self.SONNET), self.OPUS, "claude-opus-5-5", never, lambda: None), Route(self.OPUS))
self.assertEqual(chunk_start.routed(archived, None, "worker-model", never, never), Route(Rung("worker-model")))
self.assertEqual(chunk_start.routed(archived, self.OPUS, "claude-opus-5-5", never, never), archived)
self.assertEqual(chunk_start.routed(None, self.OPUS, "claude-opus-5-5", lambda: archived, never), archived)
self.assertEqual(chunk_start.routed(None, self.OPUS, "claude-opus-5-5", lambda: None, lambda: item), Route(self.HIGH, 3, 4, 5))
self.assertEqual(chunk_start.routed(Route(self.SONNET), self.OPUS, "claude-opus-5-5", never, lambda: None), Route(self.OPUS))
cheap = Route(self.SONNET, 4, 2, 3)
self.assertEqual(main.routed(cheap, self.OPUS, "claude-opus-5-5", never, never), cheap)
self.assertEqual(main.routed(Route(Rung("claude-haiku-5-5", "max"), 4, 2, 3), self.OPUS, "claude-opus-5-5", never, lambda: None), Route(self.OPUS))
self.assertEqual(chunk_start.routed(cheap, self.OPUS, "claude-opus-5-5", never, never), cheap)
self.assertEqual(chunk_start.routed(Route(Rung("claude-haiku-5-5", "max"), 4, 2, 3), self.OPUS, "claude-opus-5-5", never, lambda: None), Route(self.OPUS))
class RouteSources(unittest.TestCase):
def test_routes_on_the_plan_item_and_says_why_it_routes_on_the_projects_rung(self):
self.assertIsNone(main.plan_item(DefaultPlan("", NO_CUT), FakeEffects().effects()))
self.assertEqual(main.plan_item(DefaultPlan(TODO, LastCut("v0.1.0", "", 3, [TODO])), FakeEffects().effects()).id, 3)
self.assertIsNone(main.plan_item(DefaultPlan(TODO, LastCut("v0.2.0", "", 3, [TODO])), FakeEffects().effects()))
self.assertIsNone(chunk_start.plan_item(DefaultPlan("", NO_CUT), FakeEffects().effects()))
self.assertEqual(chunk_start.plan_item(DefaultPlan(TODO, LastCut("v0.1.0", "", 3, [TODO])), FakeEffects().effects()).id, 3)
self.assertIsNone(chunk_start.plan_item(DefaultPlan(TODO, LastCut("v0.2.0", "", 3, [TODO])), FakeEffects().effects()))
unreadable = Failure("git show origin/main:todo.md printed text that is not UTF-8")
for plan, said in ((DefaultPlan(TODO.replace("| 4 | 2 | 3 |", "| x | 2 | 3 |"), NO_CUT), "routing on the project's rung: item 3's L 'x' is not a whole number from 1 to 4\n"), (DefaultPlan(unreadable, NO_CUT), f"routing on the project's rung: {unreadable.reason}\n"), (Failure("no origin remote"), "routing on the project's rung: no origin remote\n")):
with self.subTest(said=said):
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(main.plan_item(plan, FakeEffects().effects()))
self.assertIsNone(chunk_start.plan_item(plan, FakeEffects().effects()))
self.assertTrue(stderr.getvalue().startswith(f"{STAMP} {said}"), stderr.getvalue())
def test_resumes_on_the_archived_route_and_says_why_it_routes_afresh(self):
@@ -1180,25 +1204,25 @@ class RouteSources(unittest.TestCase):
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
route = Route(Rung("claude-opus-5-5", "high"), 3, 4, 5)
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(main.archived_route(archive, opened, FakeEffects().effects()))
self.assertIsNone(chunk_start.archived_route(archive, opened, FakeEffects().effects()))
transcript.parent.mkdir(parents=True)
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
self.assertIsNone(main.archived_route(archive, opened, FakeEffects().effects()))
self.assertIsNone(chunk_start.archived_route(archive, opened, FakeEffects().effects()))
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(route.worker_line(), "working", "j", ""), NOW))
with contextlib.redirect_stderr(stderr):
self.assertEqual(main.archived_route(archive, opened, FakeEffects().effects()), route)
self.assertEqual(chunk_start.archived_route(archive, opened, FakeEffects().effects()), route)
self.assertEqual(stderr.getvalue(), "")
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.stepped_up(WorkerLine('{"model": "claude-opus-5-5", "rung": 3}')), NOW + 1))
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(main.archived_route(archive, opened, FakeEffects().effects()))
self.assertIsNone(chunk_start.archived_route(archive, opened, FakeEffects().effects()))
self.assertEqual(stderr.getvalue(), f'{STAMP} routing the chunk afresh: the archive\'s latest Worker: line names no route loop-claude can read: {{"model": "claude-opus-5-5", "rung": 3}}\n')
(chunk,) = archive.iterdir()
chunk.chmod(0)
try:
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(main.archived_route(archive, opened, FakeEffects().effects()))
self.assertIsNone(chunk_start.archived_route(archive, opened, FakeEffects().effects()))
finally:
chunk.chmod(0o700)
self.assertTrue(stderr.getvalue().startswith(f"{STAMP} routing the chunk afresh: could not read the archive"), stderr.getvalue())
@@ -1209,29 +1233,29 @@ class Steps(unittest.TestCase):
def test_a_finished_verdict_clears_the_route_and_the_evidence(self):
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="old", route=Route(self.OPUS))
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Finished("x"), run_with.route)), dataclasses.replace(run_with, route=None, unfinished_evidence=""))
self.assertEqual(main.after_verdict(dataclasses.replace(run_with, unfinished_run=3), self.OPUS, run.Concluded(Finished("x"), run_with.route)).unfinished_run, 0)
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", taken_in=TakenIn(highest_likelihood=4)), run_with.route)), dataclasses.replace(run_with, route=Route(Rung("claude-opus-5-5", "high"), unfinished=1), unfinished_evidence="new", unfinished_run=1))
self.assertEqual(close.after_verdict(run_with, self.OPUS, run.Concluded(Finished("x"), run_with.route)), dataclasses.replace(run_with, route=None, unfinished_evidence=""))
self.assertEqual(close.after_verdict(dataclasses.replace(run_with, unfinished_run=3), self.OPUS, run.Concluded(Finished("x"), run_with.route)).unfinished_run, 0)
self.assertEqual(close.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", taken_in=TakenIn(highest_likelihood=4)), run_with.route)), dataclasses.replace(run_with, route=Route(Rung("claude-opus-5-5", "high"), unfinished=1), unfinished_evidence="new", unfinished_run=1))
def test_an_unfinished_chunk_keeps_the_items_its_verdict_found_taken_in_and_a_finished_one_drops_them(self):
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=Route(self.OPUS), items_taken_in=(3,))
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", taken_in=TakenIn((3, 12))), run_with.route)).items_taken_in, (3, 12))
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Finished("x"), run_with.route)).items_taken_in, ())
self.assertEqual(close.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", taken_in=TakenIn((3, 12))), run_with.route)).items_taken_in, (3, 12))
self.assertEqual(close.after_verdict(run_with, self.OPUS, run.Concluded(Finished("x"), run_with.route)).items_taken_in, ())
def test_steps_from_the_route_its_cycle_ended_on_which_a_ruling_may_have_raised(self):
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=Route(self.OPUS, 3, 3, 4))
raised = Route(Rung("claude-opus-5-5", "high"), 3, 3, 4, escalated_at=5)
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new"), raised)).route, dataclasses.replace(raised, unfinished=1))
self.assertEqual(close.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new"), raised)).route, dataclasses.replace(raised, unfinished=1))
def test_keeps_the_route_its_cycle_ended_on_off_the_ladder(self):
off_ladder = Route(Rung("worker-model"))
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="")
self.assertEqual(main.after_verdict(run_with, None, run.Concluded(Unfinished("new", "round 2: 🔴", TakenIn(highest_likelihood=4), entries_read=9), off_ladder)).route, off_ladder)
self.assertEqual(close.after_verdict(run_with, None, run.Concluded(Unfinished("new", "round 2: 🔴", TakenIn(highest_likelihood=4), entries_read=9), off_ladder)).route, off_ladder)
def test_an_unfinished_verdicts_escalation_marks_the_transcript_entries_the_judge_read(self):
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=Route(self.OPUS, 3, 3, 4, escalated_at=2))
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", "round 2: 🔴", entries_read=9), run_with.route)).route, Route(Rung("claude-opus-5-5", "high"), 3, 3, 4, escalated_at=9))
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", entries_read=9), run_with.route)).route, Route(self.OPUS, 3, 3, 4, unfinished=1, escalated_at=2))
self.assertEqual(close.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", "round 2: 🔴", entries_read=9), run_with.route)).route, Route(Rung("claude-opus-5-5", "high"), 3, 3, 4, escalated_at=9))
self.assertEqual(close.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", entries_read=9), run_with.route)).route, Route(self.OPUS, 3, 3, 4, unfinished=1, escalated_at=2))
def test_archives_a_step_up_as_it_happens_so_a_rerun_resumes_on_it(self):
with tempfile.TemporaryDirectory() as home:
@@ -1243,18 +1267,21 @@ class Steps(unittest.TestCase):
before = Route(self.OPUS, 3, 3, 4)
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(before.worker_line(), "working", "j", ""), NOW))
after = main.after_verdict(main.LoopRun(cycle_number=1, root="/r", task="Do it.", unfinished_evidence=""), self.OPUS, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_likelihood=4)), before))
self.assertIsNone(main.archive_step(archive, opened, "Do it.", before, after.route, FakeEffects().effects()))
self.assertEqual(main.archived_route(archive, opened, FakeEffects().effects()), Route(Rung("claude-opus-5-5", "high"), 3, 3, 4))
self.assertIsNone(main.archive_step(archive, opened, "Do it.", after.route, after.route, FakeEffects().effects()))
finished = main.after_verdict(after, self.OPUS, run.Concluded(Finished("done"), after.route))
self.assertIsNone(main.archive_step(archive, opened, "Do it.", after.route, finished.route, FakeEffects().effects()))
self.assertEqual(main.archived_route(archive, opened, FakeEffects().effects()), Route(Rung("claude-opus-5-5", "high"), 3, 3, 4))
after = close.closed_cycle(main.LoopRun(cycle_number=1, root="/r", task="Do it.", unfinished_evidence=""), self.OPUS, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_likelihood=4)), before), archive, opened, FakeEffects().effects())
self.assertEqual(chunk_start.archived_route(archive, opened, FakeEffects().effects()), Route(Rung("claude-opus-5-5", "high"), 3, 3, 4))
with contextlib.redirect_stderr(io.StringIO()):
close.closed_cycle(after, self.OPUS, run.Concluded(Finished("done"), after.route), archive, opened, FakeEffects().effects())
self.assertEqual(chunk_start.archived_route(archive, opened, FakeEffects().effects()), Route(Rung("claude-opus-5-5", "high"), 3, 3, 4))
blocked = pathlib.Path(home, "blocked")
blocked.write_text("")
stop = main.archive_step(blocked, opened, "Do it.", before, after.route, FakeEffects().effects())
stop = close.closed_cycle(main.LoopRun(cycle_number=1, root="/r", task="Do it.", unfinished_evidence=""), self.OPUS, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_likelihood=4)), before), blocked, opened, FakeEffects().effects())
self.assertEqual((stop.code, stop.reason.split(":")[0]), (1, "could not archive the transcript"))
def test_names_no_step_up_where_only_the_count_of_unfinished_verdicts_changed(self):
route = Route(self.OPUS, 3, 3, 4)
self.assertIsNone(routing.step_up(route, dataclasses.replace(route, unfinished=1)))
self.assertEqual(routing.step_up(route, route.taking_in(6)), routing.StepUp(route, route.taking_in(6)))
def test_archives_a_higher_e_plus_i_taken_in_so_a_rerun_keeps_it(self):
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj loop-claude", pathlib.Path(home, "projects"))
@@ -1265,9 +1292,8 @@ class Steps(unittest.TestCase):
before = Route(self.OPUS, 3, 3, 4)
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(before.worker_line(), "working", "j", ""), NOW))
after = main.after_verdict(main.LoopRun(cycle_number=1, root="/r", task="Do it.", unfinished_evidence=""), self.OPUS, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_escape_and_impact=6)), before))
self.assertIsNone(main.archive_step(archive, opened, "Do it.", before, after.route, FakeEffects().effects()))
self.assertEqual(main.archived_route(archive, opened, FakeEffects().effects()), Route(self.OPUS, 3, 3, 4, escape_and_impact_taken_in=6))
close.closed_cycle(main.LoopRun(cycle_number=1, root="/r", task="Do it.", unfinished_evidence=""), self.OPUS, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_escape_and_impact=6)), before), archive, opened, FakeEffects().effects())
self.assertEqual(chunk_start.archived_route(archive, opened, FakeEffects().effects()), Route(self.OPUS, 3, 3, 4, escape_and_impact_taken_in=6))
class ChunkFacts(unittest.TestCase):
@@ -1281,7 +1307,7 @@ class ChunkFacts(unittest.TestCase):
def test_runs_the_preflight_at_the_checkout_root_and_quotes_what_it_printed(self):
fake = FakeEffects(python=[ran("- A fact.\n")])
said = main.chunk_facts(self.PREFLIGHT, "/r", fake.effects())
said = chunk_start.chunk_facts(self.PREFLIGHT, "/r", fake.effects())
self.assertEqual(fake.made("python"), [(["/b/chunk-preflight", "--worktree"], "/r")])
self.assertIn("\n\n- A fact.", said)
self.assertNotIn("yourself", said)
@@ -1289,7 +1315,7 @@ class ChunkFacts(unittest.TestCase):
def test_has_the_worker_find_the_facts_itself_where_the_preflight_failed(self):
for failed, why in ((ran(code=1, stderr="git could not start\n"), "git could not start"), (ran(code=1), "exited 1")):
with self.subTest(why=why):
said = main.chunk_facts(self.PREFLIGHT, "/r", FakeEffects(python=[failed]).effects())
said = chunk_start.chunk_facts(self.PREFLIGHT, "/r", FakeEffects(python=[failed]).effects())
self.assertIn("yourself", said)
self.assertIn(why, said)
@@ -1297,13 +1323,13 @@ class ChunkFacts(unittest.TestCase):
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj loop-claude", pathlib.Path(home, "projects"))
fresh = FakeEffects(python=[ran("- A fact.\n")])
facts, _ = main.chunk_start(opened, self.PREFLIGHT, "/r", fresh.effects())
facts, _ = chunk_start.chunk_start(opened, self.PREFLIGHT, "/r", fresh.effects())
self.assertEqual((fresh.made("python"), "- A fact." in facts), ([(["/b/chunk-preflight", "--worktree"], "/r")], True))
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text("")
resumed = FakeEffects()
facts, _ = main.chunk_start(opened, self.PREFLIGHT, "/r", resumed.effects())
facts, _ = chunk_start.chunk_start(opened, self.PREFLIGHT, "/r", resumed.effects())
self.assertEqual((resumed.made("python"), facts), ([], ""))
@@ -1407,7 +1433,7 @@ class TypedAnswers(unittest.TestCase):
class WorkerArgs(unittest.TestCase):
SCAN = " Run a full scan only from such a subagent, since the scan's agents run on the model that starts them."
SEATS = " Launch each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung."
MARKDOWN = " Where your branch changes only Markdown, none of it `CLAUDE.md`, `AGENTS.md` or under `agents/`, `principles/`, `rules/` or `skills/`, launch each reviewer on the project's rung instead, with the Agent tool set to model opus, effort medium."
MARKDOWN = " Where your branch changes only Markdown, none of it `CLAUDE.md`, `AGENTS.md`, `workflow-bench/*-prompt.md` or under `agents/`, `principles/`, `rules/` or `skills/`, launch each reviewer on the project's rung instead, with the Agent tool set to model opus, effort medium."
# What the preflight found, as the fresh prompt quotes it.
FACTS = "- A preflight fact."
@@ -1501,25 +1527,11 @@ class WorkerArgs(unittest.TestCase):
def test_lets_a_markdown_only_branch_review_on_the_projects_rung_where_the_reviewers_run_above_it(self):
opus = Rung("claude-opus-5-5", "medium")
markdown = " Where your branch changes only Markdown, none of it `CLAUDE.md`, `AGENTS.md` or under `agents/`, `principles/`, `rules/` or `skills/`, launch each reviewer on the project's rung instead, with the Agent tool set to model opus, effort medium."
markdown = " Where your branch changes only Markdown, none of it `CLAUDE.md`, `AGENTS.md`, `workflow-bench/*-prompt.md` or under `agents/`, `principles/`, `rules/` or `skills/`, launch each reviewer on the project's rung instead, with the Agent tool set to model opus, effort medium."
self.assertIn(markdown, prompts.fresh("Do it.", "## Goals", Route(opus, 3, 3, 7), opus, self.FACTS))
self.assertIn(markdown, prompts.resumed("", Route(Rung("claude-opus-5-5", "high"), 3, 4, 4), opus))
self.assertNotIn("only Markdown", prompts.fresh("Do it.", "## Goals", Route(opus, 3, 3, 4), opus, self.FACTS))
def test_launches_each_pass_a_replay_lowered_on_its_replays_rung_and_reruns_it_one_above(self):
opus, sonnet = Rung("claude-opus-5-5", "medium"), Rung("claude-sonnet-5-5", "high")
with unittest.mock.patch.dict(ladder.REPLAYED, {("prose", "claude-opus-5-5"): sonnet}):
below = prompts.fresh("Do it.", "## Goals", Route(sonnet, 3, 2, 4), opus, self.FACTS)
on_it = prompts.resumed("", Route(sonnet, 3, 1, 4), opus)
launched = " Launch each reviewer, each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung. Launch each prose-reviewer instead with the Agent tool set to model sonnet, effort high, the rung the replay of its pass chose."
rerun = " A reviewer or comprehension seat that asks for a rerun one rung up reruns with the Agent tool set to `model opus, effort high`. Rerun each prose-reviewer that asks with the Agent tool set to `model opus, effort medium` instead."
self.assertIn(f"whether or not it is item 3.{launched}", below)
self.assertIn(" Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model opus, effort high` from 5,", below)
self.assertIn(f"{rerun} After your first read", below)
self.assertIn(f"loop-claude runs you on claude-sonnet-5-5 at high effort.{launched}", on_it)
self.assertIn(" Where an item you take into the chunk has an E + I of 5 or more,", on_it)
self.assertTrue(on_it.endswith(rerun))
def test_names_the_rung_a_reviewer_or_seat_reruns_on_one_above_its_own(self):
opus = Rung("claude-opus-5-5", "medium")
both = " A reviewer or comprehension seat that asks for a rerun one rung up reruns with the Agent tool set to `model opus, effort high`."
@@ -1577,11 +1589,11 @@ class Records(unittest.TestCase):
with tempfile.TemporaryDirectory() as root:
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
claude_md.mkdir(parents=True)
self.assertEqual(records.read_records(root, claude_md.parent), Stop(1, f"CLAUDE.md in {claude_md.parent}: Is a directory - the judge cannot read the records."))
self.assertEqual(records.read_records(root, claude_md.parent), Failure(f"CLAUDE.md in {claude_md.parent}: Is a directory - the judge cannot read the records."))
pathlib.Path(root, "docs", "decisions.md").mkdir(parents=True)
self.assertEqual(records.read_records(root, claude_md.parent), Stop(1, f"docs/decisions.md in {root}: Is a directory - the judge cannot read the records."))
self.assertEqual(records.read_records(root, claude_md.parent), Failure(f"docs/decisions.md in {root}: Is a directory - the judge cannot read the records."))
pathlib.Path(root, "AGENTS.md").write_bytes(b"# Rules\n\xff\n")
self.assertEqual(records.read_records(root, claude_md.parent), Stop(1, f"AGENTS.md in {root} is not UTF-8 on line 2 - the judge cannot read the records."))
self.assertEqual(records.read_records(root, claude_md.parent), Failure(f"AGENTS.md in {root} is not UTF-8 on line 2 - the judge cannot read the records."))
def test_stops_on_a_rule_it_cannot_read(self):
@@ -1590,7 +1602,7 @@ class Records(unittest.TestCase):
rules = pathlib.Path(claude_md.parent, "rules")
rules.mkdir(parents=True)
pathlib.Path(rules, "todo-items.md").write_bytes(b"\xff\n")
self.assertEqual(records.read_records(root, claude_md.parent), Stop(1, f"todo-items.md in {rules} is not UTF-8 on line 1 - the judge cannot read the records."))
self.assertEqual(records.read_records(root, claude_md.parent), Failure(f"todo-items.md in {rules} is not UTF-8 on line 1 - the judge cannot read the records."))
class JudgePrompt(unittest.TestCase):
def test_shows_the_transcripts_tail_and_returns_what_it_showed(self):
@@ -1724,12 +1736,12 @@ class Verdict(unittest.TestCase):
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "A.", "round 2: … 🔴"), judge.Grounds(shown, shown, "A.")), QuotedAnswer("e", "q", "A."))
def test_puts_a_quoted_answer_the_records_do_not_hold_to_the_human_with_the_escalation_it_carried(self):
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "not there", taken_in=TakenIn((3,))), judge.Grounds("shown", "shown", "A.")), Interrupt("e", "q", taken_in=TakenIn((3,))))
self.assertEqual(judge.quotes_checked(QuotedAnswer("s", "q", "not there", taken_in=TakenIn((3,))), judge.Grounds("shown", "shown", "A.")), Interrupt("s", "q", taken_in=TakenIn((3,))))
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "not there", "round 2: 🔴", TakenIn(highest_likelihood=4)), judge.Grounds("The agent: round 2: 🔴.", "The agent: round 2: 🔴.", "A.")), Interrupt("e", "q", "round 2: 🔴", TakenIn(highest_likelihood=4)))
def test_puts_a_quoted_answer_the_records_do_not_hold_word_for_word_to_the_human(self):
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "not there"), judge.Grounds("shown", "shown", "the records")), Interrupt("e", "q"))
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "the records"), judge.Grounds("shown", "shown", "From the records: the records")), QuotedAnswer("e", "q", "the records"))
self.assertEqual(judge.quotes_checked(QuotedAnswer("s", "q", "not there"), judge.Grounds("shown", "shown", "the records")), Interrupt("s", "q"))
self.assertEqual(judge.quotes_checked(QuotedAnswer("s", "q", "the records"), judge.Grounds("shown", "shown", "From the records: the records")), QuotedAnswer("s", "q", "the records"))
def test_drops_an_escalation_quote_found_only_in_the_transcript_entries_before_the_workers_last_step_up(self):
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), assistant({"type": "text", "text": "round 3: 🔴"}), assistant({"type": "text", "text": "working"})]
@@ -1823,8 +1835,8 @@ class HelpLimits(unittest.TestCase):
self.assertIn(f"still running after {PYTHON_TIMEOUT} s", said)
self.assertIn(f"Once {measure.CYCLES_PER_CHECK} cycles have closed across all projects", said)
self.assertIn(f"run with a {worker.COMPACT_WINDOW // 1000}k auto-compact window", said)
self.assertIn(f"unfinished {main.UNFINISHED_LIMIT} cycles running", said)
self.assertIn(f"rerun to give it {main.UNFINISHED_LIMIT} more", said)
self.assertIn(f"unfinished {close.UNFINISHED_LIMIT} cycles running", said)
self.assertIn(f"rerun to give it {close.UNFINISHED_LIMIT} more", said)
class ExitContract(unittest.TestCase):
@@ -1841,26 +1853,22 @@ class ExitContract(unittest.TestCase):
("cycle/retry.py", "retried", "1", "f'the {who} failed three times running:", "the worker or the judge fails three times running"),
("cycle/run.py", "archived_stop", "1", "f'{failure.reason}", read),
("cycle/run.py", "work", "1", "f'claude left no transcript for session", "claude leaves no transcript"),
("cycle/run.py", "keep_answer", "1", "failure.reason", read),
("cycle/run.py", "rule", "1", "entries.reason", read),
("cycle/run.py", "rule", "1", "failure.reason", read),
("cycle/run.py", "keep_rejection", "1", "unkept.reason", read),
("cycle/run.py", "ask", "1", "failure.reason", read),
("cycle/run.py", "close", "1", "failure.reason", read),
("cycle/run.py", "close", "1", "failure.reason", "a finished chunk's session cannot be deleted"),
("loop/main.py", "archive_step", "1", "failure.reason", read),
("loop/close.py", "closed_cycle", "1", "failure.reason", read),
("loop/main.py", "next_cycle", "1", "premises.reason", premises_),
("loop/main.py", "next_cycle", "1", "records.reason", unreadable),
("loop/main.py", "loop_run_from", "0 if task else 1", "__doc__.strip()", "no task is given"),
("loop/main.py", "loop_run_from", "1", 'f"the edited loop-claude cannot read the state', "a reload cannot read the state the running script handed it"),
("loop/reload.py", "exec_if_edited", "1", "f'could not reload loop-claude:", "python cannot start for a reload"),
("loop/routing_check.py", "raise_calls", "1", "f'could not start claude for the routing calls:", "claude cannot start for an interrupt"),
("loop/routing_check.py", "raise_calls", "1", "f\"could not start claude for the routing calls in {paths.config_clone}", "claude cannot start for an interrupt"),
("project/checkout.py", "checkout_root", "1", "found.reason", git),
("project/model_choice.py", "get_or_ask_model", "1", "models.reason", models_),
("project/model_choice.py", "get_or_ask_model", "1", "models.reason", models_),
("project/model_choice.py", "get_or_ask_model", "1", "f'could not save the model in", models_),
("project/model_choice.py", "read_models_file", "1", 'f"could not read {saved}: {getattr(', models_),
("project/model_choice.py", "read_models_file", "1", "f'could not read {saved}: not an object", models_),
("project/premises.py", "read_premises", "1", "f'README.md has no ##", premises_),
("project/premises.py", "read_premises", "1", "f'README.md has ## {name} more than once", premises_),
("project/premises.py", "read_premises", "1", "f'README.md has nothing under", premises_),
("project/text_file.py", "read_utf8", "1", "f'{name} in {folder}:", unreadable),
("project/text_file.py", "read_utf8", "1", "f'{name} in {folder} is not UTF-8", unreadable),
]
root = pathlib.Path(main.__file__).parents[1]
found = []
@@ -1887,7 +1895,8 @@ class JudgeFailures(unittest.TestCase):
("verdict_from", "'the judge interrupted without a question'", unasked),
("verdict_from", "'the judge quoted an answer without a question'", unasked),
("run_judge", "verdict.reason", field),
("run_judge", "f'the judge quoted {verdict.evidence!r}, which is not in the transcript", "quotes a passage the transcript does not hold"),
("run_judge", "checked.reason", "quotes a passage the transcript does not hold"),
("quotes_checked", "f'the judge quoted {verdict.evidence!r}, which is not in the transcript", "quotes a passage the transcript does not hold"),
("claude_reply", "f'could not start claude:", started),
("claude_reply", "proc.stderr.strip() or f'{who} exited", exited),
("claude_reply", 'f"{who} printed no json reply', unparsed),
+6 -16
View File
@@ -3,7 +3,7 @@ import dataclasses
import re
import typing
from review_passes import ReviewedPass, REVIEWERS
from review_passes import ReviewedPass
# How Rung.__str__ spells a rung that names an effort, and Rung.parse reads it back.
WORDING = re.compile(r"(?P<model>\S+) at (?P<effort>\S+) effort")
@@ -60,8 +60,6 @@ LADDER = (
WORKER_STEPS = {1: -1, 2: -1, 3: 0, 4: 1}
# The rungs a task's E + I steps a review up from the project's rung; an E + I under 5 steps none.
REVIEW_STEPS = {5: 1, 6: 2, 7: 3, 8: 4}
# The rung a pass's replay chose for the projects on a model: docs/decisions.md → `routing: a review pass runs below the project's rung only on a rung that a replay of its rounds shows finding every 🔴 and 🟡 the project's rung finds`.
REPLAYED: dict[tuple[ReviewedPass, str], Rung] = {}
def project_rung(model: str) -> Rung | None:
@@ -94,24 +92,16 @@ def stepped_up(rung: Rung) -> Rung:
return climbed(rung, 1)
def review_rungs(project: Rung, worker: Rung, escape_and_impact: int) -> dict[ReviewKind, Rung]:
"""The rung each kind of review in a chunk runs on, where its worker runs on worker and its item has that E + I.
def review_rung(project: Rung, worker: Rung, escape_and_impact: int, kind: ReviewKind) -> Rung:
"""The rung a review of this kind runs on in a chunk whose worker runs on worker and whose item has that E + I.
A pass's reviewer runs on reviewers_rung. Where the E + I steps no review up and the worker runs no higher than the project's rung, a pass whose replay chose a rung for the project's model runs on that rung instead, even below the worker's: the replay compared reviews on the project's rung only, never on a rung an E + I stepped up, nor of a chunk whose worker climbed above the project's rung. A seat and the security pass run on the project's rung, so the seats score every chunk on one rung, and the security pass spends no more than the project chose.
A pass's reviewer runs on reviewers_rung; no replay has qualified a lower rung for any pass, docs/decisions.md → `routing: a review pass runs below the project's rung only on a rung that a replay …`. A seat and the security pass run on the project's rung, so the seats score every chunk on one rung, and the security pass spends no more than the project chose.
"""
rungs: dict[ReviewKind, Rung] = {**dict.fromkeys(REVIEWERS, reviewers_rung(project, worker, escape_and_impact)), "seat": project, "security": project}
if replay_applies(project, worker, escape_and_impact):
rungs |= {kind: rung for (kind, model), rung in REPLAYED.items() if model == project.model}
return rungs
def replay_applies(project: Rung, worker: Rung, escape_and_impact: int) -> bool:
"""Whether a pass's replay chose its rung for a chunk like this: only where the E + I steps no review up and the worker runs no higher than the project's rung."""
return escape_and_impact not in REVIEW_STEPS and position(worker) <= position(project)
return project if kind in ("seat", "security") else reviewers_rung(project, worker, escape_and_impact)
def reviewers_rung(project: Rung, worker: Rung, escape_and_impact: int) -> Rung:
"""The rung of each pass's reviewer that no replay lowered: the higher of the worker's rung and review_floor's."""
"""The rung of each pass's reviewer: the higher of the worker's rung and review_floor's."""
return highest(worker, review_floor(project, escape_and_impact))
+5 -11
View File
@@ -2,16 +2,13 @@
import dataclasses
from model_routing.route import HIGHEST_ESCAPE_AND_IMPACT, Route
from model_routing.ladder import replay_applies, review_rungs, reviewers_rung, Rung
from review_passes import ReviewedPass, REVIEWERS
from model_routing.ladder import reviewers_rung, Rung
@dataclasses.dataclass(frozen=True)
class ReviewChoice:
# The rung of each pass's reviewer that no replay lowered.
# The rung of each pass's reviewer.
reviewers: Rung
# The passes whose replay chose another rung, with that rung.
lowered: dict[ReviewedPass, Rung]
seat: Rung
security: Rung
# Each rung a higher E + I than the route's picks for the reviewers, with the lowest E + I that picks it, in rising order.
@@ -20,13 +17,10 @@ class ReviewChoice:
def chosen(project: Rung, route: Route) -> ReviewChoice:
"""The rung each kind of review runs on for a chunk whose route is route, on a project whose rung is project."""
rungs = review_rungs(project, route.rung, route.reviewed_on())
reviewers = reviewers_rung(project, route.rung, route.reviewed_on())
lowered = {kind: rungs[kind] for kind in REVIEWERS if rungs[kind] != reviewers}
higher: dict[Rung, int] = {}
for escape_and_impact in range(route.reviewed_on() + 1, HIGHEST_ESCAPE_AND_IMPACT + 1):
# A higher E + I changes a review by moving the reviewers' rung, or by ending the replay that lowered a pass.
rung = reviewers_rung(project, route.rung, escape_and_impact)
if rung != reviewers or (lowered and not replay_applies(project, route.rung, escape_and_impact)):
if (rung := reviewers_rung(project, route.rung, escape_and_impact)) != reviewers:
higher.setdefault(rung, escape_and_impact)
return ReviewChoice(reviewers, lowered, rungs["seat"], rungs["security"], higher)
# A seat and the security pass run on the project's rung, as ladder.review_rung says.
return ReviewChoice(reviewers, project, project, higher)
+6 -29
View File
@@ -1,12 +1,10 @@
import pathlib
import sys
import unittest
import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from model_routing.ladder import highest, LADDER, lowest_worker_rung, planning_rung, position, project_rung, review_floor, review_rungs, Rung, stepped_up, worker_rung, WORKER_STEPS
from model_routing import ladder
from model_routing.ladder import highest, LADDER, lowest_worker_rung, planning_rung, project_rung, review_floor, review_rung, Rung, stepped_up, worker_rung, WORKER_STEPS
from todo_table import LEVELS
from review_passes import REVIEWERS
@@ -98,31 +96,10 @@ class Ladder(unittest.TestCase):
def test_puts_reviewers_on_the_highest_of_the_projects_rung_the_workers_and_the_one_e_plus_i_picks(self):
sonnet, fable = LADDER[0], LADDER[-1]
self.assertEqual(review_rungs(OPUS, sonnet, 4)["stability"], OPUS)
self.assertEqual(review_rungs(OPUS, sonnet, 5)["stability"], Rung("claude-opus-5-5", "high"))
self.assertEqual(review_rungs(OPUS, fable, 5)["stability"], fable)
self.assertEqual(review_rungs(OPUS, Rung("claude-opus-5-5", "high"), 7)["stability"], Rung("claude-opus-5-5", "max"))
self.assertEqual(review_rung(OPUS, sonnet, 4, "stability"), OPUS)
self.assertEqual(review_rung(OPUS, sonnet, 5, "stability"), Rung("claude-opus-5-5", "high"))
self.assertEqual(review_rung(OPUS, fable, 5, "stability"), fable)
self.assertEqual(review_rung(OPUS, Rung("claude-opus-5-5", "high"), 7, "stability"), Rung("claude-opus-5-5", "max"))
def test_keeps_the_seats_and_the_security_pass_on_the_projects_rung_whatever_the_worker_or_e_plus_i(self):
self.assertEqual(review_rungs(OPUS, LADDER[-1], 8), {**dict.fromkeys(REVIEWERS, LADDER[-1]), "seat": OPUS, "security": OPUS})
def test_runs_a_replayed_pass_on_its_replays_rung_below_the_workers_where_the_e_plus_i_steps_no_review_up_and_the_worker_runs_no_higher_than_the_project(self):
sonnet, opus_high = Rung("claude-sonnet-5-5", "high"), Rung("claude-opus-5-5", "high")
with unittest.mock.patch.dict(ladder.REPLAYED, {("prose", "claude-opus-5-5"): sonnet}):
self.assertEqual(review_rungs(OPUS, sonnet, 4), {**dict.fromkeys(REVIEWERS, OPUS), "prose": sonnet, "seat": OPUS, "security": OPUS})
self.assertEqual(review_rungs(OPUS, OPUS, 4)["prose"], sonnet)
self.assertEqual(review_rungs(OPUS, opus_high, 4)["prose"], opus_high)
self.assertEqual(review_rungs(OPUS, sonnet, 5)["prose"], opus_high)
self.assertEqual(review_rungs(OPUS, LADDER[-1], 5)["prose"], LADDER[-1])
def test_holds_each_replayed_rung_below_its_models_project_rung_on_the_ladder(self):
for (kind, model), rung in ladder.REPLAYED.items():
with self.subTest(kind=kind, model=model):
project = project_rung(model)
self.assertIn(kind, REVIEWERS)
self.assertIsNotNone(project)
self.assertLess(position(rung), position(project) if project else 0)
self.assertIn(rung, LADDER)
if __name__ == "__main__":
unittest.main()
self.assertEqual([review_rung(OPUS, LADDER[-1], 8, kind) for kind in (*REVIEWERS, "seat", "security")], [LADDER[-1]] * len(REVIEWERS) + [OPUS, OPUS])
+1 -14
View File
@@ -1,26 +1,13 @@
import unittest
import unittest.mock
from model_routing.review_choice import chosen, ReviewChoice
from model_routing import ladder
from model_routing.review_choice import chosen
from model_routing.route import Route
from model_routing.ladder import Rung
HAIKU = Rung("claude-haiku-5-5", "low")
OPUS = Rung("claude-opus-5-5", "medium")
OPUS_MAX = Rung("claude-opus-5-5", "max")
FABLE = Rung("claude-fable-5-1", "max")
SONNET = Rung("claude-sonnet-5-5", "high")
class Chosen(unittest.TestCase):
def test_names_no_higher_rung_on_the_top_one(self):
self.assertEqual(chosen(OPUS, Route(FABLE, 3, 4, 5)).higher, {})
def test_names_the_e_plus_i_that_ends_a_replay_even_where_the_reviewers_are_on_the_top_rung(self):
with unittest.mock.patch.dict(ladder.REPLAYED, {("prose", "claude-fable-5-1"): HAIKU}):
self.assertEqual(chosen(FABLE, Route(OPUS_MAX, 3, 4, 4)), ReviewChoice(FABLE, {"prose": HAIKU}, FABLE, FABLE, {FABLE: 5}))
def test_puts_every_pass_on_the_reviewers_rung_once_an_e_plus_i_steps_a_review_up(self):
with unittest.mock.patch.dict(ladder.REPLAYED, {("prose", "claude-opus-5-5"): HAIKU}):
self.assertEqual(chosen(OPUS, Route(SONNET, 3, 2, 5)).lowered, {})
+1 -1
View File
@@ -8,7 +8,7 @@ from model_routing.route import Route
PROJECT_RUNG = "LOOP_CLAUDE_PROJECT_RUNG"
WORKER_RUNG = "LOOP_CLAUDE_WORKER_RUNG"
# The E + I the chunk's reviews are routed on, from which review_rungs works out each review's rung.
# The E + I the chunk's reviews are routed on, from which review_rung works out each review's rung.
ESCAPE_AND_IMPACT = "LOOP_CLAUDE_ESCAPE_AND_IMPACT"
+23 -21
View File
@@ -84,17 +84,19 @@ under the bar, or --check finding the file unranked.
import ast
import dataclasses
import decimal
import graphlib
import math
import pathlib
import re
import sys
import typing
from todo_table import BLOCKED_BY, cells, CODE_HEALTH, DETAILS, HEADER_START, Item, item_details, items_table, MOVES, OLD_SHAPE, parse_row, renames, risk, Shape, SHAPE, shape_of, Table, table_rows, version
from todo_table import BLOCKED_BY, cells, CODE_HEALTH, DETAILS, HEADER_START, Item, item_details, items_table, MOVES, OLD_SHAPE, parse_row, renames, risk, Shape, SHAPE, shape_of, Table, table_rows
from failure import Failure
from comprehension_history import FLOOR, HISTORY, latest_overall
from markdown_text import listed, sections
from file_writes import replace_file
from release import version
# What each achieve goal weighs of the one above it; docs/decisions.md records the choice.
GOAL_STEP = 0.97
@@ -416,31 +418,31 @@ def blockers_first(scored: list[tuple[Item, float]]) -> list[tuple[Item, float]]
if blocker in held and held[blocker][0].release > item.release:
later, earlier = version(held[blocker][0].release), version(item.release)
return Failure(f"item {item.id}, in release {earlier}, is blocked by item {blocker}, in later release {later}; move item {blocker} to release {earlier} or earlier, or item {item.id} to release {later} or later")
result: list[tuple[Item, float]] = []
for item, _ in scored:
placed = place(item.id, held, position, {placed.id for placed, _ in result}, [])
if isinstance(placed, Failure):
return placed
result += placed
return result
def place(id_: int, held: dict[int, tuple[Item, float]], position: dict[int, int], placed: set[int], path: list[int]) -> list[tuple[Item, float]] | Failure:
"""The item id_ preceded by those of its blockers not yet placed, each in score order, and theirs before them; path holds the items waiting on this one, to find a loop."""
if id_ in path:
loop = sorted(path[path.index(id_):])
if loop := blocking_loop(held):
return Failure(f"items {', '.join(map(str, loop[:-1]))} and {loop[-1]} block each other in a loop; drop one of the blockers")
placed: set[int] = set()
return [entry for item, _ in scored for entry in place(item.id, held, position, placed)]
def blocking_loop(held: dict[int, tuple[Item, float]]) -> list[int] | None:
"""The IDs of items that block each other in a loop, sorted; None where there is no loop."""
graph = {id_: {blocker for blocker in item.blocked_by if blocker in held} for id_, (item, _) in held.items()}
try:
graphlib.TopologicalSorter(graph).prepare()
except graphlib.CycleError as cycle:
return sorted(set(cycle.args[1]))
return None
def place(id_: int, held: dict[int, tuple[Item, float]], position: dict[int, int], placed: set[int]) -> list[tuple[Item, float]]:
"""The item id_, unless placed holds it, preceded by those of its blockers placed doesn't hold yet, each in score order, and theirs before them. Each item it returns joins placed. blocking_loop has ruled out a loop."""
if id_ in placed:
return []
placed.add(id_)
item, value = held[id_]
blockers = sorted((blocker for blocker in set(item.blocked_by) if blocker in held), key=position.__getitem__)
added: list[tuple[Item, float]] = []
for blocker in blockers:
before = place(blocker, held, position, placed | {entry.id for entry, _ in added}, [*path, id_])
if isinstance(before, Failure):
return before
added += before
return [*added, (dataclasses.replace(item, blocked_by=tuple(sorted(blockers))), value)]
before = [entry for blocker in blockers for entry in place(blocker, held, position, placed)]
return [*before, (dataclasses.replace(item, blocked_by=tuple(sorted(blockers))), value)]
def check_citations(items: list[Item], decisions: str) -> Failure | None:
+53
View File
@@ -0,0 +1,53 @@
"""A release: a semver version x.y.z the human cuts. Its tag is `v<x.y.z>`, and once cut, CHANGELOG.md heads its section `## x.y.z - YYYY-MM-DD`. Every pattern for a release lives here."""
import re
from failure import Failure
import git_repo
Release = tuple[int, int, int]
VERSION = r"[0-9]+\.[0-9]+\.[0-9]+"
# The digits of a dated heading's year, month and day, which DATED and dated_heading_pattern each render.
DATE_DIGITS = (4, 2, 2)
DATED = re.compile(rf"({VERSION}) - " + "-".join(f"[0-9]{{{digits}}}" for digits in DATE_DIGITS))
TAG = re.compile(rf"v({VERSION})")
def release_of(version: str) -> Release:
"""The release a VERSION-shaped version names."""
major, minor, patch = (int(part) for part in version.split("."))
return major, minor, patch
def version(release: Release) -> str:
"""The version a release names, such as 0.2.1: release_of's inverse."""
return ".".join(map(str, release))
def parsed(text: str) -> Release | None:
"""The release text names as a bare version, such as a CHANGELOG.md heading's `0.2.1`; None for any other text."""
return release_of(text) if re.fullmatch(VERSION, text) else None
def tagged(tag: str | None) -> Release | None:
"""The release a `v<x.y.z>` tag names; None for no tag, or a tag that names none."""
named = TAG.fullmatch(tag) if tag else None
return release_of(named[1]) if named else None
def dated(heading: str) -> Release | None:
"""The release a dated CHANGELOG.md heading, such as `0.2.1 - 2026-10-03`, names; None for any other heading, an undated one included."""
named = DATED.fullmatch(heading)
return release_of(named[1]) if named else None
def dated_heading_pattern(release: Release) -> str:
"""A basic regular expression, as `git log -G` takes, matching release's dated heading line in CHANGELOG.md and no other."""
return "^## " + re.escape(version(release)) + " - " + "-".join("[0-9]" * digits for digits in DATE_DIGITS) + "$"
def newest_tag(cwd: str | None = None, merged_into: str | None = None) -> str | None | Failure:
"""The newest `v*` tag, read in cwd, else in the current directory, and merged into merged_into where it is given; None when there is none."""
# Sorts v1.0.0-rc.1 below v1.0.0.
merged = ["--merged", merged_into] if merged_into else []
tags = git_repo.git("-c", "versionsort.suffix=-", "tag", "--list", "v*", *merged, "--sort=-v:refname", cwd=cwd)
return tags if isinstance(tags, Failure) else next(iter(tags.split()), None)
+31 -36
View File
@@ -1,10 +1,10 @@
"""The next chunk's pick: the item the chunk takes first, weighed against the last release cut, and what the default branch holds for the pick."""
"""How the plan stands to the last release cut: the next release, whether work has landed towards it, and whether it is to be planned, reviewed or cut; and, from that standing, the item the next chunk takes first, with what the default branch holds for the pick."""
import dataclasses
import re
from git_repo import commits_since, file_at, git, newest_release_tag
from git_repo import commits_since, file_at, git
from release import dated, dated_heading_pattern, newest_tag, parsed, Release, tagged
from failure import Failure
from todo_table import Item, items, next_item, release_of, renames, version
from todo_table import Item, items, next_item, renames
from markdown_text import sections
# The version a branch with no v* tag and no dated CHANGELOG.md heading counts as cut: below every release, 0.0.0 included, so that every release todo.md names comes after it.
@@ -30,18 +30,18 @@ class LastCut:
class Upcoming:
"""The next release, and whether work has landed towards it, as `~/.claude/CLAUDE.md` → "Working loop" uses both."""
# The earliest release after the last cut that todo.md has named since that cut or the plan holds; without a cut, the earliest todo.md has named or the plan holds. None where there is none.
release: tuple[int, int, int] | None
release: Release | None
# Whether it is the plan's earliest release. Also True without a cut where todo.md has never named a release and the plan holds no item.
is_earliest: bool
# Whether work has landed towards it: one or more changes are unreleased, as unreleased_changes counts them.
landed: bool | Failure
unreleased: bool | Failure
@dataclasses.dataclass(frozen=True)
class Next:
"""The plan's earliest release is the next release."""
# None where the plan holds no item.
release: tuple[int, int, int] | None
release: Release | None
# The item the next chunk takes first, next_item's pick; None where every item waits or the plan holds none.
item: Item | None
@@ -49,33 +49,33 @@ class Next:
@dataclasses.dataclass(frozen=True)
class Cut:
"""The plan's earliest release is the last cut's version or earlier."""
release: tuple[int, int, int]
release: Release
@dataclasses.dataclass(frozen=True)
class ReadyToCut:
"""The next release holds no item, and work has landed towards it."""
# The next release, None where todo.md has named none since the cut.
release: tuple[int, int, int] | None
release: Release | None
@dataclasses.dataclass(frozen=True)
class ReviewDue:
"""The next release holds no item and work has landed towards it, but no release review has landed since the cut: `~/.claude/skills/larv-review/SKILL.md` → "Release review"."""
# The next release, None where todo.md has named none since the cut.
release: tuple[int, int, int] | None
release: Release | None
@dataclasses.dataclass(frozen=True)
class Unplanned:
"""The next release holds no item, and no work has landed towards it."""
# The next release, None where todo.md has named none since the cut.
release: tuple[int, int, int] | None
release: Release | None
@dataclasses.dataclass(frozen=True)
class Unknown:
"""todo.md's texts since the cut, or the changes since it, cannot be read."""
"""The tags, todo.md's texts since the cut, or the changes since it, cannot be read."""
# How the plan stands to the last cut, with the item the next chunk takes first where there is one.
@@ -92,7 +92,7 @@ class DefaultPlan:
def default_plan(root: str, ref: str) -> DefaultPlan:
"""What ref, the default branch as last fetched, holds for the pick, read in the checkout at root."""
tag = newest_release_tag(ref, root)
tag = newest_tag(root, merged_into=ref)
changelog = file_at(ref, "CHANGELOG.md", root)
after = dated_cut(tag, changelog)
since = tag if after is None else heading_commit(after, ref, root)
@@ -101,15 +101,14 @@ def default_plan(root: str, ref: str) -> DefaultPlan:
return DefaultPlan(file_at(ref, "todo.md", root), LastCut(tag, changelog, commits, todos_since(tag, ref, root), reviewed))
def heading_commit(release: tuple[int, int, int], ref: str, cwd: str) -> str | None | Failure:
def heading_commit(release: Release, ref: str, cwd: str) -> str | None | Failure:
"""The oldest commit on ref that added release's dated heading to CHANGELOG.md, which a branch with no tag counts the unreleased commits from; None where git finds none."""
# -G matches the heading line itself, so a changelog entry that quotes the heading doesn't count.
heading = "^## " + version(release).replace(".", "\\.") + " - [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]$"
added = git("log", "--reverse", "--format=%H", "-G", heading, ref, "--", "CHANGELOG.md", cwd=cwd)
added = git("log", "--reverse", "--format=%H", "-G", dated_heading_pattern(release), ref, "--", "CHANGELOG.md", cwd=cwd)
return added if isinstance(added, Failure) else next(iter(added.split()), None)
def release_reviewed(tag: str | None, ref: str, cwd: str, after: tuple[int, int, int] | None = None) -> bool | Failure:
def release_reviewed(tag: str | None, ref: str, cwd: str, after: Release | None = None) -> bool | Failure:
"""Whether ref holds a commit titled as a release review opens, `Release review of <release>`: after tag, or anywhere where tag is None.
after is the version a branch with no tag counts as cut, or None. Where it is given, a title counts only where its release is later than after.
@@ -118,7 +117,7 @@ def release_reviewed(tag: str | None, ref: str, cwd: str, after: tuple[int, int,
if isinstance(titles, Failure):
return titles
reviews = [title.removeprefix("Release review of ") for title in titles.splitlines() if title.startswith("Release review of ")]
return any(after is None or (re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", review) and release_of(review) > after) for review in reviews)
return any(after is None or ((named := parsed(review)) is not None and named > after) for review in reviews)
def todos_since(tag: str | None | Failure, ref: str, cwd: str) -> list[str] | Failure:
@@ -146,13 +145,15 @@ def chunk_item(found: list[Item], cut: LastCut) -> Pick:
def next_release(found: list[Item], cut: LastCut) -> Upcoming | Cut | Unknown:
"""The next release after cut, and whether work has landed towards it. found holds the plan's items.
Cut where the plan files items under the last cut's version or earlier, and Unknown where todo.md's texts since the cut cannot be read.
Cut where the plan files items under the last cut's version or earlier, and Unknown where the tags, or todo.md's texts since the cut, cannot be read.
Where the branch has no tag, todos_since holds every version of todo.md, so a release todo.md named whose items are all done still reads as the next release.
"""
if isinstance(cut.tag, Failure):
return Unknown()
earliest = min((item.release for item in found), default=None)
version = cut_version(cut)
if version is None:
return Upcoming(earliest, True, landed(cut))
return Upcoming(earliest, True, has_unreleased(cut))
if earliest is not None and earliest <= version:
return Cut(earliest)
named = named_releases(cut.todos_since)
@@ -163,30 +164,24 @@ def next_release(found: list[Item], cut: LastCut) -> Upcoming | Cut | Unknown:
after.add(earliest)
release = min(after, default=None)
nothing_cut = version == BEFORE_FIRST_RELEASE
return Upcoming(release, earliest == release and (earliest is not None or nothing_cut), landed(cut))
return Upcoming(release, earliest == release and (earliest is not None or nothing_cut), has_unreleased(cut))
def cut_version(cut: LastCut) -> tuple[int, int, int] | None:
def cut_version(cut: LastCut) -> Release | None:
"""Which version is cut. It is the tag's, or None where the tag names no version. On a branch with no tag, it is the newest version under a dated CHANGELOG.md heading, else BEFORE_FIRST_RELEASE."""
if cut.tag is None:
return dated_cut(cut.tag, cut.changelog) or BEFORE_FIRST_RELEASE
return version_of(cut.tag)
return tagged(cut.tag) if isinstance(cut.tag, str) else None
def dated_cut(tag: str | None | Failure, changelog: str | Failure) -> tuple[int, int, int] | None:
def dated_cut(tag: str | None | Failure, changelog: str | Failure) -> Release | None:
"""The version a branch with no tag counts as cut: the newest CHANGELOG.md heads `## X.Y.Z - YYYY-MM-DD`, the heading a cut release takes. None where the branch has a tag, or CHANGELOG.md has no such heading; an undated heading is not cut."""
if tag is not None or isinstance(changelog, Failure):
return None
return max((release_of(dated[1]) for heading in sections(changelog) if (dated := re.fullmatch(r"([0-9]+\.[0-9]+\.[0-9]+) - [0-9]{4}-[0-9]{2}-[0-9]{2}", heading))), default=None)
return max((cut for heading in sections(changelog) if (cut := dated(heading))), default=None)
def version_of(tag: str | None | Failure) -> tuple[int, int, int] | None:
"""The version x.y.z named by tag, the newest v* tag; None where it names none."""
tagged = re.fullmatch(r"v([0-9]+\.[0-9]+\.[0-9]+)", tag) if isinstance(tag, str) else None
return None if tagged is None else release_of(tagged[1])
def landed(cut: LastCut) -> bool | Failure:
def has_unreleased(cut: LastCut) -> bool | Failure:
"""Whether work has landed since cut: one or more changes are unreleased."""
changes = unreleased_changes(cut)
return changes if isinstance(changes, Failure) else changes > 0
@@ -199,9 +194,9 @@ def standing(upcoming: Upcoming, first: Item | None, reviewed: bool | Failure) -
"""
if upcoming.is_earliest:
return Next(upcoming.release, first)
if isinstance(upcoming.landed, Failure):
if isinstance(upcoming.unreleased, Failure):
return Unknown()
if not upcoming.landed:
if not upcoming.unreleased:
return Unplanned(upcoming.release)
if isinstance(reviewed, Failure):
return Unknown()
@@ -220,13 +215,13 @@ def unreleased_entries(changelog: str) -> int | None:
return None if found is None else sum(line.startswith("- ") for line in found[0].splitlines())
def named_releases(todos: list[str] | Failure) -> set[tuple[int, int, int]] | Failure:
def named_releases(todos: list[str] | Failure) -> set[Release] | Failure:
"""Each release the todo.md texts, oldest first, file items under. A release leaves the set when a text marks it renamed, and returns when a later text files items under it again. A text whose items or marks do not parse names none, except the newest: where its marks do not parse, it returns that Failure, since the release they should have excluded would read as finished."""
if isinstance(todos, Failure):
return todos
if todos and isinstance(newest := renames(todos[-1]), Failure):
return newest
named: set[tuple[int, int, int]] = set()
named: set[Release] = set()
for text in todos:
plan, marked = items(text), renames(text)
if isinstance(plan, list) and isinstance(marked, set):
+1 -1
View File
@@ -30,5 +30,5 @@ class Types(unittest.TestCase):
self.assertEqual(ran.returncode, 0, ran.stdout + ran.stderr)
def test_mypy_checks_run_heavy_and_each_root_module(self):
self.assertLessEqual({("-p", "run_heavy"), ("-p", "machine_load"), ("-m", "chunk_pick"), ("-m", "docker_command")}, set(zip(tools()[::2], tools()[1::2])))
self.assertLessEqual({("-p", "run_heavy"), ("-p", "machine_load"), ("-m", "release_standing"), ("-m", "docker_command")}, set(zip(tools()[::2], tools()[1::2])))
self.assertNotIn("repo_checks", tools())
+13 -1
View File
@@ -1,4 +1,5 @@
"""The passes of larv-review, as skills/larv-review/SKILL.md → "Passes" lists them."""
"""The passes of larv-review, as skills/larv-review/SKILL.md → "Passes" lists them, and the Markdown that counts as config for them."""
import pathlib
import typing
# A pass an agent under agents/ reviews, in lower case. A Literal cannot be built from `PASS_AGENTS`, so test_review_passes holds the two together.
@@ -14,3 +15,14 @@ def _reviewed(name: str) -> typing.TypeGuard[ReviewedPass]:
# The passes that have an agent. A pass missing from `ReviewedPass` drops out of it, which test_review_passes fails on.
REVIEWERS: dict[ReviewedPass, str] = {name: agent for name, agent in PASS_AGENTS.items() if agent and _reviewed(name)}
# Markdown an agent follows, which counts as config: Claude Code's memory files, the bench's prompts, and the folders that hold Claude Code's skills, agents and rules and this repo's principles. skills/larv-review/SKILL.md → "Passes" and docs/decisions.md state the same lists for the agents that read them.
INSTRUCTION_FILES = ("CLAUDE.md", "AGENTS.md", "workflow-bench/*-prompt.md")
INSTRUCTION_FOLDERS = ("agents", "principles", "rules", "skills")
def prose(path: str) -> bool:
"""Whether path names Markdown that no agent follows: a branch changing only such files reviews on the project's rung, skills/larv-review/SKILL.md → "Markdown only"."""
named = pathlib.PurePosixPath(path)
return path.endswith(".md") and not any(named.match(pattern) for pattern in INSTRUCTION_FILES) and not set(INSTRUCTION_FOLDERS) & set(named.parts[:-1])
+19 -7
View File
@@ -23,9 +23,15 @@ class Findings:
def round_counts(titles: list[str]) -> dict[str, int]:
"""Each pass's round count among the branch's `Answer <pass> round <n>:` commit titles: the highest <n>, or the number of such commits where that is more, so a repeated number still counts. A pass with none has 0, and a title naming no known pass adds its name. Case is ignored."""
highest: dict[str, int] = dict.fromkeys(PASSES, 0)
commits: dict[str, int] = dict.fromkeys(PASSES, 0)
"""Each pass's round count among the branch's `Answer <pass> round <n>:` commit titles, as named_counts counts them. A pass with none has 0. A title naming no pass counts for none; rule-hook's `round-cap` refuses its commit. Case is ignored."""
counted = named_counts(titles)
return {name: counted.get(name, 0) for name in PASSES}
def named_counts(titles: list[str]) -> dict[str, int]:
"""The round count of each name the round titles give, in lower case, whether or not larv-review has a pass of that name: the highest <n>, or the number of such titles where that is more, so a repeated number still counts."""
highest: dict[str, int] = {}
commits: dict[str, int] = {}
for title in titles:
if named := ROUND.match(title):
name = named.group(1).lower()
@@ -34,9 +40,15 @@ def round_counts(titles: list[str]) -> dict[str, int]:
return {name: max(highest[name], commits[name]) for name in highest}
def round_pass(title: str) -> str | None:
"""The pass a round title names, in lower case, whether or not larv-review has it; None for any other title."""
named = ROUND.match(title)
return named.group(1).lower() if named else None
def rounds_past_first(titles: list[str]) -> int:
"""Each pass's rounds past its first among the commit titles, as round_counts counts them, summed."""
return sum(max(0, count - 1) for count in round_counts(titles).values())
"""The rounds past the first of each name the commit titles give, summed. It counts names that are no pass too, so an older title naming two passes, such as `Answer stability and prose round 2:`, still counts."""
return sum(max(0, count - 1) for count in named_counts(titles).values())
def lacks_counts(title: str) -> bool:
@@ -46,8 +58,8 @@ def lacks_counts(title: str) -> bool:
def past_cap(title: str, earlier: list[str]) -> bool:
"""Whether title is a round title that takes its pass past its cap, as round_counts counts title among the branch's earlier commit titles."""
named = ROUND.match(title)
return named is not None and round_counts([*earlier, title])[named.group(1).lower()] > cap(named.group(1).lower())
name = round_pass(title)
return name in PASSES and round_counts([*earlier, title])[name] > cap(name)
def cap(name: str) -> int:
+9 -14
View File
@@ -119,8 +119,8 @@ def measured(chunk: Chunk, home: str, history: list[BranchChunk] | None = None)
delegated[rung] += 1
reports[kind] = reports.get(kind, Reports(0, 0, 0)) + Reports(1, len(reported.get(subagent.launched_by, "")), sum(map(len, results(subagent.entries).values())))
spent += [Spent(kind, rung, message) for message in messages(subagent.entries)]
dollars, unattributed = costed(spent, chunk.project, recorded_costs(chunk.worker))
add_judge(dollars, chunk)
judged = [judged for kept in chunk.outcomes for judged in judge_costs(kept)]
dollars, unattributed = costed(spent, judged, chunk.project, recorded_costs(chunk.worker))
start = route_from(cycles[0][1]) if cycles else None
return ChunkFigures(
dollars=dollars,
@@ -136,16 +136,6 @@ def measured(chunk: Chunk, home: str, history: list[BranchChunk] | None = None)
)
def add_judge(dollars: Dollars, chunk: Chunk) -> None:
"""Adds to dollars the cost each of the chunk's outcomes records for a judge run, to the judge's rung."""
for kept in chunk.outcomes:
for rung, cost in judge_costs(kept):
dollars["project"][chunk.project] += cost
dollars["subagent type"][JUDGE] += cost
dollars["rung"][rung] += cost
dollars["tool"][PROMPT] += cost
def transcripts_lacking(launches: dict[str, tuple[str, dict]], subagents: list[Subagent]) -> int:
"""How many Agent calls, the worker's and its subagents', have no subagent transcript in the archive. Each transcript answers one call, so they are the calls less the transcripts."""
return max(0, len(launches) + sum(len(agent_calls(subagent.entries)) for subagent in subagents) - len(subagents))
@@ -185,8 +175,8 @@ def recorded_costs(worker: list[dict]) -> dict[str, float]:
return {}
def costed(spent: list[Spent], project: str, recorded: dict[str, float]) -> tuple[Dollars, float]:
"""Each message's dollars by DIMENSIONS, and the recorded cost left over, that of a recorded model with no priced message. A model's recorded cost is split across its messages in proportion to their list prices, so the calls no message records, such as compactions, are counted. A subagent transcript the archive lacks puts its cost on the archived messages of the model that ran it, in the same proportion. A model the record omits keeps its list prices."""
def costed(spent: list[Spent], judged: list[tuple[str, float]], project: str, recorded: dict[str, float]) -> tuple[Dollars, float]:
"""Each message's dollars by DIMENSIONS, with each judge run's cost in judged, by the judge's rung, under the subagent type JUDGE and the tool PROMPT; and the recorded cost left over, that of a recorded model with no priced message. A model's recorded cost is split across its messages in proportion to their list prices, so the calls no message records, such as compactions, are counted. A subagent transcript the archive lacks puts its cost on the archived messages of the model that ran it, in the same proportion. A model the record omits keeps its list prices."""
listed_by_model: collections.defaultdict[str, float] = collections.defaultdict(float)
for _, _, message in spent:
listed_by_model[message.model] += message.list_price
@@ -199,6 +189,11 @@ def costed(spent: list[Spent], project: str, recorded: dict[str, float]) -> tupl
dollars["rung"][rung] += cost
for tool in message.tools:
dollars["tool"][tool] += cost / len(message.tools)
for rung, cost in judged:
dollars["project"][project] += cost
dollars["subagent type"][JUDGE] += cost
dollars["rung"][rung] += cost
dollars["tool"][PROMPT] += cost
return dollars, sum(cost for model, cost in recorded.items() if not listed_by_model[model])
+1 -1
View File
@@ -4,7 +4,7 @@ from failure import Failure
import git_repo
def git(directory: str, *args: str) -> str | Failure:
def guarded_git(directory: str, *args: str) -> str | Failure:
"""What git printed in directory, stripped, run guarded: the hook runs git in a checkout the call has not yet been allowed to touch."""
ran = git_repo.git(*args, cwd=directory, guarded=True)
return ran if isinstance(ran, Failure) else ran.strip()
+9 -9
View File
@@ -7,7 +7,7 @@ import typing
from rule_hook.rules import Breach
from git_repo import default_ref
from rule_hook.git import existing, git
from rule_hook.git import existing, guarded_git
from failure import Failure
from utc_stamp import utc
@@ -58,12 +58,12 @@ def log_path(directory: str) -> Log | Failure | None:
directory = existing(directory)
if not any((parent / ".git").exists() for parent in [pathlib.Path(directory), *pathlib.Path(directory).parents]):
return None
common = git(directory, "rev-parse", "--path-format=absolute", "--git-common-dir")
branch = git(directory, "branch", "--show-current")
common = guarded_git(directory, "rev-parse", "--path-format=absolute", "--git-common-dir")
branch = guarded_git(directory, "branch", "--show-current")
for found in (common, branch):
if isinstance(found, Failure):
return found
head = git(directory, "rev-parse", "--verify", "--quiet", "HEAD")
head = guarded_git(directory, "rev-parse", "--verify", "--quiet", "HEAD")
return Log(pathlib.Path(str(common)) / LOG, str(branch) or "HEAD", "" if isinstance(head, Failure) else head)
@@ -89,14 +89,14 @@ def logged(log: Log, breach: Breach, reason: str, what: str, now: float) -> LogL
def print_overrides(directory: str) -> int:
common = git(directory, "rev-parse", "--path-format=absolute", "--git-common-dir")
common = guarded_git(directory, "rev-parse", "--path-format=absolute", "--git-common-dir")
if isinstance(common, Failure):
print(f"rule-hook overrides runs in a git checkout: {common.reason}", file=sys.stderr)
return 1
found = git(directory, "branch", "--show-current")
found = guarded_git(directory, "branch", "--show-current")
branch = found if isinstance(found, str) and found else "HEAD"
reflog = git(directory, "reflog", "show", "--format=%H", branch)
head = git(directory, "rev-parse", "HEAD")
reflog = guarded_git(directory, "reflog", "show", "--format=%H", branch)
head = guarded_git(directory, "rev-parse", "HEAD")
unread = unread_reflog(reflog, head)
if unread:
print(f"rule-hook overrides cannot read the branch's reflog, so it cannot tell which overrides are this chunk's: {unread.reason}", file=sys.stderr)
@@ -121,4 +121,4 @@ def unread_reflog(reflog: str | Failure, head: str | Failure) -> Failure | None:
def holds(directory: str, descendant: str, ancestor: str) -> bool:
return bool(ancestor) and not isinstance(git(directory, "merge-base", "--is-ancestor", ancestor, descendant), Failure)
return bool(ancestor) and not isinstance(guarded_git(directory, "merge-base", "--is-ancestor", ancestor, descendant), Failure)
+2 -2
View File
@@ -7,12 +7,12 @@ from rule_hook.call import Call
from review_folder import REPORTS
# Each agent under agents/, read from the folder, so an agent the routing tables miss is still confined.
REVIEWERS = frozenset(path.stem for path in (pathlib.Path(__file__).resolve().parents[2] / "agents").glob("*.md"))
AGENT_FILES = frozenset(path.stem for path in (pathlib.Path(__file__).resolve().parents[2] / "agents").glob("*.md"))
def report_breaches(call: Call, path: str, home: str) -> list[Breach]:
"""The breach of a file call from a reviewer that is not a Write under REPORTS in home; none from any other agent."""
if call.agent not in REVIEWERS:
if call.agent not in AGENT_FILES:
return []
written = os.path.realpath(os.path.join(call.cwd, path))
reports = os.path.realpath(os.path.join(home, REPORTS))
+7 -17
View File
@@ -1,17 +1,16 @@
"""Refuse an Agent or Skill call that launches a review below the rung it needs, worked out from the rungs and the E + I loop-claude hands its worker: docs/decisions.md → `routing:`."""
import pathlib
import typing
from rule_hook.rules import Breach
from rule_hook.call import Call, text
from claude_code.entries import content
from failure import Failure
from rule_hook.git import git
import git_repo
from rule_hook.git import guarded_git
from model_routing.worker_rungs import handed
from model_routing.ladder import position, review_rungs, ReviewKind, Rung
from model_routing.ladder import position, review_rung, ReviewKind, Rung
from review_passes import prose, REVIEWERS
from claude_code.transcript import read_transcript
from review_passes import REVIEWERS
# The pass each agent under agents/ reviews.
PASS_OF = {agent: kind for kind, agent in REVIEWERS.items()}
@@ -22,9 +21,6 @@ SECURITY_OPENER = "You are the security pass for this chunk."
SUBJECTS: dict[ReviewKind, str] = {"seat": "a comprehension seat", "security": "the security pass's subagent"}
# The skills that run the security pass, by their names with any plugin prefix cut.
SECURITY_SKILLS = {"claude-security", "security-review"}
# Markdown an agent follows, which counts as config: Claude Code's memory files, and the folders that hold Claude Code's skills, agents and rules and this repo's principles.
INSTRUCTION_FILES = {"CLAUDE.md", "AGENTS.md"}
INSTRUCTION_FOLDERS = {"agents", "principles", "rules", "skills"}
def reviewer_breaches(call: Call, environ: typing.Mapping[str, str]) -> list[Breach]:
@@ -33,7 +29,7 @@ def reviewer_breaches(call: Call, environ: typing.Mapping[str, str]) -> list[Bre
kind = review_kind(call.given)
if kind is None or project is None or worker is None:
return []
required = review_rungs(project, worker, escape_and_impact)[kind]
required = review_rung(project, worker, escape_and_impact, kind)
# A branch that changes only Markdown no agent follows holds no code or config for a reviewer on a higher rung to catch a defect in: skills/larv-review/SKILL.md → "Markdown only".
if kind in REVIEWERS and position(required) > position(project) and markdown_only(call.cwd):
required = project
@@ -89,16 +85,10 @@ def named_rung(given: dict[str, typing.Any], worker: Rung) -> Rung:
def markdown_only(directory: str) -> bool:
"""Whether every file the branch in directory changes, against the default branch as last fetched, is Markdown that no agent follows, counting uncommitted and untracked files; False where it changes none or git cannot say."""
default = git_repo.default_ref(directory, guarded=True)
base = default if isinstance(default, Failure) else git(directory, "merge-base", default, "HEAD")
changed = base if isinstance(base, Failure) else git(directory, "diff", "--name-only", base)
untracked = git(directory, "ls-files", "--others", "--exclude-standard")
base = default if isinstance(default, Failure) else guarded_git(directory, "merge-base", default, "HEAD")
changed = base if isinstance(base, Failure) else guarded_git(directory, "diff", "--name-only", base)
untracked = guarded_git(directory, "ls-files", "--others", "--exclude-standard")
if isinstance(changed, Failure) or isinstance(untracked, Failure):
return False
paths = [*changed.splitlines(), *untracked.splitlines()]
return bool(paths) and all(prose(path) for path in paths)
def prose(path: str) -> bool:
"""Whether path names Markdown that no agent follows."""
parts = pathlib.PurePosixPath(path).parts
return path.endswith(".md") and parts[-1] not in INSTRUCTION_FILES and not INSTRUCTION_FOLDERS & set(parts[:-1])
+11 -9
View File
@@ -1,22 +1,24 @@
from rule_hook.rules import Breach
from round_titles import cap, past_cap, ROUND, round_counts
from git_command import Commit
from round_titles import cap, past_cap, round_counts, round_pass
from git_command import CommitCall
from failure import Failure
import git_repo
from review_passes import PASSES
def round_cap_breaches(commit: Commit) -> list[Breach]:
"""A breach where the commit's title takes its pass past the cap, counted with the branch's earlier round commits."""
if not any(ROUND.match(message) for message in commit.messages):
return []
def round_cap_breaches(commit: CommitCall) -> list[Breach]:
"""A breach where the commit's title names no pass, or takes its pass past the cap, counted with the branch's earlier round commits."""
named = [(name, message) for message in commit.messages if (name := round_pass(message))]
unknown = [Breach("round-cap", f"`{name}` names no larv-review pass, so its rounds would never reach a cap; name one of: {', '.join(PASSES)}", commit.directory) for name, _ in named if name not in PASSES]
if unknown or not named:
return unknown
earlier = earlier_titles(commit.directory, commit.amends)
refused = [(named.group(1).lower(), message) for message in commit.messages if (named := ROUND.match(message)) and past_cap(message, earlier)]
return [Breach("round-cap", refusal(name, message, earlier), commit.directory) for name, message in refused]
return [Breach("round-cap", refusal(name, message, earlier), commit.directory) for name, message in named if past_cap(message, earlier)]
def refusal(name: str, title: str, earlier: list[str]) -> str:
"""Why title, a round title of the pass name, is refused, listing the pass's round commits counted, and how to commit the fixes in hand."""
counted = [earlier_title for earlier_title in earlier if (named := ROUND.match(earlier_title)) and named.group(1).lower() == name]
counted = [earlier_title for earlier_title in earlier if round_pass(earlier_title) == name]
listed = ", ".join(f"`{counted_title}`" for counted_title in counted) or "none found"
return f"this would be {name}'s round {round_counts([*counted, title])[name]}, past its {cap(name)}; {name}'s round commits on this branch: {listed}. Fold the fixes into its last round commit with `git commit --amend` where that is the newest commit, or commit them under a title that doesn't open `Answer <pass> round`; they merge once the gate passes"
+1 -1
View File
@@ -66,7 +66,7 @@ RULES: dict[RuleName, Rule] = {
),
# The cap holds with no ask, so no marker waives it.
"round-cap": Rule(
f"a larv-review pass takes at most {CAP} rounds per chunk, and the release review's architecture and principles passes {RELEASE_REVIEW_CAP}, and none after them: the fixes answering its last round merge once the gate passes",
f"a larv-review pass takes at most {CAP} rounds per chunk, and the release review's architecture and principles passes {RELEASE_REVIEW_CAP}, and none after them: the fixes answering its last round merge once the gate passes. A round commit names one of larv-review's passes, so a misspelled one cannot slip past its cap",
"a git commit whose message opens `Answer <pass> round <n>:`, counted with the pass's round commits on the branch since the default branch as last fetched, less the one an --amend replaces",
overridable=False,
),
+17 -23
View File
@@ -8,7 +8,6 @@ import subprocess
import sys
import tempfile
import unittest
import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
@@ -16,14 +15,12 @@ from rule_hook.repeat_read import Asked, modified, read_breaches, result_lines
from rule_hook.rules import blocks, Breach, check, command_breaches, file_breaches, overrides
from rule_hook.main import breaches_of
from rule_hook.call import Call, parsed_call
from git_command import Commit, commit_message, commits
from git_command import commit_message, CommitCall, commits
from test_rank_todo import DECISIONS, README, TODO
from rule_hook.git import guarded_git
from test_env import isolated_env
from model_routing import ladder
from rule_hook.override_log import LogLine, parsed_entry, this_chunks
from rule_hook.reviewer_rung import reviewer_breaches, skill_breaches
from rule_hook.git import git as rule_hook_git
from model_routing.ladder import Rung
from shell_command import segments, simple_commands
HOOK = pathlib.Path(__file__).resolve().parents[1] / "rule-hook"
@@ -35,7 +32,7 @@ def rules(breaches: list[Breach]) -> list[str]:
return [breach.rule for breach in breaches]
def committed(command: str, cwd: str = "/r") -> list[Commit]:
def committed(command: str, cwd: str = "/r") -> list[CommitCall]:
return commits(simple_commands(command, cwd, "/h"), "/h")
@@ -53,7 +50,7 @@ def git(cwd: str, *args: str) -> str:
class Commits(unittest.TestCase):
def test_cd_and_dash_capital_c_move_the_commit(self):
self.assertEqual(committed("cd a && git -C ~/b commit -am 'x'"), [Commit("/h/b", ["x"], False, False)])
self.assertEqual(committed("cd a && git -C ~/b commit -am 'x'"), [CommitCall("/h/b", ["x"], False, False)])
self.assertEqual(committed("cd a && git -c x=y -C b commit -m x")[0].directory, "/r/a/b")
def test_each_message_spelling_is_read(self):
@@ -71,7 +68,7 @@ class Commits(unittest.TestCase):
self.assertEqual(committed("git log --grep commit; echo git commit"), [])
def test_a_line_break_or_a_comment_separates_commands(self):
self.assertEqual(committed("cd a # go there\nGIT_EDITOR=true git commit -m x"), [Commit("/r/a", ["x"], False, False)])
self.assertEqual(committed("cd a # go there\nGIT_EDITOR=true git commit -m x"), [CommitCall("/r/a", ["x"], False, False)])
def test_a_commit_after_a_heredoc_or_a_blank_line_is_seen(self):
self.assertEqual(len(committed("cat <<EOF > notes\nhello\nEOF\ngit commit -m a")), 1)
@@ -87,15 +84,15 @@ class Commits(unittest.TestCase):
self.assertEqual(segments("echo hi > f arg"), [["echo", "hi", "arg"]])
for redirect in [">", ">>", "<", "<<<", "<>", ">&", "<&", ">|", "&>", "&>>"]:
with self.subTest(redirect):
self.assertEqual(committed(f"git {redirect} /tmp/x commit -m hi"), [Commit("/r", ["hi"], False, False)])
self.assertEqual(committed("git commit >&2 -m hi 2>&1"), [Commit("/r", ["hi"], False, False)])
self.assertEqual(committed("> out git commit -m hi"), [Commit("/r", ["hi"], False, False)])
self.assertEqual(committed(f"git {redirect} /tmp/x commit -m hi"), [CommitCall("/r", ["hi"], False, False)])
self.assertEqual(committed("git commit >&2 -m hi 2>&1"), [CommitCall("/r", ["hi"], False, False)])
self.assertEqual(committed("> out git commit -m hi"), [CommitCall("/r", ["hi"], False, False)])
def test_a_separator_glued_to_a_redirect_still_ends_the_command(self):
self.assertEqual(segments("echo a > f; echo b | cat &> g\nls"), [["echo", "a"], ["echo", "b"], ["cat"], ["ls"]])
for command in ["echo a &</dev/null git commit -m hi", "echo a;>f git commit -m hi"]:
with self.subTest(command):
self.assertEqual(committed(command), [Commit("/r", ["hi"], False, False)])
self.assertEqual(committed(command), [CommitCall("/r", ["hi"], False, False)])
class CommandBreaches(unittest.TestCase):
@@ -237,7 +234,7 @@ class ReviewerRung(unittest.TestCase):
self.assertEqual(rules(reviewer_breaches(Call("Agent", on_project, repo), high)), [])
self.assertEqual(rules(reviewer_breaches(Call("Agent", {**on_project, "model": "sonnet", "effort": "high"}, repo), high)), ["reviewer-rung"])
self.assertEqual(rules(reviewer_breaches(Call("Agent", on_project, repo), self.ABOVE)), [], "a worker that climbed doesn't lift a Markdown-only review either")
for instructions in ("CLAUDE.md", "AGENTS.md", "skills/review/SKILL.md", "agents/reviewer.md", "rules/todo.md", "principles/tests.md"):
for instructions in ("CLAUDE.md", "AGENTS.md", "skills/review/SKILL.md", "agents/reviewer.md", "rules/todo.md", "principles/tests.md", "workflow-bench/run-prompt.md"):
with self.subTest(instructions=instructions):
path = pathlib.Path(repo, instructions)
path.parent.mkdir(parents=True, exist_ok=True)
@@ -290,15 +287,6 @@ class ReviewerRung(unittest.TestCase):
self.assertEqual(self.refused({"subagent_type": "stability-reviewer", "model": "sonnet", "effort": "high"}, below), ["reviewer-rung"])
self.assertEqual(self.refused({"prompt": "You are one seat of a comprehension panel.", "model": "sonnet", "effort": "high"}, without), ["reviewer-rung"])
def test_a_replayed_passs_reviewer_is_held_to_its_replays_rung_below_the_workers_unless_the_e_plus_i_steps_reviews_up_or_the_worker_climbed(self):
lowered = {"subagent_type": "prose-reviewer", "model": "sonnet", "effort": "high"}
with unittest.mock.patch.dict(ladder.REPLAYED, {("prose", "claude-opus-5-5"): Rung("claude-sonnet-5-5", "high")}):
self.assertEqual(self.refused(lowered), [])
self.assertEqual(self.refused({"subagent_type": "prose-reviewer", "model": "sonnet", "effort": "medium"}), ["reviewer-rung"])
self.assertEqual(self.refused({**lowered, "subagent_type": "stability-reviewer"}), ["reviewer-rung"])
self.assertEqual(self.refused(lowered, self.ENVIRON | {"LOOP_CLAUDE_ESCAPE_AND_IMPACT": "5"}), ["reviewer-rung"])
self.assertEqual(self.refused(lowered, self.ABOVE), ["reviewer-rung"])
def test_a_seat_or_a_security_launch_is_held_to_the_project_rung_alone(self):
seat = {"prompt": "You are one seat of a comprehension panel. Tools: Read, Grep, Glob.", "model": "opus", "effort": "medium"}
self.assertEqual(self.refused(seat, self.ABOVE), [])
@@ -650,7 +638,7 @@ class Hook(unittest.TestCase):
def test_runs_git_guarded(self):
git(self.repo, "config", "core.fsmonitor", "/bin/false")
self.assertEqual(rule_hook_git(self.repo, "config", "core.fsmonitor"), "false")
self.assertEqual(guarded_git(self.repo, "config", "core.fsmonitor"), "false")
def listed(self) -> str:
return subprocess.run([str(HOOK), "overrides"], capture_output=True, check=True, cwd=self.repo, env=isolated_env(), text=True).stdout
@@ -759,6 +747,12 @@ class Hook(unittest.TestCase):
self.assertNotIn("rule-override", ran.stderr)
self.assertEqual(self.round_commit("Answer stability round 2: 0 🔴, 1 🟡 (0 misread); fix").returncode, 0)
def test_a_round_commit_naming_no_pass_is_refused_and_offers_no_override(self):
ran = self.round_commit("Answer architectur round 1: 0 🔴, 1 🟡 (0 misread); fix # rule-override round-cap: a typo")
self.assertEqual(ran.returncode, 2)
self.assertIn("Blocked by rule round-cap: `architectur` names no larv-review pass, so its rounds would never reach a cap; name one of: architecture, principles, full scan,", ran.stderr)
self.assertNotIn("rule-override", ran.stderr)
def test_a_third_round_commit_of_a_pass_on_the_branch_is_refused_whatever_its_number(self):
for title in ["Answer stability round 1: 1 🔴, 0 🟡 (0 misread); a", "Answer prose round 1: 0 🔴, 1 🟡 (0 misread); b", "Answer stability round 2: 0 🔴, 1 🟡 (0 misread); c"]:
git(self.repo, "commit", "-q", "--allow-empty", "-m", title)
+4 -4
View File
@@ -3,12 +3,12 @@ import pathlib
from rule_hook.rules import Breach
from failure import Failure
from rule_hook.git import git
from rule_hook.git import guarded_git
import rank_todo
def todo_breaches(directory: str) -> list[Breach]:
top = git(directory, "rev-parse", "--show-toplevel")
top = guarded_git(directory, "rev-parse", "--show-toplevel")
if isinstance(top, Failure) or not (pathlib.Path(top) / "todo.md").is_file():
return []
todo = pathlib.Path(top) / "todo.md"
@@ -18,8 +18,8 @@ def todo_breaches(directory: str) -> list[Breach]:
def changed(top: str, todo: pathlib.Path) -> bool:
"""Whether todo.md differs from the index or the index from HEAD: any change, staged or not, since the hook runs before a `git add` earlier in the same command."""
# Hashed here, never by `git status`, which would run a clean filter the checkout's own config names before the call is allowed.
staged, committed = git(top, "rev-parse", ":todo.md"), git(top, "rev-parse", "HEAD:todo.md")
algorithm = git(top, "rev-parse", "--show-object-format")
staged, committed = guarded_git(top, "rev-parse", ":todo.md"), guarded_git(top, "rev-parse", "HEAD:todo.md")
algorithm = guarded_git(top, "rev-parse", "--show-object-format")
content = todo.read_bytes()
worktree = hashlib.new("sha1" if isinstance(algorithm, Failure) else algorithm, b"blob %d\0" % len(content) + content).hexdigest()
return isinstance(staged, Failure) or staged != worktree or staged != committed
+1 -3
View File
@@ -34,14 +34,12 @@ import shutil
import sys
import typing
from run_heavy.capped_docker import CANNOT_RUN, NOT_FOUND, PATH_ENTRY, SHARE_VARIABLE, without_path_entry
from failure import Failure
from machine_load.memory import Memory
from run_heavy.capped_docker import PATH_ENTRY, SHARE_VARIABLE, without_path_entry
from machine_load.slots import slot_count, slot_folder, waited_slot
OWN_FAILURE = 125
CANNOT_RUN = 126
NOT_FOUND = 127
HELD = "RUN_HEAVY_SLOT"
USAGE = "usage: run-heavy <command> [<arg>...]"
+5 -5
View File
@@ -6,8 +6,8 @@ import typing
from failure import Failure
from model_routing.ladder import LADDER, position, Rung
from claude_code.reply import Limited
from loop_claude.paths import paths_from
from loop_claude.project.model_choice import read_models_file
from loop_claude.stop import Stop
# What a tool reads from its arguments, such as the project's rung of the model they name.
Parsed = typing.TypeVar("Parsed")
@@ -25,10 +25,10 @@ def arguments_or_exit(tool: str, argv: list[str], help_text: str, usage: str, re
def projects_on(model: str, home: pathlib.Path) -> list[str] | Failure:
"""The keys of the projects ~/.claude/loop-claude-models.json runs on model, sorted."""
models = read_models_file(home / ".claude" / "loop-claude-models.json")
if isinstance(models, Stop):
return Failure(models.reason)
"""The keys of the projects loop-claude's models file, under home, runs on model, sorted."""
models = read_models_file(paths_from(home).models_file)
if isinstance(models, Failure):
return models
return sorted(key for key, on in models.items() if on == model)
+17 -17
View File
@@ -14,30 +14,30 @@ AT_ONCE = 4
Case = typing.TypeVar("Case")
# What of an archived case a reply reproduces, such as a verdict's kind or a finding's index.
Item = typing.TypeVar("Item")
Reply = typing.TypeVar("Reply")
Aim = typing.TypeVar("Aim")
Answer = typing.TypeVar("Answer")
T = typing.TypeVar("T")
@dataclasses.dataclass(frozen=True)
class Miss(typing.Generic[Case, Item, Reply]):
class Miss(typing.Generic[Case, Aim, Answer]):
"""An item of case that the reference reproduced and a rung's reply missed."""
case: Case
item: Item
reply: Reply
item: Aim
reply: Answer
@dataclasses.dataclass(frozen=True)
class Tried(typing.Generic[Case, Item, Reply]):
class Tried(typing.Generic[Case, Aim, Answer]):
rung: Rung
missed: list[Miss[Case, Item, Reply]]
missed: list[Miss[Case, Aim, Answer]]
@dataclasses.dataclass(frozen=True)
class Replay(typing.Generic[Case, Item, Reply]):
class Replay(typing.Generic[Case, Aim, Answer]):
"""on_reference holds the reference's reply on each case; tried, each candidate tried, highest first; lowest, the lowest rung that, like each candidate above it, missed nothing, or None."""
on_reference: list[Reply]
tried: list[Tried[Case, Item, Reply]]
on_reference: list[Answer]
tried: list[Tried[Case, Aim, Answer]]
lowest: Rung | None
@@ -45,10 +45,10 @@ def replayed(
cases: list[Case],
reference: Rung,
candidates: list[Rung],
replay_one: typing.Callable[[Case, Rung], Reply | Limited],
reproduced: typing.Callable[[Case, Reply], list[Item]],
replay_one: typing.Callable[[Case, Rung], Answer | Limited],
reproduced: typing.Callable[[Case, Answer], list[Aim]],
what: str,
) -> Replay[Case, Item, Reply] | Limited:
) -> Replay[Case, Aim, Answer] | Limited:
"""Replays the cases on the reference rung, then each candidate, highest first, on the cases where the reference reproduced anything, until one misses an item it reproduced.
Going down, every rung at or above the one named was tried and missed nothing, so the result needs no assumption that a higher rung does at least as well; and a miss high up spares the cheaper rungs their replays.
@@ -57,14 +57,14 @@ def replayed(
if isinstance(on_reference, Limited):
return on_reference
held = [(case, aim) for case, reply in zip(cases, on_reference) if (aim := reproduced(case, reply))]
tried: list[Tried[Case, Item, Reply]] = []
tried: list[Tried[Case, Aim, Answer]] = []
lowest = None
# With nothing to hold a rung to, every rung would miss nothing.
for rung in candidates if held else []:
replies = replayed_on([case for case, _ in held], rung, replay_one, what)
if isinstance(replies, Limited):
return replies
misses: list[Miss[Case, Item, Reply]] = []
misses: list[Miss[Case, Aim, Answer]] = []
for (case, aim), reply in zip(held, replies):
got = reproduced(case, reply)
misses += [Miss(case, item, reply) for item in aim if item not in got]
@@ -75,12 +75,12 @@ def replayed(
return Replay(on_reference, tried, lowest)
def replayed_on(cases: list[Case], rung: Rung, replay_one: typing.Callable[[Case, Rung], Reply | Limited], what: str) -> list[Reply] | Limited:
def replayed_on(cases: list[Case], rung: Rung, replay_one: typing.Callable[[Case, Rung], Answer | Limited], what: str) -> list[Answer] | Limited:
"""Each case's reply on rung, AT_ONCE at a time; a usage limit cancels the rest."""
print(f"replaying {len(cases)} {what} on {rung}", file=sys.stderr, flush=True)
with concurrent.futures.ThreadPoolExecutor(AT_ONCE) as pool:
running = [pool.submit(replay_one, case, rung) for case in cases]
replies: list[Reply] = []
replies: list[Answer] = []
for future in running:
reply = future.result()
if isinstance(reply, Limited):
+1 -1
View File
@@ -131,7 +131,7 @@ def report_text(results: list[Result], anchored: Anchors | Failure, given: dict[
sections.append(["Median outcome at each guessed score, among chunks that shipped one item, never-replayed ones left out:", "", *report(results)])
sections.append(["🔴 and 🟡 findings, with how many a reviewer marked `[misread]`, per finished chunk whose round titles count them, newest first:", "", *misreads(results)])
sections.append(["Later fixes: the chunks that fixed code each finished chunk shipped, newest first:", "", *fixed_later(results)])
if any(result.landed != "merge" for result in results):
if any(result.landing != "merge" for result in results):
sections.append(UNMERGED)
return "\n\n".join("\n".join(section) for section in sections)
+6 -6
View File
@@ -14,7 +14,7 @@ ENOUGH = 10
# The tool's name and report call a Result an outcome, a word the code keeps for an entry of a `.outcomes.md` file: docs/decisions.md → `bin/: an outcome is an entry`.
@dataclasses.dataclass(frozen=True)
class Result:
"""What the chunk that shipped an item measured. findings holds the 🔴 and 🟡 counts, and how many of them a reviewer marked `[misread]`, None where a round title lacks the 🔴 and 🟡 counts or the branch has no round; its misread is None where a round title lacks that count. landed says how the chunk reached the default branch: by a merge commit, by a rebase that replayed its commits, or by commits never replayed. fixed_by names the later chunks that fixed code it shipped, newest first, as branch_chunks counts them."""
"""What the chunk that shipped an item measured. findings holds the 🔴 and 🟡 counts, and how many of them a reviewer marked `[misread]`, None where a round title lacks the 🔴 and 🟡 counts or the branch has no round; its misread is None where a round title lacks that count. landing says how the chunk reached the default branch: by a merge commit, by a rebase that replayed its commits, or by commits never replayed. fixed_by names the later chunks that fixed code it shipped, newest first, as branch_chunks counts them."""
item: Item
merge: str
merged: str
@@ -23,7 +23,7 @@ class Result:
files: int
rounds_past_first: int
findings: Findings | None
landed: Landing
landing: Landing
fixed_by: tuple[str, ...]
@@ -47,7 +47,7 @@ def results_of(chunk: BranchChunk, numstat: str) -> list[Result]:
lines = sum(int(count) for row in work for count in row[:2] if count.isdigit())
titles = list(chunk.titles)
ids = [item.id for item in chunk.shipped]
return [Result(item, chunk.label, chunk.merged, tuple(id_ for id_ in ids if id_ != item.id), lines, len(work), rounds_past_first(titles), round_findings(titles), chunk.landed, chunk.fixed_by) for item in chunk.shipped]
return [Result(item, chunk.label, chunk.merged, tuple(id_ for id_ in ids if id_ != item.id), lines, len(work), rounds_past_first(titles), round_findings(titles), chunk.landing, chunk.fixed_by) for item in chunk.shipped]
def nearest(results: list[Result], given: dict[str, int], count: int = 5) -> list[Result]:
@@ -61,7 +61,7 @@ def nearest(results: list[Result], given: dict[str, int], count: int = 5) -> lis
def report(results: list[Result]) -> list[str]:
"""Among chunks that shipped one item, since a chunk measures all its items at once, and that were merged or replayed, since a never-replayed one may hold only part of a pull request: for each score in MEDIANS, the median result at each value it was guessed at; for each level in ORDERED, how well it orders chunks by review rounds and by 🔴 findings."""
single = [result for result in results if not result.shipped_with and result.landed != "push"]
single = [result for result in results if not result.shipped_with and result.landing != "push"]
lines = []
for name, (what, measure) in MEDIANS.items():
held = [(result.item.scores[name], value) for result in single if name in result.item.scores and (value := measure(result)) is not None]
@@ -126,7 +126,7 @@ def ranks(values: list[int]) -> list[float]:
def render(result: Result) -> str:
scores = ", ".join(f"{name} {result.item.scores[name]}" for name in SHOWN if name in result.item.scores)
merged = {"merge": f"merged {result.merged} in {result.merge}", "rebase": f"merged {result.merged} by rebase in {result.merge}", "push": f"committed {result.merged} in {result.merge}, never replayed, so perhaps part of a pull request"}[result.landed]
merged = {"merge": f"merged {result.merged} in {result.merge}", "rebase": f"merged {result.merged} by rebase in {result.merge}", "push": f"committed {result.merged} in {result.merge}, never replayed, so perhaps part of a pull request"}[result.landing]
others = result.shipped_with
together = f", with item{'s' if len(others) > 1 else ''} {', '.join(map(str, others))}" if others else ""
return f"- Item {result.item.id}, {scores}: {plural(result.lines, 'line')} in {plural(result.files, 'file')}, {at_least(result)}{plural(result.rounds_past_first, 'round')} past each pass's first, {counted(result)}, {later(result)}; {merged}{together}. {result.item.tagline}"
@@ -134,7 +134,7 @@ def render(result: Result) -> str:
def at_least(result: Result) -> str:
"""The words that mark a chunk's rounds and findings as lower bounds: rebase drops a round's commit that changed no file, so history shows a chunk that left no merge commit short."""
return "" if result.landed == "merge" else "at least "
return "" if result.landing == "merge" else "at least "
def counted(result: Result) -> str:
+12 -12
View File
@@ -31,8 +31,8 @@ def item(id_: int, scores: dict[str, int]) -> Item:
return Item(id_, (0, 1, 0), "", f"**Item {id_}.**", scores, (("1", None),), ())
def result(id_: int, r: int, s: int, c: int | None, lines: int = 10, rounds: int = 1, findings: tuple[int, int] | None = (0, 0), merged: str = "2026-10-13T10:00:00Z", shipped_with: tuple[int, ...] = (), levels: dict[str, int] | None = None, landed: Landing = "merge", misread: int | None = None, merge: str = "aaaaaaa", fixed_by: tuple[str, ...] = ()) -> Result:
return Result(item(id_, {"R": r, "S": s} | ({"C": c} if c else {}) | (levels or {})), merge, merged, shipped_with, lines, 1, rounds, None if findings is None else Findings(*findings, misread), landed, fixed_by)
def result(id_: int, r: int, s: int, c: int | None, lines: int = 10, rounds: int = 1, findings: tuple[int, int] | None = (0, 0), merged: str = "2026-10-13T10:00:00Z", shipped_with: tuple[int, ...] = (), levels: dict[str, int] | None = None, landing: Landing = "merge", misread: int | None = None, merge: str = "aaaaaaa", fixed_by: tuple[str, ...] = ()) -> Result:
return Result(item(id_, {"R": r, "S": s} | ({"C": c} if c else {}) | (levels or {})), merge, merged, shipped_with, lines, 1, rounds, None if findings is None else Findings(*findings, misread), landing, fixed_by)
class ResultsOf(unittest.TestCase):
@@ -115,11 +115,11 @@ class Render(unittest.TestCase):
self.assertIn("1 round past each pass's first, 0 🔴, 0 🟡 (0 misread), 0 later fixes; merged", render(result(4, 2, 3, None, misread=0)))
def test_gives_a_rebase_merged_chunks_rounds_and_findings_as_lower_bounds(self):
self.assertEqual(render(result(4, 2, 3, None, findings=(1, 2), landed="rebase")), "- Item 4, R 2, S 3: 10 lines in 1 file, at least 1 round past each pass's first, at least 1 🔴, 2 🟡, 0 later fixes; merged 2026-10-13T10:00:00Z by rebase in aaaaaaa. **Item 4.**")
self.assertIn("at least 1 round past each pass's first, findings unknown, 0 later fixes; merged 2026-10-13T10:00:00Z by rebase in aaaaaaa", render(result(4, 2, 3, None, findings=None, landed="rebase")))
self.assertEqual(render(result(4, 2, 3, None, findings=(1, 2), landing="rebase")), "- Item 4, R 2, S 3: 10 lines in 1 file, at least 1 round past each pass's first, at least 1 🔴, 2 🟡, 0 later fixes; merged 2026-10-13T10:00:00Z by rebase in aaaaaaa. **Item 4.**")
self.assertIn("at least 1 round past each pass's first, findings unknown, 0 later fixes; merged 2026-10-13T10:00:00Z by rebase in aaaaaaa", render(result(4, 2, 3, None, findings=None, landing="rebase")))
def test_says_a_chunk_never_replayed_may_be_part_of_a_pull_request(self):
self.assertEqual(render(result(4, 2, 3, None, findings=(1, 2), landed="push")), "- Item 4, R 2, S 3: 10 lines in 1 file, at least 1 round past each pass's first, at least 1 🔴, 2 🟡, 0 later fixes; committed 2026-10-13T10:00:00Z in aaaaaaa, never replayed, so perhaps part of a pull request. **Item 4.**")
self.assertEqual(render(result(4, 2, 3, None, findings=(1, 2), landing="push")), "- Item 4, R 2, S 3: 10 lines in 1 file, at least 1 round past each pass's first, at least 1 🔴, 2 🟡, 0 later fixes; committed 2026-10-13T10:00:00Z in aaaaaaa, never replayed, so perhaps part of a pull request. **Item 4.**")
ANCHORS_TEXT = "# Score anchors\n\nWhy these.\n\n| Level | Items |\n|---|---|\n| L 1 | 4, 9 |\n| I 3 | 5 |\n"
@@ -186,7 +186,7 @@ Later fixes: the chunks that fixed code each finished chunk shipped, newest firs
- None: no later chunk fixed code a finished chunk shipped.""")
def test_lists_each_chunks_findings_with_their_misread_count_newest_first_once_per_chunk(self):
found = [result(1, 5, 3, None, findings=(1, 2), misread=1, merge="bbbbbbb", merged="2026-10-12T10:00:00Z", shipped_with=(2,)), result(2, 5, 3, None, findings=(1, 2), misread=1, merge="bbbbbbb", merged="2026-10-12T10:00:00Z", shipped_with=(1,)), result(3, 5, 3, None, findings=(0, 4), misread=0, merge="ccccccc", merged="2026-10-14T10:00:00Z", landed="rebase"), result(4, 5, 3, None)]
found = [result(1, 5, 3, None, findings=(1, 2), misread=1, merge="bbbbbbb", merged="2026-10-12T10:00:00Z", shipped_with=(2,)), result(2, 5, 3, None, findings=(1, 2), misread=1, merge="bbbbbbb", merged="2026-10-12T10:00:00Z", shipped_with=(1,)), result(3, 5, 3, None, findings=(0, 4), misread=0, merge="ccccccc", merged="2026-10-14T10:00:00Z", landing="rebase"), result(4, 5, 3, None)]
self.assertIn("""🔴 and 🟡 findings, with how many a reviewer marked `[misread]`, per finished chunk whose round titles count them, newest first:
- 2026-10-14T10:00:00Z in ccccccc: at least 0 🔴, 4 🟡 (0 misread)
@@ -201,21 +201,21 @@ Later fixes: the chunks that fixed code each finished chunk shipped, newest firs
- The other 2 finished chunks: none."""))
def test_ends_with_how_it_reads_chunks_merged_by_rebase_only_where_it_measured_one(self):
text = report_text([result(1, 5, 3, None, landed="rebase")], [], None)
text = report_text([result(1, 5, 3, None, landing="rebase")], [], None)
self.assertIn("\n\nChunks that left no merge commit:\n\n- Each is the run of commits", text.split("Median outcome")[1])
self.assertIn("a pull request that shipped nothing adds its lines, files and rounds", text)
self.assertIn("at least what history shows, and the medians count the rebased ones as shown.", text)
self.assertIn("Chunks that left no merge commit", report_text([result(1, 5, 3, None, landed="push")], [], None))
self.assertIn("Chunks that left no merge commit", report_text([result(1, 5, 3, None, landing="push")], [], None))
self.assertNotIn("Chunks that left no merge commit", report_text([result(1, 5, 3, None)], [], None))
def test_leaves_a_never_replayed_chunk_out_of_the_s_medians_and_keeps_a_rebased_one(self):
found = [result(id_, 2, 2, None, lines=10 * id_, landed="rebase" if id_ == 1 else "merge") for id_ in range(1, 11)]
self.assertIn("- S, lines changed: 2 → 55 (10 chunks)", report_text(found + [result(11, 2, 2, None, lines=100000, landed="push")], [], None))
found = [result(id_, 2, 2, None, lines=10 * id_, landing="rebase" if id_ == 1 else "merge") for id_ in range(1, 11)]
self.assertIn("- S, lines changed: 2 → 55 (10 chunks)", report_text(found + [result(11, 2, 2, None, lines=100000, landing="push")], [], None))
def test_leaves_a_never_replayed_chunk_out_of_the_r_medians_and_the_rank_correlations(self):
levels = [1, 1, 2, 2, 2, 3, 3, 3, 4, 4]
found = [result(id_, 2, 2, None, rounds=level, findings=(5 - level, 0), levels={"L": level, "E": 2, "I": level}) for id_, level in enumerate(levels, 1)]
pushed = result(11, 2, 2, None, rounds=9, findings=(9, 0), levels={"L": 1, "E": 2, "I": 1}, landed="push")
pushed = result(11, 2, 2, None, rounds=9, findings=(9, 0), levels={"L": 1, "E": 2, "I": 1}, landing="push")
text = report_text(found + [pushed], [], None)
self.assertIn("- R, 🔴 findings: 2 → 2.5 (10 chunks)", text)
self.assertIn("- L, by review rounds past each pass's first and 🔴 findings: rank correlation +1.00 and -1.00 (10 chunks)", text)
@@ -223,7 +223,7 @@ Later fixes: the chunks that fixed code each finished chunk shipped, newest firs
self.assertIn("- E, by review rounds past each pass's first and 🔴 findings: rank correlation none and none, since every chunk held one value (10 chunks)", text)
def test_keeps_a_never_replayed_chunk_in_the_nearest_and_anchors_lists(self):
pushed = result(9, 2, 3, None, landed="push")
pushed = result(9, 2, 3, None, landing="push")
text = report_text([pushed], [("L 1", (9,))], {"L": 1, "E": 1, "I": 1, "S": 3})
nearest_section, anchors_section = text.split("Anchors in ")
self.assertIn("- Item 9, R 2, S 3:", nearest_section)
+38
View File
@@ -0,0 +1,38 @@
import pathlib
import subprocess
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from release import newest_tag, tagged
from test_env import isolated_env
class NewestTag(unittest.TestCase):
def setUp(self):
self.repo = tempfile.mkdtemp()
self.addCleanup(subprocess.run, ["rm", "-rf", self.repo])
self.git("init", "-q", "-b", "main")
def git(self, *args: str) -> None:
subprocess.run(["git", "-c", "user.name=t", "-c", "user.email=t@t", *args], capture_output=True, check=True, cwd=self.repo, env=isolated_env())
def tag_commit(self, tag: str) -> None:
self.git("commit", "-q", "--allow-empty", "-m", tag)
self.git("tag", tag)
def test_takes_the_highest_version_sorting_a_release_candidate_below_its_release_and_only_tags_merged_into_the_branch_where_one_is_named(self):
self.assertIsNone(newest_tag(self.repo))
for tag in ("v0.9.0", "v0.10.0", "v1.0.0-rc.1"):
self.tag_commit(tag)
self.assertEqual(newest_tag(self.repo), "v1.0.0-rc.1")
self.git("checkout", "-q", "-b", "side")
self.tag_commit("v1.0.0")
self.git("checkout", "-q", "main")
self.assertEqual(newest_tag(self.repo), "v1.0.0")
self.assertEqual(newest_tag(self.repo, merged_into="main"), "v1.0.0-rc.1")
def test_names_a_release_only_for_a_v_tag_of_a_version(self):
self.assertEqual((tagged("v1.2.3"), tagged("1.2.3"), tagged("v1.2.3-rc.1"), tagged(None)), ((1, 2, 3), None, None, None))
@@ -6,10 +6,11 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
import chunk_pick
from failure import Failure
from test_env import isolated_env
import release_standing
import todo_table
from release import version
HEADER = "## Items\n\n| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |\n|---|---|---|---|---|---|---|---|---|---|---|---|\n"
@@ -20,7 +21,7 @@ def naming(named: set[tuple[int, int, int]] | Failure) -> list[str] | Failure:
"""LastCut.todos_since holding one todo.md text that files an item under each release in named."""
if isinstance(named, Failure):
return named
return [HEADER + "".join(ROW.format(id=id_, release=todo_table.version(release)) for id_, release in enumerate(sorted(named), 1))]
return [HEADER + "".join(ROW.format(id=id_, release=version(release)) for id_, release in enumerate(sorted(named), 1))]
def one(release: tuple[int, int, int]) -> todo_table.Item:
@@ -36,95 +37,96 @@ def plan_under(earliest: tuple[int, int, int] | None) -> list[todo_table.Item]:
class NextRelease(unittest.TestCase):
def test_names_the_earliest_release_after_the_last_cut_that_todo_md_named_since_it_or_the_plan_holds(self):
cases = (
("v1.2.3", (1, 3, 0), {(1, 2, 4), (1, 3, 0)}, chunk_pick.Upcoming((1, 2, 4), False, True)),
("v1.2.3", (1, 2, 4), {(1, 3, 0)}, chunk_pick.Upcoming((1, 2, 4), True, True)),
("v1.2.3", None, {(1, 2, 2), (1, 3, 0)}, chunk_pick.Upcoming((1, 3, 0), False, True)),
("v1.2.3", None, set(), chunk_pick.Upcoming(None, False, True)),
(None, (0, 2, 0), set(), chunk_pick.Upcoming((0, 2, 0), True, True)),
(None, None, set(), chunk_pick.Upcoming(None, True, True)),
("v1.2.3-rc.1", (1, 2, 3), set(), chunk_pick.Upcoming((1, 2, 3), True, True)),
("v1.2.3", (1, 3, 0), {(1, 2, 4), (1, 3, 0)}, release_standing.Upcoming((1, 2, 4), False, True)),
("v1.2.3", (1, 2, 4), {(1, 3, 0)}, release_standing.Upcoming((1, 2, 4), True, True)),
("v1.2.3", None, {(1, 2, 2), (1, 3, 0)}, release_standing.Upcoming((1, 3, 0), False, True)),
("v1.2.3", None, set(), release_standing.Upcoming(None, False, True)),
(None, (0, 2, 0), set(), release_standing.Upcoming((0, 2, 0), True, True)),
(None, None, set(), release_standing.Upcoming(None, True, True)),
("v1.2.3-rc.1", (1, 2, 3), set(), release_standing.Upcoming((1, 2, 3), True, True)),
)
for tag, earliest, named, upcoming in cases:
with self.subTest(tag=tag, earliest=earliest, named=named):
self.assertEqual(chunk_pick.next_release(plan_under(earliest), chunk_pick.LastCut(tag, "", 4, naming(named))), upcoming)
self.assertEqual(release_standing.next_release(plan_under(earliest), release_standing.LastCut(tag, "", 4, naming(named))), upcoming)
def test_says_work_has_landed_towards_it_where_a_change_is_unreleased(self):
cut = lambda changes: chunk_pick.LastCut("v1.2.3", "", changes, naming({(1, 3, 0)}))
self.assertEqual(chunk_pick.next_release([one((1, 3, 0))], cut(4)), chunk_pick.Upcoming((1, 3, 0), True, True))
self.assertEqual(chunk_pick.next_release([one((1, 3, 0))], cut(0)), chunk_pick.Upcoming((1, 3, 0), True, False))
self.assertEqual(chunk_pick.next_release([one((1, 3, 0))], cut(Failure("count unreadable"))), chunk_pick.Upcoming((1, 3, 0), True, Failure("count unreadable")))
cut = lambda changes: release_standing.LastCut("v1.2.3", "", changes, naming({(1, 3, 0)}))
self.assertEqual(release_standing.next_release([one((1, 3, 0))], cut(4)), release_standing.Upcoming((1, 3, 0), True, True))
self.assertEqual(release_standing.next_release([one((1, 3, 0))], cut(0)), release_standing.Upcoming((1, 3, 0), True, False))
self.assertEqual(release_standing.next_release([one((1, 3, 0))], cut(Failure("count unreadable"))), release_standing.Upcoming((1, 3, 0), True, Failure("count unreadable")))
class ChunkItem(unittest.TestCase):
def test_takes_the_earliest_release_where_it_is_the_next_and_else_says_whether_the_next_is_ready_to_cut_or_to_plan(self):
cases = (
("v1.2.3", 4, (1, 2, 4), set(), chunk_pick.Next((1, 2, 4), one((1, 2, 4)))),
("v1.2.3", 4, (1, 3, 0), {(1, 3, 0)}, chunk_pick.Next((1, 3, 0), one((1, 3, 0)))),
("v1.2.3", 4, (1, 2, 4), {(1, 3, 0)}, chunk_pick.Next((1, 2, 4), one((1, 2, 4)))),
("v1.2.3", 4, (1, 3, 0), {(1, 2, 3), (1, 3, 0)}, chunk_pick.Next((1, 3, 0), one((1, 3, 0)))),
("v0.2.0", 4, (0, 4, 0), set(), chunk_pick.Next((0, 4, 0), one((0, 4, 0)))),
(None, 4, (0, 2, 0), set(), chunk_pick.Next((0, 2, 0), one((0, 2, 0)))),
(None, 4, None, set(), chunk_pick.Next(None, None)),
("v1.2.3-rc.1", 4, (1, 2, 3), set(), chunk_pick.Next((1, 2, 3), one((1, 2, 3)))),
(Failure("tags unreadable"), 4, (0, 2, 0), set(), chunk_pick.Next((0, 2, 0), one((0, 2, 0)))),
("v1.2.3", 4, (1, 2, 3), set(), chunk_pick.Cut((1, 2, 3))),
("v1.2.3", 4, (1, 2, 2), {(1, 2, 4)}, chunk_pick.Cut((1, 2, 2))),
("v1.2.3", 4, (1, 3, 0), {(1, 2, 4), (1, 3, 0)}, chunk_pick.ReadyToCut((1, 2, 4))),
("v0.9.2", 4, (1, 0, 0), {(0, 9, 3)}, chunk_pick.ReadyToCut((0, 9, 3))),
("v1.2.3", 4, (1, 4, 0), {(1, 2, 2), (1, 3, 0)}, chunk_pick.ReadyToCut((1, 3, 0))),
("v1.2.3", 4, None, {(1, 3, 0)}, chunk_pick.ReadyToCut((1, 3, 0))),
("v1.2.3", 4, None, set(), chunk_pick.ReadyToCut(None)),
("v1.2.3", 0, (1, 3, 0), {(1, 2, 4)}, chunk_pick.Unplanned((1, 2, 4))),
("v1.2.3", 0, None, set(), chunk_pick.Unplanned(None)),
("v1.2.3", Failure("count unreadable"), None, set(), chunk_pick.Unknown()),
("v1.2.3", Failure("count unreadable"), (1, 3, 0), {(1, 3, 0)}, chunk_pick.Next((1, 3, 0), one((1, 3, 0)))),
("v1.2.3", 4, (1, 3, 0), Failure("log unreadable"), chunk_pick.Unknown()),
("v1.2.3", 4, (1, 2, 3), Failure("log unreadable"), chunk_pick.Cut((1, 2, 3))),
("v1.2.3", 4, (1, 2, 4), set(), release_standing.Next((1, 2, 4), one((1, 2, 4)))),
("v1.2.3", 4, (1, 3, 0), {(1, 3, 0)}, release_standing.Next((1, 3, 0), one((1, 3, 0)))),
("v1.2.3", 4, (1, 2, 4), {(1, 3, 0)}, release_standing.Next((1, 2, 4), one((1, 2, 4)))),
("v1.2.3", 4, (1, 3, 0), {(1, 2, 3), (1, 3, 0)}, release_standing.Next((1, 3, 0), one((1, 3, 0)))),
("v0.2.0", 4, (0, 4, 0), set(), release_standing.Next((0, 4, 0), one((0, 4, 0)))),
(None, 4, (0, 2, 0), set(), release_standing.Next((0, 2, 0), one((0, 2, 0)))),
(None, 4, None, set(), release_standing.Next(None, None)),
("v1.2.3-rc.1", 4, (1, 2, 3), set(), release_standing.Next((1, 2, 3), one((1, 2, 3)))),
(Failure("tags unreadable"), 4, (0, 2, 0), set(), release_standing.Unknown()),
(Failure("tags unreadable"), 4, None, set(), release_standing.Unknown()),
("v1.2.3", 4, (1, 2, 3), set(), release_standing.Cut((1, 2, 3))),
("v1.2.3", 4, (1, 2, 2), {(1, 2, 4)}, release_standing.Cut((1, 2, 2))),
("v1.2.3", 4, (1, 3, 0), {(1, 2, 4), (1, 3, 0)}, release_standing.ReadyToCut((1, 2, 4))),
("v0.9.2", 4, (1, 0, 0), {(0, 9, 3)}, release_standing.ReadyToCut((0, 9, 3))),
("v1.2.3", 4, (1, 4, 0), {(1, 2, 2), (1, 3, 0)}, release_standing.ReadyToCut((1, 3, 0))),
("v1.2.3", 4, None, {(1, 3, 0)}, release_standing.ReadyToCut((1, 3, 0))),
("v1.2.3", 4, None, set(), release_standing.ReadyToCut(None)),
("v1.2.3", 0, (1, 3, 0), {(1, 2, 4)}, release_standing.Unplanned((1, 2, 4))),
("v1.2.3", 0, None, set(), release_standing.Unplanned(None)),
("v1.2.3", Failure("count unreadable"), None, set(), release_standing.Unknown()),
("v1.2.3", Failure("count unreadable"), (1, 3, 0), {(1, 3, 0)}, release_standing.Next((1, 3, 0), one((1, 3, 0)))),
("v1.2.3", 4, (1, 3, 0), Failure("log unreadable"), release_standing.Unknown()),
("v1.2.3", 4, (1, 2, 3), Failure("log unreadable"), release_standing.Cut((1, 2, 3))),
)
for tag, changes, earliest, named, stands in cases:
with self.subTest(tag=tag, changes=changes, earliest=earliest, named=named):
self.assertEqual(chunk_pick.chunk_item(plan_under(earliest), chunk_pick.LastCut(tag, "", changes, naming(named), reviewed=True)), stands)
self.assertEqual(release_standing.chunk_item(plan_under(earliest), release_standing.LastCut(tag, "", changes, naming(named), reviewed=True)), stands)
def test_calls_a_release_review_due_where_the_next_release_holds_no_item_and_none_landed_since_the_cut(self):
cut = lambda reviewed: chunk_pick.LastCut("v1.2.3", "", 4, naming({(1, 3, 0)}), reviewed)
self.assertEqual(chunk_pick.chunk_item([], cut(False)), chunk_pick.ReviewDue((1, 3, 0)))
self.assertEqual(chunk_pick.chunk_item([], cut(True)), chunk_pick.ReadyToCut((1, 3, 0)))
self.assertEqual(chunk_pick.chunk_item([], cut(Failure("log unreadable"))), chunk_pick.Unknown())
self.assertEqual(chunk_pick.chunk_item(plan_under((1, 3, 0)), cut(False)), chunk_pick.Next((1, 3, 0), one((1, 3, 0))), "a review matters only once the release's items run out")
self.assertEqual(chunk_pick.chunk_item([], chunk_pick.LastCut("v1.2.3", "", 0, naming({(1, 3, 0)}))), chunk_pick.Unplanned((1, 3, 0)))
cut = lambda reviewed: release_standing.LastCut("v1.2.3", "", 4, naming({(1, 3, 0)}), reviewed)
self.assertEqual(release_standing.chunk_item([], cut(False)), release_standing.ReviewDue((1, 3, 0)))
self.assertEqual(release_standing.chunk_item([], cut(True)), release_standing.ReadyToCut((1, 3, 0)))
self.assertEqual(release_standing.chunk_item([], cut(Failure("log unreadable"))), release_standing.Unknown())
self.assertEqual(release_standing.chunk_item(plan_under((1, 3, 0)), cut(False)), release_standing.Next((1, 3, 0), one((1, 3, 0))), "a review matters only once the release's items run out")
self.assertEqual(release_standing.chunk_item([], release_standing.LastCut("v1.2.3", "", 0, naming({(1, 3, 0)}))), release_standing.Unplanned((1, 3, 0)))
def test_takes_an_item_of_any_release_on_a_branch_with_no_tag(self):
for release in ((0, 0, 0), (0, 0, 1), (0, 1, 0)):
with self.subTest(release=release):
self.assertEqual(chunk_pick.chunk_item(plan_under(release), chunk_pick.LastCut(None, "", 4, naming(set()))), chunk_pick.Next(release, one(release)))
self.assertEqual(release_standing.chunk_item(plan_under(release), release_standing.LastCut(None, "", 4, naming(set()))), release_standing.Next(release, one(release)))
def test_calls_a_release_0_0_0_finished_on_a_branch_with_no_tag(self):
history = [HEADER + ROW.format(id=1, release="0.0.0"), HEADER]
self.assertEqual(chunk_pick.chunk_item([], chunk_pick.LastCut(None, "", 4, history)), chunk_pick.ReviewDue((0, 0, 0)))
self.assertEqual(release_standing.chunk_item([], release_standing.LastCut(None, "", 4, history)), release_standing.ReviewDue((0, 0, 0)))
def test_calls_a_release_that_never_had_a_tag_finished_once_the_plan_files_no_item_under_it(self):
history = [HEADER + ROW.format(id=1, release="0.1.0") + ROW.format(id=2, release="0.1.1"), HEADER + ROW.format(id=2, release="0.1.1")]
cut = lambda reviewed: chunk_pick.LastCut(None, "", 4, history, reviewed)
self.assertEqual(chunk_pick.chunk_item(plan_under((0, 1, 1)), cut(False)), chunk_pick.ReviewDue((0, 1, 0)))
self.assertEqual(chunk_pick.chunk_item(plan_under((0, 1, 1)), cut(True)), chunk_pick.ReadyToCut((0, 1, 0)))
self.assertEqual(chunk_pick.chunk_item(plan_under((0, 1, 0)), cut(False)), chunk_pick.Next((0, 1, 0), one((0, 1, 0))))
cut = lambda reviewed: release_standing.LastCut(None, "", 4, history, reviewed)
self.assertEqual(release_standing.chunk_item(plan_under((0, 1, 1)), cut(False)), release_standing.ReviewDue((0, 1, 0)))
self.assertEqual(release_standing.chunk_item(plan_under((0, 1, 1)), cut(True)), release_standing.ReadyToCut((0, 1, 0)))
self.assertEqual(release_standing.chunk_item(plan_under((0, 1, 0)), cut(False)), release_standing.Next((0, 1, 0), one((0, 1, 0))))
def test_counts_a_dated_changelog_heading_as_cut_on_a_branch_with_no_tag(self):
changelog = "# Changelog\n\n## Unreleased\n\n## 0.0.1 - 2026-10-03\n\n- Shipped.\n"
history = [HEADER + ROW.format(id=1, release="0.0.1"), HEADER]
self.assertEqual(chunk_pick.chunk_item([], chunk_pick.LastCut(None, changelog, 4, history)), chunk_pick.Unplanned(None))
without_unreleased = chunk_pick.LastCut(None, "# Changelog\n\n## 0.0.1 - 2026-10-03\n", 4, history)
self.assertNotEqual(chunk_pick.chunk_item([], without_unreleased), chunk_pick.ReviewDue((0, 0, 1)), "0.0.1 is cut under its dated heading")
self.assertEqual(release_standing.chunk_item([], release_standing.LastCut(None, changelog, 4, history)), release_standing.Unplanned(None))
without_unreleased = release_standing.LastCut(None, "# Changelog\n\n## 0.0.1 - 2026-10-03\n", 4, history)
self.assertNotEqual(release_standing.chunk_item([], without_unreleased), release_standing.ReviewDue((0, 0, 1)), "0.0.1 is cut under its dated heading")
later = [HEADER + ROW.format(id=1, release="0.0.1") + ROW.format(id=2, release="0.1.0"), HEADER + ROW.format(id=2, release="0.1.0")]
self.assertEqual(chunk_pick.chunk_item(plan_under((0, 1, 0)), chunk_pick.LastCut(None, changelog, 4, later)), chunk_pick.Next((0, 1, 0), one((0, 1, 0))))
self.assertEqual(chunk_pick.chunk_item(plan_under((0, 0, 1)), chunk_pick.LastCut(None, changelog, 4, later)), chunk_pick.Cut((0, 0, 1)))
self.assertEqual(release_standing.chunk_item(plan_under((0, 1, 0)), release_standing.LastCut(None, changelog, 4, later)), release_standing.Next((0, 1, 0), one((0, 1, 0))))
self.assertEqual(release_standing.chunk_item(plan_under((0, 0, 1)), release_standing.LastCut(None, changelog, 4, later)), release_standing.Cut((0, 0, 1)))
def test_counts_neither_an_undated_heading_nor_a_heading_behind_a_tag_as_cut(self):
history = [HEADER + ROW.format(id=1, release="0.0.1"), HEADER]
undated = chunk_pick.LastCut(None, "# Changelog\n\n## 0.0.1\n\n- Shipped.\n", 4, history)
self.assertEqual(chunk_pick.chunk_item([], undated), chunk_pick.ReviewDue((0, 0, 1)))
tagged = chunk_pick.LastCut("v0.1.0", "# Changelog\n\n## 0.2.0 - 2026-10-03\n", 4, [HEADER + ROW.format(id=1, release="0.2.0")])
self.assertEqual(chunk_pick.chunk_item(plan_under((0, 2, 0)), tagged), chunk_pick.Next((0, 2, 0), one((0, 2, 0))))
undated = release_standing.LastCut(None, "# Changelog\n\n## 0.0.1\n\n- Shipped.\n", 4, history)
self.assertEqual(release_standing.chunk_item([], undated), release_standing.ReviewDue((0, 0, 1)))
tagged = release_standing.LastCut("v0.1.0", "# Changelog\n\n## 0.2.0 - 2026-10-03\n", 4, [HEADER + ROW.format(id=1, release="0.2.0")])
self.assertEqual(release_standing.chunk_item(plan_under((0, 2, 0)), tagged), release_standing.Next((0, 2, 0), one((0, 2, 0))))
def test_reads_every_version_of_todo_md_where_the_branch_has_no_tag(self):
repo = tempfile.mkdtemp()
@@ -138,8 +140,8 @@ class ChunkItem(unittest.TestCase):
run("commit", "-q", "--allow-empty", "-m", "Ship item 1")
pathlib.Path(repo, "todo.md").write_text(newest)
run("commit", "-q", "-a", "-m", "Ship item 1 for good")
self.assertEqual(chunk_pick.todos_since(None, "main", repo), [first, newest])
self.assertEqual(chunk_pick.todos_since(Failure("tags unreadable"), "main", repo), [])
self.assertEqual(release_standing.todos_since(None, "main", repo), [first, newest])
self.assertEqual(release_standing.todos_since(Failure("tags unreadable"), "main", repo), [])
def test_finds_a_release_review_among_the_commits_since_the_cut(self):
repo = tempfile.mkdtemp()
@@ -149,13 +151,13 @@ class ChunkItem(unittest.TestCase):
run("commit", "-q", "--allow-empty", "-m", "Release review of 0.1.0")
run("tag", "v0.1.0")
run("commit", "-q", "--allow-empty", "-m", "Ship item 4")
self.assertIs(chunk_pick.release_reviewed("v0.1.0", "main", repo), False, "a review before the cut reviewed that release")
self.assertIs(release_standing.release_reviewed("v0.1.0", "main", repo), False, "a review before the cut reviewed that release")
run("commit", "-q", "--allow-empty", "-m", "Release review of 0.2.0")
run("commit", "-q", "--allow-empty", "-m", "Answer architecture round 1: 0 🔴, 1 🟡 (0 misread); split loader")
self.assertIs(chunk_pick.release_reviewed("v0.1.0", "main", repo), True)
self.assertIs(chunk_pick.release_reviewed(None, "main", repo), True)
self.assertIsInstance(chunk_pick.release_reviewed("v9.9.9", "main", repo), Failure)
self.assertEqual(chunk_pick.default_plan(repo, "main").cut.reviewed, True)
self.assertIs(release_standing.release_reviewed("v0.1.0", "main", repo), True)
self.assertIs(release_standing.release_reviewed(None, "main", repo), True)
self.assertIsInstance(release_standing.release_reviewed("v9.9.9", "main", repo), Failure)
self.assertEqual(release_standing.default_plan(repo, "main").cut.reviewed, True)
def test_counts_no_release_review_of_a_release_cut_under_a_dated_heading_on_a_branch_with_no_tag(self):
repo = tempfile.mkdtemp()
@@ -166,13 +168,13 @@ class ChunkItem(unittest.TestCase):
pathlib.Path(repo, "CHANGELOG.md").write_text("# Changelog\n\n## 0.0.1 - 2026-10-03\n")
run("add", "CHANGELOG.md")
run("commit", "-q", "-m", "Cut 0.0.1")
self.assertIs(chunk_pick.default_plan(repo, "main").cut.reviewed, False, "that review reviewed 0.0.1, which is cut")
self.assertEqual(chunk_pick.default_plan(repo, "main").cut.commits, 0, "no commit follows the one that cut 0.0.1")
self.assertEqual(chunk_pick.chunk_item([], chunk_pick.default_plan(repo, "main").cut), chunk_pick.Unplanned(None))
self.assertIs(release_standing.default_plan(repo, "main").cut.reviewed, False, "that review reviewed 0.0.1, which is cut")
self.assertEqual(release_standing.default_plan(repo, "main").cut.commits, 0, "no commit follows the one that cut 0.0.1")
self.assertEqual(release_standing.chunk_item([], release_standing.default_plan(repo, "main").cut), release_standing.Unplanned(None))
run("commit", "-q", "--allow-empty", "-m", "Ship item 2")
self.assertEqual(chunk_pick.default_plan(repo, "main").cut.commits, 1)
self.assertEqual(release_standing.default_plan(repo, "main").cut.commits, 1)
run("commit", "-q", "--allow-empty", "-m", "Release review of 0.1.0")
self.assertIs(chunk_pick.default_plan(repo, "main").cut.reviewed, True)
self.assertIs(release_standing.default_plan(repo, "main").cut.reviewed, True)
def test_counts_from_the_commit_on_the_named_branch_that_first_added_the_dated_heading_to_changelog_md(self):
repo = tempfile.mkdtemp()
@@ -191,7 +193,7 @@ class ChunkItem(unittest.TestCase):
run("commit", "-q", "--allow-empty", "-m", "Ship item 2")
run("checkout", "-q", "-b", "side")
run("commit", "-q", "--allow-empty", "-m", "Side work")
self.assertEqual(chunk_pick.default_plan(repo, "main").cut.commits, 2, "the commits after Cut 0.0.1 on main")
self.assertEqual(release_standing.default_plan(repo, "main").cut.commits, 2, "the commits after Cut 0.0.1 on main")
def test_takes_the_tag_over_a_dated_heading_and_counts_every_commit_without_either(self):
repo = tempfile.mkdtemp()
@@ -201,32 +203,32 @@ class ChunkItem(unittest.TestCase):
pathlib.Path(repo, "todo.md").write_text(HEADER + ROW.format(id=1, release="0.2.0"))
run("add", "todo.md")
run("commit", "-q", "-m", "Plan")
self.assertEqual(chunk_pick.default_plan(repo, "main").cut, chunk_pick.LastCut(None, "", 1, [HEADER + ROW.format(id=1, release="0.2.0")], False))
self.assertEqual(release_standing.default_plan(repo, "main").cut, release_standing.LastCut(None, "", 1, [HEADER + ROW.format(id=1, release="0.2.0")], False))
run("tag", "v0.1.0")
pathlib.Path(repo, "CHANGELOG.md").write_text("# Changelog\n\n## 0.2.0 - 2026-10-03\n")
run("add", "CHANGELOG.md")
run("commit", "-q", "-m", "Date 0.2.0")
held = chunk_pick.default_plan(repo, "main")
held = release_standing.default_plan(repo, "main")
self.assertEqual(held.todo, HEADER + ROW.format(id=1, release="0.2.0"))
self.assertEqual(held.cut, chunk_pick.LastCut("v0.1.0", "# Changelog\n\n## 0.2.0 - 2026-10-03\n", 1, [HEADER + ROW.format(id=1, release="0.2.0")], False))
self.assertEqual(chunk_pick.chunk_item(plan_under((0, 2, 0)), held.cut), chunk_pick.Next((0, 2, 0), one((0, 2, 0))))
self.assertEqual(held.cut, release_standing.LastCut("v0.1.0", "# Changelog\n\n## 0.2.0 - 2026-10-03\n", 1, [HEADER + ROW.format(id=1, release="0.2.0")], False))
self.assertEqual(release_standing.chunk_item(plan_under((0, 2, 0)), held.cut), release_standing.Next((0, 2, 0), one((0, 2, 0))))
def test_counts_the_changes_unreleased_since_the_last_cut_by_unreleased_entries_and_else_by_commits(self):
changelog = "# Changelog\n\n## Unreleased\n\n- One.\n - A nested line.\n- Two.\n\n## 0.1.0\n\n- Old.\n"
self.assertEqual(chunk_pick.unreleased_changes(chunk_pick.LastCut("v0.1.0", changelog, 5, [])), 2)
self.assertEqual(chunk_pick.unreleased_changes(chunk_pick.LastCut("v0.1.0", "# Changelog\n\n## Unreleased\n\n## 0.1.0\n", 5, [])), 0)
self.assertEqual(chunk_pick.unreleased_changes(chunk_pick.LastCut("v0.1.0", "# Changelog\n\n## 0.1.0\n", 5, [])), 5)
self.assertEqual(chunk_pick.unreleased_changes(chunk_pick.LastCut("v0.1.0", Failure("unreadable"), 5, [])), 5)
self.assertEqual(release_standing.unreleased_changes(release_standing.LastCut("v0.1.0", changelog, 5, [])), 2)
self.assertEqual(release_standing.unreleased_changes(release_standing.LastCut("v0.1.0", "# Changelog\n\n## Unreleased\n\n## 0.1.0\n", 5, [])), 0)
self.assertEqual(release_standing.unreleased_changes(release_standing.LastCut("v0.1.0", "# Changelog\n\n## 0.1.0\n", 5, [])), 5)
self.assertEqual(release_standing.unreleased_changes(release_standing.LastCut("v0.1.0", Failure("unreadable"), 5, [])), 5)
def test_counts_the_top_level_entries_under_unreleased(self):
self.assertEqual(chunk_pick.unreleased_entries("# Changelog\n\n## Unreleased\n\n- One.\n - A nested line.\n- Two.\n\n## 0.1.0\n\n- Old.\n"), 2)
self.assertEqual(chunk_pick.unreleased_entries("# Changelog\n\n## Unreleased\n\n## 0.1.0\n\n- Old.\n"), 0)
self.assertIsNone(chunk_pick.unreleased_entries("# Changelog\n\n## 0.1.0\n\n- Old.\n"))
self.assertEqual(release_standing.unreleased_entries("# Changelog\n\n## Unreleased\n\n- One.\n - A nested line.\n- Two.\n\n## 0.1.0\n\n- Old.\n"), 2)
self.assertEqual(release_standing.unreleased_entries("# Changelog\n\n## Unreleased\n\n## 0.1.0\n\n- Old.\n"), 0)
self.assertIsNone(release_standing.unreleased_entries("# Changelog\n\n## 0.1.0\n\n- Old.\n"))
def test_names_each_release_a_todo_md_since_the_last_cut_filed_items_under(self):
texts = [HEADER + ROW.format(id=1, release="0.2.0"), "", "not a table | x |", HEADER + ROW.format(id=2, release="0.3.0")]
self.assertEqual(chunk_pick.named_releases(texts), {(0, 2, 0), (0, 3, 0)})
self.assertEqual(chunk_pick.named_releases(Failure("log unreadable")), Failure("log unreadable"))
self.assertEqual(release_standing.named_releases(texts), {(0, 2, 0), (0, 3, 0)})
self.assertEqual(release_standing.named_releases(Failure("log unreadable")), Failure("log unreadable"))
def test_leaves_out_a_release_the_plan_marks_renamed_and_keeps_one_whose_items_moved_unmarked(self):
plan = lambda *rows, renamed=(): HEADER + "".join(ROW.format(id=id_, release=release) for id_, release in rows) + ("\n## Renamed\n\n" + "".join(f"- {old} renamed to {new}\n" for old, new in renamed) if renamed else "")
@@ -241,8 +243,8 @@ class ChunkItem(unittest.TestCase):
)
for name, texts, named in cases:
with self.subTest(name):
self.assertEqual(chunk_pick.named_releases(texts), named)
self.assertEqual(release_standing.named_releases(texts), named)
def test_fails_to_name_releases_when_the_newest_plan_marks_one_in_a_form_it_cannot_read(self):
plan = "## Items\n\n| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |\n|---|---|---|---|---|---|---|---|---|---|---|---|\n| 1 | 0.3.0 | | | **Do 1.** | 1 | 1 | 1 | 1 | 1 | 1 by 1 | 10.0 |\n"
self.assertIsInstance(chunk_pick.named_releases([plan, plan + "\n## Renamed\n\nsoon\n"]), Failure)
self.assertIsInstance(release_standing.named_releases([plan, plan + "\n## Renamed\n\nsoon\n"]), Failure)
+9 -1
View File
@@ -5,7 +5,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from review_passes import PASS_AGENTS, PASSES, ReviewedPass, REVIEWERS
from review_passes import PASS_AGENTS, PASSES, prose, ReviewedPass, REVIEWERS
class ReviewPasses(unittest.TestCase):
@@ -16,3 +16,11 @@ class ReviewPasses(unittest.TestCase):
def test_names_each_agent_under_agents(self):
agents = pathlib.Path(__file__).resolve().parents[1] / "agents"
self.assertEqual(sorted(REVIEWERS.values()), sorted(path.stem for path in agents.glob("*.md")))
def test_counts_markdown_an_agent_follows_as_config_and_the_rest_as_prose(self):
for path in ("README.md", "docs/decisions.md", "todo.md", "workflow-bench/README.md"):
with self.subTest(path=path):
self.assertTrue(prose(path))
for path in ("CLAUDE.md", "AGENTS.md", "skills/larv-review/SKILL.md", "agents/prose-reviewer.md", "rules/todo-items.md", "principles/technical-principles.md", "workflow-bench/run-prompt.md", "workflow-bench/compare-prompt.md", ".claude/agents/reviewer.md", ".claude/skills/x/SKILL.md", "bin/rank_todo.py"):
with self.subTest(path=path):
self.assertFalse(prose(path))
+8 -1
View File
@@ -10,7 +10,14 @@ from round_titles import Findings, lacks_counts, past_cap, round_counts, round_f
class Rounds(unittest.TestCase):
def test_counts_each_passs_highest_round(self):
counts = round_counts(["Answer prose round 1: x", "Answer full scan round 2: y", "Answer full scan round 1: z", "Answer stability: no round", "Answer stabilty round 1: typo", "Answer security round 1: rename round 2: x", "Record the comprehension check: w"])
self.assertEqual((counts["prose"], counts["full scan"], counts["stability"], counts["architecture"], counts["stabilty"], counts["security"]), (1, 2, 0, 0, 1, 1))
self.assertEqual((counts["prose"], counts["full scan"], counts["stability"], counts["architecture"], counts["security"]), (1, 2, 0, 0, 1))
def test_counts_the_rounds_past_the_first_of_a_title_naming_two_passes(self):
self.assertEqual(rounds_past_first(["Answer stability and prose round 2: x", "Answer stability and prose round 1: y"]), 1)
def test_counts_no_round_for_a_title_naming_no_pass(self):
self.assertNotIn("stabilty", round_counts(["Answer stabilty round 1: typo"]))
self.assertFalse(past_cap("Answer architectur round 9: 0 🔴, 0 🟡 (0 misread)", []))
def test_counts_a_mis_cased_title_under_its_pass(self):
self.assertEqual(round_counts(["Answer Architecture round 4: x", "Answer architecture round 3: y"])["architecture"], 4)
+7 -19
View File
@@ -4,6 +4,7 @@ import typing
from failure import Failure
from markdown_text import FENCE, sections
from release import parsed, Release, release_of, VERSION
# Each score a Shape names, and each goal's move, runs from 1 to TOP_SCORE, but for the levels L, E and I, which run from 1 to their LEVELS: ~/.claude/rules/todo-items.md defines each level.
TOP_SCORE = 10
@@ -23,10 +24,8 @@ CODE_HEALTH = "code health"
HEADER_START = "| ID | Release |"
# todo.md's ## Details section, its body the first group.
DETAILS = re.compile(r"^## Details\n(.*?)(?=^## |\Z)", re.M | re.S)
# A release's version, such as 0.2.1.
RELEASE = r"[0-9]+\.[0-9]+\.[0-9]+"
# One line of todo.md's ## Renamed section: the release it renamed and the one it became.
RENAME = re.compile(rf"- ({RELEASE}) renamed to ({RELEASE})")
RENAME = re.compile(rf"- ({VERSION}) renamed to ({VERSION})")
# Each goal an item names, in its order, with how far the item moves it; None in OLD_SHAPE, whose Goals cell names no move.
Moves = tuple[tuple[str, int | None], ...]
@@ -94,7 +93,7 @@ OLD_SHAPE = Shape(("R", "S", "C", "A", "G"), (BLOCKED_BY, "C", "Score"), (), (),
@dataclasses.dataclass(frozen=True)
class Item:
id: int
release: tuple[int, int, int]
release: Release
exempt: str
tagline: str
scores: dict[str, int]
@@ -180,7 +179,7 @@ def waiting(found: list[Item], done: set[int]) -> set[int]:
def next_item(found: list[Item]) -> Item | None:
"""The earliest release's first row that does not wait, as waiting says, in the table's order; chunk_pick.chunk_item takes it as the next chunk's first item where its release is the next."""
"""The earliest release's first row that does not wait, as waiting says, in the table's order; release_standing.chunk_item takes it as the next chunk's first item where its release is the next."""
if not found:
return None
release = min(item.release for item in found)
@@ -188,9 +187,9 @@ def next_item(found: list[Item]) -> Item | None:
return next((item for item in found if item.release == release and item.id not in waits), None)
def renames(todo: str) -> set[tuple[int, int, int]] | Failure:
def renames(todo: str) -> set[Release] | Failure:
"""Each release todo.md's ## Renamed section says was renamed, as `- 0.2.1 renamed to 0.3.0`, so never finished; the tools read only the old release. `docs/decisions.md` → `todo.md: one table …` gives the rule and the spelling's pick."""
marked: set[tuple[int, int, int]] = set()
marked: set[Release] = set()
for body in sections(todo).get("Renamed", []):
for line in body.splitlines():
rename = RENAME.fullmatch(line.strip())
@@ -279,23 +278,12 @@ def parse_row(row: str, table: Table) -> Item | Failure:
return Item(int(named["ID"]), release_of(named["Release"]), named["Exempt"], named["Item"], scores, moves, blocked_by)
def release_of(version: str) -> tuple[int, int, int]:
"""The release a RELEASE-shaped version names."""
major, minor, patch = (int(part) for part in version.split("."))
return major, minor, patch
def version(release: tuple[int, int, int]) -> str:
"""The version a release names, such as 0.2.1: release_of's inverse."""
return ".".join(map(str, release))
def check_cells(row: str, named: dict[str, str]) -> Failure | None:
"""Holds the row's ID, Release, Exempt and Item cells to the format: why one breaks it, None when none does."""
id_, release, exempt, tagline = named["ID"], named["Release"], named["Exempt"], named["Item"]
if not re.fullmatch(r"[0-9]+", id_) or int(id_) < 1:
return Failure(f"the ID {id_!r} is not a positive whole number: {row}")
if not re.fullmatch(RELEASE, release):
if parsed(release) is None:
return Failure(f"item {id_}'s release {release!r} is not a semver version")
if exempt not in EXEMPTIONS:
return Failure(f"item {id_}'s Exempt {exempt!r} is not empty or one of {', '.join(EXEMPTIONS[1:])}")
+31
View File
@@ -3329,3 +3329,34 @@ Inherited architect, would restructure first:
- `bin/loop_claude/stop.py:1` `Stop`: documented as loop-claude's exit, but rung_replay exits through it and imports `cycle.retry` too
- `bin/rank_todo.py:71` `GOAL_STEP`: a separable formula evaluator inside a 658-line, 39-function module
- `bin/chunk_pick.py:1` module docstring: beside `bin/chunk_preflight/`, the names don't say which does what
## 2026-11-04T22:52:35Z, PR #415 at 37a412b, against 7681c2b
Ruling: better
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|---|---|---|---|---|---|
| Mid A | better | better | better | same | better |
| Inherited architect | better | better | better | same | better |
Mid A, decided by:
- Navigation, Shape: `bin/loop_claude/loop/main.py:159` `run_loop`: reads as a short loop of `next_cycle`, `run_cycle`, `closed_cycle` and `check_routing`, where the base held routing, preflight, step-up archiving and the unfinished limit beside it; a symptom leads to `chunk_start.py` or `close.py`
- Locality: `bin/loop_claude/cycle/run.py:40` `Cycle.questions`: the questions folder is an explicit field, where the base rebound `paths.waiting` out of sight for every later reader
- Locality, Shape: `bin/loop_claude/cycle/run.py:191` `archived_step`: both step-up paths archive through one function and one `StepUp`, and `next_route` replaces the `after_unfinished`/`after_ruling` pair, denser but in one place
- Navigation, Shape: `bin/loop_claude/waiting/talk.py:1` `talk`: the 233-line `waiting.py` is split by role, so a `/talk` bug opens a 45-line file
- Navigation: `bin/release.py:1` `release`: every release pattern has one home, where the base spread them over `chunk_pick`, `todo_table` and `git_repo`
- Locality: `bin/release_standing.py:177` `dated_cut`: what is cut on an untagged branch is worked out again at four call sites, and `LastCut.tag is None` quietly means it may be cut by a CHANGELOG heading; weaker than the gains above
- Shape: `bin/release_standing.py:1` `release_standing`: the renamed module's docstring still opens "The next chunk's pick"
Inherited architect, decided by:
- Navigation, Shape: `bin/loop_claude/loop/main.py:186` `next_cycle`: `run_loop` reads as the cycle's outline, with a cycle's start in `chunk_start.py` and its close in `close.py`, each named for its phase, where the base held about 14 functions in 338 lines
- Locality: `bin/loop_claude/loop/main.py:166` `run_loop`: passes `questions=listed.folder` explicitly, where the base rebound `paths.waiting` for every later call
- Locality: `bin/loop_claude/cycle/run.py:141` `rule`: archives every interrupt or quoted answer once, as the judge rules, with `states.to_the_human` the one rule both the transition and the archive read, where the base archived in two places through `ran_on`
- Locality, Shape: `bin/loop_claude/hand_over.py:23` `hand_over`: opening claude has one side effect, no longer a hidden write of `~/.claude.json` through `trust.py`
- Navigation, Shape: `bin/release.py:1` `release`: every release pattern lives in one module, and `Release` replaces the hand-written tuple
- Navigation, Shape: `bin/loop_claude/waiting/__init__.py:10` `waiting`: one 234-line module is four of 50 to 80 lines, each a role, at the cost of more imports in its two callers
- Locality: `bin/rule_hook/main.py:75` `breaches_of`: the repeat-read helpers moved beside their rule in `repeat_read.py`, so `main.py` only dispatches
- Locality: `bin/release_standing.py:177` `dated_cut`: whether a CHANGELOG heading counts as the cut is recomputed at four call sites and not kept in `LastCut`; worse here, outweighed by the places above, and where it rots first
- Shape: `bin/loop_claude/routing.py:84` `next_route`: fewer names to learn, a denser body to trace by hand; on balance the same
+2 -2
View File
@@ -294,7 +294,7 @@ On 2026-10-16 lilleman routed the reviewers on their item's E + I as well, on ei
Valid until the trial on real chunks shows whether a worker below the project's rung holds quality.
2026-10-29, lilleman. Serves goals 1.4 and 1.5. A branch that changes only Markdown no agent follows reviews on the project's rung, whatever its items' E + I. E + I scores an item before its chunk starts, by the change the item asks for, so one project's item, filed to build a control on a CI runner, kept its E + I of 7 when the chunk found no control the repo could build and shipped a README paragraph: five reviewers read prose on Opus 5.5 at xhigh. rule-hook's `reviewer-rung` reads the branch's files, committed or not, against the default branch as last fetched. Markdown an agent follows, `CLAUDE.md`, `AGENTS.md` and files under `agents/`, `principles/`, `rules/` or `skills/`, counts as config. Valid while a README or docs defect costs a reader no more than review on the project's rung lets through.
2026-10-29, lilleman. Serves goals 1.4 and 1.5. A branch that changes only Markdown no agent follows reviews on the project's rung, whatever its items' E + I. E + I scores an item before its chunk starts, by the change the item asks for, so one project's item, filed to build a control on a CI runner, kept its E + I of 7 when the chunk found no control the repo could build and shipped a README paragraph: five reviewers read prose on Opus 5.5 at xhigh. rule-hook's `reviewer-rung` reads the branch's files, committed or not, against the default branch as last fetched. Markdown an agent follows counts as config: `CLAUDE.md`, `AGENTS.md`, `workflow-bench/*-prompt.md` and files under `agents/`, `principles/`, `rules/` or `skills/`. On 2026-11-04 lilleman added the workflow bench's prompts, which an agent follows like a skill. `bin/review_passes.py` holds the lists, and the worker's prompt renders them. Valid while a README or docs defect costs a reader no more than review on the project's rung lets through.
## routing: a review pass runs below the project's rung only on a rung that a replay of its rounds shows finding every 🔴 and 🟡 the project's rung finds, and only where the E + I steps no review up and the worker runs no higher than the project's rung, then even below the worker's; no pass has such a rung
@@ -302,7 +302,7 @@ Valid until the trial on real chunks shows whether a worker below the project's
`bin/review-replay <pass> <model>` replays a pass's archived rounds on the code each reviewed. It replays them first on the project's rung, then on each rung below it, highest first, down to the ladder's lowest. It stops at the first rung that misses a finding the project's rung found again. A rung qualifies only where it and each rung above it find every such 🔴 and 🟡.
`REPLAYED` in `bin/model_routing/ladder.py` holds each qualifying rung, by pass and project model. Where the chunk's E + I steps no review up and the worker runs no higher than the project's rung, the pass runs on that rung, even below the worker's. Otherwise the pass runs where every other pass does: the replay compared reviews on the project's rung only, never reviews on a rung an E + I stepped up, nor of a chunk whose worker climbed above the project's rung.
A qualifying rung applies only where the chunk's E + I steps no review up and the worker runs no higher than the project's rung; there the pass runs on that rung, even below the worker's. Otherwise the pass runs where every other pass does: the replay compared reviews on the project's rung only, never reviews on a rung an E + I stepped up, nor of a chunk whose worker climbed above the project's rung. No code lowers a pass while no rung qualifies: on 2026-11-04 lilleman had the release review of 0.6.0 delete that code, which no chunk could reach. The chunk that first qualifies a rung, `todo.md` item 196's, writes that code again in `bin/model_routing/`. It records the qualifying rung for each pass and project model, and names the lowered pass in the worker's prompt and in rule-hook's `reviewer-rung`.
The runs, each on the latest 10 rounds of Opus 5.5 projects that hold a 🔴 or 🟡:
+1 -1
View File
@@ -43,7 +43,7 @@ A pass is one kind of review, a row of the table, named by its first column. Its
| Product-owner | product-owner-reviewer | changes an external surface or contract: API routes and request/response shapes, events, webhooks, published file formats, CLI flags and exit codes, UI, error or deprecation text a consumer reads, README and docs an outside reader lands on |
| Prose | prose-reviewer | changes prose a reader reads, such as docs, comments, help or error text, or the plan's text for an item the chunk ships, splits or answers; one round |
A comment is prose, not code, in any file. A change to config is one that changes what runs, so a comment in a workflow or compose file fires only the prose pass. Markdown an agent follows is config: `CLAUDE.md`, `AGENTS.md`, and files under `agents/`, `principles/`, `rules/` or `skills/`.
A comment is prose, not code, in any file. A change to config is one that changes what runs, so a comment in a workflow or compose file fires only the prose pass. Markdown an agent follows is config: `CLAUDE.md`, `AGENTS.md`, `workflow-bench/*-prompt.md`, and files under `agents/`, `principles/`, `rules/` or `skills/`.
**Plan rows.** A `todo.md` row or detail that the chunk files from a finding, re-scores or moves between releases gets no review: the chunk that ships the item reads it.
+3 -3
View File
@@ -41,7 +41,7 @@
| 102 | 0.7.0 | | | **Give loop-claude's retry delay, and the file it reads the machine's memory from, environment overrides, so the black-box test stops replacing `time.sleep` and `machine_load.memory.MEMINFO` through `sitecustomize`.** | 1 | 1 | 2 | 2 | 3 | 1.1 by 4 | 4.2 |
| 135 | 0.7.0 | | | **Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop.** | 2 | 2 | 3 | 5 | 3 | 1.1 by 4 | 4.2 |
| 171 | 0.7.0 | | decision | **Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 3, 1.4 by 3 | 4.0 |
| 196 | 0.7.0 | | decision | **Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and add any rung a run qualifies to `REPLAYED`.** | 2 | 2 | 3 | 5 | 2 | 1.4 by 3 | 4.0 |
| 196 | 0.7.0 | | decision | **Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.** | 2 | 2 | 3 | 5 | 2 | 1.4 by 3 | 4.0 |
| 68 | 0.7.0 | | principle | **Test `workflow-bench/compare.py`'s blinding and `bench.py`'s run plan.** | 2 | 1 | 1 | 2 | 4 | 1.1 by 5 | 3.5 |
| 181 | 0.7.0 | | principle | **Have the test of a quoted answer's step-up assert the stepped rung's facts as literals, not through `prompts.rungs_held`.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 3.4 |
| 192 | 0.7.0 | | principle | **Sort `.gitignore`'s allowlist block, from `!.claude/` to `!workflow-bench/test_review_git.py`, by code point, so rule-hook's `ignorefile-sorted` checks later edits to it again.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 3.4 |
@@ -203,9 +203,9 @@ The maintainability senior and the inherited architect of PR #257's scoring run
Raised by the goals audit of PR #296. `bin/routing_figures/measure.py` groups a chunk's review rounds by the rung its worker started on, and calls for a routing change when a lower rung took more of them. Since PR #296 the reviewers may run above the worker's rung, on the one their item's E + I picks, and no figure groups rounds by it, though the archived worker line names the E + I it follows from. A cheap worker on an item of high E + I gets stronger reviewers, who may raise more rounds, and a call could then blame the worker's rung for them: an interrupt of lilleman that settles nothing (goal 1.2).
### 196. Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and add any rung a run qualifies to `REPLAYED`.
### 196. Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.
Item 159's chunk, PR #322, replayed only the prose and stability passes. Sonnet 5.5 at high missed 44 to 55% of what the project's rung found again in each, so the agent judged a replay of the other four unlikely to qualify a rung. The product-owner review of PR #322 asked that the plan keep them, so the ranking decides when they run. Each run costs hours of reviewer replays (goal 1.4), so the score keeps it low until a cheaper model joins the ladder. `docs/decisions.md` → `routing: a review pass runs below the project's rung only on a rung that a replay of its rounds shows finding every 🔴 and 🟡 the project's rung finds…` records the runs so far.
Item 159's chunk, PR #322, replayed only the prose and stability passes. Sonnet 5.5 at high missed 44 to 55% of what the project's rung found again in each, so the agent judged a replay of the other four unlikely to qualify a rung. The product-owner review of PR #322 asked that the plan keep them, so the ranking decides when they run. Each run costs hours of reviewer replays (goal 1.4), so the score keeps it low until a cheaper model joins the ladder. `docs/decisions.md` → `routing: a review pass runs below the project's rung only on a rung that a replay of its rounds shows finding every 🔴 and 🟡 the project's rung finds…` records the runs so far. The release review of 0.6.0 deleted the code that lowered a pass, since no rung had qualified; the decision says where it goes back.
### 68. Test `workflow-bench/compare.py`'s blinding and `bench.py`'s run plan.