Answer architecture round 1: 2 🔴, 4 🟡 (0 misread); one branch_chunks reader with a pure fix rule, routed chunks matched by item

This commit is contained in:
2026-10-28 21:22:25 +01:00
parent 1a81b55925
commit a16f0e4625
22 changed files with 639 additions and 595 deletions
+5 -3
View File
@@ -18,6 +18,8 @@
!agents/
!agents/*.md
!bin/
!bin/branch_chunks/
!bin/branch_chunks/*.py
!bin/chunk-preflight
!bin/chunk_pick.py
!bin/chunk_preflight/
@@ -61,18 +63,18 @@
!bin/review_replay/
!bin/review_replay/*.py
!bin/round_titles.py
!bin/rule-hook
!bin/routing_figures/
!bin/routing_figures/*.py
!bin/rule-hook
!bin/rule_hook/
!bin/rule_hook/*.py
!bin/rung_replay/
!bin/rung_replay/*.py
!bin/run-heavy
!bin/run_heavy/
!bin/run_heavy/*.py
!bin/run_heavy/path_entry/
!bin/run_heavy/path_entry/docker
!bin/rung_replay/
!bin/rung_replay/*.py
!bin/score-outcomes
!bin/score_outcomes/
!bin/score_outcomes/*.py
+1 -1
View File
@@ -125,7 +125,7 @@ pytest_add_cli_args=-k
Then, in the copy: `uvx --with pytest==9.2.0 'mutmut>=3,<4' run && uvx --with pytest==9.2.0 'mutmut>=3,<4' results`, and `uvx --with pytest==9.2.0 'mutmut>=3,<4' show <mutant>` to see a mutant's change. The config keys are mutmut 3's.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py`, `settings_filter.py` and `statusline.py` are tools of one file each, tested beside them, `rule_hook` imports `rank_todo.py`'s check, `larv_review` imports `rank_todo.py`'s ranking, `score_outcomes` imports `reading_cost`'s record path, `routing_figures` imports `score_outcomes`' later fixes, `judge_replay` imports `loop_claude`'s judge, and `loop_claude` runs `chunk-preflight` as a process of its own. `judge_replay` and `review_replay` share `rung_replay/`: the rung search, the argument reader, the candidate rungs and the exits. `rung_replay/` reads `loop_claude`'s retry count and models file. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool; `model_routing/` holds the routing ladder, which loop-claude, `rule_hook`, both replay tools and `subagent-rung` read. `routing_figures/` measures loop-claude's archive, and each of its projects' default branches, for loop-claude's routing calls. `machine_load/` takes the machine's heavy slots, its shared lock files, for `run_heavy`, and reads the memory share that loop-claude waits for. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py`, `settings_filter.py` and `statusline.py` are tools of one file each, tested beside them, `rule_hook` imports `rank_todo.py`'s check, `larv_review` imports `rank_todo.py`'s ranking, `branch_chunks` imports `reading_cost`'s record path, `judge_replay` imports `loop_claude`'s judge, and `loop_claude` runs `chunk-preflight` as a process of its own. `judge_replay` and `review_replay` share `rung_replay/`: the rung search, the argument reader, the candidate rungs and the exits. `rung_replay/` reads `loop_claude`'s retry count and models file. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool; `model_routing/` holds the routing ladder, which loop-claude, `rule_hook`, both replay tools and `subagent-rung` read. `branch_chunks/` cuts a default branch into chunks, with the items each shipped and the later chunks that fixed it, for `score_outcomes` and `routing_figures`. `routing_figures/` measures loop-claude's archive, and each of its projects' default branches, for loop-claude's routing calls. `machine_load/` takes the machine's heavy slots, its shared lock files, for `run_heavy`, and reads the memory share that loop-claude waits for. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
## Tracking a new file
View File
+137
View File
@@ -0,0 +1,137 @@
import dataclasses
import typing
from comprehension_history import HISTORY
from todo_table import Item, items
from reading_cost.figures import RECORD
from utc_stamp import utc
# The plan, and the records tools append to: none of their lines are the chunk's work.
NOT_WORK = ("todo.md", HISTORY, RECORD)
# Each commit as its hash, commit and author times and parents, its short hash and title, then the files it changed; a merge's are those it changed against its first parent.
LOG = ("log", "--format=%x1e%H %ct %at %P%x1f%h%x1f%s", "--name-only", "--diff-merges=first-parent")
# How a chunk reached the default branch; rebase and push leave no merge commit to bound it.
Landing = typing.Literal["merge", "rebase", "push"]
@dataclasses.dataclass(frozen=True)
class Commit:
"""A commit as LOG prints it; committed and authored are its commit and author times, in seconds since the epoch."""
hash: str
short: str
committed: int
authored: int
parents: tuple[str, ...]
title: str
files: tuple[str, ...]
@dataclasses.dataclass(frozen=True)
class BranchChunk:
"""A chunk on a default branch. Its change runs from before, None for the branch's first chunk, to tip, its newest commit on the first-parent line. label names it as the reports do, landed says how it reached the branch, and merged is tip's commit time. titles are those of every commit it brought; shipped holds each todo.md item its change removed, as scored before it; fixed_by names the later chunks that fixed code it shipped, newest first."""
before: str | None
tip: str
label: str
landed: Landing
merged: str
titles: tuple[str, ...]
shipped: tuple[Item, ...]
fixed_by: tuple[str, ...]
@dataclasses.dataclass(frozen=True)
class Branch:
"""A default branch's commits by hash, its chunks newest first, and each commit's chunk as an index into chunks."""
commits: dict[str, Commit]
chunks: list[BranchChunk]
owner: dict[str, int]
def cut_branch(log: str) -> Branch:
"""The branch LOG printed, cut into chunks as chunk_log cuts its first-parent line; a commit belongs to the oldest chunk whose commits reach it."""
commits = parsed(log)
# git log prints the commit it starts from first.
line = [commits[next(iter(commits))]]
while line[-1].parents:
line.append(commits[line[-1].parents[0]])
runs = chunk_log(line)
owner = owners(runs, commits)
titles: list[list[str]] = [[] for _ in runs]
for commit in commits.values():
titles[owner[commit.hash]].append(commit.title)
chunks = [BranchChunk(run[-1].parents[0] if run[-1].parents else None, run[0].hash, label(run[-1].short, run[0].short), landing(run), utc(run[0].committed), tuple(titles[index]), (), ()) for index, run in enumerate(runs)]
return Branch(commits, chunks, owner)
def parsed(log: str) -> dict[str, Commit]:
"""Each commit LOG printed, by hash, in the order it printed them."""
commits = {}
for record in log.split("\x1e")[1:]:
header, short, rest = record.split("\x1f", 2)
full, committed, authored, *parents = header.split()
title, *files = rest.split("\n")
commits[full] = Commit(full, short, int(committed), int(authored), tuple(parents), title, tuple(name for name in files if name))
return commits
def chunk_log(line: list[Commit]) -> list[list[Commit]]:
"""The first-parent line, newest first, cut into chunks, newest first, each holding its commits newest first. A merge is a chunk of its own; any other commit joins the chunk of the newer commit above it when that is no merge and both share a commit time."""
runs: list[list[Commit]] = []
for commit in line:
if runs and len(commit.parents) < 2 and len(runs[-1][-1].parents) < 2 and runs[-1][-1].committed == commit.committed:
runs[-1].append(commit)
else:
runs.append([commit])
return runs
def landing(run: list[Commit]) -> Landing:
"""How a chunk chunk_log cut reached the branch: by a merge commit, by a rebase that gave a commit a commit time other than its author time, or by commits never replayed."""
if len(run[0].parents) > 1:
return "merge"
return "rebase" if any(commit.committed != commit.authored for commit in run) else "push"
def label(first: str, last: str) -> str:
"""How the report names a chunk: its one commit's short hash, or its first and last commits' as a range."""
return first if first == last else f"{first}^..{last}"
def owners(runs: list[list[Commit]], commits: dict[str, Commit]) -> dict[str, int]:
"""Each commit's chunk, as its index in runs: the oldest chunk whose commits reach it."""
owner: dict[str, int] = {}
for index in reversed(range(len(runs))):
for commit in reversed(runs[index]):
unowned = [commit.hash]
while unowned:
found = unowned.pop()
if found not in owner:
owner[found] = index
unowned += commits[found].parents
return owner
def shipped(before: str, after: str) -> list[Item]:
"""Each item the todo.md before a change holds and the todo.md after it does not, as scored before; none where either holds no items table it can read."""
planned, left = items(before), items(after)
if not isinstance(planned, list) or not isinstance(left, list):
return []
kept = {item.id for item in left}
return sorted((item for item in planned if item.id not in kept), key=lambda item: item.id)
def plan_revisions(branch: Branch) -> list[str]:
"""The revisions whose todo.md with_shipped needs: the commit before and the tip of each chunk, but the branch's first, that brought a commit changing todo.md."""
changed = sorted({branch.owner[commit.hash] for commit in branch.commits.values() if "todo.md" in commit.files})
revisions: dict[str, None] = {}
for index in changed:
before = branch.chunks[index].before
if before is not None:
revisions.update(dict.fromkeys((before, branch.chunks[index].tip)))
return list(revisions)
def with_shipped(branch: Branch, plans: dict[str, str]) -> Branch:
"""The branch with each chunk's shipped items, read from todo.md's text at each revision plan_revisions named, as plans holds it; a chunk with a revision plans lacks changed no plan."""
chunks = [dataclasses.replace(chunk, shipped=tuple(shipped(plans[chunk.before], plans[chunk.tip]))) if chunk.before is not None and chunk.before in plans and chunk.tip in plans else chunk for chunk in branch.chunks]
return dataclasses.replace(branch, chunks=chunks)
+60
View File
@@ -0,0 +1,60 @@
"""The later chunks that fixed code each chunk on a default branch shipped: its escaped defects.
A later chunk fixed code an earlier chunk shipped where one of its changes removes or rewrites lines that `git blame` traces to a commit the earlier chunk brought. Two kinds of change count: the chunk's whole change, where it ships a todo.md item marked `defect` or `security`; and a round commit of the chunk answering a 🔴 that a reviewer marked `[misread]`. Each later chunk counts at most once against each earlier chunk."""
import dataclasses
import re
from branch_chunks.cut import Branch, BranchChunk
from round_titles import title_counts
# The Exempt cells, per ~/.claude/rules/todo-items.md, that mark an item fixing a defect.
FIXES = ("defect", "security")
HUNK = re.compile(r"@@ -([0-9]+)(?:,([0-9]+))? \+[0-9]+(?:,([0-9]+))? @@")
BLAME_HEADER = re.compile(r"([0-9a-f]{40}|[0-9a-f]{64}) [0-9]+ [0-9]+")
def fixing_changes(branch: Branch) -> list[tuple[int, str, str]]:
"""Each change that fixes a defect, as its chunk's index, the commit before it and its tip: the whole change of each chunk that ships a FIXES item, and each round commit answering a 🔴 a reviewer marked `[misread]`."""
changes = [(index, chunk.before, chunk.tip) for index, chunk in enumerate(branch.chunks) if chunk.before is not None and any(item.exempt in FIXES for item in chunk.shipped)]
for commit in branch.commits.values():
counts = title_counts(commit.title)
# A title counts misread 🔴 and 🟡 findings together, so it takes one with both a 🔴 and a misread finding to answer a misread 🔴.
if counts and counts.red and counts.misread and commit.parents:
changes.append((branch.owner[commit.hash], commit.parents[0], commit.hash))
return changes
def with_fixes(branch: Branch, traced: list[tuple[int, set[str]]]) -> list[BranchChunk]:
"""The branch's chunks, each naming the later chunks that fixed code it shipped, from the commits each fixing change traced to, paired with its chunk's index."""
fixed: dict[int, set[int]] = {}
for index, commits in traced:
for commit in commits:
# A commit no chunk owns cannot be one before can reach; take it as the fixing chunk's own.
if branch.owner.get(commit, index) > index:
fixed.setdefault(branch.owner[commit], set()).add(index)
return [dataclasses.replace(chunk, fixed_by=tuple(branch.chunks[later].label for later in sorted(fixed.get(index, ())))) for index, chunk in enumerate(branch.chunks)]
def removed(diff: str) -> dict[str, list[tuple[int, int]]]:
"""The old lines each hunk of a `git diff -U0` removes or rewrites, as its first line and count, by the path the file had before; none for a path git quotes, as it does one holding a byte it cannot print."""
spans: dict[str, list[tuple[int, int]]] = {}
path, left = None, 0
for row in diff.splitlines():
if left and row[:1] in ("-", "+", " "):
left -= 1
elif row.startswith("diff --git "):
path, left = None, 0
elif row.startswith("--- "):
# Git ends a path holding a space with a tab.
path = row[len("--- a/"):].removesuffix("\t") if row.startswith("--- a/") else None
elif hunk := HUNK.match(row):
start, count = int(hunk.group(1)), int(hunk.group(2) or 1)
left = count + int(hunk.group(3) or 1)
if path and count:
spans.setdefault(path, []).append((start, count))
return spans
def blamed(porcelain: str) -> set[str]:
"""The commit each line of `git blame --porcelain` output comes from."""
return {header.group(1) for row in porcelain.splitlines() if (header := BLAME_HEADER.match(row))}
+66
View File
@@ -0,0 +1,66 @@
import typing
from branch_chunks.fixes import blamed, fixing_changes, removed, with_fixes
from branch_chunks.cut import BranchChunk, cut_branch, LOG, NOT_WORK, plan_revisions, with_shipped
from failure import Failure
# Runs git with the given arguments in the checkout's top: blame and a pathspec read a path from the current directory, and the diff names each from the top.
Run = typing.Callable[..., str | Failure]
def branch_chunks(default: str, run: Run) -> list[BranchChunk] | Failure:
"""Each chunk on the default branch's first-parent line, newest first, with the todo.md items it shipped and the later chunks that fixed code it shipped."""
# The -- keeps a file named like the branch from making it ambiguous.
log = run(*LOG, default, "--")
if isinstance(log, Failure):
return log
branch = cut_branch(log)
plans = plans_at(plan_revisions(branch), run)
if isinstance(plans, Failure):
return plans
branch = with_shipped(branch, plans)
traced = traces(fixing_changes(branch), run)
return traced if isinstance(traced, Failure) else with_fixes(branch, traced)
def plans_at(revisions: list[str], run: Run) -> dict[str, str] | Failure:
"""todo.md's text at each revision, empty where the revision holds none."""
plans = {}
for revision in revisions:
listed = run("ls-tree", "--name-only", revision, "--", "todo.md")
if isinstance(listed, Failure):
return listed
plan = run("show", f"{revision}:todo.md") if listed.strip() else ""
if isinstance(plan, Failure):
return plan
plans[revision] = plan
return plans
def traces(changes: list[tuple[int, str, str]], run: Run) -> list[tuple[int, set[str]]] | Failure:
"""The commits each change traced_to traces, paired with the change's chunk's index."""
found = []
for index, before, tip in changes:
traced = traced_to(before, tip, run)
if isinstance(traced, Failure):
return traced
found.append((index, traced))
return found
def traced_to(before: str, tip: str, run: Run) -> set[str] | Failure:
"""The commits that last changed each line the change from before to tip removes or rewrites, outside the plan and the records."""
# The prefixes and options keep the checkout's diff config out of the output.
diff = run("diff", "-U0", "-M", "--no-color", "--no-ext-diff", "--no-textconv", "--ignore-submodules", "--src-prefix=a/", "--dst-prefix=b/", before, tip)
if isinstance(diff, Failure):
return diff
traced: set[str] = set()
for path, spans in removed(diff).items():
if path in NOT_WORK:
continue
# An empty --ignore-revs-file drops a configured one, which blame fails on where the file is missing.
blame = run("blame", "--porcelain", "--no-textconv", "--ignore-revs-file=", *(f"-L{start},+{count}" for start, count in spans), before, "--", path)
if isinstance(blame, Failure):
return blame
traced |= blamed(blame)
return traced
+221
View File
@@ -0,0 +1,221 @@
import functools
import os
import pathlib
import subprocess
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from branch_chunks.fixes import blamed, fixing_changes, removed, with_fixes
from branch_chunks.cut import Branch, BranchChunk, chunk_log, Commit, landing, shipped
from branch_chunks.read import branch_chunks, traced_to
from failure import Failure
from git_repo import git
from test_env import isolated_env
from todo_table import Item
HEADER = "| ID | Release | Exempt | Item | R | S | C | A | G | Goals | Score |\n|---|---|---|---|---|---|---|---|---|---|---|\n"
WITHOUT_C = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |\n|---|---|---|---|---|---|---|---|---|---|\n"
FIRST = "| 4 | 0.1.0 | | **Do it.** | 2 | 3 | 10 | 6 | 7 | 1 | 9.0 |"
SECOND = "| 5 | 0.1.0 | | **Then this.** | 4 | 6 | 8 | 6 | 7 | 1 | 9.0 |"
FIX = "| 6 | 0.1.0 | defect | **Fix it.** | 1 | 1 | 1 | 6 | 7 | 1 | 9.0 |"
def todo(*rows: str, header: str = HEADER) -> str:
return "# todo\n\n## Items\n\n" + header + "".join(f"{row}\n" for row in rows)
def item(id_: int, exempt: str = "") -> Item:
return Item(id_, (0, 1, 0), exempt, f"**Item {id_}.**", {}, (), ())
def chunk(label: str, shipped: tuple[Item, ...] = ()) -> BranchChunk:
return BranchChunk(f"{label}-before", f"{label}-tip", label, "merge", "2026-10-13T10:00:00Z", (), shipped, ())
def commit(hash_: str, committed: int, authored: int, *parents: str, title: str = "") -> Commit:
return Commit(hash_, hash_, committed, authored, parents, title, ())
def cut(*line: Commit) -> list[tuple[str, str, int, str]]:
"""Each chunk chunk_log cuts the first-parent line into: its oldest and newest commit, its commit time and how it landed."""
return [(run[-1].hash, run[0].hash, run[0].committed, landing(run)) for run in chunk_log(list(line))]
class Shipped(unittest.TestCase):
def test_ships_each_item_the_change_removed_with_the_scores_it_had_before(self):
self.assertEqual([found.id for found in shipped(todo(FIRST, SECOND), todo(SECOND))], [4])
self.assertEqual(shipped(todo(FIRST, SECOND), todo(SECOND))[0].scores, {"R": 2, "S": 3, "C": 10, "A": 6, "G": 7})
def test_lists_the_shipped_items_by_id(self):
self.assertEqual([found.id for found in shipped(todo(SECOND, FIRST), todo())], [4, 5])
def test_reads_a_table_from_before_c_joined_it(self):
older = "| 4 | 0.1.0 | | **A \\| B.** | 2 | 3 | 6 | 7 | 1 | 9.0 |"
self.assertEqual(shipped(todo(older, header=WITHOUT_C), todo(header=WITHOUT_C))[0].scores, {"R": 2, "S": 3, "A": 6, "G": 7})
def test_ships_nothing_where_either_plan_has_no_table_it_can_read(self):
self.assertEqual(shipped(todo(FIRST), "# todo\n\n- [ ] Do it.\n"), [])
self.assertEqual(shipped(todo(FIRST, "| x | 0.1.0 | | **Bad.** | 2 | 3 | 5 | 6 | 7 | 1 | 9.0 |"), todo()), [])
class ChunkLog(unittest.TestCase):
def test_a_merge_is_a_chunk_of_its_own_and_joins_no_run_below_it(self):
self.assertEqual(cut(commit("m", 100, 100, "a", "b"), commit("c", 100, 100, "a")), [("m", "m", 100, "merge"), ("c", "c", 100, "push")])
def test_commits_sharing_a_commit_time_form_one_run_labelled_rebase_when_any_differs(self):
self.assertEqual(cut(commit("c2", 200, 150, "c1"), commit("c1", 200, 100, "r")), [("c1", "c2", 200, "rebase")])
def test_a_run_of_commits_that_all_kept_their_author_time_is_labelled_push(self):
self.assertEqual(cut(commit("c2", 200, 200, "c1"), commit("c1", 200, 200, "r")), [("c1", "c2", 200, "push")])
def test_a_run_is_rebase_when_only_its_older_commit_was_replayed(self):
self.assertEqual(cut(commit("c2", 200, 200, "c1"), commit("c1", 200, 100, "r")), [("c1", "c2", 200, "rebase")])
def test_a_run_is_rebase_when_only_its_newer_commit_was_replayed(self):
self.assertEqual(cut(commit("c2", 200, 100, "c1"), commit("c1", 200, 200, "r")), [("c1", "c2", 200, "rebase")])
def test_a_commit_at_another_time_starts_a_new_chunk(self):
self.assertEqual(cut(commit("c2", 300, 300, "c1"), commit("c1", 200, 200, "r")), [("c2", "c2", 300, "push"), ("c1", "c1", 200, "push")])
class FixingChanges(unittest.TestCase):
def test_takes_the_whole_change_of_each_chunk_that_ships_a_defect_or_a_security_item(self):
branch = Branch({}, [chunk("d", (item(9, "security"),)), chunk("c", (item(8),)), chunk("b", (item(6, "defect"), item(7))), chunk("a")], {})
self.assertEqual(fixing_changes(branch), [(0, "d-before", "d-tip"), (2, "b-before", "b-tip")])
def test_takes_each_round_commit_whose_title_counts_both_a_red_and_a_misread_finding(self):
titles = {
"r1": "Answer stability round 1: 1 🔴, 0 🟡 (1 misread); keep y's caller",
"r2": "Answer stability round 2: 1 🔴, 0 🟡 (0 misread); fix",
"r3": "Answer prose round 1: 0 🔴, 1 🟡 (1 misread); fix",
"r4": "Answer principles round 1: 1 🔴, 0 🟡; fix",
"r5": "Do the work",
}
commits = {name: commit(name, 100, 100, f"{name}-parent", title=title) for name, title in titles.items()}
branch = Branch(commits, [chunk("b"), chunk("a")], {"r1": 1, "r2": 0, "r3": 0, "r4": 0, "r5": 0})
self.assertEqual(fixing_changes(branch), [(1, "r1-parent", "r1")])
class WithFixes(unittest.TestCase):
def test_names_each_later_chunk_once_against_each_chunk_whose_commits_its_fixes_traced_to_newest_first(self):
branch = Branch({}, [chunk("d"), chunk("c"), chunk("b"), chunk("a")], {"a1": 3, "a2": 3, "b1": 2, "c1": 1, "d1": 0})
found = with_fixes(branch, [(1, {"a1", "a2"}), (0, {"a1", "b1", "c1"}), (1, {"a2"}), (0, {"a2"})])
self.assertEqual([found.fixed_by for found in found], [(), ("d",), ("d",), ("d", "c")])
def test_counts_no_fix_of_its_own_chunks_commit_a_newer_chunks_or_one_no_chunk_brought(self):
branch = Branch({}, [chunk("b"), chunk("a")], {"b1": 0, "a1": 1})
self.assertEqual([found.fixed_by for found in with_fixes(branch, [(1, {"a1", "b1", "elsewhere"})])], [(), ()])
class Removed(unittest.TestCase):
def test_reads_the_old_lines_each_hunk_removes_or_rewrites_by_the_path_they_had(self):
diff = "\n".join([
"diff --git a/a.py b/a.py",
"--- a/a.py",
"+++ b/a.py",
"@@ -2 +2 @@ def f():",
"-x",
"+y",
"@@ -5,3 +4,0 @@",
"@@ -9,0 +8,2 @@",
"diff --git a/old.py b/new.py",
"--- a/old.py",
"+++ b/new.py",
"@@ -1,2 +1,2 @@",
"diff --git a/gone.py b/gone.py",
"--- a/gone.py",
"+++ /dev/null",
"@@ -1,4 +0,0 @@",
"diff --git a/born.py b/born.py",
"--- /dev/null",
"+++ b/born.py",
"@@ -0,0 +1,3 @@",
"diff --git \"a/tab\\there.py\" \"b/tab\\there.py\"",
"--- \"a/tab\\there.py\"",
"+++ \"b/tab\\there.py\"",
"@@ -1 +1 @@",
"diff --git a/my file.py b/my file.py",
"--- a/my file.py\t",
"+++ b/my file.py\t",
"@@ -7,2 +7 @@",
])
self.assertEqual(removed(diff), {"a.py": [(2, 1), (5, 3)], "old.py": [(1, 2)], "gone.py": [(1, 4)], "my file.py": [(7, 2)]})
def test_reads_a_hunk_line_inside_the_changed_text_as_text(self):
diff = "--- a/a.md\n+++ b/a.md\n@@ -3 +3 @@\n---- a/quoted\n+text\n"
self.assertEqual(removed(diff), {"a.md": [(3, 1)]})
class Blamed(unittest.TestCase):
def test_reads_the_commit_of_each_line_from_porcelain_output(self):
first, second = "a" * 40, "b" * 40
porcelain = f"{first} 2 2 1\nauthor t\nfilename a.py\n\tx\n{second} 5 5 2\nauthor t\n\ty\n{second} 6 6\n\t{first} 1 1 1\n"
self.assertEqual(blamed(porcelain), {first, second})
class TracedTo(unittest.TestCase):
DIFF = "--- a/a.py\n+++ b/a.py\n@@ -2 +2 @@\n-x\n+y\n--- a/todo.md\n+++ b/todo.md\n@@ -5 +4,0 @@\n-| 6 |\n--- a/docs/reading-cost.md\n+++ b/docs/reading-cost.md\n@@ -1 +1 @@\n-a\n+b\n"
def test_blames_the_lines_a_change_removes_before_it_leaving_out_the_plan_and_the_records(self):
blames = []
def run(*args):
if args[0] == "diff":
return self.DIFF
blames.append(args)
return f"{'a' * 40} 2 2 1\n\tx\n"
self.assertEqual(traced_to("before", "tip", run), {"a" * 40})
self.assertEqual([(args[-1], "-L2,+1" in args, "before" in args) for args in blames], [("a.py", True, True)])
def test_fails_where_git_cannot_blame(self):
self.assertEqual(traced_to("before", "tip", lambda *args: self.DIFF if args[0] == "diff" else Failure("no blame")), Failure("no blame"))
class BranchChunks(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.checkout = pathlib.Path(os.path.realpath(temp.name))
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.checkout)}
self.git("init", "-q", "-b", "main")
self.commit("Start", {"todo.md": todo(FIRST, SECOND, FIX), "a.py": "1\n"})
def git(self, *args):
return subprocess.run(["git", "-c", "user.name=t", "-c", "user.email=t@t", *args], cwd=self.checkout, check=True, env=self.env, capture_output=True, text=True).stdout
def commit(self, title, files):
for name, text in files.items():
(self.checkout / name).write_text(text)
self.git("add", name)
self.git("commit", "-q", "-m", title)
def merge(self, branch, title, files, at):
"""Merges a branch holding one commit, at the given time, and returns the merge commit's short hash."""
self.git("switch", "-q", "-c", branch)
self.commit(title, files)
self.git("switch", "-q", "main")
self.env.update({"GIT_COMMITTER_DATE": at, "GIT_AUTHOR_DATE": at})
self.git("merge", "-q", "--no-ff", "-m", f"Merge {branch}", branch)
for name in ("GIT_COMMITTER_DATE", "GIT_AUTHOR_DATE"):
self.env.pop(name)
return self.git("rev-parse", "--short", "HEAD").strip()
def test_reads_each_chunk_with_the_items_it_shipped_and_the_later_chunks_that_fixed_it(self):
shipped = self.merge("four", "Do four", {"b.py": "x\ny\n", "todo.md": todo(SECOND, FIX)}, "2026-10-13T10:00:00Z")
fixed = self.merge("six", "Fix six", {"b.py": "x\nY\n", "todo.md": todo(SECOND)}, "2026-10-14T10:00:00Z")
found = branch_chunks("main", functools.partial(git, cwd=str(self.checkout)))
self.assertNotIsInstance(found, Failure)
self.assertEqual([(found.label, found.landed, found.merged, [item.id for item in found.shipped], found.fixed_by) for found in found[:2]], [(fixed, "merge", "2026-10-14T10:00:00Z", [6], ()), (shipped, "merge", "2026-10-13T10:00:00Z", [4], (fixed,))])
self.assertEqual(sorted(found[1].titles), ["Do four", "Merge four"])
self.assertEqual((found[1].tip, found[2].tip), (self.git("rev-parse", "HEAD^").strip(), self.git("rev-parse", "HEAD^^").strip()))
self.assertEqual((found[1].before, found[2].before), (found[2].tip, None))
def test_fails_where_git_cannot_read_the_branch(self):
self.assertEqual(branch_chunks("main", lambda *args: Failure("no log")), Failure("no log"))
if __name__ == "__main__":
unittest.main()
+17 -10
View File
@@ -4,6 +4,7 @@ Every tool in bin/ starts git through `git` here."""
import os
import re
import subprocess
import typing
from failure import Failure
@@ -42,6 +43,12 @@ def git(*args: str, cwd: str | None = None, timeout: float | None = None, guarde
return ran.stdout if ran.returncode == 0 else Failure(f"git {command}: {ran.stderr.strip()}")
class Git(typing.Protocol):
"""Runs git as `git` does."""
def __call__(self, *args: str, cwd: str | None = None, timeout: float | None = None, guarded: bool = False) -> str | Failure: ...
def top() -> str | Failure:
found = git("rev-parse", "--show-toplevel")
return found if isinstance(found, Failure) else found.strip()
@@ -60,7 +67,7 @@ def default_branch(stale_ok: bool, timeout: float | None = None) -> tuple[str, s
return ref if isinstance(ref, Failure) else (ref, note)
def default_ref(cwd: str | None = None, timeout: float | None = None, guarded: bool = False) -> str | Failure:
def default_ref(cwd: str | None = None, timeout: float | None = None, guarded: bool = False, git: Git = git) -> str | Failure:
"""The default branch's ref as last fetched, read in cwd, else in the current directory: `origin/<branch>` with that remote, else the local main or master.
Where the checkout records no default branch for origin, it asks origin, which fails after timeout seconds. Guarded, it runs git as `git` does with guarded, and fails there instead of asking origin."""
@@ -71,7 +78,7 @@ def default_ref(cwd: str | None = None, timeout: float | None = None, guarded: b
recorded = git("symbolic-ref", "--short", "refs/remotes/origin/HEAD", cwd=cwd, guarded=guarded)
if not isinstance(recorded, Failure):
return recorded.strip()
return Failure("origin records no default branch") if guarded else origin_head(cwd, timeout)
return Failure("origin records no default branch") if guarded else origin_head(cwd, timeout, git)
for branch in ("main", "master"):
if not isinstance(git("rev-parse", "--quiet", "--verify", f"refs/heads/{branch}", cwd=cwd, guarded=guarded), Failure):
return branch
@@ -86,7 +93,7 @@ def branch_titles(default: str, cwd: str | None = None, timeout: float | None =
return titles if isinstance(titles, Failure) else titles.splitlines()
def origin_head(cwd: str | None, timeout: float | None) -> str | Failure:
def origin_head(cwd: str | None, timeout: float | None, git: Git) -> str | Failure:
"""The default branch origin names, for a clone made by `git init` and `git remote add`, which records none."""
listed = git("ls-remote", "--symref", "origin", "HEAD", cwd=cwd, timeout=timeout)
if isinstance(listed, Failure):
@@ -109,18 +116,18 @@ def commits_since(tag: str | None, branch: str, cwd: str | None = None) -> int |
return count if isinstance(count, Failure) else int(count)
def commit_exists(commit: str, cwd: str | None = None, timeout: float | None = None, guarded: bool = False) -> bool:
return not isinstance(git("rev-parse", "--quiet", "--verify", f"{commit}^{{commit}}", cwd=cwd, timeout=timeout, guarded=guarded), Failure)
def commit_exists(commit: str, cwd: str | None = None) -> bool:
return not isinstance(git("rev-parse", "--quiet", "--verify", f"{commit}^{{commit}}", cwd=cwd), Failure)
def file_at(revision: str, path: str, cwd: str | None = None, timeout: float | None = None, guarded: bool = False) -> str | Failure:
"""A file's text at a revision, read in cwd, else in the current directory, each git call run as `git` runs it with timeout and guarded; empty where the revision is not a commit, such as a root commit's parent, or holds no such file."""
if not commit_exists(revision, cwd, timeout, guarded):
def file_at(revision: str, path: str, cwd: str | None = None) -> str | Failure:
"""A file's text at a revision, read in cwd, else in the current directory; empty where the revision is not a commit, such as a root commit's parent, or holds no such file."""
if not commit_exists(revision, cwd):
return ""
listed = git("ls-tree", "--name-only", revision, "--", path, cwd=cwd, timeout=timeout, guarded=guarded)
listed = git("ls-tree", "--name-only", revision, "--", path, cwd=cwd)
if isinstance(listed, Failure):
return listed
return git("show", f"{revision}:{path}", cwd=cwd, timeout=timeout, guarded=guarded) if listed.strip() else ""
return git("show", f"{revision}:{path}", cwd=cwd) if listed.strip() else ""
def full_history() -> Failure | None:
+1 -8
View File
@@ -6,7 +6,6 @@ import termios
import time
import typing
from failure import Failure
import git_repo
from machine_load.memory import Memory, read_memory
@@ -14,12 +13,6 @@ from machine_load.memory import Memory, read_memory
PYTHON_TIMEOUT = 600
class Git(typing.Protocol):
"""Runs git in cwd, as git_repo.git does."""
def __call__(self, *args: str, cwd: str) -> str | Failure: ...
class Streaming(typing.Protocol):
"""A started claude whose stream-json lines arrive as it works."""
@@ -35,7 +28,7 @@ class Effects:
# Drops what was typed on the terminal and not yet read.
discard_typed: typing.Callable[[], None]
execv: typing.Callable[[str, list[str]], None]
git: Git
git: git_repo.Git
input: typing.Callable[[str], str]
# A date and time in the machine's zone, as `2026-10-28 11:46:01`. lilleman chose this spelling and local time over UTC on 2026-10-28, since the human reads these on their own clock.
local_time: typing.Callable[[float], str]
+1 -1
View File
@@ -32,7 +32,7 @@ def new_calls(root: pathlib.Path, paths: Paths, effects: Effects) -> list[str] |
if isinstance(kept, Failure):
return kept
say(f"\n=== cycle {cycle} across loop-claude's archive: measuring the routing figures ===", effects, stdout=True)
measured = archive.measure_archive(root, str(paths.home), kept.since)
measured = archive.measure_archive(root, str(paths.home), kept.since, effects.git)
if isinstance(measured, Failure):
return measured
figures = totals(measured.chunks, measured.skipped, measured.unread)
+18 -9
View File
@@ -49,6 +49,8 @@ CWD = pathlib.Path("/c")
NOW = 1790000000
# How a line said at NOW opens, with FakeEffects' local_time.
STAMP = f"<{NOW}>"
# How git answers in a project's checkout it cannot read, such as one that is gone.
UNREAD = Failure("not a git repository")
def assistant(*parts):
@@ -77,7 +79,7 @@ class FakeEffects:
claude_on_terminal=self.on_terminal_claude,
discard_typed=lambda: self.calls.append(("discard_typed", None)),
execv=self.execv,
git=lambda *args, cwd: self.answer("git", list(args)),
git=lambda *args, cwd=None, timeout=None, guarded=False: self.answer("git", list(args)),
input=lambda prompt: self.answer("input", prompt),
local_time=lambda seconds: f"<{seconds}>",
memory=self.memory,
@@ -714,12 +716,19 @@ class RoutingCheck(unittest.TestCase):
self.assertEqual(self.check(FakeEffects()), (None, ""))
self.assertFalse((self.root / "routing-figures.md").exists())
self.chunks(6, 1, "other")
self.assertEqual(self.check(FakeEffects(on_terminal=False))[0], None)
self.assertEqual(self.check(FakeEffects(on_terminal=False, git=[UNREAD, UNREAD]))[0], None)
self.assertIn("at cycle 21 across", (self.root / "routing-figures.md").read_text())
def test_reads_each_projects_history_through_the_loops_git_and_names_each_it_cannot_read(self):
self.chunks(5, 4)
fake = FakeEffects(on_terminal=False, git=[UNREAD])
self.check(fake)
self.assertEqual(fake.made("git"), [["remote"]])
self.assertIn("## Project histories unread\n\n- log: not a git repository\n", (self.root / "routing-figures.md").read_text())
def test_hands_the_terminal_over_once_for_each_new_call(self):
self.chunks(5, 4)
fake = FakeEffects(on_terminal_claude=[0], git=[""])
fake = FakeEffects(on_terminal_claude=[0], git=[UNREAD, ""])
stop, said = self.check(fake)
self.assertIsNone(stop)
call = "Explore subagents' reports took more of the worker's context than their searches would have."
@@ -730,13 +739,13 @@ class RoutingCheck(unittest.TestCase):
self.assertIn(f"change nothing until they answer:\n- {call}", args[6])
self.assertEqual(fake.terminal_environments, [{}])
(self.root / "routing-figures.md").write_text("Measured x, at cycle 0 across")
self.assertEqual(self.check(FakeEffects())[0], None)
self.assertEqual(self.check(FakeEffects(git=[UNREAD]))[0], None)
self.assertNotIn(call, self.check(FakeEffects())[1])
def test_stamps_the_figures_and_the_calls_raised_when_it_measures(self):
self.chunks(5, 4)
(self.root / "routing-calls-raised.md").write_text("# Routing calls raised\n\n- 2026-10-14T10:02:00Z Reset\n")
self.check(FakeEffects(on_terminal_claude=[0], git=[""]))
self.check(FakeEffects(on_terminal_claude=[0], git=[UNREAD, ""]))
figures = (self.root / "routing-figures.md").read_text()
self.assertIn(f"Measured {utc(NOW)}, at cycle 20 across", figures)
self.assertIn("that finished after the reset of 2026-10-14T10:02:00Z", figures)
@@ -749,11 +758,11 @@ class RoutingCheck(unittest.TestCase):
def test_goes_on_off_a_terminal_and_raises_the_call_again_next_time(self):
self.chunks(5, 4)
stop, said = self.check(FakeEffects(on_terminal=False))
stop, said = self.check(FakeEffects(on_terminal=False, git=[UNREAD]))
self.assertIsNone(stop)
self.assertIn("Not on a terminal: the calls are raised again at the next measurement.", said)
(self.root / "routing-figures.md").write_text("Measured x, at cycle 0 across")
self.assertIn("Not on a terminal", self.check(FakeEffects(on_terminal=False))[1])
self.assertIn("Not on a terminal", self.check(FakeEffects(on_terminal=False, git=[UNREAD]))[1])
def test_leaves_the_measuring_to_a_loop_holding_the_lock(self):
self.chunks(5, 4)
@@ -762,7 +771,7 @@ class RoutingCheck(unittest.TestCase):
def test_stops_when_claude_cannot_start_for_the_calls(self):
self.chunks(5, 4)
stop, _ = self.check(FakeEffects(on_terminal_claude=[OSError(2, "No such file or directory")], git=[""]))
stop, _ = self.check(FakeEffects(on_terminal_claude=[OSError(2, "No such file or directory")], git=[UNREAD, ""]))
self.assertEqual(stop, Stop(1, "could not start claude for the routing calls: No such file or directory"))
@unittest.skipIf(os.geteuid() == 0, "root writes into a read-only directory")
@@ -771,7 +780,7 @@ class RoutingCheck(unittest.TestCase):
(self.root / "routing-figures.lock").touch()
self.root.chmod(0o555)
self.addCleanup(self.root.chmod, 0o755)
stop, said = self.check(FakeEffects())
stop, said = self.check(FakeEffects(git=[UNREAD]))
self.assertIsNone(stop)
self.assertIn("routing figures: could not write", said)
+5
View File
@@ -31,6 +31,11 @@ def round_counts(titles: list[str]) -> dict[str, int]:
return {name: max(highest[name], commits[name]) for name in highest}
def rounds_past_first(titles: list[str]) -> int:
"""Each pass's rounds past its first among the commit titles, as round_counts counts them, summed."""
return sum(max(0, count - 1) for count in round_counts(titles).values())
def lacks_counts(title: str) -> bool:
"""Whether title is a round title without all three counts after its colon, or with more misread findings than 🔴 and 🟡 findings."""
return ROUND.match(title) is not None and ((counts := title_counts(title)) is None or counts.misread is None)
+9 -7
View File
@@ -2,17 +2,19 @@
import contextlib
import dataclasses
import fcntl
import functools
import json
import pathlib
import re
import typing
from transcript_archive.read import archived_outcomes, chunk_outcomes, finished_chunks
from branch_chunks.read import branch_chunks
from branch_chunks.cut import BranchChunk
from project_key import checkout_of
from routing_figures.measure import Chunk, ChunkFigures, measured, MEASURED_AT, Subagent
from git_repo import default_ref
from git_repo import default_ref, Git
from failure import Failure
from score_outcomes.later_fixes import Landed, later_fixes
from claude_code.entries import parse_entry
from file_writes import replace_file
from utc_stamp import STAMPED
@@ -83,7 +85,7 @@ class Measured:
unread: tuple[str, ...] = ()
def measure_archive(root: pathlib.Path, home: str, since: str) -> Measured | Failure:
def measure_archive(root: pathlib.Path, home: str, since: str, git: Git) -> Measured | Failure:
"""The figures of each chunk that finished after since, measured one chunk at a time because the archive outgrows memory. Only an archive that cannot be listed fails."""
listed = archived_outcomes(root)
if isinstance(listed, Failure):
@@ -99,7 +101,7 @@ def measure_archive(root: pathlib.Path, home: str, since: str) -> Measured | Fai
skipped.append(key)
continue
chunks = [chunk for chunk in chunks if chunk.finished > since]
history = branch_history(checkout_of(key, pathlib.Path(home))) if chunks else None
history = branch_history(checkout_of(key, pathlib.Path(home)), git) if chunks else None
if isinstance(history, Failure):
unread.append(f"{key}: {history.reason}")
history = None
@@ -116,10 +118,10 @@ def measure_archive(root: pathlib.Path, home: str, since: str) -> Measured | Fai
return Measured(found, tuple(skipped), tuple(unread))
def branch_history(checkout: str) -> list[Landed] | Failure:
def branch_history(checkout: str, git: Git) -> list[BranchChunk] | Failure:
"""The chunks on the checkout's default branch as last fetched, read guarded, since the checkout's config may name a program."""
ref = default_ref(checkout, GIT_TIMEOUT, guarded=True)
return ref if isinstance(ref, Failure) else later_fixes(ref, cwd=checkout, guarded=True, timeout=GIT_TIMEOUT)
ref = default_ref(checkout, GIT_TIMEOUT, guarded=True, git=git)
return ref if isinstance(ref, Failure) else branch_chunks(ref, functools.partial(git, cwd=checkout, timeout=GIT_TIMEOUT, guarded=True))
def entries(path: pathlib.Path) -> list[dict]:
+14 -19
View File
@@ -6,12 +6,12 @@ import statistics
import typing
from claude_code.entries import assistant_parts, parts, tool_call, tool_result
from branch_chunks.cut import BranchChunk
from git_command import commits
from model_routing.route import ESCALATION_MARKER, route_from
from score_outcomes.later_fixes import Landed
from routing_figures.prices import listed
from model_routing.ladder import position, Rung
from round_titles import round_counts
from round_titles import rounds_past_first
from shell_command import simple_commands
from transcript_archive.outcomes import UNFINISHED_AT, workers
@@ -57,7 +57,7 @@ class Reports:
@dataclasses.dataclass(frozen=True)
class ChunkFigures:
"""dollars maps each of DIMENSIONS to its labels' dollars; start is the rung a routed chunk started on, None where no item routed it; escalated counts the worker's escalations by trigger; unattributed is the recorded cost of a model with no priced message in the chunk; lacking counts the Agent calls, the worker's and the subagents', that have no subagent transcript in the archive; fixed_later counts the later chunks that fixed code it shipped, None where no chunk on its project's default branch matches it."""
"""dollars maps each of DIMENSIONS to its labels' dollars; start is the rung a routed chunk started on, None where no item routed it; escalated counts the worker's escalations by trigger; unattributed is the recorded cost of a model with no priced message in the chunk; lacking counts the Agent calls, the worker's and the subagents', that have no subagent transcript in the archive; fixed_later counts the later chunks that fixed code it shipped, None where no chunk on its project's default branch shipped its item."""
dollars: Dollars
start: Rung | None
rounds_past_first: int
@@ -72,7 +72,7 @@ class ChunkFigures:
@dataclasses.dataclass(frozen=True)
class Figures:
"""escalated counts every chunk's escalations by trigger, routed or not; skipped names each chunk folder the archive could not read whole; unmatched counts the chunks whose later fixes are unknown; unread names each project whose default branch could not be read, with why."""
"""escalated counts every chunk's escalations by trigger, routed or not; skipped names each chunk folder the archive could not read whole; unmatched counts the routed chunks whose later fixes are unknown; unread names each project whose default branch could not be read, with why."""
chunks: int
dollars: Dollars
routed: dict[Rung, list[ChunkFigures]]
@@ -101,7 +101,7 @@ class Spent(typing.NamedTuple):
message: Message
def measured(chunk: Chunk, home: str, history: list[Landed] | None = None) -> ChunkFigures:
def measured(chunk: Chunk, home: str, history: list[BranchChunk] | None = None) -> ChunkFigures:
"""history is the chunks on the project's default branch, None where it could not be read."""
cycles = sorted((cycle for kept in chunk.outcomes for cycle in workers(kept)), key=lambda cycle: cycle[0])
spent = [Spent("worker", str(rung_at(cycles, message.at)), message) for message in messages(chunk.worker)]
@@ -118,29 +118,24 @@ def measured(chunk: Chunk, home: str, history: list[Landed] | None = None) -> Ch
spent += [Spent(kind, rung, message) for message in messages(subagent.entries)]
dollars, unattributed = costed(spent, chunk.project, recorded_costs(chunk.worker))
start = route_from(cycles[0][1]) if cycles else None
titles = commit_titles(chunk.worker, home)
return ChunkFigures(
dollars=dollars,
start=start.rung if start and start.item else None,
rounds_past_first=sum(max(0, rounds - 1) for rounds in round_counts(titles).values()),
rounds_past_first=rounds_past_first(commit_titles(chunk.worker, home)),
unfinished=sum(len(UNFINISHED_AT.findall(kept)) for kept in chunk.outcomes),
delegated=delegated,
reports=reports,
escalated=escalations(launches),
unattributed=unattributed,
lacking=max(0, len(launches) + sum(len(agent_calls(subagent.entries)) for subagent in chunk.subagents) - len(chunk.subagents)),
fixed_later=later_fixes_of(titles, history or []),
fixed_later=later_fixes_of(start.item, history or []) if start and start.item else None,
)
def later_fixes_of(titles: list[str], history: list[Landed]) -> int | None:
"""How many chunks in history fixed code that the chunks holding one of titles shipped, each counted once and none of those chunks itself; None where no chunk holds one. Only a title no other chunk holds counts, since one such as a round's with no findings recurs."""
# A worker's commits may land as several chunks, as a rebase cut by commit time does, and every one of them is its work.
holders = collections.Counter(title for landed in history for title in landed.titles)
shipped = [landed for landed in history if any(holders[title] == 1 for title in landed.titles.intersection(titles))]
if not shipped:
return None
return len({later for landed in shipped for later in landed.fixed_by} - {landed.label for landed in shipped})
def later_fixes_of(item: int, history: list[BranchChunk]) -> int | None:
"""How many later chunks fixed code shipped by the chunk in history that shipped item; None where no chunk shipped it."""
# lilleman chose on 2026-10-28 to match by the item a chunk's route names over commit titles, since it drops the title matching and reads projects whose commit titles the worker's transcript doesn't show; the cost is that a chunk whose PR didn't remove its item reads as unknown.
return next((len(chunk.fixed_by) for chunk in history if any(shipped.id == item for shipped in chunk.shipped)), None)
def escalations(launches: dict[str, tuple[str, dict]]) -> collections.Counter[int]:
@@ -252,7 +247,7 @@ def totals(chunks: list[ChunkFigures], skipped: tuple[str, ...] = (), unread: tu
routed.setdefault(chunk.start, []).append(chunk)
for kind, found in chunk.reports.items():
reports[kind] = reports.get(kind, Reports(0, 0, 0)) + found
return Figures(len(chunks), dollars, routed, reports, summed(chunks, lambda chunk: chunk.escalated), skipped, sum(chunk.unattributed for chunk in chunks), sum(chunk.lacking for chunk in chunks), sum(chunk.fixed_later is None for chunk in chunks), unread)
return Figures(len(chunks), dollars, routed, reports, summed(chunks, lambda chunk: chunk.escalated), skipped, sum(chunk.unattributed for chunk in chunks), sum(chunk.lacking for chunk in chunks), sum(chunk.start is not None and chunk.fixed_later is None for chunk in chunks), unread)
def calls(figures: Figures) -> list[str]:
@@ -279,9 +274,9 @@ def basis(figures: Figures) -> str:
text += f" {money(figures.unattributed)} of the recorded cost is left out, since it belongs to a model with no list price or with no priced message in the archive."
if figures.lacking:
text += f" The archive holds no subagent transcript for {figures.lacking} of the chunks' Agent calls."
text += " A chunk's later fixes are the chunks on its project's default branch, as last fetched, that fixed code it shipped, as `score-outcomes` counts them, so a recent chunk has had less time to gather them. What it shipped is each chunk there holding a title of the worker's commits that no other chunk there holds."
text += " A routed chunk's later fixes are the chunks on its project's default branch, as last fetched, that fixed code it shipped, as `score-outcomes` counts them, so a recent chunk has had less time to gather them. Its chunk there is the one that removed the item its route names from todo.md."
if figures.unmatched:
text += f" The later fixes of {figures.unmatched} of the chunks are unknown, since no chunk on the branch held such a title, or the branch could not be read."
text += f" The later fixes of {figures.unmatched} of the routed chunks are unknown, since no chunk on the branch removed their item from todo.md, or the branch could not be read."
return text
+34 -26
View File
@@ -14,18 +14,19 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from routing_figures import archive
from branch_chunks.cut import BranchChunk
from routing_figures.measure import calls, Chunk, ChunkFigures, due, measured, render, Reports, Subagent, totals
from failure import Failure
import git_repo
from test_env import isolated_env
from todo_table import Item
from model_routing.ladder import LADDER, Rung
from score_outcomes.later_fixes import Landed
from transcript_archive import outcomes, write
from model_routing.route import Route
LOW = Rung("claude-opus-5-5", "medium")
HIGH = Rung("claude-opus-5-5", "high")
LOW_ROUTE = Route(LOW, 7, 3, 4).worker_line()
ROUND = "Answer stability round 1: 0 🔴, 0 🟡"
def message(id_, at, *parts, model="claude-opus-5-5", usage=None):
@@ -45,10 +46,6 @@ def result(id_, text):
return {"type": "user", "message": {"content": [{"type": "tool_result", "tool_use_id": id_, "content": [{"type": "text", "text": text}]}]}}
def commit(id_, title):
return message(id_, "2026-10-14T10:00:00Z", call("Bash", command=f"git commit -m '{title}'"))
def stamped(at, outcome):
return f"{at} {outcome}\n"
@@ -57,6 +54,15 @@ def figures(start=LOW, rounds=0, unfinished=0, reports=None, escalated=None):
return ChunkFigures({}, start, rounds, unfinished, collections.Counter(), reports or {}, collections.Counter(escalated or {}), 0.0)
def shipping(*items, fixed_by=()):
"""A chunk on a default branch that shipped the items with these IDs."""
return BranchChunk("before", "tip", "abc1234", "merge", "2026-10-14T11:00:00Z", (), tuple(Item(id_, (0, 1, 0), "", f"**Item {id_}.**", {}, (), ()) for id_ in items), fixed_by)
def no_checkout(*args, **options):
return Failure("no checkout")
class Measured(unittest.TestCase):
def test_prices_each_message_once_by_project_type_rung_and_the_tool_it_followed(self):
worker = [
@@ -96,18 +102,19 @@ class Measured(unittest.TestCase):
self.assertEqual(measured(Chunk("log", [], worker, []), "/h").escalated, {2: 2, 3: 1})
def test_counts_review_rounds_past_each_passs_first_once_per_title(self):
commit = lambda id_, title: message(id_, "2026-10-14T10:00:00Z", call("Bash", command=f"git commit -m '{title}'"))
worker = [commit("m1", "Answer stability round 1: 1 🔴, 0 🟡"), commit("m2", "Answer stability round 2: 0 🔴, 0 🟡"), commit("m3", "Answer stability round 2: 0 🔴, 0 🟡"), commit("m4", "Answer prose round 1: 0 🔴, 1 🟡")]
self.assertEqual(measured(Chunk("log", [], worker, []), "/h").rounds_past_first, 1)
def test_counts_the_later_fixes_of_the_landed_chunks_holding_a_title_of_the_workers_commits_no_other_holds(self):
worker = [commit("m1", "Ship it"), commit("m2", ROUND)]
fixed_later = lambda *history: measured(Chunk("log", [], worker, []), "/h", list(history)).fixed_later
self.assertEqual(fixed_later(Landed("b", "2026-10-14T11:00:00Z", frozenset({ROUND}), ("c", "d", "e")), Landed("a", "2026-10-14T10:30:00Z", frozenset({"Ship it", ROUND}), ("c", "d"))), 2)
self.assertEqual(fixed_later(Landed("a", "2026-10-14T10:30:00Z", frozenset({"Ship it"}), ())), 0)
self.assertEqual(fixed_later(Landed("a", "2026-10-14T10:30:00Z", frozenset({"Ship it"}), ("c", "d")), Landed("b", "2026-10-14T10:20:00Z", frozenset({ROUND}), ("d", "a"))), 2)
self.assertIsNone(fixed_later(Landed("b", "2026-10-14T11:00:00Z", frozenset({"Ship it"}), ("c",)), Landed("a", "2026-10-14T10:30:00Z", frozenset({"Ship it"}), ())))
self.assertIsNone(fixed_later(Landed("a", "2026-10-14T10:30:00Z", frozenset({"Other"}), ())))
self.assertIsNone(measured(Chunk("log", [], worker, []), "/h").fixed_later)
def test_counts_the_later_fixes_of_the_chunk_that_shipped_the_item_its_route_names(self):
routed = [stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j"))]
fixed_later = lambda kept, *history: measured(Chunk("log", kept, [], []), "/h", list(history)).fixed_later
self.assertEqual(fixed_later(routed, shipping(6, fixed_by=("c", "d", "e")), shipping(5, 7, fixed_by=("c", "d"))), 2)
self.assertEqual(fixed_later(routed, shipping(7)), 0)
self.assertIsNone(fixed_later(routed, shipping(8, fixed_by=("c",))))
self.assertIsNone(fixed_later(routed))
self.assertIsNone(measured(Chunk("log", routed, [], []), "/h").fixed_later)
self.assertIsNone(fixed_later([stamped("2026-10-14T10:00:00Z", outcomes.finished(Route(LOW).worker_line(), "m", "j"))], shipping(7, fixed_by=("c",))))
def test_reads_the_route_of_a_line_loop_claude_wrote_before_it_archived_json(self):
older = f"{LOW}, routed by item 7's L of 3 and E + I of 4"
@@ -251,10 +258,10 @@ class Calls(unittest.TestCase):
def test_renders_the_median_later_fixes_of_the_matched_chunks_and_names_each_history_unread(self):
matched = [dataclasses.replace(figures(), fixed_later=fixed) for fixed in (0, 3)]
text = render(totals([*matched, figures()], unread=("log: no main branch",)), [], "2026-10-15T00:00:00Z", 40, "")
text = render(totals([*matched, figures(), figures(start=None)], unread=("log: no main branch",)), [], "2026-10-15T00:00:00Z", 40, "")
self.assertIn(f"| {LOW} | 3 | 0.0 | 0.0 | 1.5 | | |", text)
self.assertIn("as `score-outcomes` counts them", text)
self.assertIn("The later fixes of 1 of the chunks are unknown", text)
self.assertIn("The later fixes of 1 of the routed chunks are unknown", text)
self.assertIn("## Project histories unread\n\n- log: no main branch\n", text)
self.assertNotIn("are unknown", render(totals(matched), [], "2026-10-15T00:00:00Z", 40, ""))
self.assertNotIn("histories unread", render(totals(matched), [], "2026-10-15T00:00:00Z", 40, ""))
@@ -288,10 +295,10 @@ class Archive(unittest.TestCase):
def test_measures_each_finished_chunk_keyed_by_its_project(self):
self.chunk("home/x", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [message("m1", "2026-10-14T09:00:00Z", call("Agent", "a1", subagent_type="Explore"))], ({"agentType": "Explore", "toolUseId": "a1"}, [message("s1", "2026-10-14T09:00:01Z")]))
self.chunk("log", "c2", stamped("2026-10-14T10:00:00Z", outcomes.unfinished(LOW_ROUTE, "e", "j", "")), [message("m1", "2026-10-14T09:00:00Z")])
found = archive.measure_archive(self.root, "/h", "")
found = archive.measure_archive(self.root, "/h", "", no_checkout)
self.assertEqual([chunk.dollars["project"] for chunk in found.chunks], [{"home/x": 4.0}])
self.assertEqual(found.chunks[0].dollars["subagent type"], {"worker": 2.0, "Explore": 2.0})
self.assertEqual(archive.measure_archive(self.root, "/h", "2026-10-14T10:00:00Z"), archive.Measured([], ()))
self.assertEqual(archive.measure_archive(self.root, "/h", "2026-10-14T10:00:00Z", no_checkout), archive.Measured([], ()))
def test_counts_each_chunks_later_fixes_from_its_projects_checkout_and_names_each_history_it_cannot_read(self):
home = self.root / "home"
@@ -309,16 +316,17 @@ class Archive(unittest.TestCase):
git("commit", "-q", "-m", title)
git("switch", "-q", "main")
git("merge", "-q", "--no-ff", "-m", f"Merge {branch}", branch)
fix = "| 6 | 0.1.0 | | defect | **Fix six.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |\n"
git("init", "-q", "-b", "main")
(checkout / "todo.md").write_text(plan("| 6 | 0.1.0 | | defect | **Fix six.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |\n"))
(checkout / "todo.md").write_text(plan("| 7 | 0.1.0 | | | **Ship seven.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |\n" + fix))
(checkout / "a.py").write_text("1\n")
git("add", "todo.md", "a.py")
git("commit", "-q", "-m", "Start")
merge("ship", "Ship it", {"a.py": "1\nx\n"})
merge("ship", "Ship seven", {"a.py": "1\nx\n", "todo.md": plan(fix)})
merge("fix", "Fix six", {"a.py": "1\ny\n", "todo.md": plan("")})
self.chunk("log", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [commit("m1", "Ship it")])
self.chunk("gone", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [commit("m1", "Ship it")])
found = archive.measure_archive(self.root, str(home), "")
self.chunk("log", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [message("m1", "2026-10-14T09:00:00Z")])
self.chunk("gone", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [message("m1", "2026-10-14T09:00:00Z")])
found = archive.measure_archive(self.root, str(home), "", git_repo.git)
self.assertEqual([(chunk.dollars["project"].keys(), chunk.fixed_later) for chunk in found.chunks], [({"gone": 0}.keys(), None), ({"log": 0}.keys(), 1)])
(unread,) = found.unread
self.assertTrue(unread.startswith("gone: "), unread)
@@ -332,7 +340,7 @@ class Archive(unittest.TestCase):
self.assertIsNone(write.archive(kept / "log", str(transcript), "t", outcomes.unfinished(LOW_ROUTE, "e", "j", ""), 1790000000))
transcript.write_text(json.dumps(first) + "\n" + json.dumps(message("m2", "2099-01-01T00:00:00Z")) + "\n")
self.assertIsNone(write.archive(kept / "log", str(transcript), "t", outcomes.finished(Route(HIGH).worker_line(), "m", "j"), 4070908800))
(found,) = archive.measure_archive(kept, "/h", "").chunks
(found,) = archive.measure_archive(kept, "/h", "", no_checkout).chunks
self.assertEqual((found.start, found.unfinished), (LOW, 1))
self.assertEqual(found.dollars["rung"], {str(LOW): 2.0, str(HIGH): 2.0})
@@ -342,7 +350,7 @@ class Archive(unittest.TestCase):
self.chunk("other", "c2", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [message("m1", "2026-10-14T09:00:00Z")])
self.chunk("broken", "c3", "", [])
(self.root / "broken" / "c3" / "copy.outcomes.md").write_bytes(b"\xff")
found = archive.measure_archive(self.root, "/h", "")
found = archive.measure_archive(self.root, "/h", "", no_checkout)
self.assertEqual((len(found.chunks), found.skipped), (1, ("broken", "log/c1")))
self.assertIn("## Chunks skipped, unreadable\n\n- log/c1\n", render(totals([], ("log/c1",)), [], "2026-10-15T00:00:00Z", 20, ""))
-162
View File
@@ -1,162 +0,0 @@
"""Each chunk on a default branch, with the later chunks that fixed code it shipped: its escaped defects.
A later chunk fixed code an earlier chunk shipped where one of its changes removes or rewrites lines that `git blame` traces to a commit the earlier chunk brought. Two kinds of change count: the chunk's whole change, where it ships a todo.md item marked `defect` or `security`; and a round commit of the chunk answering a 🔴 that a reviewer marked `[misread]`. Chunks are cut as score-outcomes cuts them, and each counts at most once against each earlier chunk."""
import dataclasses
import re
import typing
from failure import Failure
from git_repo import file_at, git
from round_titles import title_counts
from score_outcomes.results import chunk_log, label, NOT_WORK, shipped
from utc_stamp import utc
# The Exempt cells, per ~/.claude/rules/todo-items.md, that mark an item fixing a defect.
FIXES = ("defect", "security")
# A todo.md row that may hold one of FIXES, as the extended regex `git log -G` takes; shipped confirms it.
FIX_ROW = r"[|] *(defect|security) *[|]"
HUNK = re.compile(r"@@ -([0-9]+)(?:,([0-9]+))? \+[0-9]+(?:,([0-9]+))? @@")
BLAME_HEADER = re.compile(r"([0-9a-f]{40}|[0-9a-f]{64}) [0-9]+ [0-9]+")
# Runs git with the given arguments, in the checkout's top.
Run = typing.Callable[..., str | Failure]
@dataclasses.dataclass(frozen=True)
class Landed:
"""A chunk: its label as score-outcomes names it, its commit time, the titles of every commit it brought to the default branch, and the labels of the later chunks that fixed code it shipped, newest first."""
label: str
landed: str
titles: frozenset[str]
fixed_by: tuple[str, ...]
def later_fixes(default: str, cwd: str | None = None, guarded: bool = False, timeout: float | None = None) -> list[Landed] | Failure:
"""Each chunk on default's first-parent line, newest first, read in the checkout holding cwd, else the current directory, each git call run as `git` runs it with timeout and guarded."""
top = git("rev-parse", "--show-toplevel", cwd=cwd, timeout=timeout, guarded=guarded)
if isinstance(top, Failure):
return top
# Blame and a pathspec read a path from the current directory, and the diff names each from the top.
root = top.strip()
def run(*args: str) -> str | Failure:
return git(*args, cwd=root, timeout=timeout, guarded=guarded)
log = run("log", "--format=%x1e%H %h %ct %at %P%x1f%s", default)
if isinstance(log, Failure):
return log
shorts: dict[str, str] = {}
parents: dict[str, list[str]] = {}
titles: dict[str, str] = {}
headers: dict[str, str] = {}
for record in log.split("\x1e")[1:]:
header, _, title = record.rstrip("\n").partition("\x1f")
full, short, committed, authored, *above = header.split()
shorts[full], parents[full], titles[full] = short, above, title
headers[full] = " ".join([full, committed, authored, *above])
# git log prints the commit it starts from first.
line = [next(iter(parents))]
while parents[line[-1]]:
line.append(parents[line[-1]][0])
chunks = chunk_log([headers[commit] for commit in line])
owner = owners(chunks, line, parents)
changes = fixing_changes(chunks, owner, parents, titles, run, lambda revision: file_at(revision, "todo.md", root, timeout, guarded), default)
if isinstance(changes, Failure):
return changes
fixed: dict[int, set[int]] = {}
for index, before, tip in changes:
traced = traced_to(before, tip, run)
if isinstance(traced, Failure):
return traced
for commit in traced:
# A commit no chunk owns cannot be one before can reach; take it as the fixing chunk's own.
if owner.get(commit, index) > index:
fixed.setdefault(owner[commit], set()).add(index)
owned: dict[int, set[str]] = {}
for commit, index in owner.items():
owned.setdefault(index, set()).add(titles[commit])
names = [label(shorts[first], shorts[last]) for first, last, _, _ in chunks]
return [Landed(names[index], utc(time), frozenset(owned[index]), tuple(names[later] for later in sorted(fixed.get(index, ())))) for index, (_, _, time, _) in enumerate(chunks)]
def owners(chunks: list, line: list[str], parents: dict[str, list[str]]) -> dict[str, int]:
"""Each commit's chunk, as its index in chunks, newest first: the oldest chunk on the first-parent line whose commits reach it."""
at = {commit: place for place, commit in enumerate(line)}
owner: dict[str, int] = {}
for index in reversed(range(len(chunks))):
first, last, _, _ = chunks[index]
for commit in reversed(line[at[last]:at[first] + 1]):
unowned = [commit]
while unowned:
found = unowned.pop()
if found not in owner:
owner[found] = index
unowned += parents[found]
return owner
def fixing_changes(chunks: list, owner: dict[str, int], parents: dict[str, list[str]], titles: dict[str, str], run: Run, plan_at: typing.Callable[[str], str | Failure], default: str) -> list[tuple[int, str, str]] | Failure:
"""Each change that fixes a defect, as its chunk's index, the commit before it and its tip: the whole change of each chunk that ships a FIXES item, and each round commit answering a 🔴 a reviewer marked `[misread]`."""
touched = run("log", "--first-parent", f"-G{FIX_ROW}", "--format=%H", default, "--", "todo.md")
if isinstance(touched, Failure):
return touched
changes = []
for index in sorted({owner[commit] for commit in touched.split()}):
first, last, _, _ = chunks[index]
if not parents[first]:
continue
before = parents[first][0]
planned, after = plan_at(before), plan_at(last)
if isinstance(planned, Failure):
return planned
if isinstance(after, Failure):
return after
if any(item.exempt in FIXES for item in shipped(planned, after)):
changes.append((index, before, last))
for commit, title in titles.items():
counts = title_counts(title)
# A title counts misread 🔴 and 🟡 findings together, so it takes one with both a 🔴 and a misread finding to answer a misread 🔴.
if counts and counts.red and counts.misread and parents[commit]:
changes.append((owner[commit], parents[commit][0], commit))
return changes
def traced_to(before: str, tip: str, run: Run) -> set[str] | Failure:
"""The commits that last changed each line the change from before to tip removes or rewrites, outside the plan and the records."""
# The prefixes and options keep the checkout's diff config out of the output.
diff = run("diff", "-U0", "-M", "--no-color", "--no-ext-diff", "--no-textconv", "--ignore-submodules", "--src-prefix=a/", "--dst-prefix=b/", before, tip)
if isinstance(diff, Failure):
return diff
traced: set[str] = set()
for path, spans in removed(diff).items():
if path in NOT_WORK:
continue
# An empty --ignore-revs-file drops a configured one, which blame fails on where the file is missing.
blame = run("blame", "--porcelain", "--no-textconv", "--ignore-revs-file=", *(f"-L{start},+{count}" for start, count in spans), before, "--", path)
if isinstance(blame, Failure):
return blame
traced |= blamed(blame)
return traced
def removed(diff: str) -> dict[str, list[tuple[int, int]]]:
"""The old lines each hunk of a `git diff -U0` removes or rewrites, as its first line and count, by the path the file had before; none for a path git quotes, as it does one holding a byte it cannot print."""
spans: dict[str, list[tuple[int, int]]] = {}
path, left = None, 0
for row in diff.splitlines():
if left and row[:1] in ("-", "+", " "):
left -= 1
elif row.startswith("diff --git "):
path, left = None, 0
elif row.startswith("--- "):
# Git ends a path holding a space with a tab.
path = row[len("--- a/"):].removesuffix("\t") if row.startswith("--- a/") else None
elif hunk := HUNK.match(row):
start, count = int(hunk.group(1)), int(hunk.group(2) or 1)
left = count + int(hunk.group(3) or 1)
if path and count:
spans.setdefault(path, []).append((start, count))
return spans
def blamed(porcelain: str) -> set[str]:
"""The commit each line of `git blame --porcelain` output comes from."""
return {header.group(1) for row in porcelain.splitlines() if (header := BLAME_HEADER.match(row))}
+21 -53
View File
@@ -35,10 +35,10 @@ holds R and S, and C too where its table also had A, G and Goals. The chunk meas
A chunk is marked `by rebase` where a commit in it has a committer time apart from its author time,
since a rebase replays a pull request's commits at once, and `never replayed` where none does, since
a pushed or fast-forwarded commit may be part of a pull request. A merge's titles are its branch's
commits'; any other chunk's are its own. Two pull requests merged in the same second read as one
chunk. Rebase drops a round's commit that changed no file, so a rebase or push chunk gives its
rounds and findings as lower bounds, `at least`, and the report ends saying all this.
a pushed or fast-forwarded commit may be part of a pull request. A merge's titles are its own and
its branch's commits'; any other chunk's are its own. Two pull requests merged in the same second
read as one chunk. Rebase drops a round's commit that changed no file, so a rebase or push chunk
gives its rounds and findings as lower bounds, `at least`, and the report ends saying all this.
Given L, E and I, on the levels ~/.claude/rules/todo-items.md defines, and S, it works out R from
L, E and I, as rank_todo.py does, and lists the 5 finished items whose guesses sit closest, closest
@@ -71,18 +71,18 @@ Exit 1, saying why, on bad arguments, when git cannot start, finds no checkout o
clone, or the default branch cannot be read; or, after the rest of the report, when the anchors are
left out.
"""
import functools
import pathlib
import re
import sys
from score_outcomes.anchors import anchor_lines, Anchors, ANCHORS_PATH, parse_anchors
from score_outcomes.results import chunk_log, fixed_later, label, Landing, misreads, nearest, render, report, Result, results_of, shipped
from git_repo import default_branch, file_at, full_history, git, top
from branch_chunks.read import branch_chunks
from git_repo import default_branch, full_history, git, top
from failure import Failure
from score_outcomes.later_fixes import later_fixes
from score_outcomes.results import fixed_later, misreads, nearest, render, report, Result, results_of
from todo_table import LEVELS, risk, TOP_SCORE
from markdown_text import listed
from utc_stamp import utc
# The scores score-outcomes takes, in order.
GIVEN = ("L", "E", "I", "S")
@@ -157,51 +157,19 @@ def read_results(default: str) -> list[Result] | Failure:
"""What each chunk on the default branch's first-parent line measured."""
if partial := full_history():
return partial
# Both logs read one commit, so a fetch that moves the branch between them cannot cut chunks two ways.
tip = git("rev-parse", "--verify", f"{default}^{{commit}}")
if isinstance(tip, Failure):
return tip
log = git("log", "--first-parent", "--format=%x1e%h %ct %at %p", "--name-only", tip.strip())
if isinstance(log, Failure):
return log
fixes = later_fixes(tip.strip())
if isinstance(fixes, Failure):
return fixes
fixed_by = {landed.label: landed.fixed_by for landed in fixes}
# Each commit's part of the log is its line, then the files it changed against its first parent.
commits = [part.splitlines() for part in log.split("\x1e")[1:]]
headers = [commit[0] for commit in commits]
hashes = [header.split()[0] for header in headers]
plan = {short for short, commit in zip(hashes, commits) if "todo.md" in commit}
root = top()
if isinstance(root, Failure):
return root
run = functools.partial(git, cwd=root)
chunks = branch_chunks(default, run)
if isinstance(chunks, Failure):
return chunks
results: list[Result] = []
for first, last, committed, landed in chunk_log(headers):
# Only a chunk that changed todo.md can ship an item, so the others skip measure's git calls.
if not any(short in plan for short in hashes[hashes.index(last):hashes.index(first) + 1]):
for chunk in chunks:
if chunk.before is None or not chunk.shipped:
continue
titled = f"{first}^1..{first}^2" if landed == "merge" else f"{first}^1..{last}"
named = label(first, last)
measured = measure(f"{first}^1", last, named, utc(committed), titled, landed, fixed_by[named])
if isinstance(measured, Failure):
return measured
results += measured
numstat = run("diff", "--numstat", chunk.before, chunk.tip)
if isinstance(numstat, Failure):
return numstat
results += results_of(chunk, numstat)
return results
def measure(before: str, tip: str, named: str, merged: str, titled: str, landed: Landing, fixed_by: tuple[str, ...]) -> list[Result] | Failure:
"""What one chunk measured for each item it shipped: its change from the commit before it to its tip, and the titles of the commits in the range titled."""
planned = file_at(before, "todo.md")
if isinstance(planned, Failure):
return planned
after = file_at(tip, "todo.md")
if isinstance(after, Failure):
return after
items = shipped(planned, after)
if not items:
return []
numstat = git("diff", "--numstat", before, tip)
if isinstance(numstat, Failure):
return numstat
titles = git("log", "--format=%s", titled)
if isinstance(titles, Failure):
return titles
return results_of(items, named, merged, numstat, titles.splitlines(), landed, fixed_by)
+9 -49
View File
@@ -2,27 +2,19 @@ import dataclasses
import statistics
import typing
from round_titles import Findings, round_counts, round_findings
from comprehension_history import HISTORY
from todo_table import Item, items
from branch_chunks.cut import BranchChunk, Landing, NOT_WORK
from round_titles import Findings, round_findings, rounds_past_first
from todo_table import Item
from markdown_text import listed
from reading_cost.figures import RECORD
# Fewest single-item chunks a score needs before its medians print: the 10 figures the reading-cost trend waits for; no published standard sets either.
ENOUGH = 10
# The plan, and the records tools append to: none of their lines are the chunk's work.
NOT_WORK = ("todo.md", HISTORY, RECORD)
# How a chunk reached the default branch; rebase and push leave no merge commit to bound it.
Landing = typing.Literal["merge", "rebase", "push"]
# A chunk's first and last commit as short hashes, its commit time, and how it reached the default branch.
Chunk = tuple[str, str, int, Landing]
# The tool's name and report call a Result an outcome, a word the code keeps for an entry of a `.outcomes.md` file: docs/decisions.md → `bin/: an outcome is an entry`.
@dataclasses.dataclass(frozen=True)
class Result:
"""What the chunk that shipped an item measured. findings holds the 🔴 and 🟡 counts, and how many of them a reviewer marked `[misread]`, None where a round title lacks the 🔴 and 🟡 counts or the branch has no round; its misread is None where a round title lacks that count. landed says how the chunk reached the default branch: by a merge commit, by a rebase that replayed its commits, or by commits never replayed. fixed_by names the later chunks that fixed code it shipped, newest first, as later_fixes counts them."""
"""What the chunk that shipped an item measured. findings holds the 🔴 and 🟡 counts, and how many of them a reviewer marked `[misread]`, None where a round title lacks the 🔴 and 🟡 counts or the branch has no round; its misread is None where a round title lacks that count. landed says how the chunk reached the default branch: by a merge commit, by a rebase that replayed its commits, or by commits never replayed. fixed_by names the later chunks that fixed code it shipped, newest first, as branch_chunks counts them."""
item: Item
merge: str
merged: str
@@ -45,49 +37,17 @@ ORDERED = ("L", "E", "I")
SHOWN = ("L", "E", "I", "R", "S", "C")
def shipped(before: str, after: str) -> list[Item]:
"""Each item the todo.md before a change holds and the todo.md after it does not, as scored before; none where either holds no items table it can read."""
planned, left = items(before), items(after)
if not isinstance(planned, list) or not isinstance(left, list):
return []
kept = {item.id for item in left}
return sorted((item for item in planned if item.id not in kept), key=lambda item: item.id)
def results_of(shipped: list[Item], merge: str, merged: str, numstat: str, titles: list[str], landed: Landing, fixed_by: tuple[str, ...]) -> list[Result]:
"""What one chunk measured for each item it shipped, from `git diff --numstat` against the commit before it and its commit titles; none where it changed nothing but the plan and the records."""
def results_of(chunk: BranchChunk, numstat: str) -> list[Result]:
"""What one chunk measured for each item it shipped, from `git diff --numstat` of its change; none where it changed nothing but the plan and the records."""
changed = [row.split("\t", 2) for row in numstat.splitlines()]
work = [row for row in changed if len(row) == 3 and row[2] not in NOT_WORK]
if not work:
return []
# Git counts a binary file's lines as "-".
lines = sum(int(count) for row in work for count in row[:2] if count.isdigit())
past_first = sum(max(0, count - 1) for count in round_counts(titles).values())
ids = [item.id for item in shipped]
return [Result(item, merge, merged, tuple(id_ for id_ in ids if id_ != item.id), lines, len(work), past_first, round_findings(titles), landed, fixed_by) for item in shipped]
def chunk_log(lines: list[str]) -> list[Chunk]:
"""The chunks `git log --first-parent --format=%h %ct %at %p` lines cut, newest first: each chunk's first and last commit as short hashes, its commit time, and how it reached the branch. A merge commit is a chunk of its own; any other commit joins the run of the newer commit above it when both share a commit time."""
chunks: list[Chunk] = []
joinable = False
for line in lines:
short, committed, authored, *parents = line.split()
if len(parents) > 1:
chunks.append((short, short, int(committed), "merge"))
joinable = False
elif joinable and chunks[-1][2] == int(committed):
_, newest, time, landed = chunks[-1]
chunks[-1] = (short, newest, time, "rebase" if landed == "rebase" or committed != authored else "push")
else:
chunks.append((short, short, int(committed), "rebase" if committed != authored else "push"))
joinable = True
return chunks
def label(first: str, last: str) -> str:
"""How the report names a chunk: its one commit's short hash, or its first and last commits' as a range."""
return first if first == last else f"{first}^..{last}"
titles = list(chunk.titles)
ids = [item.id for item in chunk.shipped]
return [Result(item, chunk.label, chunk.merged, tuple(id_ for id_ in ids if id_ != item.id), lines, len(work), rounds_past_first(titles), round_findings(titles), chunk.landed, chunk.fixed_by) for item in chunk.shipped]
def nearest(results: list[Result], given: dict[str, int], count: int = 5) -> list[Result]:
-192
View File
@@ -1,192 +0,0 @@
import os
import pathlib
import subprocess
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from score_outcomes.later_fixes import blamed, Landed, later_fixes, removed
from failure import Failure
from test_env import isolated_env
HEADER = "| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |\n|---|---|---|---|---|---|---|---|---|---|---|---|\n"
ROWS = {
4: "| 4 | 0.1.0 | | | **Do four.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |",
5: "| 5 | 0.1.0 | | | **Do five.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |",
6: "| 6 | 0.1.0 | | defect | **Fix six.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |",
7: "| 7 | 0.1.0 | | security | **Close seven.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |",
8: "| 8 | 0.1.0 | | defect | **Fix eight.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |",
9: "| 9 | 0.1.0 | | defect | **Fix nine.** | 1 | 1 | 1 | 1 | 2 | 1.1 by 3 | 1.0 |",
}
def todo(*ids: int) -> str:
return "# todo\n\n## Items\n\n" + HEADER + "".join(f"{ROWS[id_]}\n" for id_ in ids)
class Removed(unittest.TestCase):
def test_reads_the_old_lines_each_hunk_removes_or_rewrites_by_the_path_they_had(self):
diff = "\n".join([
"diff --git a/a.py b/a.py",
"--- a/a.py",
"+++ b/a.py",
"@@ -2 +2 @@ def f():",
"-x",
"+y",
"@@ -5,3 +4,0 @@",
"@@ -9,0 +8,2 @@",
"diff --git a/old.py b/new.py",
"--- a/old.py",
"+++ b/new.py",
"@@ -1,2 +1,2 @@",
"diff --git a/gone.py b/gone.py",
"--- a/gone.py",
"+++ /dev/null",
"@@ -1,4 +0,0 @@",
"diff --git a/born.py b/born.py",
"--- /dev/null",
"+++ b/born.py",
"@@ -0,0 +1,3 @@",
"diff --git \"a/tab\\there.py\" \"b/tab\\there.py\"",
"--- \"a/tab\\there.py\"",
"+++ \"b/tab\\there.py\"",
"@@ -1 +1 @@",
"diff --git a/my file.py b/my file.py",
"--- a/my file.py\t",
"+++ b/my file.py\t",
"@@ -7,2 +7 @@",
])
self.assertEqual(removed(diff), {"a.py": [(2, 1), (5, 3)], "old.py": [(1, 2)], "gone.py": [(1, 4)], "my file.py": [(7, 2)]})
def test_reads_a_hunk_line_inside_the_changed_text_as_text(self):
diff = "--- a/a.md\n+++ b/a.md\n@@ -3 +3 @@\n---- a/quoted\n+text\n"
self.assertEqual(removed(diff), {"a.md": [(3, 1)]})
class Blamed(unittest.TestCase):
def test_reads_the_commit_of_each_line_from_porcelain_output(self):
first, second = "a" * 40, "b" * 40
porcelain = f"{first} 2 2 1\nauthor t\nfilename a.py\n\tx\n{second} 5 5 2\nauthor t\n\ty\n{second} 6 6\n\t{first} 1 1 1\n"
self.assertEqual(blamed(porcelain), {first, second})
class LaterFixes(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.checkout = pathlib.Path(os.path.realpath(temp.name))
self.env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(self.checkout)}
self.git("init", "-q", "-b", "main")
self.commit("Start", {"todo.md": todo(4, 5, 6, 7, 8), "a.py": "1\n2\n3\n"})
def git(self, *args):
return subprocess.run(["git", "-c", "user.name=t", "-c", "user.email=t@t", *args], cwd=self.checkout, check=True, env=self.env, capture_output=True, text=True).stdout
def commit(self, title, files, at=None):
for name, text in files.items():
(self.checkout / name).write_text(text)
self.git("add", name)
dates = {"GIT_COMMITTER_DATE": at, "GIT_AUTHOR_DATE": at} if at else {}
self.env.update(dates)
self.git("commit", "-q", "-m", title)
for name in dates:
self.env.pop(name)
return self.git("rev-parse", "--short", "HEAD").strip()
def chunk(self, branch, *commits):
"""Merges a branch holding each (title, files) commit, and returns the merge commit's short hash."""
self.git("switch", "-q", "-c", branch)
for title, files in commits:
self.commit(title, files)
self.git("switch", "-q", "main")
self.git("merge", "-q", "--no-ff", "-m", f"Merge {branch}", branch)
return self.git("rev-parse", "--short", "HEAD").strip()
def fixes(self):
found = later_fixes("main", cwd=str(self.checkout))
self.assertNotIsInstance(found, Failure)
return {landed.label: landed.fixed_by for landed in found}
def test_counts_a_later_chunk_that_ships_a_defect_item_and_rewrites_a_line_an_earlier_chunk_added(self):
shipped = self.chunk("four", ("Do four", {"b.py": "x\ny\n", "todo.md": todo(5, 6, 7, 8)}))
fixed = self.chunk("six", ("Fix six", {"b.py": "x\nY\n", "todo.md": todo(5, 7, 8)}))
found = self.fixes()
self.assertEqual(found[shipped], (fixed,))
self.assertEqual(found[fixed], ())
def test_counts_a_chunk_that_ships_a_security_item_as_a_fix(self):
shipped = self.chunk("four", ("Do four", {"b.py": "x\ny\n", "todo.md": todo(5, 6, 7, 8)}))
fixed = self.chunk("seven", ("Close seven", {"b.py": "x\n", "todo.md": todo(5, 6, 8)}))
self.assertEqual(self.fixes()[shipped], (fixed,))
def test_counts_no_chunk_whose_shipped_items_are_neither_defect_nor_security(self):
shipped = self.chunk("four", ("Do four", {"b.py": "x\ny\n", "todo.md": todo(5, 6, 7, 8)}))
self.chunk("five", ("Do five", {"b.py": "x\nY\n", "todo.md": todo(6, 7, 8)}))
self.assertEqual(self.fixes()[shipped], ())
def test_counts_no_chunk_that_only_edits_a_defect_items_row(self):
shipped = self.chunk("four", ("Do four", {"b.py": "x\ny\n", "todo.md": todo(5, 6, 7, 8)}))
self.chunk("edit", ("Re-score six", {"b.py": "x\nY\n", "todo.md": todo(5, 6, 7, 8).replace("**Fix six.**", "**Fix six now.**")}))
self.assertEqual(self.fixes()[shipped], ())
def test_leaves_the_plans_row_out_so_a_fix_does_not_trace_to_the_chunk_that_filed_it(self):
filed = self.chunk("file", ("File item 9", {"c.py": "c\n", "todo.md": todo(4, 5, 6, 7, 8, 9)}))
self.chunk("nine", ("Fix nine", {"d.py": "d\n", "todo.md": todo(4, 5, 6, 7, 8)}))
self.assertEqual(self.fixes()[filed], ())
def test_counts_a_round_commit_that_answers_a_misread_red_finding_and_rewrites_an_earlier_chunks_line(self):
shipped = self.chunk("four", ("Do four", {"b.py": "x\ny\n", "todo.md": todo(5, 6, 7, 8)}))
fixed = self.chunk("five", ("Do five", {"c.py": "c\n", "todo.md": todo(6, 7, 8)}), ("Answer stability round 1: 1 🔴, 0 🟡 (1 misread); keep y's caller", {"b.py": "x\nY\n"}))
self.assertEqual(self.fixes()[shipped], (fixed,))
def test_counts_no_round_commit_without_both_a_red_and_a_misread_finding(self):
shipped = self.chunk("four", ("Do four", {"b.py": "w\nx\ny\n", "todo.md": todo(5, 6, 7, 8)}))
self.chunk("five", ("Do five", {"c.py": "c\n", "todo.md": todo(6, 7, 8)}), ("Answer stability round 1: 1 🔴, 0 🟡 (0 misread); fix", {"b.py": "W\nx\ny\n"}), ("Answer prose round 1: 0 🔴, 1 🟡 (1 misread); fix", {"b.py": "W\nX\ny\n"}), ("Answer principles round 1: 1 🔴, 0 🟡; fix", {"b.py": "W\nX\nY\n"}))
self.assertEqual(self.fixes()[shipped], ())
def test_counts_no_fix_of_lines_its_own_chunk_wrote(self):
self.chunk("six", ("Fix six", {"b.py": "x\ny\n", "todo.md": todo(4, 5, 7, 8)}), ("Answer stability round 1: 1 🔴, 0 🟡 (1 misread); fix", {"b.py": "x\nY\n"}))
self.assertEqual(set(self.fixes().values()), {()})
def test_counts_a_fixing_chunk_once_against_each_chunk_it_fixes_newest_first(self):
first = self.chunk("four", ("Do four", {"b.py": "x\ny\n", "d.py": "d\n", "todo.md": todo(5, 6, 7, 8)}))
second = self.chunk("five", ("Do five", {"c.py": "z\n", "todo.md": todo(6, 7, 8)}))
fixed = self.chunk("six", ("Fix six", {"b.py": "X\ny\n", "todo.md": todo(7, 8)}), ("Answer stability round 1: 1 🔴, 0 🟡 (1 misread); fix", {"b.py": "X\nY\n", "c.py": "Z\n"}))
again = self.chunk("eight", ("Fix eight", {"b.py": "XX\nY\n", "d.py": "D\n", "todo.md": todo(7)}))
found = self.fixes()
self.assertEqual(found[first], (again, fixed))
self.assertEqual(found[second], (fixed,))
self.assertEqual(found[fixed], (again,))
def test_traces_a_line_to_the_rebased_run_that_added_it_and_labels_runs_as_score_outcomes_does(self):
first = self.commit("Do four", {"b.py": "x\ny\n", "todo.md": todo(5, 6, 7, 8)}, at="2026-10-13T09:00:00Z")
last = self.commit("Answer stability round 1: 0 🔴, 0 🟡 (0 misread)", {"c.py": "c\n"}, at="2026-10-13T09:00:00Z")
fixed = self.commit("Fix six", {"b.py": "x\nY\n", "todo.md": todo(5, 7, 8)}, at="2026-10-13T10:00:00Z")
self.assertEqual(self.fixes()[f"{first}^..{last}"], (fixed,))
def test_holds_each_chunks_commit_titles_and_landing_time(self):
self.git("switch", "-q", "-c", "four")
self.commit("Do four", {"b.py": "x\n", "todo.md": todo(5, 6, 7, 8)})
self.git("switch", "-q", "main")
self.env.update({"GIT_COMMITTER_DATE": "2026-10-14T10:00:00Z", "GIT_AUTHOR_DATE": "2026-10-14T10:00:00Z"})
self.git("merge", "-q", "--no-ff", "-m", "Merge four", "four")
merged = self.git("rev-parse", "--short", "HEAD").strip()
newest = later_fixes("main", cwd=str(self.checkout))[0]
self.assertEqual(newest, Landed(merged, "2026-10-14T10:00:00Z", frozenset({"Do four", "Merge four"}), ()))
def test_reads_paths_from_the_checkouts_top_when_run_in_a_folder_of_it(self):
(self.checkout / "sub").mkdir()
shipped = self.chunk("four", ("Do four", {"sub/b.py": "x\ny\n", "todo.md": todo(5, 6, 7, 8)}))
fixed = self.chunk("six", ("Fix six", {"sub/b.py": "x\nY\n", "todo.md": todo(5, 7, 8)}))
found = later_fixes("main", cwd=str(self.checkout / "sub"))
self.assertNotIsInstance(found, Failure)
self.assertEqual({landed.label: landed.fixed_by for landed in found}[shipped], (fixed,))
def test_fails_where_git_cannot_read_the_branch(self):
self.assertIsInstance(later_fixes("missing", cwd=str(self.checkout)), Failure)
if __name__ == "__main__":
unittest.main()
+12 -51
View File
@@ -8,23 +8,23 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from score_outcomes.anchors import anchor_lines, parse_anchors
from score_outcomes.results import chunk_log, Landing, nearest, rank_correlation, render, report, Result, results_of, shipped
from branch_chunks.cut import BranchChunk, Landing
from failure import Failure
from round_titles import Findings
from test_env import isolated_env
from todo_table import Item
from score_outcomes.results import nearest, rank_correlation, render, report, Result, results_of
from score_outcomes.main import report_text
TOOL = pathlib.Path(__file__).resolve().parents[1] / "score-outcomes"
HEADER = "| ID | Release | Exempt | Item | R | S | C | A | G | Goals | Score |\n|---|---|---|---|---|---|---|---|---|---|---|\n"
WITHOUT_C = "| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |\n|---|---|---|---|---|---|---|---|---|---|\n"
FIRST = "| 4 | 0.1.0 | | **Do it.** | 2 | 3 | 10 | 6 | 7 | 1 | 9.0 |"
SECOND = "| 5 | 0.1.0 | | **Then this.** | 4 | 6 | 8 | 6 | 7 | 1 | 9.0 |"
FIX = "| 6 | 0.1.0 | defect | **Fix it.** | 1 | 1 | 1 | 6 | 7 | 1 | 9.0 |"
def todo(*rows: str, header: str = HEADER) -> str:
return "# todo\n\n## Items\n\n" + header + "".join(f"{row}\n" for row in rows)
def todo(*rows: str) -> str:
return "# todo\n\n## Items\n\n" + HEADER + "".join(f"{row}\n" for row in rows)
def item(id_: int, scores: dict[str, int]) -> Item:
@@ -35,55 +35,16 @@ def result(id_: int, r: int, s: int, c: int | None, lines: int = 10, rounds: int
return Result(item(id_, {"R": r, "S": s} | ({"C": c} if c else {}) | (levels or {})), merge, merged, shipped_with, lines, 1, rounds, None if findings is None else Findings(*findings, misread), landed, fixed_by)
class Shipped(unittest.TestCase):
def test_ships_each_item_the_change_removed_with_the_scores_it_had_before(self):
self.assertEqual([found.id for found in shipped(todo(FIRST, SECOND), todo(SECOND))], [4])
self.assertEqual(shipped(todo(FIRST, SECOND), todo(SECOND))[0].scores, {"R": 2, "S": 3, "C": 10, "A": 6, "G": 7})
def test_lists_the_shipped_items_by_id(self):
self.assertEqual([found.id for found in shipped(todo(SECOND, FIRST), todo())], [4, 5])
def test_reads_a_table_from_before_c_joined_it(self):
older = "| 4 | 0.1.0 | | **A \\| B.** | 2 | 3 | 6 | 7 | 1 | 9.0 |"
self.assertEqual(shipped(todo(older, header=WITHOUT_C), todo(header=WITHOUT_C))[0].scores, {"R": 2, "S": 3, "A": 6, "G": 7})
def test_ships_nothing_where_either_plan_has_no_table_it_can_read(self):
self.assertEqual(shipped(todo(FIRST), "# todo\n\n- [ ] Do it.\n"), [])
self.assertEqual(shipped(todo(FIRST, "| x | 0.1.0 | | **Bad.** | 2 | 3 | 5 | 6 | 7 | 1 | 9.0 |"), todo()), [])
class ChunkLog(unittest.TestCase):
# Lines are `%h %ct %at %p`, newest first: short hash, commit time, author time, parents.
def test_a_merge_is_a_chunk_of_its_own_and_joins_no_run_below_it(self):
self.assertEqual(chunk_log(["m 100 100 a b", "c 100 100 a"]), [("m", "m", 100, "merge"), ("c", "c", 100, "push")])
def test_commits_sharing_a_commit_time_form_one_run_labelled_rebase_when_any_differs(self):
self.assertEqual(chunk_log(["c2 200 150 c1", "c1 200 100 r"]), [("c1", "c2", 200, "rebase")])
def test_a_run_of_commits_that_all_kept_their_author_time_is_labelled_push(self):
self.assertEqual(chunk_log(["c2 200 200 c1", "c1 200 200 r"]), [("c1", "c2", 200, "push")])
def test_a_run_is_rebase_when_only_its_older_commit_was_replayed(self):
self.assertEqual(chunk_log(["c2 200 200 c1", "c1 200 100 r"]), [("c1", "c2", 200, "rebase")])
def test_a_run_is_rebase_when_only_its_newer_commit_was_replayed(self):
self.assertEqual(chunk_log(["c2 200 100 c1", "c1 200 200 r"]), [("c1", "c2", 200, "rebase")])
def test_a_commit_at_another_time_starts_a_new_chunk(self):
self.assertEqual(chunk_log(["c2 300 300 c1", "c1 200 200 r"]), [("c2", "c2", 300, "push"), ("c1", "c1", 200, "push")])
class ResultsOf(unittest.TestCase):
def test_measures_the_change_less_the_plan_and_the_tool_records_and_counts_binary_files(self):
numstat = "3\t1\ta.py\n-\t-\tlogo.png\n0\t1\t2026\n5\t0\ttodo.md\n9\t0\tdocs/reading-cost.md\n4\t0\tdocs/comprehension-history.md\n"
titles = ["Answer stability round 1: 1 🔴, 2 🟡; fix", "Answer stability round 2: 0 🔴, 1 🟡; fix", "Answer prose round 1: 0 🔴, 3 🟡", "Do it"]
found = results_of([item(4, {"R": 2}), item(5, {"R": 3})], "abc1234", "2026-10-13T10:00:00Z", numstat, titles, "merge", ("def5678",))
found = results_of(BranchChunk("before", "tip", "abc1234", "merge", "2026-10-13T10:00:00Z", tuple(titles), (item(4, {"R": 2}), item(5, {"R": 3})), ("def5678",)), numstat)
self.assertEqual(found[0], Result(item(4, {"R": 2}), "abc1234", "2026-10-13T10:00:00Z", (5,), 5, 3, 1, Findings(1, 6, None), "merge", ("def5678",)))
self.assertEqual(found[1].shipped_with, (4,))
def test_measures_no_chunk_that_changed_only_the_plan(self):
self.assertEqual(results_of([item(4, {"R": 2})], "abc1234", "2026-10-13T10:00:00Z", "0\t5\ttodo.md\n", [], "merge", ()), [])
self.assertEqual(results_of(BranchChunk("before", "tip", "abc1234", "merge", "2026-10-13T10:00:00Z", (), (item(4, {"R": 2}),), ()), "0\t5\ttodo.md\n"), [])
class Nearest(unittest.TestCase):
@@ -307,8 +268,8 @@ class ScoreOutcomesTool(unittest.TestCase):
self.env.pop(name, None)
return self.git("rev-parse", "--short", "HEAD").strip()
def run_tool(self, *args):
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=self.checkout, env=self.env, text=True)
def run_tool(self, *args, cwd=None):
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=cwd or self.checkout, env=self.env, text=True)
def test_measures_each_merged_chunk_that_removed_an_item(self):
self.git("switch", "-q", "-c", "chunk")
@@ -322,17 +283,17 @@ class ScoreOutcomesTool(unittest.TestCase):
self.assertIn("- Item 4, R 2, S 3, C 10: 3 lines in 1 file, 0 rounds past each pass's first, 1 🔴, 2 🟡, 0 later fixes; merged ", ran.stdout)
self.assertNotIn("Item 5", ran.stdout)
def test_counts_a_later_chunk_that_ships_a_defect_item_and_rewrites_a_line_a_chunk_shipped(self):
def test_counts_a_later_chunk_that_ships_a_defect_item_and_rewrites_a_line_a_chunk_shipped_run_from_a_folder_below_the_top(self):
self.commit("File a fix", {"todo.md": todo(FIRST, SECOND, FIX)})
self.git("switch", "-q", "-c", "chunk")
self.commit("Do it", {"a.py": "1\n2\n3\n", "todo.md": todo(SECOND, FIX)})
self.commit("Do it", {"sub/a.py": "1\n2\n3\n", "todo.md": todo(SECOND, FIX)})
self.git("switch", "-q", "main")
shipped = self.merge_branch("chunk", "2026-10-13T10:00:00Z")
self.git("switch", "-q", "-c", "fix")
self.commit("Fix it", {"a.py": "1\n2\n4\n", "todo.md": todo(SECOND)})
self.commit("Fix it", {"sub/a.py": "1\n2\n4\n", "todo.md": todo(SECOND)})
self.git("switch", "-q", "main")
fixed = self.merge_branch("fix", "2026-10-14T10:00:00Z")
ran = self.run_tool("1", "2", "1", "3")
ran = self.run_tool("1", "2", "1", "3", cwd=self.checkout / "sub")
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertIn(f"findings unknown, 1 later fix; merged 2026-10-13T10:00:00Z in {shipped}. **Do it.**", ran.stdout)
self.assertIn(f"- 2026-10-13T10:00:00Z in {shipped}: 1 later fix, by {fixed}\n- The other 1 finished chunk: none.", ran.stdout)
+5 -1
View File
@@ -4,7 +4,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from round_titles import Findings, lacks_counts, past_cap, round_counts, round_findings
from round_titles import Findings, lacks_counts, past_cap, round_counts, round_findings, rounds_past_first
class Rounds(unittest.TestCase):
@@ -21,6 +21,10 @@ class Rounds(unittest.TestCase):
def test_lists_the_principles_pass_before_its_first_round(self):
self.assertIn("principles", round_counts([]))
def test_sums_each_passs_rounds_past_its_first(self):
self.assertEqual(rounds_past_first(["Answer stability round 2: x", "Answer stability round 1: y", "Answer prose round 1: z", "Answer architecture round 3: w", "Do the work"]), 3)
self.assertEqual(rounds_past_first([]), 0)
class Cap(unittest.TestCase):
def test_finds_a_round_title_past_round_2(self):
+3 -3
View File
@@ -12,7 +12,7 @@
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 251 | 0.4.0 | | defect | **Have `file_at` in `bin/git_repo.py` read a path from the checkout's top, so `score-outcomes` run in a subfolder of a checkout reads its todo.md.** | 1 | 1 | 3 | 3 | 1 | 1.1 by 2 | 3.4 |
| 251 | 0.4.0 | | defect | **Have larv-review read the comprehension history's changes from the checkout's top, so run in a subfolder of a checkout it still finds them.** | 1 | 1 | 3 | 3 | 1 | 1.1 by 2 | 3.4 |
| 243 | 0.4.0 | | owner | **Show every loop-claude on the machine that waits for the human in one terminal app, and let the human answer each one there, `/talk` included.** | 4 | 2 | 3 | 7 | 6 | 1.5 by 4 | 0.9 |
| 226 | 0.5.0 | | question | **Answer whether rule-hook's `run-heavy` rule should also catch a mutation run or docker build behind `uv run`, `poetry run`, `python3 -m`, `sudo`, `docker image build` and `docker compose up --build`, or narrow the `machine load:` decision's wording.** | 2 | 2 | 3 | 5 | 3 | 1.2 by 2, 1.5 by 2 | 4.0 |
| 248 | 0.5.0 | | question | **Answer whether the `rule-hook:` decision should name every rule that takes no override, `repeat-read`, `reviewer-report` and `reviewer-rung`, with each one's reason.** | 1 | 2 | 1 | 2 | 1 | 1.2 by 2 | 3.3 |
@@ -78,9 +78,9 @@
## Details
### 251. Have `file_at` in `bin/git_repo.py` read a path from the checkout's top, so `score-outcomes` run in a subfolder of a checkout reads its todo.md.
### 251. Have larv-review read the comprehension history's changes from the checkout's top, so run in a subfolder of a checkout it still finds them.
Found while building item 247, 2026-10-28. `file_at` lists the path with `git ls-tree <revision> -- <path>`, which reads the path relative to the current directory. Run from `bin/`, `score-outcomes` finds no `todo.md` at any commit, and lists every anchor as shipped by no chunk instead of failing. `bin/larv_review/history_changes.py` calls `file_at` the same way. `ls-tree --full-tree` reads the path from the top, as `git show <revision>:<path>` already does.
Found while building item 247, 2026-10-28. `history_changes` in `bin/larv_review/history_changes.py` names the history by its path from the checkout's top, but git reads that path from the current directory, twice: in its `git log -- <path>`, and in `file_at` in `bin/git_repo.py`, which lists the path with `git ls-tree <revision> -- <path>`. Run from `bin/`, larv-review finds no change to the history instead of failing. Running git in the checkout's top, as `score-outcomes` does since item 247, reads both paths from the top.
### 243. Show every loop-claude on the machine that waits for the human in one terminal app, and let the human answer each one there, `/talk` included.