diff --git a/CHANGELOG.md b/CHANGELOG.md index 2dd2de35..a01e3bc3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ - A release someone else cut while it still held plan items no longer stops the loop: those items count as the next release's, `chunk-preflight` tells the worker to move them there, and `loop-claude` routes on them. Where the plan names no later release and `todo.md` named none since the cut, the loop still asks which release they belong to. - `project-record open` now writes the copy of a record kept outside a file under a folder of the worktree that ran it, `~/.local/state/project-record///`, so two chunks of one project no longer overwrite each other's unsaved edit. It also removes the folders of worktrees git no longer lists. - `chunk-pr merge` merges the chunks of one project on a machine one at a time, and `--finish ` finishes each plan item in the chunk's turn, so `todo.md` and `CHANGELOG.md` no longer conflict between open chunks. A pull request that waits for an approval leaves the queue while it waits, and merge stops where a reviewer's comment waits for an answer. -- Two loop-claude runs in one project now work side by side, each on its own chunk. Each loop holds a slot of the project, a locked file under `~/.local/state/loop-claude//`, and names its session after it: the first loop's session keeps its name, `A loop-claude`, and the second's is `A loop-claude 2`. A rerun resumes a chunk left open first. A commit in the config's folder now takes only the file it changed, as `git commit -m -- <file>`. +- `loop-claude` runs each chunk in its own window of one tmux session per project, `loop-<key>`, and needs tmux 3.2 or later. Up to the config's new optional `parallel-chunks` chunks run at once, 1 by default; a chunk starts beside the others only where its item's `Touches:` prediction overlaps none of theirs. `loop-claude <task>` starts the windows and attaches to the session, and as a chunk finishes, its loop starts the next. A window whose chunk waits for you ends its name in ` ?` and rings the bell, and the question's DM names the window. A stopped chunk keeps its window and its items until you run `loop-claude <task>` again. A chunk's claude session is named after its window, as `A <key> loop-claude item-372`. A commit in the config's folder now takes only the file it changed, as `git commit -m <title> -- <file>`. To upgrade, finish or stop every running loop-claude before `deploy-release` moves `~/.claude` to this release: a loop that reloads into it starts its project's windows afresh, and leaves its chunk's open claude session in the session picker. - `todo-item finish` finishes an item of a plan kept in a Jira epic: it moves the item's issue by the workflow transition, and to the status, that the config's new optional settings `jira-finish-transition` and `jira-finish-status` name, both as Jira shows them. It refuses a transition that leads to another status, or to one Jira doesn't count as done, and fails where the config sets neither. - Reviewers and loop-claude's judge read the review standards from the folder a project's config sets as `standards`, in place of `~/.claude/skills/larv-review/standards/`. The setting is required. To upgrade, add `standards = "~/.claude/skills/larv-review/standards"` to every config under `~/.claude-projects/` right after `deploy-release` moves `~/.claude` to this release, and not before. - A project's config may keep each of its five records on a Confluence Cloud page: set, for example, `decisions = "confluence:https://<site>/wiki/spaces/<space>/pages/<page ID>"`, with the credentials in the config's `env-file`. `decisions` and `comprehension-history` may name a page only where `plan` names a Jira epic. The loop reads and writes the page as markdown, so a save replaces a mention or a macro on it with plain text. It never overwrites a teammate's edit: where someone saved the page since the loop read it, `project-record save` refuses and the loop asks you. diff --git a/README.md b/README.md index c9f02470..a580fadc 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,7 @@ git config filter.settings.clean bin/settings_filter.py Then pick a model with `/model`. -Link `loop-claude` onto `PATH`: `ln -s ~/.claude/bin/loop-claude ~/.local/bin/loop-claude`. Every claude it starts, the interactive ones included, skips every permission prompt. +Link `loop-claude` onto `PATH`: `ln -s ~/.claude/bin/loop-claude ~/.local/bin/loop-claude`. Every claude it starts, the interactive ones included, skips every permission prompt. loop-claude runs each chunk in its own window of a tmux session per project, so it needs tmux 3.2 or later. To get loop-claude's questions and stops as DMs from a bot, write the bot's token, your username or member ID and, for Mattermost, the server into `~/.claude-projects/<key>.env`, in the keys `~/.claude/bin/notify --help` names, run `chmod 600` on it, and add `env-file = "~/.claude-projects/<key>.env"` and `notifier = "mattermost"` (or `"slack"`) to the project's config, as Install → "Set up a project" describes. @@ -105,7 +105,7 @@ A project may also set `notifier` to `"mattermost"` or `"slack"`, the service th A project's config may set four more things, each optional, as `project-config --help` describes. `language` names the language its owners write in, where that isn't English: the worker writes the project's prose in it, and the prose reviewer judges the prose in it. `commit-title`, `pull-request-title` and `branch` are templates for a project that titles and names these in another way than `CLAUDE.md` → "Git, commits, and PRs" says. `chunk-preflight` names the branch and the worktree by the `branch` template. On a branch that fits it, rule-hook refuses a commit, and `chunk-pr` a pull request, whose title doesn't fit its template; on any other branch, such as a release review's, a title is the summary alone. A tool that reads a commit title, such as a round's `Answer <pass> round <n>:`, reads the template's `{summary}`: it tries `commit-title`, then `pull-request-title`, since a squash merge titles the commit as the pull request, then takes the title whole. While a project's config holds a fault, rule-hook refuses every commit there, and the tools that read its titles fail naming the fault. -A legacy project with no container setup sets `tooling-on-host = true`, so the loop runs its language runtime, package manager, tests and task runner directly on the host. One whose code the comprehension panel scores low may lower `readability-floor` from 7.0: under it, the ranking weighs code health 4 times as much, and a scoring run asks for a floor item. +A legacy project with no container setup sets `tooling-on-host = true`, so the loop runs its language runtime, package manager, tests and task runner directly on the host. One whose code the comprehension panel scores low may lower `readability-floor` from 7.0: under it, the ranking weighs code health 4 times as much, and a scoring run asks for a floor item. To let more than one chunk run at once, set `parallel-chunks` to the most that may; left out, it is 1. Beside the plan's next item, only items whose `Touches:` predictions overlap none of the running chunks' items start. A project whose pull requests need a human's approval sets `wait-for-approval = true` in its config. `chunk-pr merge` then waits, once the checks have passed, until GitHub reports the pull request approved as its branch protection requires, and merges it. It refuses on Gitea, where a reviewer requested changes and where the protection requires no review. While it waits, other chunks merge. It stops where a reviewer's comment waits for an answer, on an approved pull request too; answer it and run it again. It gives up after 90 minutes without an approval; run it again once the pull request is approved. Left out, the setting is false. @@ -174,7 +174,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a - `deploy-release [--cut] [<checkout>]` moves `~/.claude`, or the checkout named, to the newest release, as Update says; with `--cut`, run in the development clone, it first cuts the next release, as Release says. - `judge-replay <model>` replays loop-claude's judge on archived verdicts of the projects run on the model, and prints what each rung a review climbs to misses, and the lowest rung below them that, like each candidate above it, returns them; `judge-replay --help` states the replay. - `larv-review <command>` holds larv-review's mechanical rules: the chunk's review folder, the paired ruling's base and vote count, each pass's round count, the scoring run's trigger and setup; the comprehension history's format, its latest places and its count of how often seats agree; and each reviewer's prompt header. `larv-review --help` states them. `settings.json` → `hooks` runs `larv-review reviewer-context` as each subagent starts. A reviewer under `agents/` whose frontmatter has `omitClaudeMd: true` starts without `CLAUDE.md`, and the command hands it that file's "Shared goal", "Prose" and "Vocabulary" sections. -- `loop-claude <task>` runs a project's next chunk cycle after cycle; `loop-claude --help` states its contract. +- `loop-claude <task>` runs a project's next chunks, each in its own tmux window, cycle after cycle; `loop-claude --help` states its contract. - `metric-bench [history]` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest; `metric-bench --help` states the metrics and the scores. - `mutate-changes [--base <ref>] [--prefix <shell text>] [--jobs <n>]` mutation-tests the lines a branch changes and prints each mutant its tests don't kill; `mutate-changes --help` says which tool it runs for each language. - `notify [<message>]` sends a message, or else stdin, as a DM from the bot of the project's `notifier`, Mattermost or Slack, to the human; the project's `env-file` holds the bot's token, the recipient and, for Mattermost, the server, and `notify --help` names its keys. `loop-claude --help` says when loop-claude sends one. diff --git a/bin/chunk_preflight/choices.py b/bin/chunk_preflight/choices.py index 30a76c73..8c57cb91 100644 --- a/bin/chunk_preflight/choices.py +++ b/bin/chunk_preflight/choices.py @@ -25,8 +25,8 @@ def plan_name(default: str, outside: str | None) -> str: return outside if outside else f"{default}'s todo.md" -def planned(todo: plan_stores.Plan | Failure, cut: LastCut, default: str, branch: str, outside: str | None = None) -> Plan | NoItem: - """todo, which is the project's plan as its store read it and which plan_name names, with the item the next chunk takes first, release_standing.chunk_item's pick; else why there is none. branch is the project's `branch` template.""" +def planned(todo: plan_stores.Plan | Failure, cut: LastCut, default: str, branch: str, outside: str | None = None, item_id: int | None = None) -> Plan | NoItem: + """todo, which is the project's plan as its store read it and which plan_name names, with the item the next chunk takes first: the item of item_id where one is given, else release_standing.chunk_item's pick. Else why there is none. branch is the project's `branch` template.""" where = plan_name(default, outside) if isinstance(todo, NoItemsTable): return NoItem(f"{where} holds no items table.") @@ -56,7 +56,11 @@ def planned(todo: plan_stores.Plan | Failure, cut: LastCut, default: str, branch moved = [(was, now) for was, now in zip(found, held) if was.release != now.release] # Every carried item takes one release, the next. over = CarriedOver(moved[0][1].release, tuple(was.id for was, _ in moved)) if moved else None - return Plan(held, first, f"{first.tagline}\n\n{first.detail}".strip(), branch, over) + if item_id is not None: + first = next((item for item in held if item.id == item_id), None) + if first is None: + return NoItem(f"{where} holds no item {item_id}, the one --item names; it may have shipped.") + return Plan(held, first, f"{first.tagline}\n\n{first.detail}".strip(), branch, over, item_id is not None) def worktree_on(porcelain: str, branch: str) -> pathlib.Path | None: diff --git a/bin/chunk_preflight/facts.py b/bin/chunk_preflight/facts.py index be957712..46fc1143 100644 --- a/bin/chunk_preflight/facts.py +++ b/bin/chunk_preflight/facts.py @@ -45,7 +45,7 @@ def item_lines(plan: str, item: ItemFacts) -> list[str]: first = item.plan.first worktree = item.worktree.reason if isinstance(item.worktree, Failure) else f"{item.worktree.path}, {item.worktree.how}" return [ - f"- The {'next release, after the last cut, that ' + plan + ' names' if item.plan.carried else 'earliest release in ' + plan}: {version(first.release)}. Its first item that does not wait: item {first.id}, {levels(first)}.", + f"- The {'next release, after the last cut, that ' + plan + ' names' if item.plan.carried else 'earliest release in ' + plan}: {version(first.release)}. {'The item this chunk starts from, named with --item' if item.plan.named else 'Its first item that does not wait'}: item {first.id}, {levels(first)}.", *carried_lines(item.plan), f"- Worktree: {worktree}", f"- Open pull requests from {item.plan.branch}: {pulls_text(item.pulls, item.plan.branch)}", diff --git a/bin/chunk_preflight/gathered.py b/bin/chunk_preflight/gathered.py index e7c118b2..698d5a3e 100644 --- a/bin/chunk_preflight/gathered.py +++ b/bin/chunk_preflight/gathered.py @@ -38,6 +38,8 @@ class Plan: branch_template: str # The items of a release already cut, which `items` and `first` hold as the next release's. None where every item's release is after the last cut. carried: CarriedOver | None = None + # Whether --item named first, in place of the next release's first item that does not wait. + named: bool = False @property def branch(self) -> str: diff --git a/bin/chunk_preflight/main.py b/bin/chunk_preflight/main.py index 6d66449c..385e3990 100644 --- a/bin/chunk_preflight/main.py +++ b/bin/chunk_preflight/main.py @@ -1,6 +1,6 @@ """Find the mechanical facts a chunk starts from, so its worker starts from them; with --worktree, also make its worktree. -usage: chunk-preflight [--worktree] +usage: chunk-preflight [--worktree] [--item <ID>] Run it inside a project checkout. It fetches origin, when the checkout has that remote, and takes origin's default branch, or else the local main or master. Without --worktree, it still fetches, @@ -16,12 +16,13 @@ and prints that the release reviews and the cuts are the human's to call. It pri newest version CHANGELOG.md heads `## X.Y.Z - YYYY-MM-DD`; an undated heading doesn't count. Then how many entries the default branch's CHANGELOG.md holds under ## Unreleased, or else how many commits followed the cut: the tag, or the commit that added that heading; -- the next release and its first item that does not wait, the item loop-claude routes on, with - its L and E + I. An item waits while it is marked question, or while an item still in todo.md +- the next release and its first item that does not wait, the item loop-claude routes on, with its L + and E + I; with --item <ID>, the plan item of that ID instead, which loop-claude names in a window + named for an item. An item waits while it is marked question, or while an item still in todo.md blocks it. Where the last cut is a version x.y.z, the next release is the earliest after x.y.z - that todo.md has named at that cut or since, less a release its ## Renamed section marks - renamed. Where nothing is cut, it is the earliest release todo.md has ever named, less those - renamed, so a release whose items are all done still reads as next. + that todo.md has named at that cut or since, less a release its ## Renamed section marks renamed. + Where nothing is cut, it is the earliest release todo.md has ever named, less those renamed, so a + release whose items are all done still reads as next. Where the next release holds no item, it says so, naming it where todo.md named it since the cut: its release review due where changes are unreleased (the ## Unreleased entries, or else the commits since the cut) and no commit titled `Release review of <release>` followed the tag, @@ -65,7 +66,7 @@ running after {timeout} seconds, or after the seconds CHUNK_PREFLIGHT_TIMEOUT se stopped fetch reads origin as last fetched. With no item, it makes no worktree and runs none of those four tools, --worktree or not. -Exit 1, saying why, on any argument other than --worktree, -h or --help, on a +Exit 1, saying why, on any argument other than --worktree, --item with an ID, -h or --help, on a CHUNK_PREFLIGHT_TIMEOUT that is no number of seconds above 0, when git cannot start or finds no checkout or no default branch, or when no config names the checkout or its config cannot be read. """ @@ -92,16 +93,19 @@ TIMEOUT = 60 TIMEOUT_SETTING = "CHUNK_PREFLIGHT_TIMEOUT" # The flag that makes the item's worktree and runs TOOLS in it. Three readers seated as the README's audience picked it 2-1 over --start. WORKTREE = "--worktree" +# The flag that names the chunk's item. Three readers seated as the README's audience picked it 3-0 over --for, --start and a bare ID. +ITEM = "--item" def main(argv: list[str], env: typing.Mapping[str, str]) -> int: if argv in (["-h"], ["--help"]): print(__doc__.strip().format(timeout=TIMEOUT, JOINING_DETAILS=JOINING_DETAILS)) return 0 - if argv not in ([], [WORKTREE]): - print(f"unknown arguments {' '.join(argv)}; usage: chunk-preflight [{WORKTREE}]", file=sys.stderr) + rest = [arg for arg in argv if arg != WORKTREE] + if len(argv) - len(rest) > 1 or not (rest == [] or (len(rest) == 2 and rest[0] == ITEM and re.fullmatch(r"[0-9]+", rest[1], re.ASCII))): + print(f"unknown arguments {' '.join(argv)}; usage: chunk-preflight [{WORKTREE}] [{ITEM} <ID>]", file=sys.stderr) return 1 - printed = run(env.get(TIMEOUT_SETTING), argv == [WORKTREE]) + printed = run(env.get(TIMEOUT_SETTING), WORKTREE in argv, int(rest[1]) if rest else None) if isinstance(printed, Failure): print(printed.reason, file=sys.stderr) return 1 @@ -109,13 +113,13 @@ def main(argv: list[str], env: typing.Mapping[str, str]) -> int: return 0 -def run(setting: str | None, make: bool) -> list[str] | Failure: - """The preflight's lines, given the setting TIMEOUT_SETTING holds and whether to make the item's worktree; a Failure where the setting is no number of seconds above 0, or git finds no checkout or no default branch.""" +def run(setting: str | None, make: bool, item_id: int | None = None) -> list[str] | Failure: + """The preflight's lines, given the setting TIMEOUT_SETTING holds, whether to make the item's worktree, and the ID --item names; a Failure where the setting is no number of seconds above 0, or git finds no checkout or no default branch.""" timeout = seconds(setting) if isinstance(timeout, Failure): return timeout root = top() - return root if isinstance(root, Failure) else preflight(pathlib.Path(root), timeout, make) + return root if isinstance(root, Failure) else preflight(pathlib.Path(root), timeout, make, item_id) def seconds(setting: str | None) -> float | Failure: @@ -127,7 +131,7 @@ def seconds(setting: str | None) -> float | Failure: return Failure(f"{TIMEOUT_SETTING}={setting!r} is no number of seconds above 0") -def preflight(root: pathlib.Path, timeout: float, make: bool) -> list[str] | Failure: +def preflight(root: pathlib.Path, timeout: float, make: bool, item_id: int | None) -> list[str] | Failure: """Gathers the facts step by step, each step taking what the ones before it found, and renders them; a Failure where the checkout has no default branch.""" default = default_branch(stale_ok=True, timeout=timeout) if isinstance(default, Failure): @@ -138,7 +142,7 @@ def preflight(root: pathlib.Path, timeout: float, make: bool) -> list[str] | Fai return config outside = None if config.tracked("plan") else config.plan_label held = project_plan(config, ref, str(root)) - plan = planned(held.todo, held.cut, ref, config.branch, outside) + plan = planned(held.todo, held.cut, ref, config.branch, outside, item_id) item = plan if isinstance(plan, NoItem) else item_facts(root, ref, plan, timeout, make) # A `checkout:` path lands in the worktree the chunk works in. made = located(str(item.worktree.path), pathlib.Path.home()) if isinstance(item, ItemFacts) and isinstance(item.worktree, Worktree) else config diff --git a/bin/chunk_preflight/test_chunk_preflight.py b/bin/chunk_preflight/test_chunk_preflight.py index 5c41cf0c..71314bd3 100644 --- a/bin/chunk_preflight/test_chunk_preflight.py +++ b/bin/chunk_preflight/test_chunk_preflight.py @@ -319,6 +319,29 @@ class ChunkPreflightTool(unittest.TestCase): self.assertFalse((repo.temp / "proj-item-3").exists()) self.assertNotIn("item-3", repo.git(repo.root, "branch", "--list")) + def test_item_names_the_item_the_chunk_starts_from_in_place_of_the_plans_next(self): + repo = Repo(self) + for args in (["--item", "7"], ["--worktree", "--item", "7"], ["--item", "7", "--worktree"]): + with self.subTest(args): + done = repo.preflight(*args) + self.assertEqual((done.returncode, done.stderr), (0, "")) + self.assertIn("0.2.0. The item this chunk starts from, named with --item: item 7, L 1, E + I 2.", done.stdout) + self.assertNotIn("Its first item that does not wait", done.stdout) + self.assertTrue((repo.temp / "proj-item-7").is_dir()) + self.assertFalse((repo.temp / "proj-item-3").exists()) + + def test_item_names_no_item_where_the_plan_no_longer_holds_it(self): + done = Repo(self).preflight("--worktree", "--item", "4") + self.assertEqual((done.returncode, done.stderr), (0, "")) + self.assertIn("- No plan item: origin/main's todo.md holds no item 4, the one --item names; it may have shipped. No worktree was made.", done.stdout) + + def test_exits_1_on_an_item_that_is_no_id(self): + for args in (["--item"], ["--item", "x"], ["--item", "7", "8"]): + with self.subTest(args): + done = Repo(self).preflight(*args) + self.assertEqual((done.returncode, done.stdout), (1, "")) + self.assertIn(f"unknown arguments {' '.join(args)}; usage: chunk-preflight [--worktree] [--item <ID>]", done.stderr) + def test_exits_1_on_an_argument_other_than_worktree(self): done = Repo(self).preflight("--start") self.assertEqual(done.returncode, 1) @@ -391,6 +414,10 @@ class Render(unittest.TestCase): self.assertIn("item 7, L 1, E + I 2, S 1: **Do the thing after.**", said) self.assertNotIn("item 8,", said) + def test_says_the_item_was_named_with_item(self): + said = rendered(item_facts(plan=planned(TODO, CUT, "origin/main", BRANCH, None, 7))) + self.assertIn("origin/main's todo.md: 0.2.0. The item this chunk starts from, named with --item: item 7, L 1, E + I 2.", said) + def test_states_each_fact_once_with_the_paths_between_the_worktree_and_the_open_pull_requests(self): lines = render(gathered(item_facts())) self.assertEqual([line.partition(":")[0] for line in lines[:6]], ["- Last version cut", "- The earliest release in origin/main's todo.md", "- Worktree", "- The project's config puts its files and plan at", "- The project's config sets", "- Open pull requests from item-3"]) @@ -531,6 +558,14 @@ class Plans(unittest.TestCase): self.assertIn("L 'x'", planned(TODO.replace("| 2 | 2 | 3 | 5 |", "| x | 2 | 3 | 5 |"), CUT, "origin/main", BRANCH).why) self.assertIn("## Details holds a fenced code block", planned(TODO + "```\n", CUT, "origin/main", BRANCH).why) + def test_takes_the_item_item_names_even_one_that_waits_or_says_it_left_the_plan(self): + # Item 7 waits for item 3, which another chunk runs. + named = planned(TODO, CUT, "origin/main", BRANCH, None, 7) + self.assertEqual((named.first.id, named.first.tagline, named.named), (7, "**Do the thing after.**", True)) + self.assertTrue(named.text.startswith("**Do the thing after.**"), named.text) + self.assertFalse(planned(TODO, CUT, "origin/main", BRANCH).named) + self.assertEqual(planned(TODO, CUT, "origin/main", BRANCH, None, 4), NoItem("origin/main's todo.md holds no item 4, the one --item names; it may have shipped.")) + def test_tells_a_plan_with_no_items_table_from_one_whose_table_holds_no_item(self): without = planned("", CUT, "origin/main", BRANCH).why self.assertIn("origin/main", without) diff --git a/bin/judge_replay/test_judge_replay.py b/bin/judge_replay/test_judge_replay.py index cb242e76..273452f4 100644 --- a/bin/judge_replay/test_judge_replay.py +++ b/bin/judge_replay/test_judge_replay.py @@ -126,7 +126,7 @@ class Main(unittest.TestCase): result = reply(Rung(args[args.index("--model") + 1], args[args.index("--effort") + 1]), "said Verdict: interrupt" in prompt.rsplit("latest last:", 1)[-1]) return subprocess.CompletedProcess(args, 0, json.dumps(result), "") - effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=unused, input=unused, local_time=unused, memory=unused, now=unused, on_terminal=unused, python=unused, sleep=unused, stream_claude=unused) + effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=unused, input=unused, local_time=unused, memory=unused, now=unused, on_terminal=unused, python=unused, sleep=unused, stream_claude=unused, tmux=unused, tmux_on_terminal=unused) out, err = io.StringIO(), io.StringIO() with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): code = main.main(argv, {}, self.home, effects) diff --git a/bin/loop_claude/cycle/interrupt.py b/bin/loop_claude/cycle/interrupt.py index b012de19..37c42ca5 100644 --- a/bin/loop_claude/cycle/interrupt.py +++ b/bin/loop_claude/cycle/interrupt.py @@ -10,6 +10,7 @@ from claude_code.transcript import human_turns, read_transcript from loop_claude.stop import needs_human, needs_terminal, Stop from model_routing.ladder import Rung from loop_claude.direct_message import tell +from loop_claude import tmux ANSWER_QUESTION = "answer the question above" # TALK, what the human types at loop-claude's terminal to open claude for the question, and ASK_ANSWER were picked on 2026-10-17 by a bench of three readers seated as the README's audience, as ~/.claude/CLAUDE.md → "Working loop" → "Bench a syntax with the people who write it." has it. @@ -36,17 +37,26 @@ class Talked: """The human typed in the worker's claude session.""" -def interrupt(question: str, answer_here: bool, session: WorkerSession, effects: Effects) -> str | Talked | Stop: - """Prints question for the human and returns their answer: where answer_here, the first line typed at loop-claude's terminal, stripped, or Talked once they typed in the worker's claude session, which TALK opens; else Talked once they typed in that session, which opens at once on the terminal. A Stop where loop-claude is off a terminal, or they type no answer, or the input ends.""" - say(f"\n{question}", effects) +def interrupt(question: str, answer_here: bool, session: WorkerSession, window: tmux.Window, effects: Effects) -> str | Talked | Stop: + """Prints question for the human, in window, and returns their answer. While it waits, the window's name ends in tmux.WAITS, and the bell it rings shows in tmux's status bar. The answer: where answer_here, the first line typed at loop-claude's terminal, stripped, or Talked once they typed in the worker's claude session, which TALK opens; else Talked once they typed in that session, which opens at once on the terminal. A Stop where loop-claude is off a terminal, or they type no answer, or the input ends.""" + # The bell ends the question's line, so it rings once the question can be read. + say(f"\n{question}{tmux.BELL}", effects) on_terminal = effects.on_terminal() if on_terminal: # Keys typed while the worker ran wait on the terminal, and would answer a question no one has read, here or in claude's session, which reads them too. Of a pasted answer, the first line answers; the rest wait, and the next question drops them. A longer answer goes through TALK. effects.discard_typed() + tmux.mark(window, True, effects.tmux) # After the discard, so an answer typed while the DM sends counts; before the terminal check, so a loop that cannot ask still tells the human what it would have asked. - tell(f"loop-claude in {session.root} asks:\n\n{question}", effects) - if stop := needs_terminal(ANSWER_QUESTION, on_terminal): - return stop + tell(f"loop-claude in {session.root} asks, in the tmux window {window.target}:\n\n{question}", effects) + answer = needs_terminal(ANSWER_QUESTION, on_terminal) or answered(answer_here, session, effects) + # A loop that stops unanswered leaves the mark on its window. + if not isinstance(answer, Stop): + tmux.mark(window, False, effects.tmux) + return answer + + +def answered(answer_here: bool, session: WorkerSession, effects: Effects) -> str | Talked | Stop: + """The human's answer, as interrupt returns it.""" answer = typed_answer(effects) if answer_here else TALK if answer != TALK: return answer diff --git a/bin/loop_claude/cycle/prompts.py b/bin/loop_claude/cycle/prompts.py index 287c4bee..3010ac93 100644 --- a/bin/loop_claude/cycle/prompts.py +++ b/bin/loop_claude/cycle/prompts.py @@ -53,6 +53,11 @@ def preflight_facts(printed: str) -> str: return f"loop-claude ran `~/.claude/bin/chunk-preflight` at the checkout root just before this run. Take what it printed as found rather than finding it again:\n\n{printed}" +def held_by_others(items: list[int]) -> str: + """Keeps a fresh worker off the plan items the project's other chunks hold, running or stopped.""" + return f"\n\nOther chunks of this project, running or stopped, hold {'items ' if len(items) > 1 else 'item '}{', '.join(map(str, items))}. Leave those items out of this chunk, even where the facts above list them as items that may join it." + + def preflight_failed(reason: str) -> str: """Has the worker find the facts chunk-preflight would have, its worktree among them.""" return f"loop-claude could not run `~/.claude/bin/chunk-preflight`, so find the chunk's facts yourself. Why it could not run: {reason}" diff --git a/bin/loop_claude/cycle/run.py b/bin/loop_claude/cycle/run.py index c4751010..ac780bb1 100644 --- a/bin/loop_claude/cycle/run.py +++ b/bin/loop_claude/cycle/run.py @@ -24,6 +24,7 @@ from loop_claude.cycle.retry import retried from model_routing.route import Route from loop_claude.cycle import spend from loop_claude.stop import Stop +from loop_claude.tmux import Window from model_routing import worker_rungs @@ -54,6 +55,8 @@ class Cycle: todo: Plan | Failure # Empty when the previous cycle's verdict was not Unfinished. unfinished_evidence: str + # The tmux window the loop runs in, which an interrupt marks. + window: Window @dataclasses.dataclass(frozen=True) @@ -198,7 +201,7 @@ def keep_rejection(cycle: Cycle, transcript_path: str, rejected: Rejected) -> St def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) -> HumanTyped | HumanTalked | Stop: """The human's answer to a ruling that rule archived as it ruled; the human's answer then grows the transcript.""" # Only the human may cut a release, so a release ready to cut opens claude's session at once. - answer = interrupt(question_for(ruling), not (isinstance(ruling, Interrupt) and ruling.release_ready), worker_session(cycle, transcript_path), cycle.effects) + answer = interrupt(question_for(ruling), not (isinstance(ruling, Interrupt) and ruling.release_ready), worker_session(cycle, transcript_path), cycle.window, cycle.effects) match answer: case Stop(): return answer diff --git a/bin/loop_claude/loop/chunk_slot.py b/bin/loop_claude/loop/chunk_slot.py deleted file mode 100644 index 48503add..00000000 --- a/bin/loop_claude/loop/chunk_slot.py +++ /dev/null @@ -1,46 +0,0 @@ -"""The slot a loop holds in its project, so that loops of one project, each on its own chunk, run side by side, each in a session named after its slot.""" -import dataclasses -import os -import pathlib - -from failure import Failure -from machine_load.slots import first_free_slot -from loop_claude.loop.reload import LoopRun -from loop_state.paths import Paths -from transcript_archive.project_key import project_key -from claude_code.session import Session, session_named, session_open - -# Under the home folder, a folder per project key holds a file per slot, which a running loop keeps locked with flock. -SLOTS = ".local/state/loop-claude" -# A bound no project reaches. Once todo.md item 375 lands, the project's config sets how many loops run at once. -MOST = 64 - - -def held_slot(home: pathlib.Path, key: str, claude_code_transcripts: pathlib.Path) -> int | Failure: - """The slot this loop takes, or why it could take none: the lowest free slot whose session is open, so that a rerun resumes a chunk left open, or else the lowest free slot. The loop holds it until it exits; a reload's exec keeps the lock.""" - folder = home / SLOTS / key - order = sorted(range(MOST), key=lambda slot: not session_open(session_named(session_name(key, slot), claude_code_transcripts))) - try: - folder.mkdir(parents=True, exist_ok=True) - taken = first_free_slot(folder, order) - except OSError as failure: - return Failure(f"cannot take a slot in {folder}: {failure.strerror}") - if taken is None: - return Failure(f"{MOST} loops of this project are running already; `fuser {folder}/slot-*` names them") - slot, fd = taken - os.set_inheritable(fd, True) - return slot - - -def session_name(key: str, slot: int) -> str: - """"A " is the human's cue in the session picker that the session is the loop's. Slot 0 keeps the name every loop had before slots, so a chunk left open before slots resumes in it.""" - return f"A {key} loop-claude" if slot == 0 else f"A {key} loop-claude {slot + 1}" - - -def slotted(loop_run: LoopRun, paths: Paths) -> tuple[LoopRun, Session] | Failure: - """loop_run holding its slot, and the worker's session, named after the slot. A reloaded script keeps the slot it is handed.""" - key = project_key(loop_run.root, paths.home) - slot = held_slot(paths.home, key, paths.claude_code_transcripts) if loop_run.slot is None else loop_run.slot - if isinstance(slot, Failure): - return slot - return dataclasses.replace(loop_run, slot=slot), session_named(session_name(key, slot), paths.claude_code_transcripts) diff --git a/bin/loop_claude/loop/chunk_start.py b/bin/loop_claude/loop/chunk_start.py index 388fdf3f..c58c70ae 100644 --- a/bin/loop_claude/loop/chunk_start.py +++ b/bin/loop_claude/loop/chunk_start.py @@ -8,6 +8,8 @@ from loop_effects.effects import Effects, say from failure import Failure from loop_claude.routing import first_route, kept, routed_item from plan_store.stores import held_at, Plan, store as plan_store +from loop_claude.loop.chunk_windows import held_beside, item_of +from chunk_preflight.main import ITEM from release_plan.todo_table import Item from model_routing.ladder import Ladder, Rung from transcript_archive.read import latest_worker @@ -61,15 +63,16 @@ def archived_route(archive_folder: pathlib.Path, loop: Loop) -> Route | None: def chunk_start(loop: Loop) -> tuple[str, typing.Callable[[], Item | None]]: """The facts for a fresh worker's prompt, empty where the session is open, and the reader of the item that routes the chunk. The reader reads the default branch as last fetched: for a fresh chunk, by the preflight that runs first; for an open session, which runs none, by whatever fetched last.""" facts = "" if session_open(loop.session) else chunk_facts(loop) - return facts, lambda: plan_item(fetched_plan(loop), loop.effects) + return facts, lambda: plan_item(fetched_plan(loop), item_of(loop.window.name), loop.effects) def chunk_facts(loop: Loop) -> str: - """What `chunk-preflight --worktree` printed at the checkout root, for a fresh worker's prompt, or why it could not run.""" - done = loop.effects.python([str(loop.launcher.with_name("chunk-preflight")), "--worktree"], loop.root) - if done.returncode: - return prompts.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}") - return prompts.preflight_facts(done.stdout.strip()) + """What `chunk-preflight --worktree` printed at the checkout root, for a fresh worker's prompt, or why it could not run. The chunk of a window named for an item starts from that item, and either ends with the items the project's other chunks hold.""" + item = item_of(loop.window.name) + done = loop.effects.python([str(loop.launcher.with_name("chunk-preflight")), "--worktree", *([] if item is None else [ITEM, str(item)])], loop.root) + facts = prompts.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}") if done.returncode else prompts.preflight_facts(done.stdout.strip()) + held = held_beside(loop.window.name, loop.root, loop.paths, loop.effects) + return facts + (prompts.held_by_others(held) if held else "") def fetched_plan(loop: Loop) -> DefaultPlan | Failure: @@ -83,13 +86,13 @@ def fetched_plan(loop: Loop) -> DefaultPlan | Failure: return ref if isinstance(ref, Failure) else project_plan(config, ref, loop.root, loop.effects.git) -def plan_item(plan: DefaultPlan | Failure, effects: Effects) -> Item | None: - """The item that routes a chunk, release_standing.chunk_item's pick from the default branch; None without one, so the chunk runs on the project's rung.""" +def plan_item(plan: DefaultPlan | Failure, window_item: int | None, effects: Effects) -> Item | None: + """The item that routes a chunk: the one its window is named for, or else release_standing.chunk_item's pick from the default branch; None without one, so the chunk runs on the project's rung.""" # It prints the reason itself, through effects, like the other loop-claude functions that print. todo.md item 51 moves them all to one injected effect, so changing this one alone would be undone. if isinstance(plan, Failure): item: Item | Failure | None = plan else: - item = routed_item(plan.todo, plan.cut) + item = routed_item(plan.todo, plan.cut, window_item) if isinstance(item, Failure): say(f"routing on the project's rung: {item.reason}", effects) return None diff --git a/bin/loop_claude/loop/chunk_windows.py b/bin/loop_claude/loop/chunk_windows.py new file mode 100644 index 00000000..31f88601 --- /dev/null +++ b/bin/loop_claude/loop/chunk_windows.py @@ -0,0 +1,154 @@ +"""The chunks of one project that run at once, each in its own tmux window: which may start now, and how each starts (docs/decisions.md → `loop-claude: each chunk runs in its own tmux window`).""" +import contextlib +import fcntl +import pathlib +import re +import typing + +from touch_predictions.beside import Beside, chunks_beside +from loop_claude.cycle import chunk_items +from git_cli.git_repo import default_ref +from loop_effects.effects import Effects, say +from failure import Failure +from project_config.main import located +from loop_claude.loop.reload import LoopRun +from chunk_preflight.gathered import NoItem, Plan +from touch_predictions.prediction import overlaps, predicted, Touches +from loop_state.paths import Paths +from chunk_preflight.choices import planned +from transcript_archive.project_key import project_key +from release_plan.release_standing import project_plan +from claude_code.transcript import read_transcript +from claude_code.session import Session, session_named, transcript_file +from loop_claude import tmux + +# Under the home folder, a folder per project key holds LOCK. +STATE = ".local/state/loop-claude" +# The file a loop keeps locked with flock while it lists the project's windows and starts new ones, so two loops whose chunks finish together don't start a chunk twice. +LOCK = "starting" +# The flag that tells the loop in a window its chunk, as `--chunk item-372`. Nobody types it, so no panel picked it. +CHUNK_FLAG = "--chunk" +# The chunk of a plan that names no item for the next chunk: a release to plan, a release review, or a task outside the plan. +NO_ITEM = "no-item" +# Seconds a fetch of origin may take before a chunk starts; past them, the plan reads as last fetched. +FETCH_TIMEOUT = 60 +# The prefix of the variables loop-claude reads. A window gets the tmux server's environment, not the starting terminal's, so they are handed to each window by name. +HANDED = "LOOP_CLAUDE_" + + +def window_name(item_id: int) -> str: + return f"item-{item_id}" + + +def item_of(chunk: str) -> int | None: + """The ID of the plan item a chunk starts from; None for NO_ITEM.""" + named = re.fullmatch(r"item-([0-9]+)", chunk) + return int(named[1]) if named else None + + +def chunk_session(key: str, chunk: str, claude_code_transcripts: pathlib.Path) -> Session: + """The worker's session for a chunk. A rerun finds a chunk left open by this name. "A " is the human's cue in the session picker that the session is the loop's.""" + return session_named(f"A {key} loop-claude {chunk}", claude_code_transcripts) + + +@contextlib.contextmanager +def starting(home: pathlib.Path, key: str) -> typing.Iterator[None]: + """Holds the project's LOCK. Raises OSError where the file cannot be opened or locked.""" + folder = home / STATE / key + folder.mkdir(parents=True, exist_ok=True) + with open(folder / LOCK, "a") as lock: + fcntl.flock(lock, fcntl.LOCK_EX) + yield + + +def held_items(chunks: typing.Iterable[str], key: str, claude_code_transcripts: pathlib.Path) -> set[int]: + """The plan items these chunks hold: the one each started from, and those its worker names on its latest `Chunk items:` line so far. A chunk whose transcript cannot be read holds the item it started from.""" + held: set[int] = set() + for chunk in chunks: + held.update(filter(None, [item_of(chunk)])) + transcript = transcript_file(chunk_session(key, chunk, claude_code_transcripts)) + entries = read_transcript(transcript) if transcript else [] + held.update([] if isinstance(entries, Failure) else chunk_items.named(entries)) + return held + + +def may_start(plan: Plan | NoItem, running: list[str], held: set[int], limit: int) -> list[str]: + """The chunks to start now beside the running ones, whose chunks hold the items in held: the plan's next item and the items touch_predictions picks to run beside it, in the plan's order, up to limit chunks running at once. Each leaves out an item in held, and one whose `Touches:` prediction overlaps a held item's, or where either has none. A plan without an item starts NO_ITEM, and only alone.""" + if isinstance(plan, NoItem): + return [] if running else [NO_ITEM] + beside = chunks_beside(plan.items, plan.first) + picked = [plan.first.id, *(beside.picked if isinstance(beside, Beside) else [])] + details = {item.id: item.detail for item in plan.items} + # chunks_beside checks overlaps against the plan's next item only, and a running chunk's item may no longer be that one once an item merged. A held item that left the plan overlaps nothing. + theirs = [predicted(details[item_id]) for item_id in sorted(held) if item_id in details] + free = [item_id for item_id in picked if item_id not in held and clear_of(predicted(details[item_id]), theirs)] + return [window_name(item_id) for item_id in free][: max(limit - len(running), 0)] + + +def clear_of(ours: Touches | Failure | None, theirs: list[Touches | Failure | None]) -> bool: + """Whether a prediction overlaps none of theirs; not where one of them is missing or unreadable, unless theirs is empty.""" + return not theirs or (isinstance(ours, dict) and all(isinstance(touches, dict) and not overlaps(ours, touches) for touches in theirs)) + + +def next_plan(root: str, home: pathlib.Path, effects: Effects) -> Plan | NoItem | Failure: + """The plan on the default branch, fetched now where origin answers, with its next item, as chunk-preflight reads it.""" + config = located(root, home) + if isinstance(config, Failure): + return config + # A chunk that just merged left the plan on origin. A fetch that fails leaves the plan as last fetched, which chunk-preflight then reads too. + effects.git("fetch", "--quiet", "origin", cwd=root, timeout=FETCH_TIMEOUT) + ref = default_ref(root, FETCH_TIMEOUT, run=effects.git) + if isinstance(ref, Failure): + return ref + held = project_plan(config, ref, root, effects.git) + return planned(held.todo, held.cut, ref, config.branch, None if config.tracked("plan") else config.plan_label) + + +def start_chunks(loop_run: LoopRun, leaving: bool, paths: Paths, env: typing.Mapping[str, str], launcher: pathlib.Path, effects: Effects) -> list[str] | Failure: + """Starts a loop-claude in a new window for each chunk of loop_run's project that may start now, and returns the chunks started. leaving says that loop_run's own chunk finished, so its window no longer counts as running; without it, the human started loop-claude. A plan that cannot be read starts NO_ITEM, as a plan without an item does. A Failure says why a chunk could not start; those before it run.""" + config = located(loop_run.root, paths.home) + if isinstance(config, Failure): + return config + key = project_key(loop_run.root, paths.home) + session = tmux.session_name(key) + handed = {name: value for name, value in env.items() if name.startswith(HANDED)} + try: + with starting(paths.home, key): + windows = tmux.listed(session, effects.tmux) + if windows and (owned := tmux.owner(session, effects.tmux)) != loop_run.root: + return Failure(f"the tmux session {session} runs the chunks of {owned or 'another checkout'}, whose key maps to the same session name; rename one of the two checkouts' folders") + running = [window.name for window in windows if not window.dead and not (leaving and window.name == loop_run.chunk)] + # Below 0 where the config lowered parallel-chunks while more chunks ran. + room = max(config.parallel_chunks - len(running), 0) + # A loop that the human stopped, or that failed, keeps its window, dead, with its last lines and its items. Only the human's loop-claude <task> starts it again, before any other chunk. + stopped = [window.name for window in windows if window.dead] + chunks = [] if leaving else stopped[:room] + if len(chunks) < room: + plan = next_plan(loop_run.root, paths.home, effects) + if isinstance(plan, Failure): + # Such as in a checkout without a default branch, where chunk-preflight fails too: the worker is told why and finds the facts itself. + say(f"starting a chunk without a plan item, since the plan cannot be read: {plan.reason}", effects) + plan = NoItem(plan.reason) + held = held_items(running + stopped, key, paths.claude_code_transcripts) + chunks += [chunk for chunk in may_start(plan, running + chunks, held, config.parallel_chunks) if chunk not in stopped] + for chunk in chunks: + stale = [window for window in windows if window.name == chunk and window.dead] + for window in stale: + tmux.close(window, effects.tmux) + windows = [window for window in windows if window not in stale] + command = [str(launcher), CHUNK_FLAG, chunk, loop_run.task] + if failed := tmux.start_window(tmux.Window(session, chunk, ""), loop_run.root, handed, command, not windows, effects.tmux): + return failed + windows.append(tmux.Listed(chunk, "", False)) + return chunks + except OSError as failure: + return Failure(f"cannot lock {paths.home / STATE / key / LOCK}: {failure.strerror}") + + +def held_beside(chunk: str, root: str, paths: Paths, effects: Effects) -> list[int]: + """The plan items that the project's other chunks hold now, running or stopped, which a chunk starting as `chunk` leaves to them. It waits for a loop that is starting windows, so chunks started together see each other. Where the lock cannot be taken, it reads the windows without it.""" + key = project_key(root, paths.home) + with contextlib.suppress(OSError), starting(paths.home, key): + pass + others = [window.name for window in tmux.listed(tmux.session_name(key), effects.tmux) if window.name != chunk] + return sorted(held_items(others, key, paths.claude_code_transcripts)) diff --git a/bin/loop_claude/loop/main.py b/bin/loop_claude/loop/main.py index 0013e67a..6843a8e8 100644 --- a/bin/loop_claude/loop/main.py +++ b/bin/loop_claude/loop/main.py @@ -6,11 +6,13 @@ from transcript_archive.location import archive_root, project_archive from loop_claude.loop.chunk_start import archived_route, chunk_start, fetched_todo, routed from loop_claude.loop.routing_check import check_routing from loop_claude.project.checkout import checkout_root +from loop_claude.loop.chunk_windows import CHUNK_FLAG, chunk_session, start_chunks from loop_claude.loop.close import closed_cycle from loop_claude.cycle.run import Cycle, run_cycle from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG from loop_effects.effects import Effects, say from failure import Failure +from loop_claude.verdicts import Finished from loop_claude.project.model_choice import get_or_ask_models from git_cli.git_repo import Git from loop_claude.loop.usage import HELP @@ -20,9 +22,9 @@ from transcript_archive.project_key import project_key from model_routing.ladder_file import project_ladder_file, read_ladder, SHIPPED from loop_claude.project.premises import read_premises from loop_claude.project.records import read_records -from loop_claude.loop.chunk_slot import slotted from loop_claude.stop import Stop from loop_claude.direct_message import tell +from loop_claude import tmux def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int: @@ -35,8 +37,8 @@ def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: raise if stop.reason: say(stop.reason, effects, stdout=not stop.code) - # Every stop but the human's own Ctrl-C and the help is DMed where the project's config can be read and sets `env-file`, so a loop left overnight never ends unseen. A loop given no task prints the help. - if stop.code != 130 and stop.reason != HELP: + # Every stop with a reason, but the help, is DMed where the project's config can be read and sets `env-file`, so a loop left overnight never ends unseen. A loop given no task prints the help. Ctrl-C stops without a reason. So does the loop that started the project's windows once it attaches or prints how to, and a loop whose chunk finished once it started the next chunks, since the project's chunks go on. + if stop.reason not in ("", HELP): tell(f"loop-claude in {cwd} stopped: {stop.reason}", effects) return stop.code @@ -45,12 +47,18 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc loop_run = loop_run_from(argv, cwd, effects) if isinstance(loop_run, Stop): return loop_run + # Before anything starts: a worktree, a claude session or a tmux window. + if reason := tmux.unusable(effects.tmux): + return Stop(1, reason) + if loop_run.chunk is None: + return windows_opened(loop_run, env, paths, launcher, effects) + pane = env.get("TMUX_PANE") + if not pane: + return Stop(1, f"{CHUNK_FLAG} is how loop-claude starts itself in a tmux window; run loop-claude <task>") baseline = fingerprint(launcher) - held = slotted(loop_run, paths) - if isinstance(held, Failure): - return Stop(1, held.reason) - loop_run, session = held - loop = Loop(root=loop_run.root, session=session, paths=paths, env=env, launcher=launcher, effects=effects) + key = project_key(loop_run.root, paths.home) + window = tmux.Window(tmux.session_name(key), loop_run.chunk, pane) + loop = Loop(root=loop_run.root, session=chunk_session(key, loop_run.chunk, paths.claude_code_transcripts), window=window, paths=paths, env=env, launcher=launcher, effects=effects) while True: if stop := exec_if_edited(loop, baseline, loop_run): return stop @@ -68,6 +76,31 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc # Off the ladder the worker runs on the project's model itself, which its route names. if stop := check_routing(archive_root(paths.home, env), cycle.project or cycle.route.rung, loop): return stop + if isinstance(concluded.verdict, Finished): + return handed_on(loop_run, loop) + + +def windows_opened(loop_run: LoopRun, env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects) -> Stop: + """What the loop-claude the human runs does: starts each chunk that may start now in a window of the project's tmux session, then puts that session on the terminal until the human detaches.""" + started = start_chunks(loop_run, False, paths, env, launcher, effects) + if isinstance(started, Failure): + return Stop(1, started.reason) + session = tmux.session_name(project_key(loop_run.root, paths.home)) + say(f"started {', '.join(started)} in the tmux session {session}" if started else f"the tmux session {session} runs every chunk that may run now, so `{loop_run.task}` started in no new window", effects, stdout=True) + if not effects.on_terminal(): + say(f"not on a terminal; see the chunks with `tmux attach -t ={session}`", effects, stdout=True) + return Stop(0, "") + shown = tmux.show(session, "TMUX" in env) + return Stop(1, f"`tmux {' '.join(shown)}` failed; the chunks go on in the tmux session") if effects.tmux_on_terminal(shown) else Stop(0, "") + + +def handed_on(loop_run: LoopRun, loop: Loop) -> Stop: + """The stop of a loop whose chunk finished, once it has started each chunk that may start now. Its window closes as it exits.""" + started = start_chunks(loop_run, True, loop.paths, loop.env, loop.launcher, loop.effects) + if isinstance(started, Failure): + return Stop(1, f"the chunk finished, but the next could not start: {started.reason}") + say(f"the chunk finished; started {', '.join(started) or 'no other, since the chunks running fill what may run now'}", loop.effects, stdout=True) + return Stop(0, "") def next_cycle(loop_run: LoopRun, loop: Loop) -> Cycle | Stop: @@ -96,7 +129,7 @@ def next_cycle(loop_run: LoopRun, loop: Loop) -> Cycle | Stop: if ladder_file != SHIPPED: say(own_ladder(ladder_file, models.worker, project is None), loop.effects) todo = fetched_todo(loop) - return Cycle(config=setup_version(loop.paths.claude_dir, loop.effects.git), effects=loop.effects, facts=facts, judge_model=models.judge, ladder=ladder, premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=loop_run.task, todo=todo, unfinished_evidence=loop_run.unfinished_evidence) + return Cycle(config=setup_version(loop.paths.claude_dir, loop.effects.git), effects=loop.effects, facts=facts, judge_model=models.judge, ladder=ladder, premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=loop_run.task, todo=todo, unfinished_evidence=loop_run.unfinished_evidence, window=loop.window) def own_ladder(file: pathlib.Path, model: str, unrouted: bool) -> str: @@ -107,11 +140,12 @@ def own_ladder(file: pathlib.Path, model: str, unrouted: bool) -> str: def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopRun | Stop: if argv[:1] != [RELOAD_FLAG]: - task = " ".join(argv) + chunk, words = (argv[1], argv[2:]) if len(argv) >= 2 and argv[0] == CHUNK_FLAG else (None, argv) + task = " ".join(words) if task in ("", "-h", "--help"): return Stop(0 if task else 1, HELP) root = checkout_root(cwd, effects) - return root if isinstance(root, Stop) else LoopRun(0, root, task, "") + return root if isinstance(root, Stop) else LoopRun(0, root, task, "", chunk=chunk) loop_run = decode_handed(argv[1]) if len(argv) == 2 else None if not (loop_run and loop_run.root and loop_run.task): return Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}") diff --git a/bin/loop_claude/loop/reload.py b/bin/loop_claude/loop/reload.py index 238cab82..0836d802 100644 --- a/bin/loop_claude/loop/reload.py +++ b/bin/loop_claude/loop/reload.py @@ -25,8 +25,8 @@ class LoopRun: route: Route | None = None # How many unfinished verdicts the chunk has had running; a finished chunk and a rerun each set it to 0. It stays apart from route and unfinished_evidence, though all three change together at each verdict: merging or renaming a field stops a running loop at its next reload. unfinished_run: int = 0 - # The project's slot this loop holds (chunk_slot.py); None until it takes one. A loop started before slots reloads with None, and so takes a slot then. - slot: int | None = None + # The chunk this loop runs in its tmux window, as chunk_windows names it: `item-372`, or `no-item`. None for the loop-claude the human started, which starts the windows; a loop started before chunks ran in windows reloads with None too, and so starts the project's windows; its open claude session, named for its slot, is left in the session picker (docs/decisions.md → `loop-claude: each chunk runs in its own tmux window, …`). + chunk: str | None = None class Unreadable(Exception): @@ -40,8 +40,10 @@ def decode_handed(handed: str) -> LoopRun | None: except ValueError: return None # A loop started before 0.11.0 hands the items loop-inbox listed; dropping them keeps it running across its reload. + # One started before chunks ran in tmux windows hands the slot it held. if isinstance(fields, dict): fields.pop("items_taken_in", None) + fields.pop("slot", None) try: return LoopRun(**decoded_fields(fields, LOOP_RUN_FIELDS, "cycle_number", "root", "task", "unfinished_evidence")) except Unreadable: @@ -77,7 +79,7 @@ def decoded_route(value: object) -> Route | None: # One decoder per field of LoopRun, which a test holds to the dataclass's fields: a field missing here stops a loop at the reload after it joined. -LOOP_RUN_FIELDS = {"cycle_number": exactly(int), "root": exactly(str), "task": exactly(str), "unfinished_evidence": exactly(str), "route": decoded_route, "unfinished_run": exactly(int), "slot": lambda value: None if value is None else exactly(int)(value)} +LOOP_RUN_FIELDS = {"cycle_number": exactly(int), "root": exactly(str), "task": exactly(str), "unfinished_evidence": exactly(str), "route": decoded_route, "unfinished_run": exactly(int), "chunk": lambda value: None if value is None else exactly(str)(value)} def fingerprint(launcher: pathlib.Path) -> str: diff --git a/bin/loop_claude/loop/this_loop.py b/bin/loop_claude/loop/this_loop.py index 88ba8c77..a3269bd6 100644 --- a/bin/loop_claude/loop/this_loop.py +++ b/bin/loop_claude/loop/this_loop.py @@ -5,6 +5,7 @@ import typing from loop_effects.effects import Effects from loop_state.paths import Paths from claude_code.session import Session +from loop_claude.tmux import Window @dataclasses.dataclass(frozen=True) @@ -14,6 +15,8 @@ class Loop: root: str # The worker's session. session: Session + # The tmux window the loop runs in. + window: Window paths: Paths env: typing.Mapping[str, str] # The loop-claude script, whose edits reload the loop and beside which chunk-preflight lies. diff --git a/bin/loop_claude/loop/usage.py b/bin/loop_claude/loop/usage.py index 5b5da242..437d26f2 100644 --- a/bin/loop_claude/loop/usage.py +++ b/bin/loop_claude/loop/usage.py @@ -22,35 +22,58 @@ asked again after its answer. A question the human answered earlier in the chunk worker with that answer, unless the question quotes the words of the answer that leave it open. The fourth question they answer in one cycle goes to the human too, with the quote. An unfinished chunk resumes its session next cycle, and the worker is shown the passage the judge quoted as the work -remaining. The next chunk starts a fresh session. Cycles continue until an exit below or Ctrl-C. +remaining. The next chunk starts a fresh session. Cycles continue until an exit below or Ctrl-C, +either of which stops that one chunk. Words: ~/.claude/CLAUDE.md → "Vocabulary". +loop-claude needs tmux 3.2 or later, and checks it before it starts anything. Each chunk runs in its +own window of one tmux session per project, named loop-<key>, such as loop-claude-config. <key> is +the checkout's path relative to ~/code, or to / outside it, with a dash for each character other +than a letter, a digit, _ or -. The tmux session's @loop-root option names its checkout, and a +checkout whose key maps to a tmux session another checkout runs is refused. loop-claude <task> +starts the chunks that may start now, then attaches to the tmux session, or switches to it inside +tmux; detaching leaves the chunks running. Off a terminal it prints the command that attaches. Where +no chunk may start, it says so, and the task starts in no new window. At most parallel-chunks chunks +run at once, a whole number the project's config sets, 1 by default. The first starts from the +plan's next item, and each other from an item whose `Touches:` prediction overlaps neither one +before it nor an item another chunk holds (touch-predictions --help). A window is named for the item +its chunk starts from, as item-372; a chunk without a plan item, such as planning a release or its +release review, runs alone in the window no-item. + +Each window runs a loop-claude for that one chunk, started with --chunk <window>, which nobody +types. As a chunk finishes, its loop starts each chunk that may start then, with the task it ran +itself, and exits, which closes its window. A loop that stops, by Ctrl-C, an exit below or a +failure, keeps its window, with its last lines, and its chunk keeps its items; the next loop-claude +<task> starts it again before any other. To stop every chunk, press Ctrl-C in each window. A chunk's +claude session is named "A <key> loop-claude <window>", so a rerun resumes a chunk left open, and is +deleted when the chunk finishes. A fresh worker is told the items the project's other chunks hold, +running or stopped: each one's own and those on its latest `Chunk items:` line, and leaves them out. + A question only the human can answer is an interrupt (bin/loop_claude/cycle/interrupt.py). -loop-claude prints the question on its terminal, where the human answers it: +loop-claude prints the question in the chunk's window and rings its bell. Until it is answered the +window's name ends in " ?", as in "item-372 ?", which tmux's status bar shows. The human answers it +there: - A line of text: the worker resumes headless with it. The judge reads it as the human's, so the chunk stays unfinished until the answer is recorded. -- /talk: opens the worker's session with `claude --resume` on the terminal, to read the diff or - discuss first. Once that claude exits, the judge rules on what was typed there. +- /talk: opens the worker's claude session with `claude --resume` on the terminal, to read the diff + or discuss first. Once that claude exits, the judge rules on what was typed there. - Enter alone: stops the loop. -A release ready to cut opens that session on the terminal at once, since only the human may cut it. -While claude is open, Ctrl-C reaches it alone, and the loop goes on. -Where the project's config sets notifier, loop-claude also sends the human each question, each -release ready to cut and each stop as a DM, as bin/notify sends it. Two stops are -left out: Ctrl-C, and printing this help. A DM that fails is said on stderr, and the loop goes on. +A release ready to cut opens that claude session on the terminal at once, since only the human may +cut it. While claude is open, Ctrl-C reaches it alone, and the loop goes on. Where the project's +config sets notifier, loop-claude also sends the human each question, naming its tmux window as +<session>:<window>, each release ready to cut and each stop as a DM, as bin/notify sends it. Four +stops are left out: Ctrl-C, printing this help, the loop-claude that started the windows once it +attaches or prints how to, and a loop whose chunk finished once it started the next chunks. A DM +that fails is said on stderr, and the loop goes on. -As a chunk starts, before it is routed, loop-claude runs bin/chunk-preflight --worktree and quotes -what it prints in the first prompt (prompts.preflight_facts in bin/loop_claude/cycle/prompts.py); -chunk-preflight --help says what it finds and makes. A preflight that fails, or is still running -after 600 s and is stopped, leaves the worker to find the facts itself. +As a chunk starts, before it is routed, loop-claude runs bin/chunk-preflight --worktree, adding +--item <ID> in a window named for an item, and quotes what it prints in the first prompt +(prompts.preflight_facts in bin/loop_claude/cycle/prompts.py); chunk-preflight --help says what it +finds and makes. A preflight that fails, or is still running after 600 s and is stopped, leaves the +worker to find the facts itself. -Loops of one project run side by side, each on one chunk at a time. Each holds a slot of the -project, the file slot-<n> under ~/.local/state/loop-claude/<key>/, locked while the loop runs. -<key> is the checkout's path relative to ~/code, or to / outside it. A loop takes the lowest free -slot whose session is open, so a rerun resumes a chunk left open, or else the lowest free slot. -slot-0's session is named "A <key> loop-claude", and slot-<n>'s "A <key> loop-claude <n+1>". -The session is deleted when its chunk finishes. At every -interrupt, and as every cycle ends, Ctrl-C and a failure included, the session's transcript is -copied into a folder per chunk under $LOOP_CLAUDE_TRANSCRIPTS/<key> (default root +At every interrupt, and as every cycle ends, Ctrl-C and a failure included, the claude session's +transcript is copied into a folder per chunk under $LOOP_CLAUDE_TRANSCRIPTS/<key> (default root ~/.claude/loop-claude-transcripts), named by its hash, with an outcomes file beside it: the task, then each verdict, quoted answer, rejected judge reply, step-up and stop, each opened by its UTC time. The layout, and when a copy is replaced, are in bin/transcript_archive/__init__.py. @@ -116,7 +139,11 @@ loop-claude --help succeeds: the loop reloads into the edited script. Exit 1, saying why, when: - no task is given - git cannot start or finds no checkout -- its slot in the project cannot be taken +- tmux is missing or older than 3.2 +- --chunk is given outside a tmux window +- a chunk's window cannot be started, as when the project's config cannot be read, or the tmux + session runs another checkout's chunks +- the tmux session cannot be put on the terminal - the goals file, or AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/*.md or the project's review standards where they exist, cannot be read or is not UTF-8 - no config names the checkout, its config cannot be read, or the goals file holds no premises @@ -146,4 +173,5 @@ Exit 2, when the human is needed to go on: - the claude opened for /talk, or for a release ready to cut, ends without anything typed - the judge rules a chunk unfinished 6 cycles running; rerun to give it 6 more -Rerunning resumes an open session that no running loop holds, the lowest one first.""" +Rerunning loop-claude <task> starts the chunks that may start, a stopped one first, and each +resumes its claude session where it is open.""" diff --git a/bin/loop_claude/routing.py b/bin/loop_claude/routing.py index e5da7906..52f769a9 100644 --- a/bin/loop_claude/routing.py +++ b/bin/loop_claude/routing.py @@ -45,14 +45,17 @@ ESCALATION_CAP = 2 JUDGE_REPLAYED = ("claude-opus-5-5",) -def routed_item(plan: Plan | Failure, cut: LastCut) -> Item | Failure | None: - """The item release_standing.chunk_item picks from plan, weighed against the last cut; None when there is none, or it lacks L, E or I.""" +def routed_item(plan: Plan | Failure, cut: LastCut, item_id: int | None = None) -> Item | Failure | None: + """The plan's item item_id, or without one the item release_standing.chunk_item picks from plan, weighed against the last cut; None when there is none, or it lacks L, E or I.""" if isinstance(plan, NoItemsTable): return None if isinstance(plan, Failure): return plan - pick = chunk_item(plan.items, cut) - takes = pick.item if isinstance(pick, Next) else None + if item_id is None: + pick = chunk_item(plan.items, cut) + takes = pick.item if isinstance(pick, Next) else None + else: + takes = next((item for item in plan.items if item.id == item_id), None) return takes if takes and takes.likelihood and takes.escape_and_impact else None diff --git a/bin/loop_claude/test_black_box.py b/bin/loop_claude/test_black_box.py index 5505ba2c..c490bd4a 100644 --- a/bin/loop_claude/test_black_box.py +++ b/bin/loop_claude/test_black_box.py @@ -29,6 +29,26 @@ OFF_LADDER = '{"model": "worker-model"}' OFF_LADDER_ROUTE = {"rung": {"model": "worker-model", "effort": ""}, "item": 0, "likelihood": 0, "escape_and_impact": 0, "unfinished": 0, "escape_and_impact_taken_in": 0, "escalated_at": 0} +# A tmux that logs each command. It prints $FAKE_TMUX_VERSION for -V, $FAKE_TMUX_OWNER for show-options, and the file $FAKE_TMUX_WINDOWS for list-windows, where it finds no session without the file, and for any other command the ID of a new window. +FAKE_TMUX = """#!{python} +import json, os, pathlib, sys + +argv = sys.argv[1:] +with open(os.environ["FAKE_TMUX_LOG"], "a") as log: + log.write(json.dumps(argv) + "\\n") +windows = pathlib.Path(os.environ["FAKE_TMUX_WINDOWS"]) +if argv[0] == "-V": + print(os.environ.get("FAKE_TMUX_VERSION", "tmux 3.2")) +elif argv[0] == "show-options": + print(os.environ["FAKE_TMUX_OWNER"]) +elif argv[0] == "list-windows": + if not windows.exists(): + sys.exit("no server running") + print(windows.read_text(), end="") +else: + print("@7") +""" + # Plays claude from FAKE_CLAUDE_SCENARIO, one step per call, logging each call; a step for another role, # or none left, is Ctrl-C to loop-claude, which otherwise loops forever. FAKE_CLAUDE = """#!{python} @@ -166,9 +186,11 @@ class Run: code: int sleeps: list starts: list - # stderr and stdout are without the local time that opens each message loop-claude prints; raw_stderr and raw_stdout keep it. + # stderr and stdout are without the local time that opens each message loop-claude prints, and stderr without the bell an interrupt rings; raw_stderr and raw_stdout keep them. stderr: str stdout: str + # Each tmux command run, as its arguments. + tmux: list raw_stderr: str raw_stdout: str @@ -195,6 +217,8 @@ class Checkout: fake.mkdir() (fake / "claude").write_text(FAKE_CLAUDE.format(python=sys.executable)) (fake / "claude").chmod(0o755) + (fake / "tmux").write_text(FAKE_TMUX.format(python=sys.executable)) + (fake / "tmux").chmod(0o755) (fake / "git").symlink_to(shutil.which("git")) site = self.temp / "site" site.mkdir() @@ -203,6 +227,11 @@ class Checkout: self.env.update({ "FAKE_CLAUDE_LOG": str(self.temp / "calls.jsonl"), "FAKE_MEMINFO": str(self.temp / "meminfo"), + "FAKE_TMUX_LOG": str(self.temp / "tmux.jsonl"), + "FAKE_TMUX_WINDOWS": str(self.temp / "tmux-windows"), + "FAKE_TMUX_OWNER": str(self.root), + # The pane of the window the loop under test runs in. + "TMUX_PANE": "%3", "FAKE_CLAUDE_SCENARIO": str(self.temp / "scenario.json"), "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1", @@ -224,15 +253,20 @@ class Checkout: self.git("add", "todo.md") self.git("commit", "--quiet", "-m", "plan") - def run(self, steps, argv=(TASK,), cwd=None, terminal_input=None, stdout_on_terminal=True, before_typing=None, typed_early=""): - for name in ("FAKE_CLAUDE_LOG", "LOOP_TEST_LOG"): + def run(self, steps, argv=None, cwd=None, terminal_input=None, stdout_on_terminal=True, before_typing=None, typed_early="", chunk="no-item"): + """Runs loop-claude with argv, or without one as the loop of the window `chunk`, on TASK. Off a terminal, a window's loop whose chunk finished is run again while steps remain, as the window it starts for the next chunk would.""" + for name in ("FAKE_CLAUDE_LOG", "LOOP_TEST_LOG", "FAKE_TMUX_LOG"): pathlib.Path(self.env[name]).unlink(missing_ok=True) pathlib.Path(self.env["FAKE_CLAUDE_SCENARIO"]).write_text(json.dumps(steps)) - command = [sys.executable, str(self.script), *argv] + command = [sys.executable, str(self.script), *(("--chunk", chunk, TASK) if argv is None else argv)] cwd = cwd or self.root if terminal_input is None: - done = subprocess.run(command, capture_output=True, cwd=cwd, env=self.env, stdin=subprocess.DEVNULL, text=True, timeout=60) - code, stdout, stderr = done.returncode, done.stdout, done.stderr + code, stdout, stderr = 0, "", "" + runs = 0 + while code == 0 and (runs == 0 or (argv is None and json.loads(pathlib.Path(self.env["FAKE_CLAUDE_SCENARIO"]).read_text()))): + runs += 1 + done = subprocess.run(command, capture_output=True, cwd=cwd, env=self.env, stdin=subprocess.DEVNULL, text=True, timeout=60) + code, stdout, stderr = done.returncode, stdout + done.stdout, stderr + done.stderr else: code, stdout, stderr = self.run_on_terminal(command, cwd, terminal_input, stdout_on_terminal, before_typing, typed_early) started = lines(pathlib.Path(self.env["LOOP_TEST_LOG"])) @@ -241,8 +275,9 @@ class Checkout: code=code, sleeps=[event["sleep"] for event in started if "sleep" in event], starts=[event["argv"] for event in started if "argv" in event], - stderr=unstamped(stderr), + stderr=unstamped(stderr).replace("\a", ""), stdout=unstamped(stdout), + tmux=lines(pathlib.Path(self.env["FAKE_TMUX_LOG"])), raw_stderr=stderr, raw_stdout=stdout, ) @@ -394,7 +429,7 @@ class Premises(unittest.TestCase): write_config(checkout.home, root) checkout.git("init", "--quiet", str(root)) run = checkout.run([worker()], cwd=root) - self.assertEqual(argument(run.calls[0], "--name"), f"A {str(root).lstrip('/')} loop-claude") + self.assertEqual(argument(run.calls[0], "--name"), f"A {str(root).lstrip('/')} loop-claude no-item") class Cycles(unittest.TestCase): @@ -414,7 +449,7 @@ class Cycles(unittest.TestCase): self.assertEqual(run.roles(), ["worker", "judge", "worker", "judge", "worker", "judge"]) workers = [call for call in run.calls if call["role"] == "worker"] session = argument(workers[0], "--session-id") - self.assertEqual(argument(workers[0], "--name"), "A proj loop-claude") + self.assertEqual(argument(workers[0], "--name"), "A proj loop-claude no-item") self.assertEqual(argument(workers[1], "--resume"), session) self.assertIsNone(argument(workers[1], "--session-id")) self.assertIsNone(argument(workers[1], "--name")) @@ -433,8 +468,10 @@ class Cycles(unittest.TestCase): self.assertIn("landed-it", run.calls[3]["stdin"]) self.assertNotIn("landed-it", run.calls[5]["stdin"]) self.assertEqual([path.name for path in checkout.transcripts()[0].parent.iterdir()], [checkout.transcripts()[0].name]) - self.assertIn("=== cycle 3, worker on worker-model ===", run.stdout) - self.assertEqual(len(re.findall(rf"(?m)^{STAMP} === cycle 3, worker on worker-model ===$", run.raw_stdout)), 1) + self.assertIn("=== cycle 2, worker on worker-model ===", run.stdout) + self.assertEqual(len(re.findall(rf"(?m)^{STAMP} === cycle 2, worker on worker-model ===$", run.raw_stdout)), 1) + # The finished chunk's loop started the next chunk's window and ended; the checkout has no default branch to read a plan from. + self.assertIn("landed-it\nthe chunk finished; started no-item\n\n=== cycle 1, worker on worker-model ===\nnext chunk\n", run.stdout) copies = {verdict.removeprefix(f"Task: {TASK}\n\n"): copy for copy, verdict in checkout.archived().items()} self.assertEqual(sorted(copies), [f"<UTC> Stopped: Ctrl-C\n\nWorker: {OFF_LADDER}\n", f"<UTC> Verdict: finished\n\nEvidence:\n> landed-it\n\nConfig: <config>\nJudge: judge-model\nWorker: {OFF_LADDER}\n", f"<UTC> Verdict: unfinished\n\nEvidence:\n> started\n\nConfig: <config>\nJudge: judge-model\nWorker: {OFF_LADDER}\n"]) self.assertIn('"text": "landed-it"', copies[f"<UTC> Verdict: finished\n\nEvidence:\n> landed-it\n\nConfig: <config>\nJudge: judge-model\nWorker: {OFF_LADDER}\n"]) @@ -608,19 +645,15 @@ class Cycles(unittest.TestCase): self.assertIsNotNone(argument(run.calls[0], "--resume")) self.assertEqual((argument(run.calls[0], "--model"), argument(run.calls[0], "--effort")), ("claude-fable-5-2", "medium")) - def test_a_second_loop_in_the_project_runs_its_own_session_and_leaves_the_first_loops_open_chunk_alone(self): + def test_a_second_window_of_the_project_runs_its_own_session_and_leaves_the_first_ones_open_chunk_alone(self): checkout = Checkout(self) first = checkout.run([worker(say="first chunk"), judge(evidence="first chunk")]) session = argument(first.calls[0], "--session-id") transcript = checkout.transcripts()[0] kept, archived = transcript.read_bytes(), checkout.archived() - slot = checkout.home / ".local" / "state" / "loop-claude" / "proj" / "slot-0" - slot.parent.mkdir(parents=True, exist_ok=True) - # The test holds the first loop's slot, as the first loop does while it runs. - with open(slot, "a") as running: - fcntl.flock(running, fcntl.LOCK_EX | fcntl.LOCK_NB) - second = checkout.run([worker(say="second chunk"), judge(evidence="second chunk")]) - self.assertEqual(argument(second.calls[0], "--name"), "A proj loop-claude 2") + self.assertEqual(argument(first.calls[0], "--name"), "A proj loop-claude no-item") + second = checkout.run([worker(say="second chunk"), judge(evidence="second chunk")], chunk="item-2") + self.assertEqual(argument(second.calls[0], "--name"), "A proj loop-claude item-2") self.assertNotIn(session, [argument(call, "--session-id") or argument(call, "--resume") for call in second.calls if call["role"] == "worker"]) self.assertNotIn("first chunk", second.calls[1]["stdin"]) self.assertEqual(transcript.read_bytes(), kept) @@ -771,6 +804,111 @@ class Cycles(unittest.TestCase): self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"] * 12)) +# Items 4 and 9 with predictions that do not overlap, so both may run at once, and item 12, which overlaps item 4. +BESIDE = TAKEN_IN_AT_L_4 + "| 12 | 0.1.0 | | **Do most.** | 2 | 1 | 2 | 3 | 2 | 1 by 9 | 10 |\n\n## Details\n\n### 4. Do it.\n\nTouches:\n\n- `a.py`\n\n### 9. Do more.\n\nTouches:\n\n- `b.py`\n\n### 12. Do most.\n\nTouches:\n\n- `a.py`\n" + + +class Windows(unittest.TestCase): + def checkout(self, parallel=None): + """A checkout whose plan is BESIDE, with parallel-chunks set where given.""" + checkout = Checkout(self) + checkout.commit_plan(BESIDE) + if parallel: + config = checkout.home / ".claude-projects" / "proj" / "config.toml" + config.write_text(f"parallel-chunks = {parallel}\n" + config.read_text()) + return checkout + + def started(self, run): + """The window each new-session or new-window made, with the command it runs.""" + return [(command[0], command[command.index("-n") + 1], command[command.index("--") + 1:]) for command in run.tmux if command[0] in ("new-session", "new-window")] + + def test_starts_one_chunk_in_a_window_of_the_projects_session_by_default(self): + checkout = self.checkout() + run = checkout.run([], argv=["ship", "it"]) + self.assertEqual((run.code, run.calls, run.stderr), (0, [], "")) + self.assertEqual(run.tmux, [ + ["-V"], + ["list-windows", "-t", "=loop-proj", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"], + ["new-session", "-d", "-s", "loop-proj", "-n", "item-4", "-c", str(checkout.root), "-e", "LOOP_CLAUDE_JUDGE_MODEL=judge-model", "-e", "LOOP_CLAUDE_WORKER_MODEL=worker-model", "-P", "-F", "#{window_id}", "--", str(checkout.script), "--chunk", "item-4", "ship it"], + ["set-option", "-t", "=loop-proj:", "@loop-root", str(checkout.root)], + ["set-option", "-w", "-t", "@7", "remain-on-exit", "failed"], + ]) + self.assertEqual(run.stdout, "started item-4 in the tmux session loop-proj\nnot on a terminal; see the chunks with `tmux attach -t =loop-proj`\n") + + def test_starts_up_to_the_configured_number_of_chunks_whose_predictions_do_not_overlap(self): + run = self.checkout(parallel=3).run([], argv=[TASK]) + self.assertEqual([(made, window) for made, window, _ in self.started(run)], [("new-session", "item-4"), ("new-window", "item-9")]) + self.assertIn("started item-4, item-9 in the tmux session loop-proj\n", run.stdout) + + def test_starts_only_what_the_running_windows_leave_room_for(self): + checkout = self.checkout(parallel=2) + # Item 4's chunk runs and waits for the human; item 9's loop failed, so its window stays, dead. + pathlib.Path(checkout.env["FAKE_TMUX_WINDOWS"]).write_text("item-4 ?\t0\t@1\nitem-9\t1\t@2\n") + run = checkout.run([], argv=[TASK]) + self.assertEqual([command for command in run.tmux if command[0] == "kill-window"], [["kill-window", "-t", "@2"]]) + self.assertEqual([(made, window) for made, window, _ in self.started(run)], [("new-window", "item-9")]) + pathlib.Path(checkout.env["FAKE_TMUX_WINDOWS"]).write_text("item-4\t0\t@1\nitem-9\t0\t@2\n") + run = checkout.run([], argv=[TASK]) + self.assertEqual((self.started(run), run.code), ([], 0)) + self.assertIn(f"the tmux session loop-proj runs every chunk that may run now, so `{TASK}` started in no new window\n", run.stdout) + + def test_a_finished_chunk_starts_the_next_one_its_window_leaves_room_for(self): + checkout = self.checkout(parallel=2) + pathlib.Path(checkout.env["FAKE_TMUX_WINDOWS"]).write_text("item-4\t0\t@1\nitem-9\t0\t@2\n") + run = checkout.run([worker(say="merged"), judge(finished=True, evidence="merged")], chunk="item-9") + self.assertEqual((run.code, run.roles()), (0, ["worker", "judge"])) + # Item 9 is still in the checkout's plan, since nothing merged here; the window that leaves holds it no longer, and item 4's chunk still runs. + self.assertEqual(self.started(run), [("new-window", "item-9", [str(checkout.script), "--chunk", "item-9", TASK])]) + self.assertIn("the chunk finished; started item-9\n", run.stdout) + + def test_a_finished_chunk_starts_the_next_item_from_the_plan_its_merge_left_on_origin(self): + checkout = Checkout(self) + origin = checkout.temp / "origin.git" + checkout.git("init", "--quiet", "--bare", "--initial-branch=main", str(origin)) + checkout.git("symbolic-ref", "HEAD", "refs/heads/main") + checkout.commit_plan(TODO) + checkout.git("remote", "add", "origin", str(origin)) + checkout.git("push", "--quiet", "--set-upstream", "origin", "main") + # Item 4's chunk runs a cycle and stays open, so the rerun below resumes it and runs no chunk-preflight, which would fetch. + self.assertEqual(checkout.run([worker(), judge()], chunk="item-4").code, 130) + # Item 4's chunk merges, which leaves item 6 next on origin; the checkout fetched before it. + elsewhere = checkout.temp / "elsewhere" + checkout.git("clone", "--quiet", str(origin), str(elsewhere)) + (elsewhere / "todo.md").write_text(TODO.replace("| 4 | 0.1.0 |", "| 6 | 0.1.0 |")) + checkout.git("-C", str(elsewhere), "commit", "--quiet", "--all", "-m", "Finish item 4") + checkout.git("-C", str(elsewhere), "push", "--quiet", "origin", "main") + run = checkout.run([worker(say="merged"), judge(finished=True, evidence="merged")], chunk="item-4") + self.assertEqual(self.started(run), [("new-session", "item-6", [str(checkout.script), "--chunk", "item-6", TASK])]) + + def test_tells_a_fresh_worker_the_items_the_other_running_chunks_hold_and_starts_from_its_windows_item(self): + checkout = self.checkout(parallel=2) + checkout.env["LOOP_CLAUDE_WORKER_MODEL"] = "claude-opus-5-5" + pathlib.Path(checkout.env["FAKE_TMUX_WINDOWS"]).write_text("item-4\t0\t@1\nitem-9\t0\t@2\n") + run = checkout.run([worker()], chunk="item-9") + prompt = argument(run.calls[0], "-p") + self.assertIn("The item this chunk starts from, named with --item: item 9, L 4, E + I 3.", prompt) + self.assertIn("Other chunks of this project, running or stopped, hold item 4. Leave those items out of this chunk, even where the facts above list them as items that may join it.", prompt) + self.assertIn("routed by item 9's L of 4", prompt) + + def test_refuses_to_start_on_a_tmux_older_than_needed_or_without_one(self): + checkout = self.checkout() + checkout.env["FAKE_TMUX_VERSION"] = "tmux 3.1c" + run = checkout.run([], argv=[TASK]) + self.assertEqual((run.code, run.stderr, run.tmux), (1, "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 3.1c\n", [["-V"]])) + (pathlib.Path(checkout.env["PATH"]) / "tmux").unlink() + run = checkout.run([], argv=[TASK]) + self.assertEqual((run.code, run.stderr), (1, "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found no tmux\n")) + + def test_marks_the_window_while_an_interrupt_waits_and_rings_its_bell(self): + checkout = self.checkout() + run = checkout.run([worker(say="A or B?"), judge(interrupt=True, question="A or B?")], chunk="item-4") + self.assertEqual(run.code, 2) + self.assertEqual([command for command in run.tmux if command[0] == "rename-window"], [["rename-window", "-t", "%3", "item-4 ?"]]) + self.assertEqual(run.raw_stderr.count("\a"), 1) + run = checkout.run([worker(say="A or B?"), judge(interrupt=True, question="A or B?"), worker()], chunk="item-4", terminal_input="B\n", before_typing=ANSWERING) + self.assertEqual([command for command in run.tmux if command[0] == "rename-window"], [["rename-window", "-t", "%3", "item-4 ?"], ["rename-window", "-t", "%3", "item-4"]]) + + class Interrupt(unittest.TestCase): def test_exits_2_off_a_terminal_and_resumes_on_the_next_run(self): checkout = Checkout(self) @@ -971,17 +1109,28 @@ class Reload(unittest.TestCase): self.assertEqual(run.roles(), ["worker", "judge"] * 5) self.assertIn("keeping the running script", run.stderr) reloads = [json.loads(argv[1]) for argv in run.starts if argv[:1] == ["--reloaded"]] - self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working", "route": OFF_LADDER_ROUTE, "unfinished_run": cycle, "slot": 0} for cycle in (2, 3, 4)]) + self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working", "route": OFF_LADDER_ROUTE, "unfinished_run": cycle, "chunk": "no-item"} for cycle in (2, 3, 4)]) self.assertIn("=== cycle 5,", run.stdout) outside = checkout.temp / "outside" outside.mkdir() - state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "", "route": None} + state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "", "route": None, "chunk": "no-item"} run = checkout.run([worker()], argv=["--reloaded", json.dumps(state)], cwd=outside) self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"])) self.assertEqual(run.calls[0]["cwd"], str(checkout.root)) self.assertIn("=== cycle 4,", run.stdout) + def test_a_loop_started_before_chunks_ran_in_windows_reloads_into_starting_the_projects_windows(self): + checkout = Checkout(self) + state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "", "route": None, "slot": 0} + run = checkout.run([worker()], argv=["--reloaded", json.dumps(state)]) + self.assertEqual((run.code, run.calls), (0, [])) + self.assertEqual([command[0] for command in run.tmux], ["-V", "list-windows", "new-session", "set-option", "set-option"]) + self.assertIn("started no-item in the tmux session loop-proj\n", run.stdout) + + def test_stops_where_a_reload_hands_no_state(self): + checkout = Checkout(self) + run = checkout.run([], argv=["--reloaded", "{}"]) self.assertEqual((run.code, run.calls), (1, [])) self.assertIn("rerun loop-claude <task>", run.stderr) diff --git a/bin/loop_claude/test_loop_claude.py b/bin/loop_claude/test_loop_claude.py index d5bc624c..30c2ecd5 100644 --- a/bin/loop_claude/test_loop_claude.py +++ b/bin/loop_claude/test_loop_claude.py @@ -21,13 +21,14 @@ from routing_figures import archive, state from routing_figures.calls import Call from loop_claude.project import checkout, model_choice, records from loop_claude.cycle import chunk_items -from loop_claude.loop import chunk_slot, chunk_start, close +from loop_claude.loop import chunk_start, chunk_windows, close from project_config.test_project_config import configured from release_plan.release_standing import DefaultPlan, LastCut, UNSEEN_RELEASES from loop_effects.effects import Effects, PYTHON_TIMEOUT from claude_code import entries, reply, session from failure import Failure from loop_claude.verdicts import Finished, Interrupt, Judged, QuotedAnswer, TakenIn, Unfinished +from chunk_preflight import gathered from git_cli.git_repo import git from loop_claude.cycle import interrupt, judge, prompts, retry, run, spend, states, worker from claude_code.reply import Limited @@ -42,7 +43,9 @@ from model_routing.route import Route, route_from, WorkerLine from loop_claude import routing from model_routing.ladder import Rung from loop_claude.stop import Stop +from loop_claude import tmux from claude_code.tokens import Tokens, tokens_of +from touch_predictions.test_touch_predictions import plan as touch_plan from text_files.utc_stamp import utc from loop_claude.cycle.judge import verdict_from @@ -76,18 +79,28 @@ def human(content): return {"type": "user", "message": {"content": content}} +# The window the loops under test run in. +WINDOW = tmux.Window("loop-proj", "item-7", "%4") + + def a_loop(effects, **fields): """A loop in /r on effects, with fields in place of its defaults.""" home = pathlib.Path("/h") - defaults = {"root": "/r", "session": session.session_named("A proj loop-claude", home / ".claude" / "projects"), "paths": paths.paths_from(home), "env": {}, "launcher": pathlib.Path("/b/loop-claude")} + defaults = {"root": "/r", "session": session.session_named("A proj loop-claude", home / ".claude" / "projects"), "paths": paths.paths_from(home), "env": {}, "launcher": pathlib.Path("/b/loop-claude"), "window": WINDOW} return this_loop.Loop(**{**defaults, **fields}, effects=effects) class FakeEffects: """Effects that record each call and answer from a script, one answer per call.""" - def __init__(self, claude=(), on_terminal_claude=(), git=(), inputs=(), python=(), streams=(), on_terminal=True, free_memory=(), dm_failures=()): + def __init__(self, claude=(), on_terminal_claude=(), git=(), inputs=(), python=(), streams=(), on_terminal=True, free_memory=(), dm_failures=(), tmux_version="tmux 3.2", windows=None, owner="/r"): self.calls = [] + # Each tmux command run. They stay out of self.calls, as the DMs do. tmux prints `windows` for list-windows, or without them finds no session. + self.tmux_calls = [] + self.tmux_version = tmux_version + self.windows = windows + # The checkout the session's @loop-root option names. + self.owner = owner # Each direct message sent. They stay out of self.calls, so a test that lists every call doesn't list them. Without a script, each is sent. self.dms = [] self.dm_failures = list(dm_failures) @@ -115,8 +128,16 @@ class FakeEffects: python=lambda args, cwd: self.answer("python", (args, cwd)), sleep=lambda seconds: self.calls.append(("sleep", seconds)), stream_claude=self.stream, + tmux=self.tmux, + tmux_on_terminal=lambda args: self.tmux(args).returncode, ) + def tmux(self, args): + self.tmux_calls.append(args) + if args[0] == "list-windows": + return subprocess.CompletedProcess(args, 1 if self.windows is None else 0, self.windows or "", "no server") + return subprocess.CompletedProcess(args, 0, {"-V": self.tmux_version + "\n", "show-options": self.owner + "\n"}.get(args[0], "@9\n"), "") + def dm(self, text): self.dms.append(text) return self.dm_failures.pop(0) if self.dm_failures else None @@ -265,7 +286,7 @@ class Cycles(unittest.TestCase): self.archive = project_archive(str(self.home / "code" / "proj"), self.home, {}) def cycle(self, fake, records="", unfinished_evidence="", facts="- A preflight fact."): - return run.Cycle(config=CONFIG, effects=fake.effects(), facts=facts, judge_model="judge-model", ladder=LADDER, premises="## Goals", project=None, project_archive=self.archive, records=records, root=str(self.home), route=Route(Rung("worker-model")), session=self.session, task="Do it.", todo=plan_of(""), unfinished_evidence=unfinished_evidence) + return run.Cycle(config=CONFIG, window=WINDOW, effects=fake.effects(), facts=facts, judge_model="judge-model", ladder=LADDER, premises="## Goals", project=None, project_archive=self.archive, records=records, root=str(self.home), route=Route(Rung("worker-model")), session=self.session, task="Do it.", todo=plan_of(""), unfinished_evidence=unfinished_evidence) def archived_outcomes(self): return sorted(path.read_text() for path in self.archive.glob("*/*.outcomes.md")) @@ -560,7 +581,7 @@ Spent since 2026-09-21T14:13:20Z, in tokens of input, cache write 5m, cache writ fake = FakeEffects(streams=[worker, worker], claude=[judged("interrupt"), judged("unfinished")], inputs=["Use B."]) with contextlib.redirect_stderr(io.StringIO()) as said: self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Unfinished("working", entries_read=6)) - self.assertIn(f"\n{STAMP} Which one?\n", said.getvalue()) + self.assertIn(f"\n{STAMP} Which one?\a\n", said.getvalue()) self.assertEqual(fake.made("claude_on_terminal"), []) resumed = fake.made("stream_claude")[1] self.assertEqual(resumed[resumed.index("--resume") + 1], str(self.session.id)) @@ -627,7 +648,7 @@ Spent since 2026-09-21T14:13:20Z, in tokens of input, cache write 5m, cache writ with contextlib.redirect_stderr(io.StringIO()) as said: self.assertEqual(run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse B. Use A.")), Stop(2, "the human is needed to answer the question above, but nothing was typed")) self.assertEqual(len(fake.made("stream_claude")), 1) - self.assertIn(f"\n{STAMP} Which one?\n", said.getvalue()) + self.assertIn(f"\n{STAMP} Which one?\a\n", said.getvalue()) self.assertNotIn("Use ... A.", said.getvalue()) @unittest.skipIf(os.geteuid() == 0, "root writes into a read-only directory") @@ -733,10 +754,11 @@ class Interrupts(unittest.TestCase): self.session = interrupt.WorkerSession(str(self.home), uuid.uuid7(), Rung("m", "high"), str(self.transcript), {"LOOP_CLAUDE_WORKER_RUNG": "m at high effort"}) def put(self, fake, answer_here=True, effects=None, question="Which one?"): - """interrupt's answer and what it said.""" + """interrupt's answer and what it said, without the bell it rang, which self.bells counts.""" with contextlib.redirect_stderr(io.StringIO()) as said: - answer = interrupt.interrupt(question, answer_here, self.session, effects or fake.effects()) - return answer, said.getvalue() + answer = interrupt.interrupt(question, answer_here, self.session, WINDOW, effects or fake.effects()) + self.bells = said.getvalue().count(tmux.BELL) + return answer, said.getvalue().replace(tmux.BELL, "") def typed_there(self, args): """The claude opened on the terminal: the human types a line in the worker's session, and it ends.""" @@ -754,7 +776,19 @@ class Interrupts(unittest.TestCase): with self.subTest(on_terminal=on_terminal): fake = FakeEffects(inputs=["Use B."], on_terminal=on_terminal) self.put(fake) - self.assertEqual(fake.dms, [f"loop-claude in {self.home} asks:\n\nWhich one?"]) + self.assertEqual(fake.dms, [f"loop-claude in {self.home} asks, in the tmux window loop-proj:item-7:\n\nWhich one?"]) + + def test_marks_its_window_and_rings_the_bell_while_it_waits_and_clears_the_mark_once_answered(self): + fake = FakeEffects(inputs=["Use B."]) + answer, said = self.put(fake) + self.assertEqual(answer, "Use B.") + self.assertEqual(self.bells, 1) + self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"], ["rename-window", "-t", "%4", "item-7"]]) + + def test_a_loop_that_stops_unanswered_leaves_its_window_marked(self): + fake = FakeEffects(on_terminal=False) + self.put(fake) + self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"]]) def test_drops_the_keys_typed_before_the_question_ahead_of_the_dm_so_an_answer_typed_while_it_sends_counts(self): fake = FakeEffects(inputs=["Use B."]) @@ -1125,14 +1159,33 @@ class Routing(unittest.TestCase): class RouteSources(unittest.TestCase): def test_routes_on_the_plan_item_and_says_why_it_routes_on_the_projects_rung(self): - self.assertIsNone(chunk_start.plan_item(DefaultPlan(plan_of(""), NO_CUT), FakeEffects().effects())) - self.assertEqual(chunk_start.plan_item(DefaultPlan(plan_of(TODO), LastCut("v0.1.0", "", 3, [TODO])), FakeEffects().effects()).id, 3) - self.assertIsNone(chunk_start.plan_item(DefaultPlan(plan_of(TODO), LastCut("v0.3.0", "", 3, [TODO])), FakeEffects().effects())) + self.assertIsNone(chunk_start.plan_item(DefaultPlan(plan_of(""), NO_CUT), None, FakeEffects().effects())) + self.assertEqual(chunk_start.plan_item(DefaultPlan(plan_of(TODO), LastCut("v0.1.0", "", 3, [TODO])), None, FakeEffects().effects()).id, 3) + self.assertIsNone(chunk_start.plan_item(DefaultPlan(plan_of(TODO), LastCut("v0.3.0", "", 3, [TODO])), None, FakeEffects().effects())) + + def test_routes_a_window_named_for_an_item_on_that_item_whatever_the_plan_takes_next(self): + plan = DefaultPlan(plan_of(TODO), LastCut("v0.1.0", "", 3, [TODO])) + later = next(item.id for item in plan.todo.items if item.id != 3) + self.assertEqual(chunk_start.plan_item(plan, later, FakeEffects().effects()).id, later) + # An item that left the plan, as once it shipped. + self.assertIsNone(chunk_start.plan_item(plan, 999, FakeEffects().effects())) + with unittest.mock.patch.object(chunk_start, "session_open", return_value=True), unittest.mock.patch.object(chunk_start, "fetched_plan", return_value=plan) as fetched: + for window, routed in ((tmux.Window("loop-proj", f"item-{later}", "%4"), later), (tmux.Window("loop-proj", "no-item", "%4"), 3)): + with self.subTest(window=window.name): + loop = a_loop(FakeEffects().effects(), window=window) + facts, reader = chunk_start.chunk_start(loop) + self.assertEqual((facts, reader().id), ("", routed)) + fetched.assert_called_with(loop) + # Where the plan cannot be read, the reader says so through the loop's effects, and routes on the project's rung. + fetched.return_value = Failure("no origin remote") + with contextlib.redirect_stderr(io.StringIO()) as said: + self.assertIsNone(chunk_start.chunk_start(a_loop(FakeEffects().effects()))[1]()) + self.assertIn("routing on the project's rung: no origin remote", said.getvalue()) unreadable = Failure("git show origin/main:todo.md printed text that is not UTF-8") for plan, said in ((DefaultPlan(plan_of(TODO.replace("| 4 | 2 | 3 |", "| x | 2 | 3 |")), NO_CUT), "routing on the project's rung: item 3's L 'x' is not a whole number from 1 to 4\n"), (DefaultPlan(unreadable, NO_CUT), f"routing on the project's rung: {unreadable.reason}\n"), (Failure("no origin remote"), "routing on the project's rung: no origin remote\n")): with self.subTest(said=said): with contextlib.redirect_stderr(io.StringIO()) as stderr: - self.assertIsNone(chunk_start.plan_item(plan, FakeEffects().effects())) + self.assertIsNone(chunk_start.plan_item(plan, None, FakeEffects().effects())) self.assertTrue(stderr.getvalue().startswith(f"{STAMP} {said}"), stderr.getvalue()) def test_says_why_the_items_the_worker_names_lift_no_rung(self): @@ -1210,61 +1263,272 @@ class RouteSources(unittest.TestCase): self.assertTrue(stderr.getvalue().startswith(f"{STAMP} routing the chunk afresh: could not read the archive"), stderr.getvalue()) -class ChunkSlots(unittest.TestCase): +class TmuxCommands(unittest.TestCase): + def test_refuses_a_tmux_older_than_needed_naming_both_versions(self): + for found, reason in (("tmux 3.2", None), ("tmux 3.8", None), ("tmux next-3.4", None), ("tmux 10.0", None), + ("tmux 3.1c", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 3.1c"), + ("tmux 2.9a", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 2.9a"), + ("", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found no tmux")): + with self.subTest(found=found): + fake = FakeEffects(tmux_version=found) + self.assertEqual(tmux.unusable(fake.tmux), reason) + self.assertEqual(fake.tmux_calls, [["-V"]]) + + def test_names_the_session_for_the_project(self): + self.assertEqual(tmux.session_name("claude-config"), "loop-claude-config") + # tmux reads `.` and `:` in a target as separators. + self.assertEqual(tmux.session_name("my.site:2 x"), "loop-my-site-2-x") + self.assertEqual(tmux.session_name("Acme/Billing"), "loop-Acme-Billing") + + def test_lists_the_windows_without_the_waiting_mark(self): + fake = FakeEffects(windows="item-372 ?\t0\t@1\nitem-380\t1\t@2\nno-item\t0\t@5\n") + self.assertEqual(tmux.listed("loop-proj", fake.tmux), [tmux.Listed("item-372", "@1", False), tmux.Listed("item-380", "@2", True), tmux.Listed("no-item", "@5", False)]) + self.assertEqual(fake.tmux_calls, [["list-windows", "-t", "=loop-proj", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"]]) + self.assertEqual(tmux.listed("loop-proj", FakeEffects().tmux), []) + failed = lambda args: subprocess.CompletedProcess(args, 1, "item-1\t0\t@1\n", "no server") + self.assertEqual(tmux.listed("loop-proj", failed), []) + + def test_the_first_window_makes_the_session_and_each_later_one_joins_it(self): + fake = FakeEffects() + window = tmux.Window("loop-proj", "item-7", "") + self.assertIsNone(tmux.start_window(window, "/r", {"LOOP_CLAUDE_X": "1"}, ["/b/loop-claude", "--chunk", "item-7", "ship it;"], True, fake.tmux)) + self.assertIsNone(tmux.start_window(window, "/r", {}, ["/b/loop-claude"], False, fake.tmux)) + self.assertEqual(fake.tmux_calls, [ + ["new-session", "-d", "-s", "loop-proj", "-n", "item-7", "-c", "/r", "-e", "LOOP_CLAUDE_X=1", "-P", "-F", "#{window_id}", "--", "/b/loop-claude", "--chunk", "item-7", "ship it\;"], + ["set-option", "-t", "=loop-proj:", "@loop-root", "/r"], + ["set-option", "-w", "-t", "@9", "remain-on-exit", "failed"], + ["new-window", "-d", "-t", "=loop-proj:", "-n", "item-7", "-c", "/r", "-P", "-F", "#{window_id}", "--", "/b/loop-claude"], + ["set-option", "-w", "-t", "@9", "remain-on-exit", "failed"], + ]) + + def test_says_why_tmux_started_no_window(self): + failed = lambda args: subprocess.CompletedProcess(args, 1, "", "duplicate session: loop-proj\n") + silent = lambda args: subprocess.CompletedProcess(args, 2, "", "") + window = tmux.Window("loop-proj", "item-7", "") + self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, failed), Failure("tmux could not start the window loop-proj:item-7: duplicate session: loop-proj")) + self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, silent), Failure("tmux could not start the window loop-proj:item-7: it exited 2")) + + def test_reads_the_checkout_the_session_names(self): + fake = FakeEffects(owner="/c/proj") + self.assertEqual(tmux.owner("loop-proj", fake.tmux), "/c/proj") + self.assertEqual(fake.tmux_calls, [["show-options", "-qv", "-t", "=loop-proj:", "@loop-root"]]) + + def test_marks_a_waiting_window_by_its_pane_and_clears_the_mark(self): + fake = FakeEffects() + tmux.mark(WINDOW, True, fake.tmux) + tmux.mark(WINDOW, False, fake.tmux) + tmux.close(tmux.Listed("item-7", "@3", True), fake.tmux) + self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"], ["rename-window", "-t", "%4", "item-7"], ["kill-window", "-t", "@3"]]) + self.assertEqual(WINDOW.target, "loop-proj:item-7") + + def test_shows_the_session_by_attaching_or_inside_tmux_by_switching(self): + self.assertEqual(tmux.show("loop-proj", False), ["attach-session", "-t", "=loop-proj"]) + self.assertEqual(tmux.show("loop-proj", True), ["switch-client", "-t", "=loop-proj"]) + + +class ChunkWindows(unittest.TestCase): + def plan(self, *touched): + """A plan of items 1, 2 and on, each predicted to touch the path given for it.""" + items = touch_plan(*((number, "1.0.0", "", f"- `{path}`") for number, path in enumerate(touched, 1))) + return gathered.Plan(items=items, first=items[0], text="", branch_template="") + + def test_starts_the_next_item_and_those_picked_beside_it_up_to_the_limit(self): + # Item 2 overlaps item 1, so it waits for it; items 3 and 4 run beside it. + plan = self.plan("a.py", "a.py", "c.py", "d.py") + self.assertEqual(chunk_windows.may_start(plan, [], set(), 1), ["item-1"]) + self.assertEqual(chunk_windows.may_start(plan, [], set(), 2), ["item-1", "item-3"]) + self.assertEqual(chunk_windows.may_start(plan, [], set(), 9), ["item-1", "item-3", "item-4"]) + + def test_leaves_the_items_running_chunks_hold_and_counts_them_against_the_limit(self): + plan = self.plan("a.py", "a.py", "c.py", "d.py") + self.assertEqual(chunk_windows.may_start(plan, ["item-1"], {1, 3}, 2), ["item-4"]) + self.assertEqual(chunk_windows.may_start(plan, ["item-1"], {1}, 3), ["item-3", "item-4"]) + self.assertEqual(chunk_windows.may_start(plan, ["item-1", "item-3"], {1, 3}, 2), []) + self.assertEqual(chunk_windows.may_start(plan, ["item-1", "item-3", "item-4"], {1, 3, 4}, 2), []) + + def test_starts_no_item_that_overlaps_a_running_chunks_items(self): + # Item 1 merged, so 2 is the plan's next item; it overlaps item 3, which still runs, at c.py. + items = touch_plan((2, "1.0.0", "", "- `a.py`\n- `c.py`"), (3, "1.0.0", "", "- `c.py`"), (4, "1.0.0", "", "- `d.py`")) + plan = gathered.Plan(items=items, first=items[0], text="", branch_template="") + self.assertEqual(chunk_windows.may_start(plan, ["item-3"], {3}, 3), ["item-4"]) + # A running chunk's item without a prediction leaves no other item free to start beside it. + unpredicted = touch_plan((2, "1.0.0", "", "- `a.py`"), (3, "1.0.0", "", None)) + self.assertEqual(chunk_windows.may_start(gathered.Plan(items=unpredicted, first=unpredicted[0], text="", branch_template=""), ["item-3"], {3}, 3), []) + # A held item that left the plan, as once it merged, overlaps nothing. + self.assertEqual(chunk_windows.may_start(plan, ["item-9"], {9}, 3), ["item-2", "item-4"]) + + def test_a_plan_without_an_item_starts_one_chunk_and_only_alone(self): + none = gathered.NoItem("release 1.0.0 has no items") + self.assertEqual(chunk_windows.may_start(none, [], set(), 3), ["no-item"]) + self.assertEqual(chunk_windows.may_start(none, ["item-1"], {1}, 3), []) + + def test_names_a_window_for_its_item_and_reads_the_item_back(self): + self.assertEqual(chunk_windows.window_name(372), "item-372") + self.assertEqual([chunk_windows.item_of(chunk) for chunk in ("item-372", "no-item", "item-", "item-3x", "xitem-3")], [372, None, None, None, None]) + # Only ASCII digits name an item, as window_name writes them. + self.assertIsNone(chunk_windows.item_of("item-\u0663")) + + def test_starts_only_the_next_item_where_the_touch_predictions_cannot_be_read(self): + with unittest.mock.patch.object(chunk_windows, "chunks_beside", return_value="item 1's Touches: list cannot be read"): + self.assertEqual(chunk_windows.may_start(self.plan("a.py", "c.py"), [], set(), 3), ["item-1"]) + + def test_a_chunk_whose_transcript_cannot_be_read_holds_the_item_it_started_from(self): + with tempfile.TemporaryDirectory() as temp: + transcripts = pathlib.Path(temp) + folder = transcripts / "-r" + folder.mkdir() + # A folder in place of the transcript file, so reading it fails. + (folder / f"{chunk_windows.chunk_session('proj', 'item-7', transcripts).id}.jsonl").mkdir() + self.assertEqual(chunk_windows.held_items(["item-7"], "proj", transcripts), {7}) + + def test_tells_a_chunk_the_items_the_projects_other_live_chunks_hold(self): + with tempfile.TemporaryDirectory() as temp: + home = pathlib.Path(temp) + held = paths.paths_from(home) + folder = held.claude_code_transcripts / "-r" + folder.mkdir(parents=True) + lines = [assistant({"type": "text", "text": "Chunk items: 8 (L 1, E + I 2), 11 (L 2, E + I 3)"})] + (folder / f"{chunk_windows.chunk_session('r', 'item-8', held.claude_code_transcripts).id}.jsonl").write_text("".join(json.dumps(line) + "\n" for line in lines)) + # Its own window, item-7, holds nothing for it; item-9's stopped chunk holds its item. + fake = FakeEffects(windows="item-7\t0\t@1\nitem-8 ?\t0\t@2\nitem-9\t1\t@3\nitem-10\t0\t@4\n") + self.assertEqual(chunk_windows.held_beside("item-7", "/r", held, fake.effects()), [8, 9, 10, 11]) + self.assertEqual(fake.tmux_calls, [["list-windows", "-t", "=loop-r", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"]]) + + +class StartChunks(unittest.TestCase): + """start_chunks: the windows a loop opens for the chunks that may start now.""" + LAUNCHER = pathlib.Path("/b/loop-claude") + def setUp(self): - temp = tempfile.TemporaryDirectory() - self.addCleanup(temp.cleanup) - self.home = pathlib.Path(temp.name) - self.paths = paths.paths_from(self.home) - self.addCleanup(self.release) + made = tempfile.TemporaryDirectory() + self.addCleanup(made.cleanup) + self.root = pathlib.Path(made.name, "r") + self.home = configured(self, self.root).parents[1] + self.session = "loop-" + str(self.root).lstrip("/").replace("/", "-").replace(".", "-") + # Items 1 and 2 overlap, so 2 waits; 3 runs beside 1. + self.plan = ChunkWindows.plan(self, "a.py", "a.py", "c.py") - def held(self): - """This process's open files on a slot, each with whether an exec keeps it.""" - folder = self.home / chunk_slot.SLOTS - found = {} - for fd in os.listdir("/proc/self/fd"): - with contextlib.suppress(OSError): - if pathlib.Path(os.readlink(f"/proc/self/fd/{fd}")).is_relative_to(folder): - found[os.readlink(f"/proc/self/fd/{fd}")] = os.get_inheritable(int(fd)) - return found + def limit(self, chunks): + config = self.home / ".claude-projects" / self.root.name / "config.toml" + config.write_text(config.read_text() + f"parallel-chunks = {chunks}\n") - def release(self): - folder = str(self.home / chunk_slot.SLOTS) - for fd in os.listdir("/proc/self/fd"): - with contextlib.suppress(OSError): - if os.readlink(f"/proc/self/fd/{fd}").startswith(folder): - os.close(int(fd)) + def start(self, fake, chunk=None, leaving=False, plan=None, env=None, owner=None): + """start_chunks' answer, for a session whose @loop-root names owner, by default this checkout.""" + fake.owner = str(self.root) if owner is None else owner + run = main.LoopRun(0, str(self.root), "ship it", "", chunk=chunk) + with unittest.mock.patch.object(chunk_windows, "next_plan", return_value=plan or self.plan) as planned, contextlib.redirect_stderr(io.StringIO()) as said: + effects = fake.effects() + started = chunk_windows.start_chunks(run, leaving, paths.paths_from(self.home), env or {}, self.LAUNCHER, effects) + self.said = said.getvalue() + self.planned = planned.call_count + if planned.call_count: + planned.assert_called_once_with(str(self.root), self.home, effects) + return started - def test_each_loop_takes_the_lowest_free_slot_and_names_its_session_after_it(self): - loop_run = main.LoopRun(cycle_number=0, root=str(self.home / "code" / "proj"), task="do it", unfinished_evidence="") - first, second = chunk_slot.slotted(loop_run, self.paths), chunk_slot.slotted(loop_run, self.paths) - self.assertEqual((first[0], first[1].name), (dataclasses.replace(loop_run, slot=0), "A proj loop-claude")) - self.assertEqual((second[0].slot, second[1].name), (1, "A proj loop-claude 2")) - self.assertEqual(second[1], session.session_named("A proj loop-claude 2", self.paths.claude_code_transcripts)) - folder = self.home / ".local" / "state" / "loop-claude" / "proj" - self.assertEqual(self.held(), {str(folder / "slot-0"): True, str(folder / "slot-1"): True}) + def starts(self, fake): + """The window starts of fake's tmux calls, each as (command, window name, what it runs).""" + return [(call[0], call[call.index("-n") + 1], call[call.index("--") + 1:]) for call in fake.tmux_calls if call[0] in ("new-session", "new-window")] - def test_a_rerun_takes_the_lowest_free_slot_whose_session_is_open_before_a_lower_closed_one(self): - loop_run = main.LoopRun(cycle_number=0, root=str(self.home / "code" / "proj"), task="do it", unfinished_evidence="") - for name in ("A proj loop-claude 3", "A proj loop-claude 4"): - transcript = self.paths.claude_code_transcripts / "-home-proj" / f"{session.session_named(name, self.paths.claude_code_transcripts).id}.jsonl" - transcript.parent.mkdir(parents=True, exist_ok=True) - transcript.write_text("") - self.assertEqual([chunk_slot.slotted(loop_run, self.paths)[0].slot for _ in range(4)], [2, 3, 0, 1]) + def test_opens_a_window_for_each_chunk_that_may_start_the_first_making_the_session(self): + self.limit(2) + fake = FakeEffects() + self.assertEqual(self.start(fake, env={"LOOP_CLAUDE_WORKER_MODEL": "m", "PATH": "/bin"}), ["item-1", "item-3"]) + self.assertEqual(self.starts(fake), [("new-session", "item-1", [str(self.LAUNCHER), "--chunk", "item-1", "ship it"]), ("new-window", "item-3", [str(self.LAUNCHER), "--chunk", "item-3", "ship it"])]) + first = next(call for call in fake.tmux_calls if call[0] == "new-session") + self.assertEqual((first[first.index("-s") + 1], first[first.index("-c") + 1]), (self.session, str(self.root))) + # Only loop-claude's own variables are handed on, since a window gets the tmux server's environment. + self.assertEqual([word for word in first if "=" in word], ["LOOP_CLAUDE_WORKER_MODEL=m"]) + self.assertEqual(fake.tmux_calls[0], ["list-windows", "-t", f"={self.session}", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"]) + # The session names the checkout whose chunks it runs. + self.assertIn(["set-option", "-t", f"={self.session}:", "@loop-root", str(self.root)], fake.tmux_calls) - def test_a_reload_keeps_the_slot_it_is_handed_and_takes_no_other(self): - loop_run = main.LoopRun(cycle_number=3, root="/srv/proj", task="do it", unfinished_evidence="", slot=2) - self.assertEqual(chunk_slot.slotted(loop_run, self.paths), (loop_run, session.session_named("A srv/proj loop-claude 3", self.paths.claude_code_transcripts))) - self.assertEqual(self.held(), {}) + def test_refuses_a_session_that_runs_another_checkouts_chunks(self): + for owner, named in (("/c/acme.api", "/c/acme.api"), ("", "another checkout")): + with self.subTest(owner=owner): + fake = FakeEffects(windows="item-5\t0\t@1\n") + self.assertEqual(self.start(fake, owner=owner), Failure(f"the tmux session {self.session} runs the chunks of {named}, whose key maps to the same session name; rename one of the two checkouts' folders")) + self.assertEqual(self.starts(fake), []) + self.assertIn(["show-options", "-qv", "-t", f"={self.session}:", "@loop-root"], fake.tmux_calls) - def test_says_why_it_takes_no_slot(self): - loop_run = main.LoopRun(cycle_number=0, root="/srv/proj", task="do it", unfinished_evidence="") - with unittest.mock.patch.object(chunk_slot, "MOST", 1): - chunk_slot.slotted(loop_run, self.paths) - self.assertEqual(chunk_slot.slotted(loop_run, self.paths), Failure(f"1 loops of this project are running already; `fuser {self.home}/.local/state/loop-claude/srv/proj/slot-*` names them")) - (self.home / ".local" / "state" / "loop-claude").mkdir(parents=True, exist_ok=True) - (self.home / ".local" / "state" / "loop-claude" / "other").write_text("") - self.assertEqual(chunk_slot.slotted(dataclasses.replace(loop_run, root="/other"), self.paths), Failure(f"cannot take a slot in {self.home}/.local/state/loop-claude/other: File exists")) + def test_leaves_out_the_items_a_running_chunk_names_on_its_chunk_items_line(self): + self.limit(2) + transcripts = paths.paths_from(self.home).claude_code_transcripts + folder = transcripts / "-r" + folder.mkdir(parents=True) + opened = chunk_windows.chunk_session(str(self.root).lstrip("/"), "item-5", transcripts) + (folder / f"{opened.id}.jsonl").write_text(json.dumps(assistant({"type": "text", "text": "Chunk items: 5 (L 1, E + I 2), 1 (L 1, E + I 2)"})) + "\n") + self.assertEqual(self.start(FakeEffects(windows="item-5\t0\t@1\n")), ["item-3"]) + + def test_starts_one_chunk_without_parallel_chunks(self): + fake = FakeEffects() + self.assertEqual(self.start(fake), ["item-1"]) + + def test_starts_none_and_reads_no_plan_while_the_running_chunks_fill_the_limit(self): + self.limit(2) + fake = FakeEffects(windows="item-5\t0\t@1\nitem-6 ?\t0\t@2\n") + self.assertEqual((self.start(fake), self.planned, self.starts(fake)), ([], 0, [])) + # More chunks run than the limit allows, as after it was lowered; a stopped chunk still waits. + fake = FakeEffects(windows="item-5\t0\t@1\nitem-6\t0\t@2\nitem-7\t0\t@3\nitem-8\t1\t@4\nitem-9\t1\t@5\n") + self.assertEqual((self.start(fake), self.planned), ([], 0)) + + def test_a_finishing_chunks_own_window_counts_as_no_longer_running(self): + fake = FakeEffects(windows="item-5\t0\t@1\n") + self.assertEqual(self.start(fake, chunk="item-5"), []) + fake = FakeEffects(windows="item-5\t0\t@1\n") + self.assertEqual(self.start(fake, chunk="item-5", leaving=True), ["item-1"]) + # The session exists, since the finishing loop's window is still in it. + self.assertEqual(self.starts(fake)[0][0], "new-window") + + def test_the_humans_loop_claude_starts_a_stopped_chunk_again_first_closing_its_dead_window(self): + fake = FakeEffects(windows="item-5\t1\t@3\n") + self.assertEqual((self.start(fake), self.planned), (["item-5"], 0)) + self.assertIn(["kill-window", "-t", "@3"], fake.tmux_calls) + # Its dead window was the session's last, so the session is made again. + self.assertEqual(self.starts(fake)[0][0], "new-session") + self.limit(2) + fake = FakeEffects(windows="item-5\t1\t@3\n") + self.assertEqual(self.start(fake), ["item-5", "item-1"]) + + def test_a_finishing_chunk_leaves_a_stopped_chunk_and_its_items_to_the_human(self): + self.limit(2) + transcripts = paths.paths_from(self.home).claude_code_transcripts + folder = transcripts / "-r" + folder.mkdir(parents=True) + opened = chunk_windows.chunk_session(str(self.root).lstrip("/"), "item-5", transcripts) + (folder / f"{opened.id}.jsonl").write_text(json.dumps(assistant({"type": "text", "text": "Chunk items: 5 (L 1, E + I 2), 1 (L 1, E + I 2)"})) + "\n") + fake = FakeEffects(windows="item-7\t0\t@1\nitem-5\t1\t@3\n") + # Item 1 is the stopped chunk's, so item 3 starts; the stopped one doesn't count against parallel-chunks. + self.assertEqual(self.start(fake, chunk="item-7", leaving=True), ["item-3"]) + self.assertEqual([call for call in fake.tmux_calls if call[0] == "kill-window"], []) + fake = FakeEffects(windows="item-7\t0\t@1\nno-item\t1\t@3\n") + self.assertEqual(self.start(fake, chunk="item-7", leaving=True, plan=gathered.NoItem("release 1.0.0 has no items")), []) + + def test_starts_the_chunk_without_a_plan_item_where_the_plan_cannot_be_read(self): + fake = FakeEffects() + self.assertEqual(self.start(fake, plan=Failure("no origin/main")), ["no-item"]) + self.assertIn("starting a chunk without a plan item, since the plan cannot be read: no origin/main", self.said) + + def test_says_why_a_window_could_not_start(self): + with unittest.mock.patch.object(tmux, "start_window", return_value=Failure("tmux could not start the window")): + self.assertEqual(self.start(FakeEffects()), Failure("tmux could not start the window")) + + def test_says_why_it_cannot_take_the_lock_on_starting_windows(self): + (self.home / ".local" / "state").mkdir(parents=True) + (self.home / ".local" / "state" / "loop-claude").write_text("") + started = self.start(FakeEffects()) + self.assertIsInstance(started, Failure) + self.assertTrue(started.reason.startswith(f"cannot lock {self.home}/.local/state/loop-claude/"), started.reason) + + def test_a_chunk_holds_its_own_item_and_those_on_its_latest_chunk_items_line(self): + with tempfile.TemporaryDirectory() as temp: + transcripts = pathlib.Path(temp) + opened = chunk_windows.chunk_session("proj", "item-7", transcripts) + self.assertEqual(opened.name, "A proj loop-claude item-7") + folder = transcripts / "-r" + folder.mkdir() + lines = [assistant({"type": "text", "text": "Chunk items: 7 (L 1, E + I 2)"}), assistant({"type": "text", "text": "Chunk items: 7 (L 1, E + I 2), 9 (L 2, E + I 3)"})] + (folder / f"{opened.id}.jsonl").write_text("".join(json.dumps(line) + "\n" for line in lines)) + self.assertEqual(chunk_windows.held_items(["item-7", "item-8", "no-item"], "proj", transcripts), {7, 8, 9}) class Steps(unittest.TestCase): @@ -1337,10 +1601,27 @@ class ChunkFacts(unittest.TestCase): def test_runs_the_preflight_at_the_checkout_root_and_quotes_what_it_printed(self): fake = FakeEffects(python=[ran("- A fact.\n")]) said = chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER)) - self.assertEqual(fake.made("python"), [(["/b/chunk-preflight", "--worktree"], "/r")]) - self.assertIn("\n\n- A fact.", said) + self.assertEqual(fake.made("python"), [(["/b/chunk-preflight", "--worktree", "--item", "7"], "/r")]) + self.assertTrue(said.endswith("\n\n- A fact."), said) self.assertNotIn("yourself", said) + def test_a_chunk_without_an_item_runs_the_preflight_without_naming_one(self): + fake = FakeEffects(python=[ran("- A fact.\n")]) + chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER, window=tmux.Window("loop-proj", "no-item", "%4"))) + self.assertEqual(fake.made("python"), [(["/b/chunk-preflight", "--worktree"], "/r")]) + + def test_the_facts_end_with_the_items_the_projects_other_running_chunks_hold(self): + # The loop's own window, a stopped one and two that run, one of them waiting for the human. + fake = FakeEffects(python=[ran("- A fact.\n")], windows="item-7\t0\t@1\nitem-5\t1\t@2\nitem-12 ?\t0\t@3\nitem-9\t0\t@4\n") + said = chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER)) + self.assertTrue(said.endswith("- A fact.\n\nOther chunks of this project, running or stopped, hold items 5, 9, 12. Leave those items out of this chunk, even where the facts above list them as items that may join it."), said) + self.assertEqual(prompts.held_by_others([9]), "\n\nOther chunks of this project, running or stopped, hold item 9. Leave those items out of this chunk, even where the facts above list them as items that may join it.") + self.assertTrue(prompts.held_by_others([4, 9]).startswith("\n\nOther chunks of this project, running or stopped, hold items 4, 9.")) + # A worker whose preflight failed is told them too. + fake = FakeEffects(python=[ran(code=1, stderr="git could not start\n")], windows="item-9\t0\t@4\n") + said = chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER)) + self.assertTrue(said.endswith("Other chunks of this project, running or stopped, hold item 9. Leave those items out of this chunk, even where the facts above list them as items that may join it."), said) + def test_has_the_worker_find_the_facts_itself_where_the_preflight_failed(self): for failed, why in ((ran(code=1, stderr="git could not start\n"), "git could not start"), (ran(code=1), "exited 1")): with self.subTest(why=why): @@ -1353,7 +1634,7 @@ class ChunkFacts(unittest.TestCase): opened = session.session_named("A proj loop-claude", pathlib.Path(home, "projects")) fresh = FakeEffects(python=[ran("- A fact.\n")]) facts, _ = chunk_start.chunk_start(a_loop(fresh.effects(), session=opened, launcher=self.LAUNCHER)) - self.assertEqual((fresh.made("python"), "- A fact." in facts), ([(["/b/chunk-preflight", "--worktree"], "/r")], True)) + self.assertEqual((fresh.made("python"), "- A fact." in facts), ([(["/b/chunk-preflight", "--worktree", "--item", "7"], "/r")], True)) transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl") transcript.parent.mkdir(parents=True) transcript.write_text("") @@ -1924,6 +2205,10 @@ class LoopRuns(unittest.TestCase): state = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="") self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(state) | {"items_taken_in": [12, 14]})), state) + def test_drops_the_slot_that_a_loop_started_before_chunks_ran_in_windows_hands_on(self): + state = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="") + self.assertEqual(reload.decode_handed(json.dumps({key: value for key, value in dataclasses.asdict(state).items() if key != "chunk"} | {"slot": 1})), state) + def test_stops_when_the_reload_cannot_start_python(self): with tempfile.TemporaryDirectory() as root: launcher = pathlib.Path(root, "loop-claude") @@ -1932,6 +2217,106 @@ class LoopRuns(unittest.TestCase): self.assertEqual(reload.exec_if_edited(a_loop(fake.effects(), launcher=launcher), "stale", main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="")), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>")) +class WindowStarts(unittest.TestCase): + """The loop-claude the human runs, which starts the windows, and the stop of a window's loop whose chunk finished.""" + HOME = pathlib.Path("/h") + + def run_loop(self, argv, fake, env=None, started=("item-3", "item-4")): + """run_loop's stop and what it said on stdout, with start_chunks answering `started`; self.asked holds what start_chunks was asked.""" + with unittest.mock.patch.object(main, "start_chunks", return_value=list(started) if isinstance(started, tuple) else started) as start, contextlib.redirect_stdout(io.StringIO()) as said: + stop = main.run_loop(argv, env or {}, paths.paths_from(self.HOME), pathlib.Path("/b/loop-claude"), CWD, fake.effects()) + self.asked = [call.args[:2] for call in start.call_args_list] + return stop, said.getvalue() + + def test_refuses_to_start_anything_on_a_tmux_older_than_needed(self): + fake = FakeEffects(git=["/r\n"], tmux_version="tmux 3.1c") + self.assertEqual(self.run_loop(["do it"], fake), (Stop(1, "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 3.1c"), "")) + self.assertEqual((fake.tmux_calls, self.asked), ([["-V"]], [])) + + def test_prints_the_help_without_tmux(self): + fake = FakeEffects(tmux_version="") + self.assertEqual(self.run_loop(["--help"], fake)[0], Stop(0, usage.HELP)) + self.assertEqual(fake.tmux_calls, []) + + def test_starts_the_chunks_then_attaches_to_the_session_or_inside_tmux_switches_to_it(self): + for env, shown in (({}, ["attach-session", "-t", "=loop-r"]), ({"TMUX": "/tmp/tmux-1000/default,1,0"}, ["switch-client", "-t", "=loop-r"])): + with self.subTest(shown=shown): + fake = FakeEffects(git=["/r\n"]) + stop, said = self.run_loop(["do", "it"], fake, env) + self.assertEqual((stop, said), (Stop(0, ""), f"{STAMP} started item-3, item-4 in the tmux session loop-r\n")) + self.assertEqual(fake.tmux_calls, [["-V"], shown]) + self.assertEqual(self.asked, [(main.LoopRun(0, "/r", "do it", ""), False)]) + + def test_hands_the_window_starts_its_paths_environment_launcher_and_effects(self): + fake = FakeEffects(git=["/r\n"], on_terminal=False) + effects, env, launcher = fake.effects(), {"LOOP_CLAUDE_X": "1"}, pathlib.Path("/b/loop-claude") + with unittest.mock.patch.object(main, "start_chunks", return_value=["item-3"]) as start, contextlib.redirect_stdout(io.StringIO()): + main.run_loop(["do it"], env, paths.paths_from(self.HOME), launcher, CWD, effects) + self.assertEqual(start.call_args.args, (main.LoopRun(0, "/r", "do it", ""), False, paths.paths_from(self.HOME), env, launcher, effects)) + + def test_a_window_loop_runs_its_chunk_in_its_own_session_and_window_and_hands_on_once_it_finished(self): + fake = FakeEffects(git=["/r\n"]) + effects, env = fake.effects(), {"TMUX_PANE": "%4"} + launcher = pathlib.Path("/b/loop-claude") + cycle = unittest.mock.Mock(project="claude-opus-5-5") + unfinished = unittest.mock.Mock(verdict=unittest.mock.Mock(spec=Unfinished)) + finished = unittest.mock.Mock(verdict=unittest.mock.Mock(spec=Finished)) + closed = [main.LoopRun(1, "/r", "do it", "", chunk="item-3"), main.LoopRun(2, "/r", "do it", "", chunk="item-3")] + with unittest.mock.patch.object(main, "fingerprint", return_value=""), unittest.mock.patch.object(main, "exec_if_edited", return_value=None), unittest.mock.patch.object(main, "next_cycle", return_value=cycle) as next_cycle, \ + unittest.mock.patch.object(main, "run_cycle", side_effect=[unfinished, finished]), unittest.mock.patch.object(main, "closed_cycle", side_effect=closed), \ + unittest.mock.patch.object(main, "check_routing", return_value=None), unittest.mock.patch.object(main, "handed_on", return_value=Stop(0, "")) as handed_on: + stop = main.run_loop(["--chunk", "item-3", "do it"], env, paths.paths_from(self.HOME), launcher, CWD, effects) + loop = this_loop.Loop(root="/r", session=session.session_named("A r loop-claude item-3", self.HOME / ".claude" / "projects"), window=tmux.Window("loop-r", "item-3", "%4"), paths=paths.paths_from(self.HOME), env=env, launcher=launcher, effects=effects) + self.assertEqual(stop, Stop(0, "")) + self.assertEqual([call.args for call in next_cycle.call_args_list], [(main.LoopRun(1, "/r", "do it", "", chunk="item-3"), loop), (main.LoopRun(2, "/r", "do it", "", chunk="item-3"), loop)]) + handed_on.assert_called_once_with(closed[1], loop) + + def test_a_window_loop_given_no_task_prints_the_help(self): + self.assertEqual(main.loop_run_from(["--chunk", "item-3"], CWD, FakeEffects(git=["/r\n"]).effects()), Stop(1, usage.HELP)) + + def test_a_window_loop_outside_a_checkout_stops(self): + self.assertIsInstance(main.loop_run_from(["--chunk", "item-3", "do it"], CWD, FakeEffects(git=[Failure("git cannot start")]).effects()), Stop) + + def test_says_that_nothing_started_where_the_session_already_runs_every_chunk_that_may(self): + self.assertEqual(self.run_loop(["do it"], FakeEffects(git=["/r\n"]), started=())[1], f"{STAMP} the tmux session loop-r runs every chunk that may run now, so `do it` started in no new window\n") + + def test_off_a_terminal_it_prints_how_to_attach(self): + fake = FakeEffects(git=["/r\n"], on_terminal=False) + stop, said = self.run_loop(["do it"], fake) + self.assertEqual(stop, Stop(0, "")) + self.assertTrue(said.endswith(f"{STAMP} not on a terminal; see the chunks with `tmux attach -t =loop-r`\n"), said) + self.assertEqual(fake.tmux_calls, [["-V"]]) + + def test_stops_where_no_chunk_could_start(self): + self.assertEqual(self.run_loop(["do it"], FakeEffects(git=["/r\n"]), started=Failure("no plan"))[0], Stop(1, "no plan")) + + def test_stops_where_tmux_cannot_show_the_session(self): + fake = FakeEffects(git=["/r\n"]) + effects = dataclasses.replace(fake.effects(), tmux_on_terminal=lambda args: 1) + with unittest.mock.patch.object(main, "start_chunks", return_value=["item-3"]), contextlib.redirect_stdout(io.StringIO()): + stop = main.run_loop(["do it"], {}, paths.paths_from(self.HOME), pathlib.Path("/b/loop-claude"), CWD, effects) + self.assertEqual(stop, Stop(1, "`tmux attach-session -t =loop-r` failed; the chunks go on in the tmux session")) + + def test_a_window_loop_outside_tmux_stops(self): + fake = FakeEffects(git=["/r\n"]) + self.assertEqual(self.run_loop(["--chunk", "item-3", "do it"], fake)[0], Stop(1, "--chunk is how loop-claude starts itself in a tmux window; run loop-claude <task>")) + self.assertEqual(self.asked, []) + + def test_reads_the_chunk_a_window_loop_is_started_with(self): + self.assertEqual(main.loop_run_from(["--chunk", "item-3", "do", "it"], CWD, FakeEffects(git=["/r\n"]).effects()), main.LoopRun(0, "/r", "do it", "", chunk="item-3")) + self.assertEqual(main.loop_run_from(["do", "it"], CWD, FakeEffects(git=["/r\n"]).effects()), main.LoopRun(0, "/r", "do it", "")) + + def test_a_finished_chunk_starts_each_chunk_that_may_start_and_stops_without_a_reason(self): + loop_run = main.LoopRun(4, "/r", "do it", "", chunk="item-7") + for started, said, stop in ((["item-9", "item-12"], "the chunk finished; started item-9, item-12", Stop(0, "")), ([], "the chunk finished; started no other, since the chunks running fill what may run now", Stop(0, "")), + (Failure("no plan"), "", Stop(1, "the chunk finished, but the next could not start: no plan"))): + with self.subTest(started=started), unittest.mock.patch.object(main, "start_chunks", return_value=started) as start, contextlib.redirect_stdout(io.StringIO()) as printed: + loop = a_loop(FakeEffects().effects(), env={"LOOP_CLAUDE_X": "1"}) + self.assertEqual(main.handed_on(loop_run, loop), stop) + self.assertEqual(printed.getvalue(), f"{STAMP} {said}\n" if said else "") + self.assertEqual(start.call_args.args, (loop_run, True, loop.paths, loop.env, loop.launcher, loop.effects)) + + class StopDms(unittest.TestCase): def run_main(self, argv, fake): with tempfile.TemporaryDirectory() as home, contextlib.redirect_stderr(io.StringIO()) as said, contextlib.redirect_stdout(io.StringIO()): @@ -1944,6 +2329,12 @@ class StopDms(unittest.TestCase): self.assertEqual(self.run_main(["do it"], fake)[0], 2) self.assertEqual(fake.dms, [f"loop-claude in {CWD} stopped: the human is needed to look at the chunk, but it stayed unfinished"]) + def test_skips_the_dm_of_a_stop_without_a_reason_as_once_the_windows_started(self): + fake = FakeEffects() + with unittest.mock.patch.object(main, "run_loop", return_value=Stop(0, "")): + self.assertEqual(self.run_main(["do it"], fake), (0, "")) + self.assertEqual(fake.dms, []) + def test_skips_the_dm_where_ctrl_c_stopped_the_loop(self): fake = FakeEffects() with unittest.mock.patch.object(main, "run_loop", side_effect=KeyboardInterrupt): @@ -2021,7 +2412,11 @@ class ExitContract(unittest.TestCase): ("cycle/run.py", "close", "1", "failure.reason", read), ("cycle/run.py", "close", "1", "failure.reason", "a finished chunk's session cannot be deleted"), ("loop/close.py", "closed_cycle", "1", "failure.reason", read), - ("loop/main.py", "run_loop", "1", "held.reason", "its slot in the project cannot be taken"), + ("loop/main.py", "run_loop", "1", "reason", "tmux is missing or older than 3.2"), + ("loop/main.py", "run_loop", "1", "f'{CHUNK_FLAG} is how loop-claude starts itself", "--chunk is given outside a tmux window"), + ("loop/main.py", "windows_opened", "1", "started.reason", "a chunk's window cannot be started"), + ("loop/main.py", "windows_opened", "1", "f\"`tmux {' '.join(shown)}` failed", "the tmux session cannot be put on the terminal"), + ("loop/main.py", "handed_on", "1", "f'the chunk finished, but the next could not start", "a chunk's window cannot be started"), ("loop/main.py", "next_cycle", "1", "premises.reason", premises_), ("loop/main.py", "next_cycle", "1", "records.reason", unreadable), ("loop/main.py", "next_cycle", "1", "ladder_file.reason", premises_), diff --git a/bin/loop_claude/tmux.py b/bin/loop_claude/tmux.py new file mode 100644 index 00000000..1f2e0519 --- /dev/null +++ b/bin/loop_claude/tmux.py @@ -0,0 +1,101 @@ +"""The tmux commands loop-claude runs: each chunk's loop runs in its own window of one tmux session named for the project (docs/decisions.md → `loop-claude: each chunk runs in its own tmux window`).""" +import dataclasses +import re +import subprocess +import typing + +from failure import Failure + +# Runs tmux with these arguments and returns what it printed; exit 127 where tmux cannot start. +Tmux = typing.Callable[[list[str]], subprocess.CompletedProcess[str]] + +# The oldest tmux that has every command and option used here. Its CHANGES file dates the newest two to 3.2: `remain-on-exit failed` and `new-session -e`. +NEEDED = (3, 2) +# What a window's name ends in while its chunk waits for the human. tmux's status bar shows the name, and `!` after it once the bell rang. +WAITS = " ?" +BELL = "\a" +# The session option that names the checkout whose chunks the session runs, since two checkouts' keys may map to one session name. +OWNER = "@loop-root" + + +@dataclasses.dataclass(frozen=True) +class Window: + """The tmux window a chunk's loop runs in.""" + session: str + # Without WAITS. + name: str + # The ID of the loop's pane, which $TMUX_PANE holds. It targets the window whatever its name is. + pane: str + + @property + def target(self) -> str: + """What `tmux attach -t` and `tmux select-window -t` take to reach the window.""" + return f"{self.session}:{self.name}" + + +@dataclasses.dataclass(frozen=True) +class Listed: + """A window of the project's session, as tmux lists it.""" + # Without WAITS. + name: str + # tmux's ID of the window, such as `@3`. + id: str + # Whether its loop has exited: `remain-on-exit failed` keeps the window of a loop that failed, so its last lines stay readable. + dead: bool + + +def unusable(run: Tmux) -> str | None: + """Why loop-claude cannot use this machine's tmux, naming the version found and the one needed; None where it is NEEDED or later.""" + found = run(["-V"]).stdout.strip() + version = re.search(r"([0-9]+)\.([0-9]+)", found) + if version and (int(version[1]), int(version[2])) >= NEEDED: + return None + return f"loop-claude runs each chunk in a tmux window, and needs tmux {NEEDED[0]}.{NEEDED[1]} or later; it found {found or 'no tmux'}" + + +def session_name(key: str) -> str: + """The tmux session of the project with this key, such as `loop-claude-config`. tmux reads `.` and `:` in a target as separators, so every character but a letter, a digit, `_` and `-` becomes a dash.""" + return "loop-" + re.sub(r"[^A-Za-z0-9_-]", "-", key) + + +def listed(session: str, run: Tmux) -> list[Listed]: + """The windows of session; none where tmux lists none, as while the session doesn't exist or no tmux server runs.""" + done = run(["list-windows", "-t", f"={session}", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"]) + rows = [line.split("\t") for line in done.stdout.splitlines()] if done.returncode == 0 else [] + return [Listed(row[0].removesuffix(WAITS), row[2], row[1] == "1") for row in rows if len(row) == 3] + + +def start_window(window: Window, cwd: str, env: typing.Mapping[str, str], command: list[str], first: bool, run: Tmux) -> Failure | None: + """Starts command in a new window of window's session, in cwd with env added to its environment, and leaves the client where it is. first makes the session too. A Failure says why tmux started none.""" + make = ["new-session", "-d", "-s", window.session] if first else ["new-window", "-d", "-t", f"={window.session}:"] + variables = [flag for name, value in env.items() for flag in ("-e", f"{name}={value}")] + # tmux reads a `;` that ends an argument as the end of the command, unless a backslash precedes it. + words = [re.sub(r";$", r"\\;", word) for word in command] + done = run([*make, "-n", window.name, "-c", cwd, *variables, "-P", "-F", "#{window_id}", "--", *words]) + if done.returncode: + return Failure(f"tmux could not start the window {window.target}: {done.stderr.strip() or f'it exited {done.returncode}'}") + if first: + # set-option and show-options read a target as a pane's, so `=session` alone names no session. + run(["set-option", "-t", f"={window.session}:", OWNER, cwd]) + # By the ID, since a window of the same name may still be closing. A loop that fails before the option is set closes its window, and its stop is still sent as a DM. + run(["set-option", "-w", "-t", done.stdout.strip(), "remain-on-exit", "failed"]) + return None + + +def owner(session: str, run: Tmux) -> str: + """The checkout whose chunks session runs, as OWNER names it; empty where it names none.""" + return run(["show-options", "-qv", "-t", f"={session}:", OWNER]).stdout.strip() + + +def close(window: Listed, run: Tmux) -> None: + run(["kill-window", "-t", window.id]) + + +def mark(window: Window, waits: bool, run: Tmux) -> None: + """Renames window to end in WAITS while its chunk waits for the human, and back once it doesn't.""" + run(["rename-window", "-t", window.pane, window.name + (WAITS if waits else "")]) + + +def show(session: str, inside: bool) -> list[str]: + """The tmux arguments that put session on the human's terminal: inside tmux the client switches to it, and outside a new client attaches.""" + return ["switch-client" if inside else "attach-session", "-t", f"={session}"] diff --git a/bin/loop_effects/effects.py b/bin/loop_effects/effects.py index 625f8507..457ef24e 100644 --- a/bin/loop_effects/effects.py +++ b/bin/loop_effects/effects.py @@ -51,6 +51,10 @@ class Effects: python: typing.Callable[[list[str], str], subprocess.CompletedProcess[str]] sleep: typing.Callable[[float], None] stream_claude: typing.Callable[[list[str], str, dict[str, str]], Streaming] + # Runs tmux with these arguments, as run_tmux does. + tmux: typing.Callable[[list[str]], subprocess.CompletedProcess[str]] + # Runs tmux on the terminal, as to attach a session, and returns its exit code once the human detaches. + tmux_on_terminal: typing.Callable[[list[str]], int] class StreamingClaude: @@ -82,6 +86,8 @@ def system_effects(backends: typing.Mapping[Notifier, Backend[typing.Any]] = BAC python=lambda args, cwd: run_python(args, cwd, PYTHON_TIMEOUT), sleep=time.sleep, stream_claude=StreamingClaude, + tmux=run_tmux, + tmux_on_terminal=lambda args: subprocess.run(["tmux", *args]).returncode, ) @@ -106,6 +112,15 @@ def run_python(args: list[str], cwd: str, timeout: float) -> subprocess.Complete return subprocess.CompletedProcess(command, 1, "", f"stopped after {timeout} s") +def run_tmux(args: list[str]) -> subprocess.CompletedProcess[str]: + """Runs tmux and returns what it printed. A tmux that cannot start, as where none is installed, reads as exit 127, saying why on stderr.""" + command = ["tmux", *args] + try: + return subprocess.run(command, capture_output=True, encoding="utf-8", errors="replace", stdin=subprocess.DEVNULL) + except OSError as failure: + return subprocess.CompletedProcess(command, 127, "", f"could not start: {failure}") + + def say(text: str, effects: Effects, *, stdout: bool = False) -> None: """Prints text on stderr, or on stdout where stdout, opening the first line of words with the machine's local time. Blank lines before those words stay blank, and later lines get no time.""" body = text.lstrip("\n") diff --git a/bin/loop_effects/test_effects.py b/bin/loop_effects/test_effects.py index 93b43067..45a0c479 100644 --- a/bin/loop_effects/test_effects.py +++ b/bin/loop_effects/test_effects.py @@ -84,6 +84,36 @@ class ClaudeOnTerminal(unittest.TestCase): +class TmuxEffect(unittest.TestCase): + """Each test runs a fake tmux off its own PATH, without $TMUX, so the human's tmux server is never reached.""" + + def fake_tmux(self, folder): + tmux = pathlib.Path(folder, "tmux") + tmux.write_text(f"#!{sys.executable}\nimport sys\nprint(' '.join(sys.argv[1:]))\nsys.exit(4)\n") + tmux.chmod(0o755) + return unittest.mock.patch.dict(os.environ, {"PATH": folder, "TMUX": "", "TMUX_PANE": ""}) + + def test_runs_tmux_and_returns_what_it_printed(self): + with tempfile.TemporaryDirectory() as folder, self.fake_tmux(folder): + done = system_effects().tmux(["list-windows", "-t", "=loop-proj"]) + self.assertEqual((done.returncode, done.stdout), (4, "list-windows -t =loop-proj\n")) + + def test_reads_output_that_is_not_utf_8_with_each_bad_byte_replaced(self): + with tempfile.TemporaryDirectory() as folder, self.fake_tmux(folder): + pathlib.Path(folder, "tmux").write_text(f"#!{sys.executable}\nimport sys\nsys.stdout.buffer.write(b'caf\\xe9')\n") + self.assertEqual(system_effects().tmux(["list-windows"]).stdout, "caf\ufffd") + + def test_reads_a_tmux_that_cannot_start_as_exit_127(self): + with tempfile.TemporaryDirectory() as folder, unittest.mock.patch.dict(os.environ, {"PATH": folder}): + done = system_effects().tmux(["-V"]) + self.assertEqual((done.args, done.returncode, done.stdout), (["tmux", "-V"], 127, "")) + self.assertIn("could not start", done.stderr) + + def test_runs_tmux_on_the_terminal_and_returns_its_exit_code(self): + with tempfile.TemporaryDirectory() as folder, self.fake_tmux(folder), contextlib.redirect_stdout(io.StringIO()): + self.assertEqual(system_effects().tmux_on_terminal(["attach-session", "-t", "=loop-proj"]), 4) + + class PythonEffect(unittest.TestCase): def test_runs_a_script_in_the_given_directory(self): with tempfile.TemporaryDirectory() as cwd: diff --git a/bin/machine_load/slots.py b/bin/machine_load/slots.py index 0ad47408..9b2e9d64 100644 --- a/bin/machine_load/slots.py +++ b/bin/machine_load/slots.py @@ -28,12 +28,7 @@ def slot_folder(env: typing.Mapping[str, str], home: pathlib.Path) -> pathlib.Pa def taken_slot(folder: pathlib.Path, slots: int) -> tuple[int, int] | None: """The number and open file of the lowest slot this call locked; None while every slot is held. Raises OSError where a slot file cannot be opened or locked.""" - return first_free_slot(folder, range(slots)) - - -def first_free_slot(folder: pathlib.Path, order: typing.Iterable[int]) -> tuple[int, int] | None: - """The number and open file of the first slot in order that this call locked; None while every slot in order is held. Raises OSError where a slot file cannot be opened or locked.""" - for slot in order: + for slot in range(slots): fd = os.open(folder / f"slot-{slot}", os.O_RDWR | os.O_CREAT, 0o644) try: fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) diff --git a/bin/project_config/main.py b/bin/project_config/main.py index aba22ab6..f5e35141 100644 --- a/bin/project_config/main.py +++ b/bin/project_config/main.py @@ -91,8 +91,12 @@ aims for. While the panel's overall score is under it, code health weighs 4 time ranking, and a scoring run with a mean under it asks for a floor item. A legacy project, whose code scores low, may lower it. Left out, it is 7.0. -`project-config tooling-on-host` and `project-config readability-floor` print their values; the full -listing shows each only where the config changes it from what it is when left out. +`parallel-chunks` is an optional setting, a whole number of 1 or more: the most chunks of the project +loop-claude runs at once, each in its own tmux window. Left out, it is 1. + +`project-config tooling-on-host`, `project-config readability-floor` and `project-config parallel-chunks` +print their values; the full listing shows each only where the config changes it from what it is when +left out. Four optional settings say how the project writes, for a project whose owners don't write as ~/.claude/CLAUDE.md does: @@ -130,13 +134,13 @@ setting, names an unknown one, misspells a path, a language or a template, sets without `env-file`, sets `jira-finish-transition` or `jira-finish-status` to anything but a name, without the other or without a `plan` in Jira, names a Jira `plan` and leaves out `jira-score-formula` or `jira-score-bar`, sets one of those two without a `plan` in Jira, sets the formula to one that doesn't parse or the bar to anything -but a number, or sets `readability-floor` to anything but a number -from 1 to 10, and where `env-file`, `jira-finish-transition`, `jira-finish-status`, `notifier` or `touch-predictions` is asked for and not set. No path setting has a default. +but a number, sets `readability-floor` to anything but a number +from 1 to 10, or sets `parallel-chunks` to anything but a whole number of 1 or more, and where `env-file`, `jira-finish-transition`, `jira-finish-status`, `notifier` or `touch-predictions` is asked for and not set. No path setting has a default. """ import pathlib import sys -from project_config.settings import CONFIG, DEFAULT_FLOOR, ENV_FILE, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION, JIRA_SCORE_BAR, JIRA_SCORE_FORMULA, load, NOTIFIER, OPTIONAL, ProjectConfig, READABILITY_FLOOR, SETTINGS, TOOLING_ON_HOST, Unnamed, WAIT_FOR_APPROVAL +from project_config.settings import CONFIG, DEFAULT_FLOOR, ENV_FILE, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION, JIRA_SCORE_BAR, JIRA_SCORE_FORMULA, load, NOTIFIER, OPTIONAL, PARALLEL_CHUNKS, ProjectConfig, READABILITY_FLOOR, SETTINGS, TOOLING_ON_HOST, Unnamed, WAIT_FOR_APPROVAL from failure import Failure from git_cli.git_repo import git, Git from project_config import templates @@ -160,6 +164,9 @@ def main(argv: list[str]) -> int: if argv == [READABILITY_FLOOR]: print(config.readability_floor) return 0 + if argv == [PARALLEL_CHUNKS]: + print(config.parallel_chunks) + return 0 if argv and argv[0] in config.texts: print(config.texts[argv[0]]) return 0 @@ -168,7 +175,7 @@ def main(argv: list[str]) -> int: print(f"{config.folder / CONFIG} sets no `{argv[0]}`", file=sys.stderr) return 1 defaults = ProjectConfig(config.key, config.folder, config.top, {}).texts - listed = [f"{name}: {path}" for name, path in paths.items()] + [f"{name}: true" for name, on in switches.items() if on] + ([f"{READABILITY_FLOOR}: {config.readability_floor}"] if config.readability_floor != DEFAULT_FLOOR else []) + [f"{name}: {text}" for name, text in config.texts.items() if text != defaults[name]] + listed = [f"{name}: {path}" for name, path in paths.items()] + [f"{name}: true" for name, on in switches.items() if on] + ([f"{READABILITY_FLOOR}: {config.readability_floor}"] if config.readability_floor != DEFAULT_FLOOR else []) + ([f"{PARALLEL_CHUNKS}: {config.parallel_chunks}"] if config.parallel_chunks != 1 else []) + [f"{name}: {text}" for name, text in config.texts.items() if text != defaults[name]] print(paths[argv[0]] if argv else "\n".join(listed)) return 0 diff --git a/bin/project_config/settings.py b/bin/project_config/settings.py index 7a20e1fd..5948e4cf 100644 --- a/bin/project_config/settings.py +++ b/bin/project_config/settings.py @@ -35,6 +35,8 @@ TOOLING_ON_HOST = "tooling-on-host" READABILITY_FLOOR = "readability-floor" # The floor where the config leaves it out, which every dimension and the overall should reach; docs/decisions.md → "comprehension floor" says it doesn't block anything. DEFAULT_FLOOR = decimal.Decimal("7.0") +# The most chunks of the project loop-claude runs at once, each in its own tmux window; 1 where left out. Three readers seated as the README's audience picked the spelling 2-1 over concurrent-chunks. +PARALLEL_CHUNKS = "parallel-chunks" # The optional text settings. `language` is the language the project's prose is written in, by its English name. The others are templates, as `templates` says. LANGUAGE = "language" COMMIT_TITLE = "commit-title" @@ -50,7 +52,7 @@ JIRA_FINISH = (JIRA_FINISH_TRANSITION, JIRA_FINISH_STATUS) JIRA_SCORE_FORMULA = "jira-score-formula" JIRA_SCORE_BAR = "jira-score-bar" JIRA_SCORE = (JIRA_SCORE_FORMULA, JIRA_SCORE_BAR) -OPTIONAL = (BRANCH, COMMIT_TITLE, ENV_FILE, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION, JIRA_SCORE_BAR, JIRA_SCORE_FORMULA, LANGUAGE, NOTIFIER, PULL_REQUEST_TITLE, READABILITY_FLOOR, TOOLING_ON_HOST, TOUCH_PREDICTIONS, WAIT_FOR_APPROVAL) +OPTIONAL = (BRANCH, COMMIT_TITLE, ENV_FILE, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION, JIRA_SCORE_BAR, JIRA_SCORE_FORMULA, LANGUAGE, NOTIFIER, PARALLEL_CHUNKS, PULL_REQUEST_TITLE, READABILITY_FLOOR, TOOLING_ON_HOST, TOUCH_PREDICTIONS, WAIT_FOR_APPROVAL) # What opens a setting that names a Jira Cloud epic in place of a path: `jira:`, then the epic's address as the browser shows it. The settings below may name an epic; each reads its own part of it. JIRA = "jira:" EPIC_SPELLINGS = { @@ -117,6 +119,7 @@ class ProjectConfig: wait_for_approval: bool = False tooling_on_host: bool = False readability_floor: decimal.Decimal = DEFAULT_FLOOR + parallel_chunks: int = 1 language: str = "English" commit_title: str = templates.TITLE pull_request_title: str = templates.TITLE @@ -194,6 +197,7 @@ def load(top: pathlib.Path, main: pathlib.Path, home: pathlib.Path) -> ProjectCo waits = values.get(WAIT_FOR_APPROVAL, False) on_host = values.get(TOOLING_ON_HOST, False) floor = floor_of(values[READABILITY_FLOOR]) if READABILITY_FLOOR in values else DEFAULT_FLOOR + parallel = values.get(PARALLEL_CHUNKS, 1) finish = [name for setting in JIRA_FINISH if (name := name_of(values.get(setting)))] formula, bar = values.get(JIRA_SCORE_FORMULA), values.get(JIRA_SCORE_BAR) scoring = JiraScoring(formula, bar_value) if isinstance(formula, str) and (bar_value := bar_of(bar)) is not None else None @@ -221,14 +225,16 @@ def load(top: pathlib.Path, main: pathlib.Path, home: pathlib.Path) -> ProjectCo *([f"`{JIRA_SCORE_BAR}` is {bar!r}, not a bare number, such as `{JIRA_SCORE_BAR} = 1`"] if JIRA_SCORE_BAR in values and bar_of(bar) is None else []), *(f"`{name}` is {value!r}, not true or false" for name, value in ((WAIT_FOR_APPROVAL, waits), (TOOLING_ON_HOST, on_host)) if not isinstance(value, bool)), *([f"`{READABILITY_FLOOR}` is {values[READABILITY_FLOOR]!r}, not a number from 1 to 10; write a bare number, such as `{READABILITY_FLOOR} = 7.0`"] if floor is None else []), + *([f"`{PARALLEL_CHUNKS}` is {parallel!r}, not a whole number of 1 or more, such as `{PARALLEL_CHUNKS} = 2`"] if type(parallel) is not int or parallel < 1 else []), *(f"`{name}` is {values[name]!r}, {fault}" for name, check in checks.items() if name in values and (fault := check(values[name]))), ] if faults: return Failure(f"{file}: {'; '.join(faults)}. The settings are {', '.join(SETTINGS)}, and optionally {', '.join(OPTIONAL[:-1])} and {OPTIONAL[-1]}.") - # faults names a floor the config misspells, and a formula or bar a plan in Jira lacks or misspells. + # faults names a floor or a `parallel-chunks` value the config misspells, and a formula or bar a plan in Jira lacks or misspells. assert floor is not None + assert isinstance(parallel, int) assert scoring is not None or not epics["plan"] - return ProjectConfig(file.parent.name, file.parent, top, {name: path for name, path in paths.items() if path}, env_file, waits is True, on_host is True, floor, **{name.replace("-", "_"): str(values[name]) for name in checks if name in values}, epic=epics["plan"], goals_epic=epics["goals-and-audience"], notifier=notifier, jira_finish=Finish(*finish) if finish else None, jira_scoring=scoring, pages=pages) + return ProjectConfig(file.parent.name, file.parent, top, {name: path for name, path in paths.items() if path}, env_file, waits is True, on_host is True, floor, parallel, **{name.replace("-", "_"): str(values[name]) for name in checks if name in values}, epic=epics["plan"], goals_epic=epics["goals-and-audience"], notifier=notifier, jira_finish=Finish(*finish) if finish else None, jira_scoring=scoring, pages=pages) def formula_fault(value: object) -> str | None: diff --git a/bin/project_config/test_project_config.py b/bin/project_config/test_project_config.py index 26c4e828..7a07c131 100644 --- a/bin/project_config/test_project_config.py +++ b/bin/project_config/test_project_config.py @@ -462,6 +462,28 @@ class LoadTest(ConfigCase): self.assertIn("`readability-floor` is ", found.reason) self.assertIn(", not a number from 1 to 10; write a bare number, such as `readability-floor = 7.0`", found.reason) + def test_parallel_chunks_is_an_optional_whole_number_from_1_that_is_1_where_left_out(self) -> None: + folder = self.write("acme", CLIENT) + left_out = load(self.checkout, self.checkout, self.home) + assert isinstance(left_out, ProjectConfig), left_out + self.assertEqual(left_out.parallel_chunks, 1) + for spelled in (1, 3, 12): + with self.subTest(spelled): + (folder / "config.toml").write_text(CLIENT + f"parallel-chunks = {spelled}\n") + found = load(self.checkout, self.checkout, self.home) + assert isinstance(found, ProjectConfig), found + self.assertEqual(found.parallel_chunks, spelled) + + def test_parallel_chunks_that_is_no_whole_number_from_1_fails_naming_it(self) -> None: + folder = self.write("acme", CLIENT) + for spelled in ('"2"', "0", "-1", "2.0", "true", "inf"): + with self.subTest(spelled): + (folder / "config.toml").write_text(CLIENT + f"parallel-chunks = {spelled}\n") + found = load(self.checkout, self.checkout, self.home) + assert isinstance(found, Failure), found + self.assertIn("`parallel-chunks` is ", found.reason) + self.assertIn(", not a whole number of 1 or more, such as `parallel-chunks = 2`", found.reason) + def test_the_language_and_the_templates_are_optional_and_default_to_english_bare_summaries_and_item_branches(self) -> None: folder = self.write("acme", CLIENT) left_out = load(self.checkout, self.checkout, self.home) @@ -497,7 +519,7 @@ class LoadTest(ConfigCase): def test_the_failure_for_a_misplaced_env_file_and_a_missing_setting_names_both_and_lists_every_setting(self) -> None: folder = self.write("acme", "".join(line + "\n" for line in CLIENT.splitlines() if not line.startswith("plan")) + 'env-file = "checkout:.env"\n') - self.assertEqual(self.failure(), f"{folder / 'config.toml'}: `plan` is missing; `env-file` is 'checkout:.env', not {OUTSIDE}. The settings are {', '.join(SETTINGS)}, and optionally branch, commit-title, env-file, jira-finish-status, jira-finish-transition, jira-score-bar, jira-score-formula, language, notifier, pull-request-title, readability-floor, tooling-on-host, touch-predictions and wait-for-approval.") + self.assertEqual(self.failure(), f"{folder / 'config.toml'}: `plan` is missing; `env-file` is 'checkout:.env', not {OUTSIDE}. The settings are {', '.join(SETTINGS)}, and optionally branch, commit-title, env-file, jira-finish-status, jira-finish-transition, jira-score-bar, jira-score-formula, language, notifier, parallel-chunks, pull-request-title, readability-floor, tooling-on-host, touch-predictions and wait-for-approval.") def test_two_configs_naming_one_checkout_fail_naming_both(self) -> None: self.write("acme", CLIENT) @@ -652,6 +674,14 @@ class ToolTest(ConfigCase): self.assertEqual((self.run_tool("tooling-on-host").stdout, self.run_tool("readability-floor").stdout), ("true\n", "5\n")) self.assertEqual(self.run_tool().stdout.splitlines()[-2:], ["tooling-on-host: true", "readability-floor: 5"]) + def test_prints_parallel_chunks_and_lists_it_only_where_the_config_changes_it(self) -> None: + folder = self.write("acme", CLIENT) + self.assertEqual(self.run_tool("parallel-chunks").stdout, "1\n") + self.assertNotIn("parallel-chunks", self.run_tool().stdout) + (folder / "config.toml").write_text(CLIENT + "parallel-chunks = 3\n") + self.assertEqual(self.run_tool("parallel-chunks").stdout, "3\n") + self.assertIn("parallel-chunks: 3", self.run_tool().stdout.splitlines()) + def test_prints_the_language_and_each_template_and_lists_them_only_where_the_config_sets_them(self) -> None: folder = self.write("acme", CLIENT) defaults = {"language": "English", "commit-title": "{summary}", "pull-request-title": "{summary}", "branch": "item-{id}"} diff --git a/bin/review_replay/test_review_replay.py b/bin/review_replay/test_review_replay.py index 93d4daae..32ed8595 100644 --- a/bin/review_replay/test_review_replay.py +++ b/bin/review_replay/test_review_replay.py @@ -267,7 +267,7 @@ class Main(unittest.TestCase): result = result if "api_error_status" in result else {"type": "result", "structured_output": {"found": [n + 1 for n in result["found"]]}} return subprocess.CompletedProcess(args, 0, json.dumps(result), "") - effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=system_effects().git, input=unused, local_time=unused, memory=unused, now=unused, on_terminal=unused, python=unused, sleep=unused, stream_claude=unused) + effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=system_effects().git, input=unused, local_time=unused, memory=unused, now=unused, on_terminal=unused, python=unused, sleep=unused, stream_claude=unused, tmux=unused, tmux_on_terminal=unused) out, err = io.StringIO(), io.StringIO() with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): code = main.main(argv, {}, self.home, effects) diff --git a/docs/comprehension-history.md b/docs/comprehension-history.md index ce39f55e..c43bae3a 100644 --- a/docs/comprehension-history.md +++ b/docs/comprehension-history.md @@ -3873,3 +3873,109 @@ Inherited architect, would restructure first: - Shape: `bin/deploy_release/forge.py:1` module: the forge lives in `deploy_release` and in `chunk_preflight.choices.forge_of`, neither a home its name gives - Shape: `bin/rule_hook/rules.py:26` `RULES`: the file is the registry and also four rules' checkers, so where a new rule goes is undecided - Self-sufficiency: `bin/plan_store/stores.py:5` module docstring: cites plan items 365, 366 and 368 as part of the contract, as routing and ladder code cite decisions + +## 2026-11-16T12:09:27Z, PR #560 at 55235a0 + +| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | +|---|---|---|---|---|---| +| Junior A | 7 | 5.5 | 6.5 | 6 | 6 | +| Mid A | 7.5 | 6.5 | 7 | 6.5 | 7 | +| Maintainability senior | 7 | 6 | 6.5 | 6 | 6.5 | +| Inherited architect | 7 | 6.5 | 6 | 6 | 6.5 | +| Mean | 7.13 | 6.13 | 6.50 | 6.13 | 6.50 | + +Junior A, hardest first: + +- Locality: `skills/larv-review/SKILL.md:90` "Re-entry": six rules held at once with the round cap, the release-review caps and the mechanical-answers exception, as dense prose rather than a decision table +- Locality, Self-sufficiency: `bin/release_plan/release_standing.py:176` `next_release`: three sources of the cut version and release names replayed from todo.md's history; `is_earliest` carries the meaning of "the plan is current" without naming it +- Locality: `bin/loop_claude/routing.py:116` `unfinished_route`: five rungs, a counter and an escalation mark from another module; the docstring table makes it tractable +- Shape: `bin/release_plan/ranking.py:239` `score`: the OLD_SHAPE and SHAPE split runs through every layer, and `weight_of` packs a three-way rule into one line +- Self-sufficiency: `bin/loop_claude/cycle/judge.py:197` `quotes_checked`: six quote kinds each with its own fallback; `INTERRUPT_RULES` is behaviour the code cannot verify +- Locality: `bin/loop_claude/cycle/run.py:154` `rule`: the order of stamping, archiving and stepping matters across three functions +- Shape: `bin/loop_claude/cycle/prompts.py:66` `review_launches`: a prompt assembled from five helpers that may each return nothing + +Mid A, hardest first: + +- Locality: `bin/loop_claude/routing.py:116` `unfinished_route`: five interacting rungs, a counter and an escalation mark; the module table resolves it, the escalation-mark rule must stay open beside it +- Locality: `bin/release_plan/release_standing.py:176` `next_release`: four cut cases plus renamed releases; the order of the checks carries meaning the docstrings don't explain +- Locality: `bin/chunk_pr/main.py:298` `merged`: a loop around a flock with approval and comment checks and two exception exits; `done` assigned only under branches +- Self-sufficiency: `bin/loop_claude/cycle/judge.py:197` `quotes_checked`: the prompt text at `judge.py:22` is behaviour written as English +- Shape: `bin/release_plan/ranking.py:239` `score`: two table shapes double the branches, and `weight_of`'s defect rule needs a second read +- Self-sufficiency: `bin/rule_hook/repeat_read.py:114` `shown_lines`: ISO-string time comparisons and harness behaviour nothing else lets a reader check +- Locality: `skills/larv-review/SKILL.md:82` "Round cap": Done, Cap and Re-entry cross-reference each other and point out to three other files +- Shape: `bin/chunk_pr/main.py:407` `finished`: five subprocess results destructured by position + +Maintainability senior, hardest first: + +- Locality: `bin/release_plan/release_standing.py:176` `next_release`: four regimes at once; `carried` reapplied in three places, one relying on `zip` keeping order +- Locality: `bin/release_plan/ranking.py:239` `score`: one Score depends on five inputs held in other files, and the two shapes fork every function +- Locality: `bin/loop_claude/routing.py:116` `unfinished_route`: rungs, a counter and a mark interact; the table resolves most of it +- Locality: `bin/loop_claude/loop/chunk_windows.py:107` `start_chunks`: about 40 lines under one flock that check the owner, split running from stopped, compute room, read the plan and filter, with local state tracked by hand +- Self-sufficiency: `bin/loop_claude/cycle/judge.py:197` `quotes_checked`: behaviour split between English rules for a model and verbatim checks +- Shape: `bin/plan_store/jira_epic.py:356` `revision_of`: indexes an untyped tuple from `scores` at `jira_epic.py:530` +- Locality: `skills/larv-review/SKILL.md:82` "Round cap": dense rules with exceptions pointing to other sections and files +- Self-sufficiency: `bin/larv_review/main.py:1` the module docstring: the whole contract of ten subcommands in one block + +Inherited architect, would restructure first: + +- Navigation, Shape: `bin/release_plan/todo_table.py:1` the plan model: lives under `release_plan/`, while the plan is also in `plan_store/`, `todo_item/`, `rank_todo.py` and `larv_review/plan.py`, under three names +- Locality: `bin/review_rules/round_titles.py:63` `past_cap`: the round cap is split across this, `rule_hook/round_cap.py:10`, `larv-review round-count` and SKILL.md's "Round cap" +- Shape: `bin/release_plan/named_code.py:5` `named_paths`: nothing in the module is code +- Shape: `bin/loop_claude/verdicts.py:11` `TakenIn.highest_likelihood`: L is a likelihood in the field and a level in the prose +- Shape: `bin/loop_claude/loop/routing_check.py:20` the routing check: "routing" names four concepts across `model_routing/`, `routing_figures/` and this module +- Navigation: `README.md:197` the `bin/` package list: the 72-entry `bin/` level has only the README's two lists as its index +- Self-sufficiency: `bin/review_rules/round_titles.py:7` `CAP`: the why is deferred to a decision-log title in a 200 KB file + +## 2026-11-16T12:13:26Z, PR #560 rerun at 55235a0 + +| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall | +|---|---|---|---|---|---| +| Junior A | 7.5 | 6 | 7 | 6.5 | 6.5 | +| Mid A | 7 | 5.5 | 6 | 5.5 | 6 | +| Maintainability senior | 8 | 6.5 | 7 | 7 | 7 | +| Inherited architect | 7 | 6.5 | 6 | 7 | 6.5 | +| Mean | 7.38 | 6.13 | 6.50 | 6.50 | 6.50 | + +Junior A, hardest first: + +- Locality, Self-sufficiency: `bin/release_plan/release_standing.py:176` `next_release`: five `LastCut` fields, a sentinel, renames and carried items at once; the `is_earliest` expression stayed opaque, and the full case table lives only in chunk-preflight's help +- Locality: `bin/loop_claude/routing.py:116` `unfinished_route`: the escalation mark spans `escalation_mark.py`, `judge.py`, `verdicts.py`, `route.py` and `reload.py` +- Self-sufficiency: `bin/rule_hook/folder_search.py:89` `skips`: mirrors rg's and ag's flag handling, which nothing in the repo lets a reader check +- Shape: `bin/release_plan/ranking.py:239` `score`: the old-shape branches run through every function of ranking +- Locality: `bin/loop_claude/cycle/judge.py:197` `quotes_checked`: the verdict changes shape in three places after the checks +- Navigation: `bin/larv_review/main.py:1` the help docstring: 155 lines of rules, with no pointer to the module that enforces each +- Locality: `bin/rule_hook/reviewer_rung.py:19` `SEAT`: a launch counts as a review by matching prompt sentences kept in Markdown +- Locality: `bin/loop_claude/loop/reload.py:97` `exec_if_edited`: a running loop re-executes itself on any edit and hands its state over as JSON + +Mid A, hardest first: + +- Locality: `bin/chunk_pr/main.py:298` `merged`: `done` bound only under some branches, and the approval wait is a fall-through rather than a named branch; `levelled_and_finished` returns a finish failure only after the push +- Locality: `bin/loop_claude/routing.py:116` `unfinished_route`: five rungs, a counter and a mark; only the module table makes the step line readable +- Self-sufficiency: `bin/release_plan/release_standing.py:176` `next_release`: three kinds of "none", and the reason `carried` rewrites releases is a dated call with no rule in the code +- Shape: `bin/release_plan/ranking.py:239` `score`: two table shapes through every function, and `MAINTAIN_WEIGHT`'s reason in the decision log +- Self-sufficiency: `bin/loop_claude/cycle/judge.py:197` `quotes_checked`: the judge's prompt is spread over field metadata, rule constants and a sentence from `escalation_mark.py` +- Locality: `skills/larv-review/SKILL.md:35` "§3. Passes": control flow as prose whose sub-sections point at each other, enforced only partly by code +- Locality: `bin/rule_hook/rules.py:26` `RULES`: each rule's spec is a paragraph apart from the module that implements it +- Self-sufficiency: `bin/larv_review/main.py:1` the module docstring: the only full spec of a dozen subcommands + +Maintainability senior, hardest first: + +- Locality: `bin/release_plan/release_standing.py:176` `next_release`: state from git history, a sentinel, and `carried` run twice, whose agreement must be checked by hand +- Locality: `bin/loop_claude/routing.py:116` `unfinished_route`: five rungs, a count and a mark; table cells need the `steps` docstring to parse +- Shape: `bin/release_plan/ranking.py:144` `weigh`: two table shapes through every step, and `Scoring.weights` means different things per shape +- Shape: `bin/plan_store/jira_epic.py:356` `revision_of`: indexes the untyped tuple `scores` returns by position +- Locality: `bin/loop_claude/loop/chunk_windows.py:107` `start_chunks`: a flock, running and dead windows, `leaving`, `room`, the restart of stopped windows, held items and a NoItem fallback in one function, with the limit applied again in `may_start` +- Locality: `bin/chunk_pr/main.py:298` `merged`: whether `done` is bound when read depends on three `if ready` branches lining up +- Self-sufficiency: `bin/loop_claude/cycle/judge.py:22` `INTERRUPT_RULES`: what the model is asked and what code checks need both files open +- Self-sufficiency: `bin/rule_hook/rules.py:44` `RULES`: the env-file-read spec is a 120-word string kept in step with code elsewhere + +Inherited architect, would restructure first: + +- Locality, Shape: `bin/plan_store/stores.py:1` the plan store: `release_plan` and `plan_store` import each other, and the goals file sits under `release_plan` +- Shape: `bin/larv_review/plan.py:22` `scoring_run_due`: the scoring-run trigger is spread over `plan.py`, `owed.py`, `unmeasured.py` and `scoring.py`, and `plan.py` isn't about the plan +- Locality: `bin/review_rules/round_titles.py:45` the round cap: constants, commit enforcement in `rule_hook/round_cap.py:10`, the launch refusal in `larv_review/reviewer_header.py:73`, the printout and SKILL.md's prose +- Shape: `bin/loop_claude/cycle/run.py:55` `Cycle.todo`: says todo.md where the plan may be a Jira epic, as `fetched_todo` and `INTERRUPT_RULES` do +- Shape: `bin/loop_claude/hand_over.py:23` `hand_over`: shares its name with the reload's handover of state in `loop/reload.py:36` `decode_handed` +- Locality: `bin/loop_claude/loop/close.py:18` `closed_cycle`: a second "close" beside `cycle/run.py:234` `close` +- Navigation, Shape: `README.md:197` the `bin/` package list: 74 entries at one level with the README's list as the only index +- Self-sufficiency: `bin/loop_claude/loop/routing_check.py:1` the module docstring: names `state.CYCLES_PER_CHECK` without saying which module `state` is diff --git a/docs/decisions.md b/docs/decisions.md index 2d30f57f..7d384661 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -18,7 +18,7 @@ Each project principle states one rule that the decisions below apply; their ent - chunk-pr: the chunks of a project merge one at a time on a machine, each finishing its plan items in its turn, and a pull request that waits for an approval leaves the queue while it waits - deploy: this repo keeps no `AGENTS.md`, since `~/.claude` is its checkout; its principles and decision titles head `docs/decisions.md`, its working rules follow them there, and its `.claude/settings.json` excludes every `.claude/CLAUDE.md`, so a session in the clone loads only the clone's `CLAUDE.md` - loop-claude: each verdict records the claude-config release and commit that ran it, the raw tokens each thread spent by model and kind since the previous outcome that recorded a spend, and the human's typed answer the judge quotes as reversing a choice the worker took on its own recommendation -- loop-claude: one session per running loop of a project, named after the slot the loop holds, its id derived from that name; a chunk's commit in the config's folder takes only the file it changed +- loop-claude: each chunk runs in its own tmux window, of one tmux session per project, up to `parallel-chunks` at once, and needs tmux 3.2; its claude session is named after its window, its id derived from that name; a chunk's commit in the config's folder takes only the file it changed - machine load: a heavy command waits for one of the machine's heavy slots, one for every 4 CPUs, which every session shares, and loop-claude starts a worker only while at least a quarter of the machine's memory is available - principles: about seven things per level, four or five where they act on each other, more where a reader finds them by name; crossing a limit asks for a grouping or a recorded reason - project config: a record may name a Confluence Cloud page as `confluence:` and its address, the loop reads and writes it as markdown, and a write never overwrites a save made since the loop's read @@ -157,17 +157,25 @@ Valid while `~/.claude` is a checkout of this repo. 2026-09-30, lilleman. Applies "Always grant explicit access to data": the sandbox `loop-claude` runs in is where access is granted, so the worker and the claude an interrupt opens run with `bypassPermissions`. The worker may read `~/.claude/.credentials.json`: it holds Claude Code's own credentials, which claude reads to run anyway. Its one seam to production is the forge: what it pushes or merges can publish through CI. Valid while `loop-claude` runs only in the sandbox README → Install names. -## loop-claude: one session per running loop of a project, named after the slot the loop holds, its id derived from that name; a chunk's commit in the config's folder takes only the file it changed +## loop-claude: each chunk runs in its own tmux window, of one tmux session per project, up to `parallel-chunks` at once, and needs tmux 3.2; its claude session is named after its window, its id derived from that name; a chunk's commit in the config's folder takes only the file it changed 2026-09-21, lilleman; revised 2026-09-30 by lilleman. Serves goal 1. Ids derived from names keep the session picker at one entry per running loop and make any transcript under those ids loop-claude's own to delete. A checkout outside `~/code` mirroring a path inside it shares that id, and a second loop on it is not refused. Once `todo.md` item 5 lands, the id comes from the full path, so no two checkouts share one, and `loop-claude` refuses to start while another session on that id is active. Moving the checkout orphans its open sessions, archive folder and models entry: sessions stay in the picker until the human deletes them, migration code not being worth a one-human audience. 2026-11-15, lilleman's call of 2026-11-13 in item 372, which split item 361: loops of one project, each on its own chunk, run side by side without touching each other's state. The worker's choices, for what the call left open: -- **A slot, not the chunk, names the session.** A loop names its session before it knows its chunk, since a rerun resumes the open session in place of running `chunk-preflight`. So each loop holds a slot, a file it keeps locked with flock under `~/.local/state/loop-claude/<key>/` (`bin/loop_claude/loop/chunk_slot.py`): the lowest free slot whose session is open, so a rerun resumes a chunk left open, or else the lowest free slot. A session lives one chunk, so one slot holds one chunk at a time. Slot 0 keeps the name every loop had before slots, so a chunk left open before slots still resumes. A reload's exec keeps the lock and hands on the slot. - **The archive and the route needed no change.** The archive keys each chunk by its own transcript's first line, and a rerun's route comes from the latest verdict on that transcript, so both follow the session. The routing figures are measured across all projects, under their own lock. - **git's own lock serializes the config folder's commits, in place of the file lock the item asked for.** `git commit` holds `.git/index.lock`, so of two commits at once the second fails, and the worker tries again. A wrapper lock would only make it wait, and a command spelled `flock … git commit` would pass rule-hook unchecked, since it reads only a command that starts with `git`. What two chunks could still break is one committing the file the other staged, so each commits only its own file, as `git commit -m <title> -- <file>` (`CLAUDE.md` → "Git, commits, and PRs"). -Valid while claude takes `--session-id` and resumes by it, and a loop names its session before it knows its chunk. +2026-11-16, lilleman's second call of 2026-11-13 splitting item 361: lilleman knows tmux and prefers it. Each chunk runs in its own window of one tmux session per project, `loop-<key>`, up to the number the config's `parallel-chunks` sets, 1 by default. tmux runs even at 1, so every project runs the same workflow (goal 1.3). A window waiting for the human ends its name in ` ?` and rings the bell, and the DM names `<session>:<window>`. The worker's choices, for what the call left open (`bin/loop_claude/tmux.py`, `bin/loop_claude/loop/chunk_windows.py`): + +- **The window, in place of item 372's lock-file slot, names the claude session**, as `A <key> loop-claude item-372`. The loop-claude the human runs starts the windows and attaches; each window runs `loop-claude --chunk <window>`, so a loop knows its chunk before it names its claude session, and a rerun still resumes the chunk left open. +- **A finishing chunk starts the next; only the human restarts a stopped one.** As its chunk finishes, a window's loop starts each chunk that may start then and exits, so its window closes. `remain-on-exit failed` keeps the window of a loop that stopped any other way, Ctrl-C included, with its last lines, and its chunk keeps its items. A finishing chunk never restarts it, since the human may have stopped it on purpose; the next `loop-claude <task>` starts it again before any other chunk. One flock under `~/.local/state/loop-claude/<key>/` serializes the starts, so two chunks finishing together don't start one chunk twice. +- **The other chunks' items are told, not locked.** A fresh worker is told the items each other chunk holds, running or stopped: its window's item and those on its latest `Chunk items:` line. A chunk starts beside the others only where its item's `Touches:` prediction overlaps none of those items' predictions, since the plan's next item, which chunk-preflight's pick checks against, may already have merged. +- **The tmux session names its checkout.** Two checkouts whose keys differ only in a character other than a letter, a digit, `_` or `-` map to one session name, so the session's `@loop-root` option names the checkout that made it, and another checkout's loop-claude refuses it. +- **tmux 3.2 is the oldest that has every command and option used.** tmux's `CHANGES` dates the newest two to "CHANGES FROM 3.1c TO 3.2": `new-session -e`, and the `remain-on-exit` value that keeps only a failed program's pane. +- **A loop started before this release isn't moved into a window.** One that reloads into this release starts its project's windows afresh, and leaves its chunk's open claude session, named for its slot, in the session picker. Carrying a slot's session into a window is not worth the code for a one-human audience, so the changelog asks the human to finish or stop every running loop before taking the release. + +Valid while claude takes `--session-id` and resumes by it, and the human runs loop-claude where tmux is installed. ## loop-claude: the archive keeps the raw transcript, and a failed archive stops the loop @@ -294,7 +302,7 @@ Valid while the later fixes per finished chunk, which `score-outcomes` and the r ## project config: a record may name a Confluence Cloud page as `confluence:` and its address, the loop reads and writes it as markdown, and a write never overwrites a save made since the loop's read -2026-11-15, larv-review, on lilleman's calls of 2026-11-12. Serves goal 1.6: a client's team keeps its records in its own wiki. Valid while one record fits one page, and while one session per project opens a record's copy, which todo.md's item 375 ends. +2026-11-15, larv-review, on lilleman's calls of 2026-11-12. Serves goal 1.6: a client's team keeps its records in its own wiki. Valid while one record fits one page, and while each worktree opens its own copy of a record, as loop-claude's side-by-side chunks need. - **A bench of readers picked the spelling.** Three fresh readers, seated as the person who writes a client's `config.toml`, each picked `decisions = "confluence:https://<site>/wiki/spaces/<space>/pages/<page ID>"`, pasted from the browser with or without the title at its end, over the bare address, a page ID beside a `confluence-site` key, and a table. Each named the match with `jira:`. Any of the five records may name a page; the goals file, the plan, the working rules and the principles may not. - **`decisions` and `comprehension-history` may name a page only where `plan` names a Jira epic.** rule-hook ranks a plan kept as a file before tool calls, inside its 30-second limit, and ranking reads those two records. A Confluence read and two adf-codec container runs would hold up every guarded tool call, and fail it while Confluence is down. `load` refuses goals in Jira beside a plan in a file for the same reason. @@ -703,12 +711,12 @@ Valid while the worker writes the tests that larv-review's reviewers hold to the - **The spelling.** A `Touches:` line, then one bullet per file: the path in backticks, then its parts, each a name in backticks or `lines <first>-<last>`, with `(small)` after a part the item changes only a little; a path or a name that holds a backtick takes a longer run of backticks with a space inside each end, as a Markdown code span does. A path alone is the whole file. Three readers seated as the README's audience picked the bulleted list over one line and over a table, since backticks hold a heading's commas, colons and parentheses. Two of the three put `(small)` after the part and outside the backticks, and two left line ranges unquoted. Each picked a different config key: `touch-predictions` was one reader's first choice and the second choice of the other two. - **A part's boundaries.** Python's `ast` module names the innermost function or class, a top-level assignment by the name it sets, and `(module)` for the rest. A Markdown file's parts are its headings outside fenced code. Any other file, and Python that doesn't parse, falls back to line ranges. A table row is no part of its own: the item named one, but the tables chunks collide on are the plan's and the changelog's, which don't count. - **What doesn't count.** The plan and every record the config names, the changelog among them, since every chunk writes them: `chunk-pr merge --finish` writes the plan and the changelog at merge time, and the other records are appended to. `touch-predictions record` leaves them out of what a chunk touched, and the rule keeps them out of predictions. -- **The pick.** `chunk-preflight` takes the items of the next release that don't wait, in the plan's order, each where it overlaps neither the next item nor an item taken before it. The call asked for the set that keeps the most chunks running. The plan's order comes first all the same: the largest set could start a low-ranked item in place of one ranked above it, and with more ready items than the parallel limit, either set fills the limit. `todo.md` item 375 takes as many of the picked items as its limit allows. +- **The pick.** `chunk-preflight` takes the items of the next release that don't wait, in the plan's order, each where it overlaps neither the next item nor an item taken before it. The call asked for the set that keeps the most chunks running. The plan's order comes first all the same: the largest set could start a low-ranked item in place of one ranked above it, and with more ready items than the parallel limit, either set fills the limit. loop-claude starts as many of the picked items as `parallel-chunks` allows. - **An overlap that is taken.** Two items may share a part where either marks it `(small)`. The call takes an overlap only for a small change to the same part, and a few lines re-apply onto whatever the other chunk did to it. - **Which items share a chunk** stays the worker's choice, by the review passes they fire, so the pick names items, and an item the next chunk takes starts no chunk of its own. - **The record is optional.** No path setting has a default, and a mandatory one would stop every project until its owner edits the config (goal 1.6). A project that leaves `touch-predictions` out is still picked for, and keeps no record. The worker runs `touch-predictions record` before it removes the chunk's items (`skills/larv-review/SKILL.md` §5), since the prediction leaves the plan with the item, and a plan in Jira keeps no history to read it back from. -Valid while `todo.md` item 375 runs chunks by this pick. Once the record holds enough chunks to show how often the predictions miss, an item, not filed yet, revisits the pick's order and the `(small)` rule. +Valid while loop-claude starts a project's chunks side by side by this pick (`bin/loop_claude/loop/chunk_windows.py` `may_start`), which also checks each new chunk against the running chunks' items. Once the record holds enough chunks to show how often the predictions miss, an item, not filed yet, revisits the pick's order and the `(small)` rule. ## todo.md: one table of items, scored by the formula it states, held to its bar and ranked by `bin/rank_todo.py`, which ranks each blocker above the items it blocks; each achieve goal weighs by its position, code health first and four times as heavy while the comprehension panel's overall is under the project's readability floor or unmeasured, then the goals file's goals, and a maintain goal weighs 0.5 per point of the confidence an item's test, gate leg or probe adds to it; the table's order is the plan's only order diff --git a/todo.md b/todo.md index 77618f58..f40f17a8 100644 --- a/todo.md +++ b/todo.md @@ -12,7 +12,6 @@ | ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score | |---|---|---|---|---|---|---|---|---|---|---|---| -| 375 | 0.11.0 | | decision | **Run up to the project's configured number of chunks at once, each in its own window of one tmux session named for the project, mark a window whose chunk waits for the human, and refuse to start without tmux at the needed version.** | 4 | 2 | 3 | 7 | 7 | 1.5 by 7 | 0.6 | | 391 | 0.12.0 | | | **Read and write a Jira epic issue's description and a Confluence record page through the lossless pair, so saving an unedited document leaves it unchanged.** | 3 | 2 | 3 | 6 | 5 | 1.6 by 6, 1.1 by 3 | 1.9 | | 319 | 0.12.0 | | | **Start a fresh session with a brief of the chunk when a rung step changes the worker's model, in place of resuming the old session.** | 3 | 2 | 3 | 6 | 4 | 1.1 by 2, 1.4 by 2 | 1.2 | | 309 | 0.12.0 | | | **Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.** | 2 | 2 | 2 | 4 | 2 | 1.4 by 3 | 1.2 | @@ -44,7 +43,7 @@ | 273 | 0.14.0 | | decision | **Re-tune the E + I steps of `REVIEW_STEPS` for 6 to 8 once 10 finished chunks have scored an E + I of 6 or more, from how well E + I orders their review rounds and 🔴 findings.** | 1 | 2 | 2 | 3 | 2 | 1.1 by 2, 1.4 by 3 | 2.0 | | 268 | 0.14.0 | | | **End a run whose release is ready to cut with a notice on loop-claude's terminal, in place of an interrupt that waits for an answer.** | 2 | 2 | 3 | 5 | 3 | 1 by 5 | 1.9 | | 171 | 0.14.0 | | decision | **Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 3, 1.4 by 3 | 1.8 | -| 300 | 0.14.0 | | owner | **Reach the comprehension floor: restructure the places the comprehension seats found hardest to follow, so the project reads at 7.0 or above on every dimension, Locality and Self-sufficiency first; today the scoring run at bcb04e3 scored Navigation 6.75, Locality 5.63, Shape 5.88, Self-sufficiency 5.63 and Overall 6.00.** | 4 | 2 | 3 | 7 | 8 | code health by 8 | 1.7 | +| 300 | 0.14.0 | | owner | **Reach the comprehension floor: restructure the places the comprehension seats found hardest to follow, so the project reads at 7.0 or above on every dimension, Locality first; today the scoring run at 55235a0 scored Navigation 7.38, Locality 6.13, Shape 6.50, Self-sufficiency 6.50 and Overall 6.50.** | 4 | 2 | 3 | 7 | 8 | code health by 8 | 1.7 | | 264 | 0.14.0 | | | **Report the comprehension seats and the escalated subagents as subagent types of their own in `routing-figures.md`, with their dollars.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 5 | 1.7 | | 323 | 0.14.0 | | | **Count a release review as landed once its pull request has merged, whatever its commits are titled.** | 3 | 2 | 3 | 6 | 3 | 1.4 by 3, 1.5 by 2 | 1.7 | | 342 | 0.14.0 | | | **Research the tooling agents read, search and run commands with once more, and file an item for each tool that cuts tokens a replay shows keep quality.** | 1 | 2 | 1 | 2 | 3 | 1.4 by 3, 1.5 by 2 | 1.7 | @@ -76,14 +75,6 @@ ## Details -### 375. Run up to the project's configured number of chunks at once, each in its own window of one tmux session named for the project, mark a window whose chunk waits for the human, and refuse to start without tmux at the needed version. - -lilleman's call, 2026-11-13, splitting item 361. lilleman knows tmux and prefers it. loop-claude runs each chunk in its own window of one tmux session named for the project, such as session `loop-claude-config` with windows `item-372` and `item-373`. It runs up to the number of chunks the project's config allows at once, with 1 as the default, and starts the next chunk item 373's pass allows as each one merges. It uses tmux even at 1, so every project runs the same workflow (goal 1.3). Each window holds a loop-claude for its one chunk, so the interrupt and `/talk` work there as on any terminal. When a chunk waits for the human, loop-claude marks its window, renaming it to end in ` ?` and ringing the bell so tmux's status bar shows it, and clears the mark once it is answered. The DM each interrupt sends names the session and window to answer in, such as `loop-claude-config:item-372`, so the human goes straight to it. - -Before starting anything — a worktree, a session, a window — loop-claude checks that tmux is installed at the needed version or later. Where it is missing or older, it stops and names the version it found and the one it needs. The needed version is the oldest that has every tmux command and option loop-claude uses, read from tmux's `CHANGES` file and cited in the decision that records it. - -The config key for the parallel limit is a syntax users write, so a panel seated from the README's audience picks its spelling (`CLAUDE.md` → "Working loop" → "Bench a syntax"). run-heavy already sizes its slots from the machine, at one per 4 CPUs and with memory from `MemTotal`, so a larger VM runs more heavy commands at once without a change here. - ### 391. Read and write a Jira epic issue's description and a Confluence record page through the lossless pair, so saving an unedited document leaves it unchanged. `plan_store/jira_epic.py` reads an issue's description and writes it back on an edit, and `record_store/confluence_page.py` reads a page and saves it back; both use the portable pair, which drops each block's `localId` and the empty paragraphs Jira uses for spacing. `record_store/jira_description.py` only reads, to show a person: it stays on the portable pair. @@ -301,30 +292,27 @@ Releases are the human's, and `CLAUDE.md` → "Working loop" already ends the lo Raised by the goals audit of PR #296. `bin/routing_figures/calls.py` groups a chunk's review rounds by the rung its worker started on, and calls for a routing change when a lower rung took more of them. Since PR #296 the reviewers may run above the worker's rung, on the one their item's E + I picks, and no figure groups rounds by it, though the archived worker line names the E + I it follows from. A cheap worker on an item of high E + I gets stronger reviewers, who may raise more rounds, and a call could then blame the worker's rung for them: an interrupt of lilleman that settles nothing (goal 1.2). -### 300. Reach the comprehension floor: restructure the places the comprehension seats found hardest to follow, so the project reads at 7.0 or above on every dimension, Locality and Self-sufficiency first; today the scoring run at bcb04e3 scored Navigation 6.75, Locality 5.63, Shape 5.88, Self-sufficiency 5.63 and Overall 6.00. +### 300. Reach the comprehension floor: restructure the places the comprehension seats found hardest to follow, so the project reads at 7.0 or above on every dimension, Locality first; today the scoring run at 55235a0 scored Navigation 7.38, Locality 6.13, Shape 6.50, Self-sufficiency 6.50 and Overall 6.50. -Filed by the comprehension check of PR #473 at 13d1550 (`larv-review record-scoring`), which printed `Floor:` and `Next:`, and refreshed from the run of PR #554 at bcb04e3. The lowest means are Locality and Self-sufficiency, 5.63 each: the seats name state shared across modules, such as the route and the archive order, and reasons that sit in `docs/decisions.md`, `CLAUDE.md` or `skills/larv-review/SKILL.md` rather than beside the code. Refresh this item's places and aim from the newest run before the chunk starts; do not file a second floor item. +Filed by the comprehension check of PR #473 at 13d1550 (`larv-review record-scoring`), which printed `Floor:` and `Next:`, and refreshed from the two runs of PR #560 at 55235a0. The lowest mean is Locality, 6.13 in both runs: the seats name rules held partly in agent-facing prose and partly in code, and state that one unit can't be read without three others open. Self-sufficiency and Shape follow at 6.50. Refresh this item's places and aim from the newest run before the chunk starts; do not file a second floor item. Aim: one ambitious refactor of the areas below, sized so every mean reaches 7.0 on the chunk's own scoring run. Where a place is already easier to read when the chunk starts, drop it. -Places the seats named, hardest first (`larv-review places`): +Places the seats named, hardest first (`larv-review places`), Locality first: -- Locality, Self-sufficiency: `bin/loop_claude/routing.py:113` `unfinished_route`, with `:128` `escalated_route`: five rungs, the escalation mark from `escalation_mark.py`, the unfinished count from `loop/close.py` and two step-up counters are held at once, and why they combine sits in the decision log. Three seats named it; one would least want to change it. -- Locality, Shape: `bin/chunk_pr/main.py:298` `merged`, with `:384` `levelled_and_finished`, `:349` `unplanned`, `:407` `finished` and `:438` `remeasure`: a while loop around the queue turn binds `ready`, `asked` and `done` only on some paths and states no invariant; `levelled_and_finished` does five things under a name that gives two; `unplanned` and `finished` match the error text of `todo-item show` where `plan_store` reads the plan; `remeasure` keeps its control flow in chained boolean expressions. Three seats named it; two would least want to change it, and one expects the next defect there. -- Shape, Self-sufficiency: `bin/release_plan/release_standing.py:162` `next_release`, with `:207` `standing`: `str | None | Failure` fields in `LastCut`, the `BEFORE_FIRST_RELEASE` sentinel and `nothing_cut` meet in the `is_earliest` test at `:184`, and the comments explain each branch alone. -- Locality, Shape: `bin/loop_claude/cycle/run.py:151` `rule`, with the 17-field `Cycle` at `:31`: judges, stamps, annotates, steps the rung up and archives in an order that matters and that no single unit owns, and `Cycle` is passed around whole. -- Shape: `bin/release_plan/ranking.py:48` `rank`, `:136` `weigh` and `:231` `score`: every function forks on `SHAPE` versus `OLD_SHAPE`, `Shape` is built from positional tuples of callables, and the defect rule sits in `weight_of`, far from where scores are read. -- Shape: `bin/chunk_preflight/main.py:128` `preflight`: lines 142-143 pack a conditional re-`located` and an eight-argument `Gathered` into two lines. -- Locality, Self-sufficiency: `skills/larv-review/SKILL.md:146` "Comprehension check", with `:82` "Round cap", `bin/larv_review/main.py:1` and `bin/larv_review/plan.py:77` `plan_reasons`: one rule is spread over skill prose, a 155-line help, a hook and the code, so rebuilding it means three files open. -- Navigation, Shape: `bin/larv_review/main.py:222` `COMMANDS` and `bin/rule_hook/round_cap.py:10` `round_cap_breaches`: the package named for the review mostly holds the comprehension check, part of which sits in `review_rules/comprehension_history.py`, and the round cap is enforced in the hook, where a reader looks last. -- Self-sufficiency: `bin/loop_claude/cycle/judge.py:22` `INTERRUPT_RULES`, with `:197` `quotes_checked`: behaviour held partly in prompt text and schema descriptions and partly in Python checks, with nothing saying which the code enforces. -- Locality, Shape: `bin/deploy_release/forge.py:1` and `bin/chunk_preflight/choices.py` `forge_of`: forge identity and forge calls live in two tool packages, neither a home the name gives. -- Navigation, Shape: `README.md:193` `bin/`: about 70 entries at one level, the shared libraries told apart only by README prose. -- Shape: `bin/rule_hook/rules.py:26` `RULES`: the registry and four rules' checkers in one file, so where a new rule goes is undecided. -- Locality: `bin/loop_claude/loop/reload.py:95` `exec_if_edited`, with `:67` `decoded_route`: an edit under `bin/` makes a running loop re-exec, and renaming a `LoopRun` field stops it. -- Shape: `bin/rule_hook/env_file.py:34` `reaching`: `and` and `or` mixed without parentheses, and four path spellings in one lookaround regex. -- Shape: `bin/loop_claude/loop/chunk_start.py:24` `fetched_todo`: nearly the same plan read as `:75` `fetched_plan`, told apart only by a docstring. -- Self-sufficiency: every seat names the decision-log titles and plan items that code cites for a reason (`routing.py`, `ladder.py`, `run.py:31`, `plan_store/stores.py:5` and `:50`, `chunk_start.py:86`) as the largest cost; state each reason beside its constant and keep the citation as a footnote. +- Locality, Self-sufficiency: `bin/release_plan/release_standing.py:176` `next_release`, with `:162` `carried`: the cut from a tag, a dated heading or `BEFORE_FIRST_RELEASE`, release names replayed from the plan's history, and renames meet in the `is_earliest` test at `:198`; `carried` is reapplied in `chunk_item` and in `chunk_preflight/choices.py` `planned`, one relying on `zip` keeping order. All eight seats named it; four would least want to change it. +- Locality: `bin/loop_claude/routing.py:116` `unfinished_route`, with `:131` `escalated_route`: five rungs, the unfinished count and the escalation mark, which spans `escalation_mark.py`, `judge.py`, `verdicts.py`, `route.py` and `reload.py`. Seven seats named it; the module table is what makes it readable. +- Locality: `bin/chunk_pr/main.py:298` `merged`, with `:384` `levelled_and_finished` and `:407` `finished`: a loop around the queue turn binds `done` only under some `if ready` branches, the approval wait is a fall-through, and `finished` destructures five subprocess results by position. +- Locality: `skills/larv-review/SKILL.md:82` "Round cap" and `:90` "Re-entry", with `bin/review_rules/round_titles.py:45` and `:63` `past_cap`, `bin/rule_hook/round_cap.py:10` and `bin/larv_review/reviewer_header.py:73`: the round cap lives in five places, and SKILL.md's Done, Cap, Re-entry and Mechanical answers point at each other as prose. +- Locality: `bin/loop_claude/loop/chunk_windows.py:107` `start_chunks`: under one flock it checks the session's owner, splits running from stopped windows, computes the room, reads the plan with a fallback, gathers held items and filters again, with the limit applied a second time in `may_start`. +- Shape, Locality: `bin/release_plan/ranking.py:239` `score`, `:144` `weigh` and `:250` `weight_of`: every function forks on `SHAPE` versus `OLD_SHAPE`, and one Score depends on five inputs held in other files. +- Locality, Shape: `bin/plan_store/stores.py:1` and `bin/release_plan/todo_table.py:1`: `release_plan` and `plan_store` import each other, the plan has three names, and the goals file sits under `release_plan`. +- Shape: `bin/larv_review/plan.py:22` `scoring_run_due`, with `owed.py`, `unmeasured.py` and `scoring.py`: the scoring-run trigger is spread over four modules, three of whose names don't say it, in a flat 14-module package that mixes rounds, the comprehension check and its triggers. +- Self-sufficiency: `bin/loop_claude/cycle/judge.py:22` `INTERRUPT_RULES`, with `:197` `quotes_checked`: the judge's behaviour is held partly in prompt text and field metadata and partly in Python checks. +- Shape: `bin/plan_store/jira_epic.py:356` `revision_of`: indexes the untyped tuple `scores` at `:530` returns by position. +- Shape: `bin/loop_claude/cycle/run.py:55` `Cycle.todo`, `bin/loop_claude/hand_over.py:23` `hand_over` and `bin/release_plan/named_code.py:5`: names that say todo.md where the plan may be a Jira epic, a handover that is not the reload's, and a module with no code in it. +- Navigation, Shape: `README.md:197` the `bin/` package list: 74 entries at one level, with the README's list as the only index. +- Self-sufficiency: every seat names the decision-log titles that code cites for a reason (`routing.py:1`, `ranking.py:20`, `loop/routing_check.py:1`) as a cost; state each reason beside its constant and keep the citation as a footnote. Scored L 4 because the refactor touches several components that hold known defects, E 2 because review and a manual check see a regression, I 3 because every session runs these tools. The chunk runs a scoring run, since it is a floor item.