Release review of 0.12.0 #634

Merged
lilleman merged 14 commits from release-review-0.12.0 into main 2026-11-19 14:12:13 +01:00
194 changed files with 6603 additions and 4459 deletions
+4
View File
@@ -39,6 +39,10 @@
!bin/branch_chunks/*.py
!bin/chunk-pr
!bin/chunk-preflight
!bin/chunk_driver/
!bin/chunk_driver/*.py
!bin/chunk_driver/project/
!bin/chunk_driver/project/*.py
!bin/chunk_pr/
!bin/chunk_pr/*.py
!bin/chunk_preflight/
+8 -8
View File
@@ -219,21 +219,21 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
### Packages under `bin/`
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests. Beside the packages, `bin/`'s root holds only the launchers, the tools of one file with their tests, `test_env.py`, `failure.py` and `setup.cfg`. `failure.py`'s `Failure`, which modules across `bin/` return for a failed step, belongs to no single package. `agent_cycle/`, `atlassian/`, `branch_chunks/`, `claude_code/`, `command_lines/`, `forge/`, `git_cli/`, `loop_effects/`, `loop_state/`, `machine_load/`, `model_routing/`, `notifier/`, `plan/`, `plan_store/`, `record_store/`, `release_plan/`, `review_rules/`, `rung_replay/`, `text_files/` and `transcript_archive/` serve more than one tool. No package imports one that imports it back, which `repo_checks/test_package_cycles.py` checks.
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests. Beside the packages, `bin/`'s root holds only the launchers, the tools of one file with their tests, `test_env.py`, `failure.py` and `setup.cfg`. `failure.py`'s `Failure`, which modules across `bin/` return for a failed step, belongs to no single package. `agent_cycle/`, `atlassian/`, `branch_chunks/`, `chunk_driver/`, `claude_code/`, `command_lines/`, `forge/`, `git_cli/`, `loop_effects/`, `loop_state/`, `machine_load/`, `model_routing/`, `notifier/`, `plan/`, `plan_store/`, `record_store/`, `release_plan/`, `review_rules/`, `rung_replay/`, `text_files/` and `transcript_archive/` serve more than one tool. No package imports one that imports it back, and a package's `doubles.py` holds the test doubles that other packages' tests use, so no module imports a `test_*` module of another package; `repo_checks/test_package_cycles.py` checks both. No Python file of the repo, `workflow-bench/`'s included, uses `mock.patch`, which `repo_checks/test_no_patched_globals.py` checks.
- `adf_codec/` converts markdown to ADF, the rich text of Jira Cloud and Confluence, and back, in a throwaway container running a pinned `@larvit/adf-codec`. It builds the container's image inside a heavy slot.
- `agent_cycle/` runs one cycle (`run.py`): a headless claude worker, then the judge of its transcript. It also holds the interrupt, the tmux window and the routing of the worker's rung. The tool that drives the cycles supplies the prompts and file names of its own workflow (`worker_prompts.py`), and reads which items a worker took in. `loop_claude` drives it, and `do_epic` will; `agent_cycle` doesn't import either.
- `atlassian/` holds what the Jira and Confluence backends share (`client.py`): one Atlassian account's credentials, the request both backends send, and each service's error messages.
- `agent_cycle/` runs one cycle (`run.py`): a headless claude worker, then the judge of its transcript. It also holds the interrupt, the tmux window and the routing of the worker's rung. The entry point that runs the cycles supplies the prompts and file names of its own workflow (`worker_prompts.py`), and reads which items a worker took in. `loop_claude` and `do_epic` drive it; `agent_cycle` doesn't import either.
- `atlassian/` holds what the Jira and Confluence backends share (`client.py`): one Atlassian account's credentials, the request both backends send, and each service's error messages. `jira.py` holds what every reader of Jira shares: a search read page by page, the key, fields, links and property of each issue it finds, checked for their shape, and the transition that finishes an issue, for `plan_store` and `do_epic`.
- `branch_chunks/` cuts a default branch into chunks and finds, for each, the items it shipped and the later chunks that fixed its code, for `score_outcomes` and `routing_figures`. It imports `reading_cost`'s record path.
- `chunk_driver/` holds what the entry points share. An entry point is a tool that drives chunks through `agent_cycle`: `loop_claude` and `do_epic` are the two. They share how a cycle starts (`cycle_start.py`) and closes (`chunk_run.py`), the claim on a checkout before a chunk starts in it (`claims.py`), the prompts every worker gets (`prompts.py`), and a checkout's premises, records and saved models (`project/`). It imports neither entry point, and `repo_checks` fails any package that imports one of the two. `agent_cycle` is the layer below it, and `repo_checks` fails `agent_cycle` where it imports `chunk_driver`, tests included.
- `command_lines/` reads a shell command: its simple commands, and a docker call's subcommand and memory limits, for `command_patterns`, `reading_cost`, `routing_figures`, `rule_hook` and `run_heavy`.
- `deploy_release` imports `release_plan`'s `release_standing`.
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `planning.py` settles, before the chunks start, where each open issue is worked and the order of the issues do-epic filed, through `split.py`, `repo_pick.py` and `ranking.py`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. `gates.py` reads once whether a chunk's pull request waits on its checks, an approval or a new review, and `parked.py` keeps each chunk that does in a state file, so it takes no running slot while it waits and goes on waiting after a reboot. `chunk_prompt.py` words what a chunk's worker reads. Through `findings.py` that worker files a finding as an issue of the epic, with `--file`, and appends a decision to its description, with `--record-decision`. `finish.py` ends an epic that holds no open issue: its finishing chunk files a release review of each repo the epic changed, has the human confirm each goal, and with `--close-epic` posts the summary on the epic and closes it. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, posts comments, moves a finished chunk's issue to done, and closes the epic. `e2e/` holds `do-epic-e2e`.
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and reads its checks, once or until they settle (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), finds a pull request and reads its checks, once or until they settle (`pulls.py`), and reads a pull request's review state (`reviews.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
- `git_cli/` runs git for every tool (`git_repo.py`), and reads the git calls in a shell command (`git_command.py`).
- `judge_replay` imports `agent_cycle`'s judge and verdicts, and `loop_claude`'s premises, records and `ANSWER_DESTINATIONS`.
- `judge_replay` imports `agent_cycle`'s judge and verdicts, and `chunk_driver`'s premises, records and `ANSWER_DESTINATIONS`.
- `loop_claude` runs `chunk-preflight` as a process of its own.
- `loop_effects/` holds `Effects`: the processes, terminal, clock and git a tool reaches, injected so its tests can script them. loop-claude, both replay tools and `reviewer-merge-trials`' driver use it.
- `loop_state/` holds where loop-claude keeps its files (`paths.py`), which `rung_replay` reads too.
- `loop_effects/` holds `Effects`: the processes, terminal, clock and git a tool reaches, injected so its tests can script them. loop-claude, do-epic, both replay tools and `reviewer-merge-trials`' driver use it.- `loop_state/` holds where loop-claude and do-epic keep their files (`paths.py`), which `rung_replay` reads too.
- `machine_load/` takes the machine's heavy slots, its shared lock files, for `run_heavy` and `adf_codec`, and reads the memory share that loop-claude waits for. It also gives `mutate_changes` a slot's CPU count.
- `model_routing/` holds the shipped routing ladder (`routing-ladder.json`) and its steps (`ladder.py`), and reads a ladder from that file or a project's own (`ladder_file.py`), for loop-claude, `rule_hook`, `do_epic`, both replay tools and `subagent-rung`. It finds a project's config folder through `project_config`. It also holds the list prices, which `routing_figures` and `transcript_archive` read.
- `notifier/` sends the human a DM through the notifier a project's config names, for `notify` and for loop-claude through `loop_effects`. Its backends are `mattermost.py` and `slack.py`.
@@ -246,7 +246,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `rest_client/` sends a JSON request to a REST API and reads its credentials from an env file, for `notifier`, `atlassian` and `record_store`.
- `review_rules/` holds the larv-review rules that several tools apply: its passes, its review folder, its round commit titles and the comprehension history's format.
- `routing_figures/` measures loop-claude's archive, and each of its projects' default branches, for loop-claude's routing calls.
- `rung_replay/` holds what `judge_replay` and `review_replay` share: the parallel replay runner, the argument reader, the candidate rungs and the exits, and `judge_replay`'s rung search. It reads `agent_cycle`'s retry count and `loop_claude`'s models file, and finds that file through `loop_state`'s paths. Its `archive.py` keeps each replay's runs and its report in a `replays/` folder of loop-claude's archive.
- `rung_replay/` holds what `judge_replay` and `review_replay` share: the parallel replay runner, the argument reader, the candidate rungs and the exits, and `judge_replay`'s rung search. It reads `agent_cycle`'s retry count and `chunk_driver`'s models file, and finds that file through `loop_state`'s paths. Its `archive.py` keeps each replay's runs and its report in a `replays/` folder of loop-claude's archive.
- `test_env.py` serves the tests.
- `text_files/` reads a Markdown file's `##` sections, quotes a passage and joins names into a list, reads the text piped into a command, stamps times in UTC, replaces a file in one step, private where it is new, and gzips a file beside its plain name, by which it reads it afterwards.
- `transcript_archive/` stores loop-claude's transcripts. Its `project_key.py` names each project's archive folder, loop-claude session and models-file entry.
+22 -8
View File
@@ -8,16 +8,18 @@ adf-codec has two pairs of calls, and each function here names the one it runs:
"""
import contextlib
import dataclasses
import fcntl
import hashlib
import json
import os
import pathlib
import subprocess
import sys
import time
import typing
from failure import Failure
from machine_load.slots import HELD, slot_count, slot_folder, usable_cpus, waited_slot
from machine_load.slots import HELD, Lock, slot_count, slot_folder, usable_cpus, waited_slot
SHIM = pathlib.Path(__file__).resolve().parent / "node_shim"
SHIM_FILES = ("Dockerfile", "package.json", "package-lock.json", "shim.mjs")
@@ -62,7 +64,13 @@ def docker_args(image: str, mode: str) -> list[str]:
return ["docker", "run", "--rm", "-i", "--memory", MEMORY, "--memory-swap", MEMORY, "--network", "none", "--read-only", image, mode]
def docker(args: list[str], text: str, seconds: int, call: typing.Callable[..., typing.Any] = subprocess.run) -> Ran | Failure:
def docker(args: list[str], text: str, seconds: int) -> Ran | Failure:
"""docker_ran with subprocess.run: the one place that starts docker as a process."""
return docker_ran(args, text, seconds, subprocess.run)
def docker_ran(args: list[str], text: str, seconds: int, call: typing.Callable[..., typing.Any]) -> Ran | Failure:
"""Runs a docker command line through call, with text on its stdin: what it exited with and printed, or a Failure where docker is missing or takes more than seconds. call runs a command line as subprocess.run does."""
try:
done = call(args, input=text, capture_output=True, text=True, encoding="utf-8", timeout=seconds)
except FileNotFoundError:
@@ -73,16 +81,22 @@ def docker(args: list[str], text: str, seconds: int, call: typing.Callable[...,
@contextlib.contextmanager
def heavy_slot(env: typing.Mapping[str, str] = os.environ, slots: int | None = None) -> typing.Iterator[Failure | None]:
"""Holds one of the machine's heavy slots, which run-heavy shares, for the with block. Under run-heavy the process already holds one, so it doesn't wait."""
def machine_slot() -> typing.Iterator[Failure | None]:
"""heavy_slot with this process's environment, home folder, processor count, stderr, sleep and flock."""
with heavy_slot(os.environ, pathlib.Path.home(), slot_count(usable_cpus()), lambda text: print(text, file=sys.stderr, flush=True), time.sleep, fcntl.flock) as refused:
yield refused
@contextlib.contextmanager
def heavy_slot(env: typing.Mapping[str, str], home: pathlib.Path, slots: int, say: typing.Callable[[str], None], sleep: typing.Callable[[float], None], lock: Lock) -> typing.Iterator[Failure | None]:
"""Holds one of the machine's heavy slots, which run-heavy shares, for the with block, telling say where it waits for one. Under run-heavy the process already holds one, so it doesn't wait."""
if HELD in env:
yield None
return
folder = slot_folder(env, pathlib.Path.home())
count = slot_count(usable_cpus()) if slots is None else slots
folder = slot_folder(env, home)
try:
folder.mkdir(parents=True, exist_ok=True)
_, fd = waited_slot(folder, count, lambda: print(f"adf-codec: waiting for a heavy slot, of {count} on this machine, in {folder}, to build the shim's image; `fuser {folder}/slot-*` names the processes that hold them", file=sys.stderr, flush=True))
_, fd = waited_slot(folder, slots, lambda: say(f"adf-codec: waiting for a heavy slot, of {slots} on this machine, in {folder}, to build the shim's image; `fuser {folder}/slot-*` names the processes that hold them"), sleep, lock)
except OSError as failure:
yield Failure(f"cannot take a heavy slot in {folder}: {failure.strerror}")
return
@@ -92,7 +106,7 @@ def heavy_slot(env: typing.Mapping[str, str] = os.environ, slots: int | None = N
os.close(fd)
def ensure_image(shim: pathlib.Path, run_docker: Docker = docker, slot: Slot = heavy_slot) -> str | Failure:
def ensure_image(shim: pathlib.Path, run_docker: Docker = docker, slot: Slot = machine_slot) -> str | Failure:
"""The tag of the image for the shim folder, built from that folder first where this machine has no image under that tag.
The build waits for a heavy slot. Once this process holds the slot, it inspects again, as another process may have built the image meanwhile."""
+18 -9
View File
@@ -1,4 +1,5 @@
import contextlib
import fcntl
import json
import os
import subprocess
@@ -161,7 +162,7 @@ def test_docker_passes_the_text_on_stdin_and_returns_the_output():
seen.update(kwargs, args=args)
return subprocess.CompletedProcess(args, 3, "out", "err")
assert convert.docker(["docker", "x"], "text", 7, call) == convert.Ran(3, "out", "err")
assert convert.docker_ran(["docker", "x"], "text", 7, call) == convert.Ran(3, "out", "err")
assert seen == {"args": ["docker", "x"], "input": "text", "capture_output": True, "text": True, "encoding": "utf-8", "timeout": 7}
@@ -169,7 +170,7 @@ def test_docker_missing_becomes_a_failure():
def call(args, **kwargs):
raise FileNotFoundError
failure = convert.docker(["docker", "x"], "", 7, call)
failure = convert.docker_ran(["docker", "x"], "", 7, call)
assert failure == Failure("docker is not installed, and the adf-codec shim runs in a container")
@@ -177,7 +178,7 @@ def test_docker_that_takes_too_long_becomes_a_failure():
def call(args, **kwargs):
raise subprocess.TimeoutExpired(args, 7)
assert convert.docker(["docker", "build", "x"], "", 7, call) == Failure("`docker build` took more than 7 seconds")
assert convert.docker_ran(["docker", "build", "x"], "", 7, call) == Failure("`docker build` took more than 7 seconds")
def shim_dir(tmp_path):
@@ -260,19 +261,27 @@ def test_a_failure_to_start_docker_passes_through_inspect_and_build(tmp_path, an
assert convert.ensure_image(tmp_path, run_docker, Slot(run_docker)) == DOWN
def not_said(text):
raise AssertionError(f"said {text}")
def not_slept(seconds):
raise AssertionError(f"slept {seconds} seconds")
def test_a_heavy_slot_is_held_until_the_block_ends(tmp_path):
with convert.heavy_slot({"XDG_STATE_HOME": str(tmp_path)}, 1) as held:
with convert.heavy_slot({"XDG_STATE_HOME": str(tmp_path)}, tmp_path, 1, not_said, not_slept, fcntl.flock) as held:
assert held is None
assert slots.taken_slot(tmp_path / "run-heavy", 1) is None
slot, fd = slots.taken_slot(tmp_path / "run-heavy", 1)
assert slots.taken_slot(tmp_path / "run-heavy", 1, fcntl.flock) is None
slot, fd = slots.taken_slot(tmp_path / "run-heavy", 1, fcntl.flock)
os.close(fd)
def test_a_process_under_run_heavy_takes_no_second_slot(tmp_path):
(tmp_path / "run-heavy").mkdir()
slot, fd = slots.taken_slot(tmp_path / "run-heavy", 1)
slot, fd = slots.taken_slot(tmp_path / "run-heavy", 1, fcntl.flock)
try:
with convert.heavy_slot({"XDG_STATE_HOME": str(tmp_path), slots.HELD: "0"}, 1) as held:
with convert.heavy_slot({"XDG_STATE_HOME": str(tmp_path), slots.HELD: "0"}, tmp_path, 1, not_said, not_slept, fcntl.flock) as held:
assert held is None
finally:
os.close(fd)
@@ -280,7 +289,7 @@ def test_a_process_under_run_heavy_takes_no_second_slot(tmp_path):
def test_a_heavy_slot_folder_that_cannot_be_made_yields_a_failure(tmp_path):
(tmp_path / "run-heavy").write_text("a file where the folder goes")
with convert.heavy_slot({"XDG_STATE_HOME": str(tmp_path)}, 1) as held:
with convert.heavy_slot({"XDG_STATE_HOME": str(tmp_path)}, tmp_path, 1, not_said, not_slept, fcntl.flock) as held:
assert isinstance(held, Failure) and held.reason.startswith(f"cannot take a heavy slot in {tmp_path / 'run-heavy'}: ")
+2 -2
View File
@@ -1,4 +1,4 @@
"""One cycle of a headless claude worker and the judge of its transcript, for every entry point that drives such cycles: loop-claude today, do-epic next (docs/decisions.md → `do-epic: drives one Jira epic …`).
"""One cycle of a headless claude worker and the judge of its transcript, for every entry point that drives such cycles: loop-claude and do-epic (docs/decisions.md → `do-epic: drives one Jira epic …`).
run.py runs a cycle through the states in states.py: worker.py runs the worker, judge.py rules on its transcript, interrupt.py puts a question to the human in the tmux window tmux.py names, and routing.py picks the rung the worker runs on next. An entry point hands the cycle what is its own: its prompts and file names (worker_prompts.py) and how it reads the plan items a worker took in (run.py's Cycle.taken_in). No module here imports an entry point's package: loop_claude imports this one, so bin/repo_checks/test_package_cycles.py fails on such an import. A reader finds a module by its name, so the modules stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows that.
run.py runs a cycle through the states in states.py: worker.py runs the worker, judge.py rules on its transcript, interrupt.py puts a question to the human in the tmux window tmux.py names, and routing.py picks the rung the worker runs on next. An entry point hands the cycle what is its own: its prompts and file names (worker_prompts.py) and how it reads the plan items a worker took in (run.py's Cycle.taken_in). What two entry points share beyond the cycle lives in bin/chunk_driver/. No module here imports an entry point's package, and bin/repo_checks/test_package_cycles.py fails on such an import. A reader finds a module by its name, so the modules stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows that.
"""
+15
View File
@@ -0,0 +1,15 @@
"""Test doubles for the suites that run a cycle: the tmux window it runs in, and the prompts of an entry point that exists only in tests."""
from agent_cycle import tmux
from agent_cycle.worker_prompts import WorkerPrompts
WINDOW = tmux.Window("loop-proj", "item-7", "%4")
# The prompts of an entry point that exists only in tests, so a cycle's tests read no real entry point's wording.
PROMPTS = WorkerPrompts(
driver="a-driver",
fresh=lambda task, premises, route, project, facts: f"Fresh: {task}\n\n{premises}\n\n{facts}",
resumed=lambda evidence, route, project: f"Resumed: {evidence}",
rungs_held=lambda route, project: f" You hold {route.rung}.",
quote_sources="the project's records",
answer_destinations="the plan or a comment",
records_named="the project's decisions",
)
+2 -2
View File
@@ -34,7 +34,7 @@ class Cycle:
Each step reads the few fields it needs by name, so the fields stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows more than seven where a reader finds them by name.
"""
# The claude-config release and commit the cycle runs under, such as `v0.8.0 (<sha>)`, or why they are unknown: loop/main.py's setup_version.
# The claude-config release and commit the cycle runs under, such as `v0.8.0 (<sha>)`, or why they are unknown: chunk_driver/cycle_start.py's setup_version.
config: str
effects: Effects
# What chunk-preflight found, or why it could not run, for a fresh worker's prompt; empty when the session was open as the cycle began.
@@ -101,7 +101,7 @@ def archived_stop(project_archive: pathlib.Path, session: Session, task: str, ro
def archived(project_archive: pathlib.Path, transcript_path: str, task: str, outcome: str, effects: Effects, judge_run: JudgeRun | None = None) -> Failure | None:
"""transcript_archive.write.archive, stamped now and saying its notes through effects. A cycle and close.py, which has no Cycle, both archive through it."""
return archive(project_archive, transcript_path, task, outcome, effects.now(), lambda text: say(text, effects), judge_run)
return archive(project_archive, transcript_path, task, outcome, effects.now(), lambda text: say(text, effects), judge_run, replace=effects.replace_file)
def step(cycle: Cycle, state: Active) -> Event | Stop:
+229
View File
@@ -0,0 +1,229 @@
import contextlib
import dataclasses
import io
import json
import pathlib
import shutil
import sys
import tempfile
import unittest
import uuid
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from loop_effects.doubles import assistant, FakeEffects, human, NOW, STAMP, Streamed
from failure import Failure
from agent_cycle import interrupt
from machine_load import memory
from loop_state import paths
from model_routing.ladder import Rung
from agent_cycle.stop import Stop
from agent_cycle import tmux
from text_files.utc_stamp import utc
from agent_cycle.doubles import WINDOW
class Interrupts(unittest.TestCase):
def setUp(self):
self.home = pathlib.Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.home)
self.paths = paths.paths_from(self.home)
self.transcript = self.home / "t.jsonl"
self.transcript.write_text(json.dumps(human("a")) + "\n")
self.session = interrupt.WorkerSession(str(self.home), uuid.uuid7(), Rung("m", "high"), Rung("m", "high"), str(self.transcript), {"LOOP_CLAUDE_WORKER_RUNG": "m at high effort"})
def put(self, fake, answer_here=True, effects=None, question="Which one?"):
"""interrupt's answer and what it said, without the bell it rang, which self.bells counts."""
with contextlib.redirect_stderr(io.StringIO()) as said:
answer = interrupt.interrupt(question, answer_here, self.session, WINDOW, "loop-claude", effects or fake.effects())
self.bells = said.getvalue().count(tmux.BELL)
return answer, said.getvalue().replace(tmux.BELL, "")
def typed_there(self, args):
"""The claude opened on the terminal: the human types a line in the worker's session, and it ends."""
with open(self.transcript, "a") as handle:
handle.write(json.dumps(human("cut it")) + "\n")
return 0
def worked(self, minutes_ago, written_1h=5000, read=70000, written_5m=0):
"""Appends the worker's last message, sent minutes_ago before NOW, whose usage wrote written_1h tokens to the hour-long cache, written_5m to the five-minute one, and read read tokens from the cache."""
said = assistant({"type": "text", "text": "Which one?"})
said["timestamp"] = utc(NOW - minutes_ago * 60)
said["message"] |= {"id": "msg-1", "model": "m", "usage": {"input_tokens": 2, "cache_creation": {"ephemeral_5m_input_tokens": written_5m, "ephemeral_1h_input_tokens": written_1h}, "cache_read_input_tokens": read, "output_tokens": 100}}
with open(self.transcript, "a") as handle:
handle.write(json.dumps(said) + "\n")
def fired(self, fake, answer):
"""An input that first fires every timer started, then returns answer as the line the human typed."""
def typed(prompt):
for _, action in fake.timers:
action()
return answer
return typed
def test_compacts_the_workers_session_ten_minutes_before_its_hour_long_prompt_cache_expires_and_keeps_waiting(self):
self.worked(minutes_ago=10)
fake = FakeEffects(streams=[Streamed(0)])
fake.scripts["input"] = [self.fired(fake, "Use B.")]
answer, said = self.put(fake)
self.assertEqual(answer, "Use B.")
self.assertEqual([delay for delay, _ in fake.timers], [40 * 60])
self.assertEqual([what for seam, what in fake.calls if seam == "stream_claude"], [["--model", "m", "--effort", "high", "--permission-mode", "bypassPermissions", "-p", "/compact", "--resume", str(self.session.id), "--output-format", "stream-json", "--verbose"]])
self.assertEqual(fake.worker_environments, [self.session.env])
self.assertEqual(fake.worker_folders, [self.session.root])
self.assertIn(f"\n{STAMP} compacting the worker's claude session before its prompt cache expires; an answer typed meanwhile is read once that ends\n\n{STAMP} compacted the worker's claude session\n", said)
self.assertNotIn("still waiting", said)
def test_compacts_on_the_rung_the_worker_ran_on_where_a_ruling_stepped_it_up_and_talk_opens_claude_on_the_new_rung(self):
self.worked(minutes_ago=10)
self.session = dataclasses.replace(self.session, rung=Rung("n", "medium"))
fake = FakeEffects(streams=[Streamed(0)], on_terminal_claude=[self.typed_there])
fake.scripts["input"] = [self.fired(fake, "/talk")]
self.assertEqual(self.put(fake)[0], interrupt.Talked())
(compaction,), (opened,) = fake.made("stream_claude"), fake.made("claude_on_terminal")
self.assertEqual(compaction[:4], ["--model", "m", "--effort", "high"])
self.assertEqual(opened[:4], ["--model", "n", "--effort", "medium"])
def test_compacts_until_the_prompt_cache_expires_and_not_from_then_on(self):
for name, late, compactions in (("a second before", 50 * 60 - 1, 1), ("as it expires", 50 * 60, 0)):
with self.subTest(name):
self.transcript.write_text(json.dumps(human("a")) + "\n")
self.worked(minutes_ago=10)
fake = FakeEffects(streams=[Streamed(0)])
clock = [NOW]
def typed(prompt):
clock[0] = NOW + late
for _, action in fake.timers:
action()
return "Use B."
fake.scripts["input"] = [typed]
answer, said = self.put(fake, effects=dataclasses.replace(fake.effects(), now=lambda: clock[0]))
self.assertEqual(answer, "Use B.")
self.assertEqual(len(fake.made("stream_claude")), compactions)
self.assertEqual(said.count("compact"), 2 * compactions, said)
def test_a_compaction_doesnt_wait_for_memory(self):
self.worked(minutes_ago=10)
fake = FakeEffects(streams=[Streamed(0)], free_memory=[memory.Memory(available=3, total=16)])
fake.scripts["input"] = [self.fired(fake, "Use B.")]
said = self.put(fake)[1]
self.assertEqual([seam for seam, _ in fake.calls if seam in ("memory", "sleep", "stream_claude")], ["stream_claude"])
self.assertNotIn("waiting to start the worker", said)
def test_an_answer_before_the_timer_fires_stops_it_and_doesnt_compact_anything(self):
self.worked(minutes_ago=10)
fake = FakeEffects(inputs=["Use B."])
self.assertEqual(self.put(fake)[0], "Use B.")
self.assertEqual([call for call in fake.calls if call[0] in ("stop_timer", "stream_claude")], [("stop_timer", 40 * 60)])
def test_stops_the_timer_before_talk_opens_the_workers_session(self):
self.worked(minutes_ago=10)
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[self.typed_there])
self.assertEqual(self.put(fake)[0], interrupt.Talked())
self.assertEqual([call[0] for call in fake.calls if call[0] in ("stop_timer", "claude_on_terminal")], ["stop_timer", "claude_on_terminal"])
def test_starts_no_timer_under_60k_tokens_of_context_past_the_margin_on_a_five_minute_cache_or_where_claude_opens_at_once(self):
for name, minutes_ago, written_1h, written_5m, read, answer_here in (("small", 10, 5000, 0, 54997, True), ("margin begun", 50, 5000, 0, 70000, True), ("five minutes", 1, 0, 5000, 70000, True), ("nothing written", 1, 0, 0, 70000, True), ("opens at once", 10, 5000, 0, 70000, False)):
with self.subTest(name):
self.transcript.write_text(json.dumps(human("a")) + "\n")
self.worked(minutes_ago, written_1h, read, written_5m)
fake = FakeEffects(inputs=["Use B."], on_terminal_claude=[self.typed_there])
self.put(fake, answer_here=answer_here)
self.assertEqual(fake.timers, [])
def test_starts_the_timer_at_60k_tokens_of_context(self):
self.worked(minutes_ago=10, read=54998)
fake = FakeEffects(inputs=["Use B."])
self.put(fake)
self.assertEqual([delay for delay, _ in fake.timers], [40 * 60])
def test_starts_no_timer_where_the_transcript_cannot_be_read_and_still_asks(self):
self.transcript.unlink()
fake = FakeEffects(inputs=["Use B."])
self.assertEqual(self.put(fake)[0], "Use B.")
self.assertEqual(fake.timers, [])
def test_a_compaction_that_hits_a_usage_limit_says_the_limits_message_and_keeps_waiting(self):
self.worked(minutes_ago=10)
fake = FakeEffects(streams=[Streamed(0, {"type": "result", "api_error_status": 429, "result": "You've hit your session limit"})])
fake.scripts["input"] = [self.fired(fake, "Use B.")]
answer, said = self.put(fake)
self.assertEqual(answer, "Use B.")
self.assertIn(f"\n{STAMP} could not compact the worker's claude session: You've hit your session limit\n", said)
def test_a_compaction_that_fails_says_why_and_keeps_waiting(self):
self.worked(minutes_ago=10)
fake = FakeEffects(streams=[Streamed(1)])
fake.scripts["input"] = [self.fired(fake, "Use B.")]
answer, said = self.put(fake)
self.assertEqual(answer, "Use B.")
self.assertIn(f"\n{STAMP} could not compact the worker's claude session: claude exited 1\n", said)
def test_stops_off_a_terminal_before_asking(self):
fake = FakeEffects(on_terminal=False)
self.assertEqual(self.put(fake), (Stop(2, "the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"), f"\n{STAMP} Which one?\n"))
self.assertEqual(fake.calls, [])
def test_dms_the_question_naming_the_project_on_a_terminal_or_off_one(self):
for on_terminal in (True, False):
with self.subTest(on_terminal=on_terminal):
fake = FakeEffects(inputs=["Use B."], on_terminal=on_terminal)
self.put(fake)
self.assertEqual(fake.dms, [f"loop-claude in {self.home} asks, in the tmux window loop-proj:item-7:\n\nWhich one?"])
def test_marks_its_window_and_rings_the_bell_while_it_waits_and_clears_the_mark_once_answered(self):
fake = FakeEffects(inputs=["Use B."])
answer, said = self.put(fake)
self.assertEqual(answer, "Use B.")
self.assertEqual(self.bells, 1)
self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"], ["rename-window", "-t", "%4", "item-7"]])
def test_a_loop_that_stops_unanswered_leaves_its_window_marked(self):
fake = FakeEffects(on_terminal=False)
self.put(fake)
self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"]])
def test_drops_the_keys_typed_before_the_question_ahead_of_the_dm_so_an_answer_typed_while_it_sends_counts(self):
fake = FakeEffects(inputs=["Use B."])
seen = []
effects = dataclasses.replace(fake.effects(), dm=lambda text, directory: seen.append(list(fake.calls)))
self.assertEqual(self.put(fake, effects=effects)[0], "Use B.")
self.assertEqual(seen, [[("discard_typed", None)]])
def test_says_why_a_dm_failed_and_still_takes_the_answer(self):
fake = FakeEffects(inputs=["Use B."], dm_failures=[Failure("cannot reach https://chat.example: timed out")])
self.assertEqual(self.put(fake), ("Use B.", f"\n{STAMP} Which one?\n{STAMP} could not DM you: cannot reach https://chat.example: timed out\n"))
def test_returns_the_line_typed_after_the_question_stripped_dropping_what_was_typed_before(self):
fake = FakeEffects(inputs=[" Use B. "])
self.assertEqual(self.put(fake), ("Use B.", f"\n{STAMP} Which one?\n"))
self.assertEqual(fake.calls, [("discard_typed", None), ("input", "Answer (/talk opens claude, Enter stops the loop): ")])
def test_talks_on_talk_or_at_once_where_the_answer_is_not_asked_for_here(self):
for typed, answer_here in (([" /talk "], True), ([], False)):
with self.subTest(typed=typed):
fake = FakeEffects(inputs=typed, on_terminal_claude=[self.typed_there])
self.assertEqual(self.put(fake, answer_here)[0], interrupt.Talked())
self.assertEqual(len(fake.made("input")), len(typed))
self.assertEqual(fake.calls[0], ("discard_typed", None))
def test_stops_on_an_empty_answer_or_the_inputs_end(self):
for typed, why, said in ((" ", "nothing was typed", ""), (EOFError(), "the input ended", "\n")):
with self.subTest(why=why):
self.assertEqual(self.put(FakeEffects(inputs=[typed])), (Stop(2, f"the human is needed to answer the question above, but {why}"), f"\n{STAMP} Which one?\n{said}"))
def test_stops_when_nothing_was_typed_in_the_session(self):
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[0])
stop, _ = self.put(fake)
self.assertEqual(fake.made("claude_on_terminal")[0][:4], ["--model", "m", "--effort", "high"])
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended without anything typed"))
def test_hands_the_claude_that_takes_the_terminal_the_workers_environment(self):
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[0])
self.put(fake)
self.assertEqual(fake.terminal_environments, [{"LOOP_CLAUDE_WORKER_RUNG": "m at high effort", "CLAUDE_CODE_SANDBOXED": "1"}])
if __name__ == "__main__":
unittest.main()
+176
View File
@@ -0,0 +1,176 @@
import dataclasses
import json
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from loop_effects.doubles import assistant, FakeEffects, human, judged, kinds
from failure import Failure
from agent_cycle.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished
from agent_cycle import judge
from agent_cycle.doubles import PROMPTS
from model_routing.route import Route
from model_routing.ladder import Rung
from agent_cycle.judge import verdict_from
class Grounded(unittest.TestCase):
def test_every_verdict_must_quote_the_transcript(self):
self.assertTrue(judge.grounded("Needs you: x", "so.\n\nNeeds you: x\n"))
self.assertFalse(judge.grounded("invented", "the transcript"))
self.assertFalse(judge.grounded("", "the transcript"))
self.assertFalse(judge.grounded(" ", "the transcript"))
transcript = "done.\n\n**Your step:**\n```\n./run.sh\n```\nRun it."
self.assertTrue(judge.grounded("Your step:\n```\n./run.sh\n```\nRun it.", transcript))
self.assertTrue(judge.grounded("Your step: ``` ./run.sh ``` Run it.", transcript))
self.assertFalse(judge.grounded("**", transcript))
def test_an_elided_quote_holds_when_its_fragments_appear_in_order(self):
transcript = "**Decision for you:** security, or not.\n\nI recommend one more round of each.\n\nI recommend skipping it: it costs."
self.assertTrue(judge.grounded("**Decision for you:** security, … ... I recommend one more round of each ... I recommend skipping it: …", transcript))
self.assertFalse(judge.grounded("I recommend skipping it ... Decision for you", transcript))
self.assertFalse(judge.grounded("Decision for you ... invented", transcript))
self.assertFalse(judge.grounded("... …", transcript))
self.assertTrue(judge.grounded("a ... b ... c", "x a b a c"))
class Verdict(unittest.TestCase):
def test_needs_every_field_the_schema_requires(self):
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Finished("e"))
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": False, "question": "ignored", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Unfinished("e"))
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": True, "evidence": "e", "finished": True, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Interrupt("e", "Which one?"))
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": True, "evidence": "e", "finished": False, "question": "Cut 0.2.0?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": True}), Interrupt("e", "Cut 0.2.0?", release_ready=True))
self.assertEqual(verdict_from({"escalation": "", "answered_before": "", "left_open": "", "reversed": "", "answer": "", "interrupt": True, "evidence": "e", "finished": False, "question": "Cut 0.2.0?"}), Failure("the judge's reply has no release_ready of type boolean"))
self.assertEqual(verdict_from({"escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), QuotedAnswer("e", "Which one?", "A."))
self.assertEqual(verdict_from({"escalation": "", "answer": "A.", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Interrupt("e", "Which one?"))
self.assertEqual(verdict_from({"escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge quoted an answer without a question"))
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": "no", "evidence": "", "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge's reply has no interrupt of type boolean"))
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": False, "evidence": None, "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge's reply has no evidence of type string"))
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": True, "evidence": "x", "finished": False, "question": " ", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge interrupted without a question"))
self.assertEqual(verdict_from(None), Failure("the judge returned no reply"))
def test_carries_the_earlier_answer_an_interrupt_names(self):
reply = {"answer": "", "escalation": "", "interrupt": True, "evidence": "e", "finished": False, "release_ready": False, "answered_before": "Grant one more round.", "left_open": "one more", "question": "Which one?", "reversed": ""}
self.assertEqual(verdict_from(reply), Interrupt("e", "Which one?", answered_before="Grant one more round.", left_open="one more"))
self.assertEqual(verdict_from({**reply, "interrupt": False, "finished": True}), Finished("e"))
def test_takes_an_escalation_quote_on_an_unfinished_verdict(self):
reply = {"answer": "", "escalation": "round 2: 🔴", "interrupt": False, "evidence": "e", "finished": False, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}
self.assertEqual(verdict_from(reply), Unfinished("e", "round 2: 🔴"))
self.assertEqual(verdict_from({**reply, "finished": True}), Finished("e"))
self.assertEqual(verdict_from({**reply, "escalation": None}), Failure("the judge's reply has no escalation of type string"))
def test_takes_an_escalation_quote_on_an_interrupt(self):
reply = {"answer": "", "escalation": "round 2: 🔴", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}
self.assertEqual(verdict_from(reply), Interrupt("e", "Which one?", "round 2: 🔴"))
def test_drops_an_escalation_quote_the_transcript_does_not_hold_word_for_word_from_an_interrupt_or_an_unfinished_verdict(self):
shown = "The agent: round 2: 🔴. Which one?"
self.assertEqual(judge.quotes_checked(Interrupt("e", "q", "round 2: 🔴"), judge.Grounds(shown, shown, "")), Interrupt("e", "q", "round 2: 🔴"))
self.assertEqual(judge.quotes_checked(Interrupt("e", "q", "round 2: … 🔴"), judge.Grounds(shown, shown, "")), Interrupt("e", "q"))
self.assertEqual(judge.quotes_checked(Unfinished("e", "round 3: 🔴"), judge.Grounds(shown, shown, "")), Unfinished("e"))
def test_turns_an_interrupt_that_re_asks_a_human_answer_into_that_answer_quoted_back(self):
answer = "Grant one more round; ask again if a 🔴 survives it."
shown = f"[loop] Asked.\n\n[human] {answer}\n\n[step] Round 2: a 🔴 survived."
grounds = judge.Grounds(shown, shown, "")
asking = "Grant a third round? Your answer left open: a 🔴 survives it"
kept = Interrupt("[step] Round 2", asking, answered_before=answer, left_open="a 🔴 survives it")
self.assertEqual(judge.quotes_checked(kept, grounds), kept)
re_asked = Interrupt("[step] Round 2", "Grant a third round?", "Round 2: a 🔴", answered_before=answer, left_open="a 🔴 survives it")
self.assertEqual(judge.quotes_checked(re_asked, grounds), Unfinished(answer, "Round 2: a 🔴"))
self.assertEqual(judge.quotes_checked(dataclasses.replace(kept, left_open=""), grounds), Unfinished(answer))
self.assertEqual(judge.quotes_checked(dataclasses.replace(kept, left_open="a 🔴 survived"), grounds), Unfinished(answer))
invented = dataclasses.replace(re_asked, answered_before="Grant two more rounds.")
self.assertEqual(judge.quotes_checked(invented, grounds), invented)
unnamed = Interrupt("[step] Round 2", "Grant a third round?")
self.assertEqual(judge.quotes_checked(unnamed, grounds), unnamed)
def test_takes_an_escalation_quote_on_a_quoted_answer(self):
reply = {"answer": "A.", "escalation": "round 2: 🔴", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}
self.assertEqual(verdict_from(reply), QuotedAnswer("e", "Which one?", "A.", "round 2: 🔴"))
def test_drops_an_escalation_quote_the_transcript_does_not_hold_word_for_word_from_a_quoted_answer(self):
shown = "The agent: round 2: 🔴. Which one?"
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "A.", "round 2: 🔴"), judge.Grounds(shown, shown, "A.")), QuotedAnswer("e", "q", "A.", "round 2: 🔴"))
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "A.", "round 2: … 🔴"), judge.Grounds(shown, shown, "A.")), QuotedAnswer("e", "q", "A."))
def test_puts_a_quoted_answer_the_records_do_not_hold_to_the_human_with_the_escalation_it_carried(self):
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "not there", "round 2: 🔴"), judge.Grounds("The agent: round 2: 🔴.", "The agent: round 2: 🔴.", "A.")), Interrupt("e", "q", "round 2: 🔴"))
def test_puts_a_quoted_answer_the_records_do_not_hold_word_for_word_to_the_human(self):
self.assertEqual(judge.quotes_checked(QuotedAnswer("s", "q", "not there"), judge.Grounds("shown", "shown", "the records")), Interrupt("s", "q"))
self.assertEqual(judge.quotes_checked(QuotedAnswer("s", "q", "the records"), judge.Grounds("shown", "shown", "From the records: the records")), QuotedAnswer("s", "q", "the records"))
def test_drops_an_escalation_quote_found_only_in_the_transcript_entries_before_the_workers_last_step_up(self):
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), assistant({"type": "text", "text": "round 3: 🔴"}), assistant({"type": "text", "text": "working"})]
def judged_on(mark, escalation):
return judge.judge_once("/r", judge.Brief("Do it.", "## Goals", "", "the project's decisions", "the plan or a comment", kinds(*entries), mark, "loop-claude"), Rung("judge-model"), FakeEffects(claude=[judged("unfinished", escalation=escalation)]).effects())
self.assertEqual(judged_on(0, "round 2: 🔴"), Unfinished("working", "round 2: 🔴", entries_read=3))
self.assertEqual(judged_on(1, "round 2: 🔴"), Unfinished("working", entries_read=3))
self.assertEqual(judged_on(1, "round 3: 🔴"), Unfinished("working", "round 3: 🔴", entries_read=3))
self.assertEqual(judged_on(1, "round 2: 🔴\n\nround 3: 🔴"), Unfinished("working", entries_read=3))
self.assertEqual(judged_on(3, "round 3: 🔴"), Unfinished("working", entries_read=3))
def test_drops_an_escalation_quote_that_only_loop_claudes_prompts_or_the_humans_lines_repeat_after_the_step_up(self):
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), {**human("as the work remaining: round 2: 🔴"), "promptSource": "sdk"}, human("round 2: 🔴 again"), assistant({"type": "text", "text": "working"})]
verdict = judge.judge_once("/r", judge.Brief("Do it.", "## Goals", "", "the project's decisions", "the plan or a comment", kinds(*entries), 1, "loop-claude"), Rung("judge-model"), FakeEffects(claude=[judged("unfinished", escalation="round 2: 🔴")]).effects())
self.assertEqual(verdict, Unfinished("working", entries_read=4))
def test_takes_evidence_from_before_the_workers_last_step_up(self):
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), assistant({"type": "text", "text": "working"})]
verdict = judge.judge_once("/r", judge.Brief("Do it.", "## Goals", "", "the project's decisions", "the plan or a comment", kinds(*entries), 1, "loop-claude"), Rung("judge-model"), FakeEffects(claude=[judged("unfinished", evidence="round 2: 🔴")]).effects())
self.assertEqual(verdict, Unfinished("round 2: 🔴", entries_read=2))
class JudgePrompt(unittest.TestCase):
def brief(self, entries, escalated_at, answer_destinations="the plan or a comment"):
"""What the judge of a-driver reads of a transcript holding entries."""
return judge.Brief("Do it.", "## Goals", "", "the project's decisions", answer_destinations, kinds(*entries), escalated_at, "a-driver")
def test_shows_the_transcripts_tail_and_returns_what_it_showed(self):
entries = [assistant({"type": "text", "text": "old " + "x" * judge.JUDGE_PROMPT_CHARS}), assistant({"type": "text", "text": "new"})]
prompt, grounds = judge.judge_prompt(self.brief(entries, 0))
self.assertTrue(grounds.shown.endswith("\n\nnew"))
self.assertNotIn("old", grounds.shown)
self.assertIn(grounds.shown, prompt)
self.assertLessEqual(len(prompt), judge.JUDGE_PROMPT_CHARS)
self.assertEqual(grounds.after_mark, "old " + "x" * judge.JUDGE_PROMPT_CHARS + "\n\nnew")
self.assertEqual(grounds.quotable, "## Goals\n\n")
def test_the_judge_prompt_and_schema_call_the_worker_the_worker(self):
prompt, _ = judge.judge_prompt(self.brief([assistant({"type": "text", "text": "done"})], 1))
self.assertNotRegex(prompt + json.dumps(judge.judge_schema("a-driver")), r"\b[Aa]gent")
def test_the_judge_prompt_names_the_entry_points_answer_destinations_in_both_rules_on_a_recorded_answer(self):
prompt, _ = judge.judge_prompt(self.brief([assistant({"type": "text", "text": "done"})], 0, "the ledger"))
self.assertIn("is recorded in the ledger at the one place it governs", prompt)
self.assertIn("the worker writing it into the ledger, but not into one of the human's rules", prompt)
def test_shows_the_judge_where_the_worker_last_stepped_up_and_checks_an_escalation_only_after_it(self):
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), assistant({"type": "text", "text": "round 3: 🔴"})]
prompt, grounds = judge.judge_prompt(self.brief(entries, 1))
self.assertRegex(prompt, "round 2: 🔴\n\n\\[step\\] [^\n]+\n\nround 3: 🔴\n\n")
self.assertEqual(grounds.after_mark, "round 3: 🔴")
prompt, grounds = judge.judge_prompt(self.brief(entries, 0))
self.assertNotIn("stepped the worker up", prompt)
self.assertEqual(grounds.shown, "round 2: 🔴\n\nround 3: 🔴")
self.assertEqual(grounds.after_mark, "round 2: 🔴\n\nround 3: 🔴")
def test_drops_an_escalation_quote_from_after_the_step_up_that_the_tail_it_shows_cuts_off(self):
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), assistant({"type": "text", "text": "old " + "x" * judge.JUDGE_PROMPT_CHARS}), assistant({"type": "text", "text": "new"})]
_, grounds = judge.judge_prompt(self.brief(entries, 1))
self.assertEqual(judge.quotes_checked(Unfinished("e", "old x", entries_read=3), grounds), Unfinished("e", entries_read=3))
self.assertEqual(judge.quotes_checked(Unfinished("e", "new", entries_read=3), grounds), Unfinished("e", "new", entries_read=3))
def test_shows_a_typed_answer_before_the_step_up_as_the_humans(self):
typed = PROMPTS.typed("Which one?", "Use B.", Route(Rung("worker-model")), None, False)
entries = [{**human(typed), "promptSource": "sdk"}, assistant({"type": "text", "text": "round 3: 🔴"})]
_, grounds = judge.judge_prompt(self.brief(entries, 1))
self.assertIn("\n\n[human] Use B.\n\n[step] ", grounds.shown)
if __name__ == "__main__":
unittest.main()
+50
View File
@@ -0,0 +1,50 @@
import contextlib
import io
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from failure import Failure
from loop_effects.doubles import FakeEffects, NOW, STAMP
from claude_code.reply import Limited
from agent_cycle import retry
from agent_cycle.stop import Stop
class Retried(unittest.TestCase):
def test_retries_a_minute_apart_and_stops_on_the_third_failure(self):
fake = FakeEffects()
results = [Failure("one"), "done"]
self.assertEqual(retry.retried("worker", lambda: results.pop(0), fake.effects()), "done")
self.assertEqual(fake.calls, [("sleep", 60)])
fake = FakeEffects()
self.assertEqual(retry.retried("judge", lambda: Failure("no"), fake.effects()), Stop(1, "the judge failed three times running: no"))
self.assertEqual(fake.calls, [("sleep", 60), ("sleep", 60)])
def test_hands_each_failure_to_the_keeper_and_stops_where_it_stops(self):
kept = []
def keep(failure):
kept.append(failure)
return Stop(1, "no room") if len(kept) == 2 else None
fake = FakeEffects()
self.assertEqual(retry.retried("judge", lambda: Failure("no"), fake.effects(), keep), Stop(1, "no room"))
self.assertEqual(kept, [Failure("no")] * 2)
self.assertEqual(fake.calls, [("sleep", 60)])
def test_waits_out_a_usage_limit_which_ends_a_run_of_failures(self):
fake = FakeEffects()
results = [Failure("one"), Failure("two"), Limited(NOW + 3600, "limit"), Failure("three"), Limited(None, "limit"), Limited(NOW - 600, "limit"), Failure("four"), Failure("five"), Failure("six")]
self.assertEqual(retry.retried("worker", lambda: results.pop(0), fake.effects()), Stop(1, "the worker failed three times running: six"))
self.assertEqual(fake.calls, [("sleep", 60), ("sleep", 60), ("sleep", 3720), ("sleep", 60), ("sleep", 600), ("sleep", 600), ("sleep", 60), ("sleep", 60)])
def test_says_what_claude_said_and_when_the_next_try_is_due(self):
results = [Limited(NOW + 3600, "You've hit your session limit"), "done"]
with contextlib.redirect_stderr(io.StringIO()) as said:
retry.retried("judge", lambda: results.pop(0), FakeEffects().effects())
self.assertEqual(said.getvalue(), f"{STAMP} the judge hit a usage limit (You've hit your session limit); retrying at <{NOW + 3720}>\n")
if __name__ == "__main__":
unittest.main()
+109
View File
@@ -0,0 +1,109 @@
import dataclasses
import pathlib
import sys
import types
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from agent_cycle.verdicts import Interrupt, TakenIn, Unfinished
from model_routing.ladder import ProjectRung, Rung
from model_routing.ladder_file import read_ladder, SHIPPED
from model_routing.route import Route
from agent_cycle import routing
LADDER = read_ladder(SHIPPED)
OPUS_PROJECT = ProjectRung(LADDER, Rung("claude-opus-5-5", "medium"))
# The ladder's rungs around the project's: the one below it, its own, and those one to four steps up.
SONNET = Rung("claude-sonnet-5-5", "high")
OPUS = Rung("claude-opus-5-5", "medium")
ONE_UP = Rung("claude-fable-5-2", "low")
TWO_UP = Rung("claude-fable-5-2", "medium")
THREE_UP = Rung("claude-opus-5-5", "xhigh")
FOUR_UP = Rung("claude-fable-5-2", "high")
def scored(likelihood, escape_and_impact):
"""Item 3 with that L and E + I, as routing.Scored reads an item."""
return types.SimpleNamespace(id=3, likelihood=likelihood, escape_and_impact=escape_and_impact)
def unfinished(likelihood=0, escalated_at=None):
"""An unfinished verdict: the worker took in an item of that L, 0 for none, and the judge found an escalation after reading escalated_at transcript entries, None for none."""
return Unfinished("e", escalation="round 2: 🔴" if escalated_at is not None else "", taken_in=TakenIn(highest_likelihood=likelihood), entries_read=escalated_at)
def interrupted(likelihood=0, escalated_at=None):
"""An interrupt, its arguments as unfinished's."""
return Interrupt("e", "q", escalation="round 2: 🔴" if escalated_at is not None else "", taken_in=TakenIn(highest_likelihood=likelihood), entries_read=escalated_at)
class FirstRoute(unittest.TestCase):
def test_starts_a_chunk_on_the_rung_its_items_l_picks_below_the_projects_too(self):
self.assertEqual(routing.first_route(OPUS_PROJECT, scored(4, 5)), Route(TWO_UP, 3, 4, 5))
self.assertEqual(routing.first_route(OPUS_PROJECT, scored(1, 5)), Route(SONNET, 3, 1, 5))
self.assertEqual(routing.first_route(OPUS_PROJECT, None), Route(OPUS))
class NextRoute(unittest.TestCase):
def test_steps_up_a_rung_after_every_second_unfinished_verdict(self):
once = routing.next_route(OPUS_PROJECT, Route(ONE_UP, 3, 3, 4), unfinished())
self.assertEqual(once, Route(ONE_UP, 3, 3, 4, unfinished=1))
self.assertEqual(routing.next_route(OPUS_PROJECT, once, unfinished()), Route(TWO_UP, 3, 3, 4))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(LADDER.rungs[-1], unfinished=1), unfinished()).rung, LADDER.rungs[-1])
def test_steps_up_to_the_rung_a_taken_in_items_l_picks_and_never_down(self):
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(OPUS), unfinished(4)).rung, TWO_UP)
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(THREE_UP), unfinished(4)).rung, THREE_UP)
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(OPUS, unfinished=1), unfinished(4)).rung, THREE_UP)
def test_steps_up_from_a_rung_below_the_projects_one_rung_at_a_time(self):
low = Route(SONNET, 3, 1, 3, unfinished=1)
self.assertEqual(routing.next_route(OPUS_PROJECT, low, unfinished()).rung, OPUS)
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(low.rung, 3, 1, 3), unfinished()).rung, low.rung)
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(low.rung, 3, 1, 3), unfinished(3)).rung, ONE_UP)
def test_steps_up_a_rung_on_an_escalation_the_judge_found_up_to_two_above_the_items_rung_marking_the_transcript_entries_it_read(self):
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(TWO_UP, 3, 4, 5), unfinished(escalated_at=7)), Route(THREE_UP, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(THREE_UP, 3, 4, 5, escalated_at=7), unfinished(escalated_at=9)), Route(FOUR_UP, 3, 4, 5, escalated_at=9))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(FOUR_UP, 3, 4, 5, escalated_at=9), unfinished(escalated_at=12)), Route(FOUR_UP, 3, 4, 5, unfinished=1, escalated_at=9))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(TWO_UP, 3, 4, 5, unfinished=1), unfinished(escalated_at=7)), Route(THREE_UP, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(TWO_UP), unfinished(escalated_at=7)), Route(TWO_UP, unfinished=1))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(SONNET, 3, 2, 4), unfinished(4, escalated_at=7)), Route(TWO_UP, 3, 2, 4, unfinished=1))
def test_keeps_the_mark_through_a_step_no_escalation_took(self):
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(ONE_UP, 3, 3, 4, unfinished=1, escalated_at=5), unfinished()), Route(TWO_UP, 3, 3, 4, escalated_at=5))
def test_steps_up_a_rung_as_the_judge_interrupts_up_to_two_above_the_items_rung_without_counting_an_unfinished_verdict(self):
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(TWO_UP, 3, 4, 5, unfinished=1), interrupted(escalated_at=7)), Route(THREE_UP, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(THREE_UP, 3, 4, 5, escalated_at=7), interrupted(escalated_at=9)), Route(FOUR_UP, 3, 4, 5, escalated_at=9))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(FOUR_UP, 3, 4, 5, unfinished=1, escalated_at=9), interrupted(escalated_at=12)), Route(FOUR_UP, 3, 4, 5, unfinished=1, escalated_at=9))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(SONNET, 3, 2, 4), interrupted(escalated_at=7)), Route(OPUS, 3, 2, 4, escalated_at=7))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(TWO_UP, 3, 4, 5, unfinished=1, escalated_at=2), interrupted(4)), Route(TWO_UP, 3, 4, 5, unfinished=1, escalated_at=2))
def test_raises_the_worker_to_a_taken_in_items_rung_above_the_escalation_cap_without_a_step_as_the_judge_interrupts(self):
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(OPUS, 3, 1, 3, escalated_at=2), interrupted(4, escalated_at=7)), Route(TWO_UP, 3, 1, 3, escalated_at=2))
def test_steps_up_from_the_rung_a_taken_in_item_picks_as_the_judge_interrupts(self):
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(ONE_UP, 3, 3, 4), interrupted(4, escalated_at=7)), Route(THREE_UP, 3, 3, 4, escalated_at=7))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(TWO_UP, 3, 4, 5), interrupted(4, escalated_at=7)), Route(THREE_UP, 3, 4, 5, escalated_at=7))
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(FOUR_UP, 3, 4, 5, unfinished=1), interrupted(4, escalated_at=7)), Route(FOUR_UP, 3, 4, 5, unfinished=1))
def test_a_rung_an_edit_took_off_the_ladder_counts_as_the_projects(self):
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(Rung("claude-opus-5-5", "low")), unfinished()).rung, OPUS)
self.assertEqual(routing.next_route(OPUS_PROJECT, Route(Rung("claude-opus-5-5", "low")), unfinished(2)).rung, OPUS)
class StepUp(unittest.TestCase):
def test_names_no_step_up_where_only_the_count_of_unfinished_verdicts_changed(self):
route = Route(OPUS, 3, 3, 4)
self.assertIsNone(routing.step_up(route, dataclasses.replace(route, unfinished=1)))
self.assertEqual(routing.step_up(route, route.taking_in(6)), routing.StepUp(route, route.taking_in(6)))
class JudgeRung(unittest.TestCase):
def test_runs_the_judge_of_a_replayed_model_on_its_models_highest_rung(self):
self.assertEqual(routing.judge_rung(LADDER, "claude-opus-5-5"), THREE_UP)
# A model off the ladder, or not replayed, gets no effort flag.
self.assertEqual(routing.judge_rung(LADDER, "judge-model"), Rung("judge-model"))
self.assertEqual(routing.judge_rung(LADDER, "claude-sonnet-5-5"), Rung("claude-sonnet-5-5"))
+518
View File
@@ -0,0 +1,518 @@
import contextlib
import dataclasses
import io
import json
import os
import pathlib
import re
import shutil
import stat
import sys
import tempfile
import types
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from test_env import assert_not_root
from failure import Failure
from loop_effects.doubles import FakeEffects, human, judged, NOW, ran, STAMP, Streamed, Worker
from agent_cycle.verdicts import Finished, Interrupt, Judged, QuotedAnswer, TakenIn, Unfinished
from transcript_archive import outcomes, write
from loop_state import paths
from transcript_archive.location import project_archive
from model_routing.ladder import ProjectRung, Rung
from agent_cycle.doubles import PROMPTS, WINDOW
from model_routing.ladder_file import read_ladder, SHIPPED
from text_files.file_writes import replace_file
from model_routing.route import Route
from agent_cycle import routing
from agent_cycle import run, stuck_workers, worker_prompts
from claude_code import session
from loop_effects.child_shells import Shell
from agent_cycle.stop import Stop
from text_files.utc_stamp import utc
LADDER = read_ladder(SHIPPED)
OPUS_PROJECT = ProjectRung(LADDER, Rung("claude-opus-5-5", "medium"))
# The ladder's rungs the cycles run on, and those one and three steps up from OPUS.
SONNET = Rung("claude-sonnet-5-5", "high")
OPUS = Rung("claude-opus-5-5", "medium")
ONE_UP = Rung("claude-fable-5-2", "low")
THREE_UP = Rung("claude-opus-5-5", "xhigh")
# How the archive records a worker that runs off the ladder.
OFF_LADDER = '{"model": "worker-model"}'
# The claude-config release and commit each cycle the tests run is under.
CONFIG = "v0.8.0 (abc123)"
# How it records routes by item 179's L of 2 and E + I of 5.
SONNET_179 = '{"model": "claude-sonnet-5-5", "effort": "high", "item": 179, "likelihood": 2, "escape_and_impact": 5}'
OPUS_179_AT_3 = '{"model": "claude-opus-5-5", "effort": "medium", "item": 179, "likelihood": 2, "escape_and_impact": 5, "escalated_at": 3}'
OPUS_179_AT_6 = '{"model": "claude-opus-5-5", "effort": "medium", "item": 179, "likelihood": 2, "escape_and_impact": 5, "escalated_at": 6}'
JUDGE_RUN_LINE = re.compile(r"^Judge run: judge/[0-9a-f]{64}\.json\n", re.MULTILINE)
class QuestionFor(unittest.TestCase):
def test_puts_an_interrupts_question_as_asked_and_a_quoted_answer_past_the_cap_with_its_quote(self):
cycle = types.SimpleNamespace(prompts=PROMPTS)
self.assertEqual(run.question_for(cycle, Interrupt("e", "Which one?")), "Which one?")
self.assertEqual(run.question_for(cycle, QuotedAnswer("e", "Which one?", "A.")), "Which one?\n\nthe project's records says:\nA.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.")
class Cycles(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.home = pathlib.Path(temp.name)
self.paths = paths.paths_from(self.home)
self.session = session.session_named("A proj a-driver", self.paths.claude_code_transcripts)
self.archive = project_archive(str(self.home / "code" / "proj"), self.home, {})
def cycle(self, fake, records="", unfinished_evidence="", facts="- A preflight fact."):
return run.Cycle(config=CONFIG, window=WINDOW, effects=fake.effects(), facts=facts, judge=Rung("judge-model"), premises="## Goals", project=None, project_archive=self.archive, records=records, root=str(self.home), route=Route(Rung("worker-model")), session=self.session, task="Do it.", prompts=PROMPTS, taken_in=lambda entries: TakenIn(), unfinished_evidence=unfinished_evidence)
def archived_outcomes(self):
"""Each outcomes file's text without its `Judge run:` lines, which name a file by the hash of a prompt and its reply: the tests of the judge's runs check them."""
return sorted(JUDGE_RUN_LINE.sub("", path.read_text()) for path in self.archive.glob("*/*.outcomes.md"))
def test_a_finished_chunk_is_archived_and_its_session_deleted(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Finished("working"))
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n"])
def test_says_where_it_archived_the_transcript(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
with contextlib.redirect_stderr(io.StringIO()) as said:
run.run_cycle(self.cycle(fake))
copy, = self.archive.glob("*/*.jsonl")
self.assertIn(f"transcript: {copy}\n", said.getvalue())
def test_archives_the_cost_the_judges_reply_records_beside_its_verdict(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished", cost=0.25)])
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Finished("working", judged=Judged(cost=0.25)))
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nJudge cost: 0.2500\nWorker: {OFF_LADDER}\n"])
self.assertIsNone(session.transcript_file(self.session))
def test_archives_what_the_worker_and_the_judge_spent_beside_the_verdict(self):
worker = Worker(self.paths.claude_code_transcripts.parent, usage={"input_tokens": 1, "cache_creation": {"ephemeral_5m_input_tokens": 2, "ephemeral_1h_input_tokens": 3}, "cache_read_input_tokens": 4, "output_tokens": 5})
answer = json.loads(judged("finished").stdout)
fake = FakeEffects(streams=[worker], claude=[ran(json.dumps({**answer, "usage": {"input_tokens": 6, "output_tokens": 7}}))])
run.run_cycle(self.cycle(fake))
self.assertEqual(self.archived_outcomes(), [f"""Task: Do it.
2026-09-21T14:13:20Z Verdict: finished
Evidence:
> working
Spent since the chunk began, in tokens of input, cache write 5m, cache write 1h, cache read, output:
- worker, claude-opus-5-5: 1, 2, 3, 4, 5
- judge, judge-model: 6, 0, 0, 0, 7
List prices in dollars per million tokens of input, cache write 5m, cache write 1h, cache read, output:
- claude-opus-5-5: 4, 5, 8, 0.2, 20
Config: {CONFIG}
Judge: judge-model
Worker: {OFF_LADDER}
"""])
def test_a_fresh_worker_reads_the_preflights_facts_and_a_resumed_one_does_not(self):
worker = Worker(self.paths.claude_code_transcripts.parent)
fake = FakeEffects(streams=[worker, worker], claude=[judged("unfinished"), judged("finished")])
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Unfinished("working", entries_read=3))
self.assertEqual(run.run_cycle(self.cycle(fake, unfinished_evidence="working")).verdict, Finished("working"))
fresh, resumed = (args[args.index("-p") + 1] for args in fake.made("stream_claude"))
self.assertIn("- A preflight fact.", fresh)
self.assertNotIn("- A preflight fact.", resumed)
def test_a_worker_is_handed_the_projects_rung_its_own_its_reviews_e_plus_i_and_its_compaction_window_in_its_environment(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
cycle = dataclasses.replace(self.cycle(fake), project=OPUS_PROJECT, route=Route(Rung("claude-haiku-5-5", "max"), 163, 1, 5))
self.assertEqual(run.run_cycle(cycle).verdict, Finished("working"))
self.assertEqual(fake.worker_environments, [{"LOOP_CLAUDE_PROJECT_RUNG": "claude-opus-5-5 at medium effort", "LOOP_CLAUDE_WORKER_RUNG": "claude-haiku-5-5 at max effort", "LOOP_CLAUDE_ESCAPE_AND_IMPACT": "5", "CLAUDE_CODE_AUTO_COMPACT_WINDOW": "200000", "BASH_MAX_OUTPUT_LENGTH": "6000"}])
def test_a_stuck_check_of_the_worker_runs_on_the_projects_rung_and_off_the_ladder_on_the_workers_own(self):
for project, model in ((OPUS_PROJECT, "claude-opus-5-5"), (None, "claude-haiku-5-5")):
with self.subTest(project=project):
clock = [NOW]
quiet = Shell(pid=7, started=NOW, command="bin/run-tests", written=NOW, runs_busy_tool=False)
events = ({"type": "assistant", "message": {"content": []}}, None, None, {"type": "result"})
stream = lambda args: (Worker(self.paths.claude_code_transcripts.parent)(args), Streamed(0, *events, clock=clock, held=[quiet]))[1]
verdict = ran(json.dumps({"type": "result", "structured_output": {"verdict": "stuck", "process": 7, "why": "it waits"}}))
fake = FakeEffects(streams=[stream], claude=[verdict, judged("finished")])
cycle = dataclasses.replace(self.cycle(fake), project=project, route=Route(Rung("claude-haiku-5-5", "max")))
with contextlib.redirect_stdout(io.StringIO()):
run.run_cycle(dataclasses.replace(cycle, effects=dataclasses.replace(cycle.effects, now=lambda: clock[0])))
[investigator, _] = [args for args, _ in fake.made("claude")]
self.assertEqual(investigator[investigator.index("--model") + 1], model)
self.assertEqual(fake.worker_folders, [str(self.home)])
self.assertIn("judged the worker stuck", (self.archive / stuck_workers.RECORD).read_text())
def test_the_claude_an_interrupt_resumes_is_handed_the_same_rungs_as_the_worker(self):
def answer(args):
with open(session.transcript_file(self.session), "a") as handle:
handle.write(json.dumps(human("the second")) + "\n")
return 0
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt"), judged("finished")], inputs=["/talk"], on_terminal_claude=[answer])
cycle = dataclasses.replace(self.cycle(fake), project=OPUS_PROJECT, route=Route(Rung("claude-haiku-5-5", "max"), 163, 1, 5))
with contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(run.run_cycle(cycle).verdict, Finished("working"))
self.assertEqual(fake.terminal_environments, [{**environment, "CLAUDE_CODE_SANDBOXED": "1"} for environment in fake.worker_environments])
self.assertEqual(fake.terminal_environments[0]["LOOP_CLAUDE_WORKER_RUNG"], "claude-haiku-5-5 at max effort")
def test_an_unfinished_chunks_transcript_is_archived_and_the_next_cycle_resumes_its_session_with_the_judges_evidence(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[judged("unfinished")] * 2)
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Unfinished("working", entries_read=3))
self.assertTrue(session.transcript_file(self.session))
self.assertEqual(run.run_cycle(self.cycle(fake, unfinished_evidence="working")).verdict, Unfinished("working", entries_read=4))
workers = fake.made("stream_claude")
self.assertIn("--session-id", workers[0])
self.assertIn("--resume", workers[1])
self.assertEqual(workers[1][workers[1].index("-p") + 1], "Resumed: working")
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n"] * 2)
def test_a_cycle_stopped_by_a_failure_or_ctrl_c_archives_what_its_session_holds(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[ran(code=1)] * 3)
self.assertEqual(run.run_cycle(self.cycle(fake)).code, 1)
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[KeyboardInterrupt()])
self.assertEqual(run.run_cycle(self.cycle(fake)), Stop(130, ""))
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n\nWorker: {OFF_LADDER}\n", f"Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: the judge failed three times running: the judge exited 1\n\nWorker: {OFF_LADDER}\n"])
def test_each_judge_reply_the_loop_rejects_is_archived_with_why_beside_the_verdict_that_follows(self):
unfilled = ran(json.dumps({"structured_output": {"interrupt": False}}))
misquoted = judged("finished", evidence="never said")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[unfilled, misquoted, judged("finished")])
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Finished("working"))
stamp = "2026-09-21T14:13:20Z"
self.assertEqual(self.archived_outcomes(), [
"Task: Do it.\n\n"
f"{stamp} Judge reply rejected: the judge's reply has no answer of type string\n\nReply:\n> {unfilled.stdout}\n\nJudge: judge-model\n\n"
f"{stamp} Judge reply rejected: the judge quoted 'never said', which is not in the transcript\n\nReply:\n> {misquoted.stdout}\n\nJudge: judge-model\n\n"
f"{stamp} Verdict: finished\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n"])
def test_archives_each_judge_runs_prompt_and_reply_beside_the_rejection_interrupt_and_verdict_it_gave(self):
rejected, interrupted, finished = ran(json.dumps({"structured_output": {"interrupt": False}})), judged("interrupt", cost=0.25), judged("finished")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[rejected, interrupted, finished], inputs=["Use B."])
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Finished("working"))
prompts = [stdin for args, stdin in fake.made("claude")[:3]]
[chunk] = self.archive.iterdir()
kept = "\n".join(path.read_text() for path in chunk.glob("*.outcomes.md"))
named = re.findall(r"^Judge run: (judge/[0-9a-f]{64}\.json)$", kept, re.MULTILINE)
self.assertEqual(len(named), 3, kept)
runs = [json.loads((chunk / name).read_text()) for name in named]
replies = [json.loads(reply.stdout) for reply in (rejected, interrupted, finished)]
self.assertEqual(sorted((found["prompt"], json.dumps(found["reply"])) for found in runs), sorted((prompt, json.dumps(reply)) for prompt, reply in zip(prompts, replies)))
self.assertEqual(sorted(path.name for path in (chunk / "judge").iterdir()), sorted(name.removeprefix("judge/") for name in named))
def test_stops_when_a_judge_run_cannot_be_archived(self):
self.archive.mkdir(parents=True)
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
fake.replace_file = lambda path, data: Failure("disk full") if "/judge/" in str(path) else replace_file(path, data)
stop = run.run_cycle(self.cycle(fake))
self.assertEqual(stop.code, 1)
self.assertRegex(stop.reason, r"^could not archive the transcript: .*/judge/[0-9a-f]{64}\.json: disk full$")
def test_archives_what_each_judge_run_cost_on_an_interrupt_a_quoted_answer_and_a_rejected_misquote(self):
for first, then, inputs in (
(judged("interrupt", cost=0.25), judged("finished"), ["Use B."]),
(judged("quoted", answer="Use A.", cost=0.25), judged("finished"), []),
(judged("unfinished", cost=0.25), judged("finished"), []),
(judged("finished", evidence="never said", cost=0.25), judged("finished"), []),
):
with self.subTest(first=first.stdout), tempfile.TemporaryDirectory() as archive:
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[first, then], inputs=inputs)
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
run.run_cycle(dataclasses.replace(self.cycle(fake, records="From AGENTS.md:\nUse A."), project_archive=pathlib.Path(archive)))
kept = "".join(path.read_text() for path in pathlib.Path(archive).glob("*/*.outcomes.md"))
self.assertEqual(kept.count("Judge cost: 0.2500\n"), 1, kept)
self.assertIn("Judge cost: 0.2500\nJudge run: judge/", kept)
def test_stops_with_exit_1_where_a_subagents_transcript_cannot_be_read_for_the_spend(self):
for verdict in ("interrupt", "finished"):
with self.subTest(verdict=verdict), tempfile.TemporaryDirectory() as archive:
unreadable = self.paths.claude_code_transcripts / "-fake" / str(self.session.id) / "subagents" / "agent-x.jsonl"
unreadable.mkdir(parents=True)
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged(verdict)], inputs=["Use B."])
with contextlib.redirect_stderr(io.StringIO()):
stop = run.run_cycle(dataclasses.replace(self.cycle(fake), project_archive=pathlib.Path(archive)))
self.assertEqual((stop.code, stop.reason), (1, f"could not read the transcript {unreadable}: Is a directory"))
shutil.rmtree(unreadable.parent.parent)
(self.paths.claude_code_transcripts / "-fake" / f"{self.session.id}.jsonl").unlink()
def test_counts_the_spend_since_the_last_outcome_that_recorded_one_and_archives_the_reversal(self):
transcript = self.paths.claude_code_transcripts / "-fake" / f"{self.session.id}.jsonl"
transcript.parent.mkdir(parents=True)
old = {"type": "assistant", "timestamp": "2026-09-21T14:13:19Z", "message": {"id": "old", "model": "claude-opus-5-5", "content": [], "usage": {"output_tokens": 100}}}
new = {"type": "assistant", "timestamp": "2026-09-21T14:13:21Z", "message": {"id": "new", "model": "claude-opus-5-5", "content": [], "usage": {"output_tokens": 5}}}
transcript.write_text("\n".join(json.dumps(entry) for entry in [{"type": "queue-operation", "timestamp": "2026-09-21T14:00:00Z"}, old, human("No, keep the old name."), new]) + "\n")
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(self.archive, str(transcript), "Do it.", outcomes.unfinished("w", "e", "j", "", annotation=outcomes.Annotation("c", "Spent since the chunk began, in tokens of x:\n- worker, m: 1")), NOW, replace=replace_file))
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("unfinished", reversed="No, keep the old name.")])
run.run_cycle(self.cycle(fake))
latest = max(self.archived_outcomes(), key=len)
self.assertIn(f"""Reversed:
> No, keep the old name.
Spent since 2026-09-21T14:13:20Z, in tokens of input, cache write 5m, cache write 1h, cache read, output:
- worker, claude-opus-5-5: 0, 0, 0, 0, 5
""", latest)
def test_stops_with_exit_1_asking_nothing_where_an_interrupt_cannot_be_archived(self):
self.archive.parent.mkdir(parents=True)
self.archive.write_text("")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt")], inputs=["Use B."])
with contextlib.redirect_stderr(io.StringIO()) as said:
stop = run.run_cycle(self.cycle(fake))
self.assertEqual((stop.code, stop.reason.split(":")[0]), (1, "could not archive the transcript"))
self.assertNotIn("Which one?", said.getvalue())
self.assertEqual(fake.made("claude_on_terminal"), [])
def test_stops_when_a_rejected_reply_cannot_be_archived(self):
self.archive.parent.mkdir(parents=True)
self.archive.write_text("")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[ran(json.dumps({"structured_output": {"interrupt": False}})), judged("finished")])
stop = run.run_cycle(self.cycle(fake))
self.assertRegex(stop.reason, f"^could not archive the transcript: {self.archive}/[0-9a-f]{{64}}: Not a directory$")
self.assertEqual(stop.code, 1)
self.assertEqual(len(fake.made("claude")), 1)
def test_archives_the_judges_rejected_replies_though_it_fails_three_times_running(self):
unfilled = ran(json.dumps({"structured_output": {"interrupt": False}}))
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[unfilled] * 3)
self.assertEqual(run.run_cycle(self.cycle(fake)).code, 1)
[archived] = self.archived_outcomes()
self.assertEqual(archived.count(f"Reply:\n> {unfilled.stdout}\n"), 3)
self.assertIn("Stopped: the judge failed three times running", archived)
def test_archives_each_chunks_session_folder_replacing_a_copy_only_with_what_grew_from_it(self):
folder = self.paths.claude_code_transcripts / "-fake" / str(self.session.id)
(folder / "subagents" / "workflows").mkdir(parents=True)
(folder / "subagents" / "agent-a.jsonl").write_text("a\n")
(folder / "subagents" / "workflows" / "journal.jsonl").write_text("j\n")
(folder / "custom-title.json").write_text("first")
(folder.parent / f"{self.session.id}.orphaned-1.jsonl").write_text("o\n")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 3, claude=[judged("unfinished"), judged("finished"), judged("unfinished")])
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Unfinished("working", entries_read=3))
[chunk] = self.archive.iterdir()
kept = chunk / "session" / str(self.session.id)
self.assertEqual((chunk / "session" / f"{self.session.id}.orphaned-1.jsonl").read_text(), "o\n")
os.utime(kept / "subagents" / "workflows" / "journal.jsonl", (0, 0))
with open(folder / "subagents" / "agent-a.jsonl", "a") as handle:
handle.write("more\n")
(folder / "custom-title.json").write_text("second")
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Finished("working"))
self.assertFalse(folder.exists())
self.assertEqual((kept / "subagents" / "agent-a.jsonl").read_text(), "a\nmore\n")
self.assertEqual((kept / "subagents" / "workflows" / "journal.jsonl").stat().st_mtime, 0)
self.assertEqual(sorted(path.read_text() for path in kept.glob("custom-title.json*")), ["first", "second"])
self.assertEqual(stat.S_IMODE((kept / "subagents" / "agent-a.jsonl").stat().st_mode), 0o600)
folder.mkdir()
(folder / "custom-title.json").write_text("third")
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Unfinished("working", entries_read=3))
[next_chunk] = set(self.archive.iterdir()) - {chunk}
self.assertEqual(sorted(path.read_text() for path in (next_chunk / "session" / str(self.session.id)).glob("custom-title.json*")), ["third"])
def test_stops_on_a_session_folder_it_cannot_read_whole(self):
assert_not_root(self)
locked = self.paths.claude_code_transcripts / "-fake" / str(self.session.id) / "subagents"
locked.mkdir(parents=True)
locked.chmod(0)
self.addCleanup(locked.chmod, 0o700)
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("unfinished")])
self.assertEqual(run.run_cycle(self.cycle(fake)), Stop(1, f"could not archive the transcript: {locked}: Permission denied"))
def test_an_interrupt_talked_over_in_claude_is_archived_answered_and_judged_again(self):
def answer(args):
with open(session.transcript_file(self.session), "a") as handle:
handle.write(json.dumps(human("the second")) + "\n")
return 0
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt"), judged("unfinished")], inputs=["/talk"], on_terminal_claude=[answer])
with contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Unfinished("working", entries_read=4))
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n", f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n"])
self.assertIn(str(self.session.id), fake.made("claude_on_terminal")[0])
judge_prompts = [stdin for args, stdin in fake.made("claude")]
self.assertEqual(len(judge_prompts), 2)
self.assertNotIn("the second", judge_prompts[0])
self.assertIn("[human] the second", judge_prompts[1])
self.assertIn("From the goals file, ## Goals", judge_prompts[0])
self.assertIn("writing it into the plan or a comment,", judge_prompts[0])
self.assertIn("The task the worker was given:\nDo it.", judge_prompts[0])
def test_a_question_the_records_answer_resumes_the_worker_with_the_answer_three_times_a_cycle(self):
answered = judged("quoted", answer="Use *A*.")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 4, claude=[answered] * 4, inputs=[""])
with contextlib.redirect_stderr(io.StringIO()) as said, contextlib.redirect_stdout(io.StringIO()) as printed:
self.assertEqual(run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse A.")), Stop(2, "the human is needed to answer the question above, but nothing was typed"))
resumed = fake.made("stream_claude")[1]
self.assertEqual(resumed[resumed.index("-p") + 1], "Your question: Which one?\n\nthe project's records answers it:\n\nUse *A*.\n\nContinue the task.")
self.assertEqual(len(fake.made("stream_claude")), 4)
self.assertEqual(printed.getvalue().count("=== the project's records answered: Which one? ===\nUse *A*."), 3)
self.assertIn("Which one?\n\nthe project's records says:\nUse *A*.\n\nThose files have answered 3 questions this cycle", said.getvalue())
quoted = f"2026-09-21T14:13:20Z Verdict: quoted answer\n\nWhich one?\n\nthe project's records says:\n> Use *A*.\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n"
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nthe project's records says:\nUse *A*.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n\n2026-09-21T14:13:20Z Stopped: the human is needed to answer the question above, but nothing was typed\n\nWorker: {OFF_LADDER}\n", *[f"Task: Do it.\n\n{quoted}"] * 3])
def test_a_quoted_answer_that_cites_an_escalation_trigger_steps_the_worker_up_before_it_resumes_and_archives_the_step(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[judged("quoted", answer="Use A.", escalation="working"), judged("finished")])
cycle = dataclasses.replace(self.cycle(fake, records="From AGENTS.md:\nUse A."), project=OPUS_PROJECT, route=Route(SONNET, 179, 2, 5))
with contextlib.redirect_stdout(io.StringIO()):
concluded = run.run_cycle(cycle)
self.assertEqual(concluded.route, Route(OPUS, 179, 2, 5, escalated_at=3))
first, resumed = fake.made("stream_claude")
self.assertEqual([args[args.index("--effort") + 1] for args in (first, resumed)], ["high", "medium"])
self.assertTrue(resumed[resumed.index("-p") + 1].endswith("Continue the task. Your request for a higher rung went through: a-driver runs you on claude-opus-5-5 at medium effort. The escalation triggers that fired before this step are spent. You hold claude-opus-5-5 at medium effort."))
self.assertNotIn("went through", first[first.index("-p") + 1])
answered = f"Verdict: quoted answer\n\nWhich one?\n\nthe project's records says:\n> Use A.\n\nEvidence:\n> working\n\nEscalation:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {SONNET_179}\n"
stepped = f"Stepped up\n\nWorker: {OPUS_179_AT_3}\n"
kept = "".join(self.archived_outcomes())
self.assertIn(f"Z {answered}\n20", kept)
self.assertIn(f"Z {stepped}", kept)
def test_a_quoted_answer_past_the_cap_archives_the_escalation_it_carried_with_the_interrupt_and_steps_up(self):
steps = [judged("quoted", answer="Use A.")] * 3 + [judged("quoted", answer="Use A.", escalation="working")]
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 4, claude=steps, inputs=[""])
cycle = dataclasses.replace(self.cycle(fake, records="From AGENTS.md:\nUse A."), project=OPUS_PROJECT, route=Route(SONNET, 179, 2, 5))
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(run.run_cycle(cycle), Stop(2, "the human is needed to answer the question above, but nothing was typed"))
kept = "".join(self.archived_outcomes())
self.assertIn(f"Evidence:\n> working\n\nEscalation:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {SONNET_179}\n", kept)
self.assertIn(f"Stepped up\n\nWorker: {OPUS_179_AT_6}\n", kept)
def test_a_ruling_steps_the_route_only_on_an_escalation_from_the_rung_its_taken_in_l_picks(self):
cycle = dataclasses.replace(self.cycle(FakeEffects()), project=OPUS_PROJECT, route=Route(ONE_UP, 3, 3, 4))
for ruling in (Interrupt("e", "q", entries_read=3), QuotedAnswer("e", "q", "A.", entries_read=3)):
with self.subTest(ruling=ruling):
self.assertIsNone(run.ruling_step_up(cycle, ruling))
self.assertEqual(run.ruling_step_up(cycle, Interrupt("e", "q", "round 2: 🔴", TakenIn(highest_likelihood=4), entries_read=3)), routing.StepUp(cycle.route, Route(THREE_UP, 3, 3, 4, escalated_at=3)))
def test_an_escalation_after_an_interrupt_stepped_the_worker_steps_it_again_in_the_same_cycle(self):
streams = [Worker(self.paths.claude_code_transcripts.parent, text="round 2: 🔴"), Worker(self.paths.claude_code_transcripts.parent), Worker(self.paths.claude_code_transcripts.parent)]
rulings = [judged("interrupt", evidence="round 2: 🔴", escalation="round 2: 🔴"), judged("quoted", answer="Use A.", escalation="working"), judged("finished")]
fake = FakeEffects(streams=streams, claude=rulings, inputs=["Use B."])
cycle = dataclasses.replace(self.cycle(fake, records="From AGENTS.md:\nUse A."), project=OPUS_PROJECT, route=Route(SONNET, 179, 2, 5))
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
concluded = run.run_cycle(cycle)
self.assertEqual(concluded.route, Route(ONE_UP, 179, 2, 5, escalated_at=4))
self.assertEqual([args[args.index("--effort") + 1] for args in fake.made("stream_claude")], ["high", "medium", "low"])
def test_an_escalation_quoted_from_before_the_workers_last_step_up_steps_nothing_and_a_later_one_reaches_the_verdict(self):
for escalation, verdict in (("round 2: 🔴", Unfinished("working", entries_read=4)), ("working", Unfinished("working", "working", entries_read=4))):
with self.subTest(escalation=escalation):
self.session = session.session_named(f"A proj {escalation} a-driver", self.paths.claude_code_transcripts)
streams = [Worker(self.paths.claude_code_transcripts.parent, text="round 2: 🔴"), Worker(self.paths.claude_code_transcripts.parent)]
rulings = [judged("interrupt", evidence="round 2: 🔴", escalation="round 2: 🔴"), judged("unfinished", escalation=escalation)]
fake = FakeEffects(streams=streams, claude=rulings, inputs=["Use B."])
cycle = dataclasses.replace(self.cycle(fake), project=OPUS_PROJECT, route=Route(SONNET, 179, 2, 5))
with contextlib.redirect_stderr(io.StringIO()):
concluded = run.run_cycle(cycle)
self.assertEqual(concluded, run.Concluded(verdict, Route(OPUS, 179, 2, 5, escalated_at=3)))
def test_an_answer_typed_at_the_terminal_resumes_the_worker_headless_and_the_judge_reads_it_as_the_humans(self):
worker = Worker(self.paths.claude_code_transcripts.parent, prompted=True)
fake = FakeEffects(streams=[worker, worker], claude=[judged("interrupt"), judged("unfinished")], inputs=["Use B."])
with contextlib.redirect_stderr(io.StringIO()) as said:
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Unfinished("working", entries_read=6))
self.assertIn(f"\n{STAMP} Which one?\a\n", said.getvalue())
self.assertEqual(fake.made("claude_on_terminal"), [])
resumed = fake.made("stream_claude")[1]
self.assertEqual(resumed[resumed.index("--resume") + 1], str(self.session.id))
typed = resumed[resumed.index("-p") + 1]
self.assertEqual(worker_prompts.typed_answer(typed).answer, "Use B.")
self.assertIn("Which one?", typed)
self.assertIn("Continue the task. The chunk is finished only once every answer the human typed in it", typed)
judge_prompts = [stdin for args, stdin in fake.made("claude")]
self.assertRegex(judge_prompts[1], r"(?s)\[loop\] Your question was: Which one\?.*\n\n\[human\] Use B\.\n\nworking\n\nDecide whether")
self.assertNotIn("The human answered", judge_prompts[1])
def test_an_interrupt_that_steps_the_worker_up_names_its_new_rung_in_the_prompt_that_resumes_it_with_the_typed_answer(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[judged("interrupt", escalation="working"), judged("finished")], inputs=["Use B."])
cycle = dataclasses.replace(self.cycle(fake), project=OPUS_PROJECT, route=Route(SONNET, 179, 2, 5))
with contextlib.redirect_stderr(io.StringIO()):
concluded = run.run_cycle(cycle)
self.assertEqual(concluded.route, Route(OPUS, 179, 2, 5, escalated_at=3))
first, resumed = fake.made("stream_claude")
self.assertEqual([args[args.index("--effort") + 1] for args in (first, resumed)], ["high", "medium"])
typed = resumed[resumed.index("-p") + 1]
self.assertIn("Your request for a higher rung went through: a-driver runs you on claude-opus-5-5 at medium effort.", typed)
self.assertIn(" You hold claude-opus-5-5 at medium effort.", typed)
self.assertNotIn("went through", first[first.index("-p") + 1])
def test_an_interrupt_that_steps_the_worker_up_compacts_its_session_on_the_rung_it_ran_on(self):
usage = {"input_tokens": 2, "cache_creation": {"ephemeral_5m_input_tokens": 0, "ephemeral_1h_input_tokens": 5000}, "cache_read_input_tokens": 70000, "output_tokens": 100}
worker = Worker(self.paths.claude_code_transcripts.parent, usage=usage, sent=utc(NOW - 600))
fake = FakeEffects(streams=[worker, Streamed(0), worker], claude=[judged("interrupt", escalation="working"), judged("finished")])
def typed(prompt):
for _, action in fake.timers:
action()
return "Use B."
fake.scripts["input"] = [typed]
cycle = dataclasses.replace(self.cycle(fake), project=OPUS_PROJECT, route=Route(SONNET, 179, 2, 5))
with contextlib.redirect_stderr(io.StringIO()):
run.run_cycle(cycle)
_, compaction, resumed = fake.made("stream_claude")
self.assertEqual(compaction[compaction.index("-p") + 1], "/compact")
self.assertEqual([(args[args.index("--model") + 1], args[args.index("--effort") + 1]) for args in (compaction, resumed)], [("claude-sonnet-5-5", "high"), ("claude-opus-5-5", "medium")])
def test_a_typed_answer_past_the_quoted_answers_cap_reaches_the_worker_with_its_own_question(self):
worker = Worker(self.paths.claude_code_transcripts.parent)
fake = FakeEffects(streams=[worker] * 5, claude=[judged("quoted", answer="Use A.")] * 4 + [judged("finished")], inputs=["Use B."])
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse A.")).verdict, Finished("working"))
typed = fake.made("stream_claude")[4]
self.assertIn("Your question was: Which one?\n\nContinue the task.", typed[typed.index("-p") + 1])
self.assertNotIn("Those files have answered", typed[typed.index("-p") + 1])
def test_a_release_ready_to_cut_opens_claudes_session_without_asking_for_an_answer(self):
def answer(args):
with open(session.transcript_file(self.session), "a") as handle:
handle.write(json.dumps(human("cut")) + "\n")
return 0
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt", release_ready=True), judged("finished")], on_terminal_claude=[answer])
with contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(run.run_cycle(self.cycle(fake)).verdict, Finished("working"))
self.assertEqual(fake.made("input"), [])
self.assertEqual(len(fake.made("claude_on_terminal")), 1)
def test_stops_when_an_interrupt_cannot_be_archived(self):
self.archive.parent.mkdir(parents=True)
self.archive.write_text("")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt")], inputs=["Use B."])
stop = run.run_cycle(self.cycle(fake))
self.assertRegex(stop.reason, f"^could not archive the transcript: {self.archive}/[0-9a-f]{{64}}: Not a directory$")
self.assertEqual(fake.made("input"), [])
def test_stops_when_a_quoted_answer_cannot_be_archived(self):
self.archive.parent.mkdir(parents=True)
self.archive.write_text("")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("quoted", answer="Use A.")])
with contextlib.redirect_stdout(io.StringIO()):
stop = run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse A."))
self.assertRegex(stop.reason, f"^could not archive the transcript: {self.archive}/[0-9a-f]{{64}}: Not a directory$")
self.assertEqual(len(fake.made("stream_claude")), 1)
def test_an_answer_the_records_do_not_hold_is_dropped_and_the_question_goes_to_the_human(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("quoted", answer="Use ... A.")], inputs=[""])
with contextlib.redirect_stderr(io.StringIO()) as said:
self.assertEqual(run.run_cycle(self.cycle(fake, records="From AGENTS.md:\nUse B. Use A.")), Stop(2, "the human is needed to answer the question above, but nothing was typed"))
self.assertEqual(len(fake.made("stream_claude")), 1)
self.assertIn(f"\n{STAMP} Which one?\a\n", said.getvalue())
self.assertNotIn("Use ... A.", said.getvalue())
def test_stops_keeping_the_transcript_when_the_finished_chunks_session_cannot_be_deleted(self):
assert_not_root(self)
folder = self.paths.claude_code_transcripts / "-fake" / str(self.session.id)
(folder / "subagents").mkdir(parents=True)
folder.chmod(0o500)
self.addCleanup(folder.chmod, 0o700)
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
stop = run.run_cycle(self.cycle(fake))
self.assertEqual(stop, Stop(1, f"could not delete {folder}: Permission denied; delete every file and folder matching {folder}*, then run it again; a run before that resumes the finished chunk's session"))
self.assertTrue(session.session_open(self.session))
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n\nConfig: {CONFIG}\nJudge: judge-model\nWorker: {OFF_LADDER}\n\n2026-09-21T14:13:20Z Stopped: {stop.reason}\n\nWorker: {OFF_LADDER}\n"])
def test_stops_when_the_worker_leaves_no_transcript(self):
fake = FakeEffects(streams=[Streamed(0)])
self.assertEqual(run.run_cycle(self.cycle(fake)), Stop(1, f"claude left no transcript for session {self.session.id}"))
+89
View File
@@ -0,0 +1,89 @@
import json
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from claude_code import entries
from loop_effects.doubles import FakeEffects, judged, ran
from agent_cycle import judge, spend
from agent_cycle.verdicts import Judged, Unfinished
from transcript_archive import outcomes
from claude_code.tokens import Tokens, tokens_of
def turn(message_id, model, at, **usage):
return entries.AssistantTurn(message_id=message_id, model=model, timestamp=at, usage=usage)
class Spend(unittest.TestCase):
def test_reads_the_cache_writes_by_lifetime_and_counts_one_without_the_split_as_five_minutes(self):
self.assertEqual(tokens_of({"input_tokens": 1, "cache_creation_input_tokens": 9, "cache_creation": {"ephemeral_5m_input_tokens": 2, "ephemeral_1h_input_tokens": 3}, "cache_read_input_tokens": 4, "output_tokens": 5}), Tokens(1, 2, 3, 4, 5))
self.assertEqual(tokens_of({"input_tokens": 1, "cache_creation_input_tokens": 9}), Tokens(1, 9, 0, 0, 0))
self.assertEqual(tokens_of(None), Tokens())
def test_counts_a_messages_context_as_every_prompt_token_and_no_output(self):
self.assertEqual(Tokens(1, 2, 3, 4, 5).context(), 10)
def test_counts_a_message_written_in_the_stamped_second_or_without_a_readable_time_as_before_since(self):
thread = [turn("m1", "a", "2026-11-01T10:00:01Z", output_tokens=7), turn("m2", "a", "not a time", output_tokens=3), turn("m3", "a", "2026-11-01T10:00:02Z", output_tokens=1)]
self.assertEqual(spend.spent("2026-11-01T10:00:01Z", thread, [], "j", Tokens()), {("worker", "a"): Tokens(output=1)})
def test_sums_two_subagents_of_one_name_on_one_model(self):
helpers = [spend.Subagent("Explore", [turn("s1", "b", "2026-11-01T10:00:00Z", input_tokens=3)]), spend.Subagent("Explore", [turn("s2", "b", "2026-11-01T10:00:00Z", input_tokens=4)])]
self.assertEqual(spend.spent("", [], helpers, "j", Tokens()), {("Explore", "b"): Tokens(input=7)})
def test_counts_each_message_once_by_its_id_and_only_after_since(self):
thread = [
turn("m1", "a", "2026-11-01T10:00:00Z", output_tokens=7),
turn("m2", "a", "2026-11-01T10:00:02Z", input_tokens=1, output_tokens=2),
turn("m2", "a", "2026-11-01T10:00:02Z", input_tokens=1, output_tokens=2),
turn("m3", "b", "2026-11-01T10:00:03Z", cache_read_input_tokens=5),
]
self.assertEqual(spend.thread_spend(thread, "2026-11-01T10:00:01Z"), {"a": Tokens(1, 0, 0, 0, 2), "b": Tokens(0, 0, 0, 5, 0)})
self.assertEqual(spend.thread_spend(thread, ""), {"a": Tokens(1, 0, 0, 0, 9), "b": Tokens(0, 0, 0, 5, 0)})
def test_names_each_spender_and_leaves_out_one_that_spent_nothing(self):
worker = [turn("m1", "a", "2026-11-01T10:00:00Z", output_tokens=7)]
helper = spend.Subagent('Explore "Search"', [turn("s1", "b", "2026-11-01T10:00:00Z", input_tokens=3)])
self.assertEqual(spend.spent("", worker, [helper], "j", Tokens()), {("worker", "a"): Tokens(output=7), ('Explore "Search"', "b"): Tokens(input=3)})
self.assertEqual(spend.spent("", [], [], "j", Tokens(input=1)), {("judge", "j"): Tokens(input=1)})
def test_names_a_subagent_by_its_agent_type_and_description(self):
with tempfile.TemporaryDirectory() as root:
folder = pathlib.Path(root, "s", "subagents")
folder.mkdir(parents=True)
for name, meta in [("agent-a", {"agentType": "Explore", "description": "Search the hooks"}), ("agent-b", None), ("agent-c", ["Explore"]), ("agent-d", {"description": "Läs"}), ("agent-e", {"agentType": "Ex\nConfig: x"})]:
(folder / f"{name}.jsonl").write_text(json.dumps({"type": "assistant", "message": {"id": name, "model": "m", "content": [], "usage": {"output_tokens": 1}}, "timestamp": "2026-11-01T10:00:00Z"}) + "\n")
if meta:
(folder / f"{name}.meta.json").write_text(json.dumps(meta))
found = spend.subagents(str(pathlib.Path(root, "s.jsonl")))
self.assertEqual([subagent.name for subagent in found], ['Explore "Search the hooks"', "subagent", "subagent", 'subagent "Läs"', "Ex Config: x"])
self.assertEqual(spend.spent("", [], found[:2], "j", Tokens()), {('Explore "Search the hooks"', "m"): Tokens(output=1), ("subagent", "m"): Tokens(output=1)})
self.assertEqual(spend.subagents(str(pathlib.Path(root, "none.jsonl"))), [])
def test_renders_the_spend_then_the_list_prices_of_each_model_it_knows(self):
found = {("worker", "claude-opus-5-5"): Tokens(1, 2, 3, 4, 5), ("judge", "unpriced"): Tokens(input=6)}
self.assertEqual(outcomes.spent_block("2026-11-01T10:00:00Z", found), """Spent since 2026-11-01T10:00:00Z, in tokens of input, cache write 5m, cache write 1h, cache read, output:
- worker, claude-opus-5-5: 1, 2, 3, 4, 5
- judge, unpriced: 6, 0, 0, 0, 0
List prices in dollars per million tokens of input, cache write 5m, cache write 1h, cache read, output:
- claude-opus-5-5: 4, 5, 8, 0.2, 20""")
self.assertEqual(outcomes.spent_block("", {("judge", "unpriced"): Tokens(input=6)}), "Spent since the chunk began, in tokens of input, cache write 5m, cache write 1h, cache read, output:\n- judge, unpriced: 6, 0, 0, 0, 0")
self.assertEqual(outcomes.spent_block("", {}), "")
def test_keeps_the_whole_typed_answer_that_holds_the_judges_quote_and_its_tokens(self):
shown = "[human] Use A.\n[human] No, keep the old name. Use X.\nworking: keep the new name"
typed = ("Use A.", "No, keep the old name. Use X.")
for quoted, kept in [("No, keep the old name. Use X.", "No, keep the old name. Use X."), ("keep the old name", "No, keep the old name. Use X."), ("Use", "No, keep the old name. Use X."), ("keep the new name", ""), ("No, keep the name.", "")]:
with self.subTest(quoted=quoted):
answer = json.loads(judged("unfinished", reversed=quoted).stdout)
fake = FakeEffects(claude=[ran(json.dumps({**answer, "usage": {"input_tokens": 2, "output_tokens": 3}}))])
self.assertEqual(judge.run_judge("/r", [], "", judge.Grounds(shown, shown, "", typed), fake.effects()), Unfinished("working", judged=Judged(tokens=Tokens(input=2, output=3), reversed=kept)))
if __name__ == "__main__":
unittest.main()
+68
View File
@@ -0,0 +1,68 @@
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from agent_cycle.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished
from model_routing.route import Route
from agent_cycle import routing
from model_routing.ladder import Rung
from agent_cycle import states
TYPED = states.TypedAnswer("Which one?", "Use B.")
class Transitions(unittest.TestCase):
def test_each_state_moves_on_its_events_and_only_on_them(self):
answered = states.Ruled(QuotedAnswer("e", "Which one?", "A."))
legal = (
(states.Working(), states.WorkerRan("/t"), states.Judging("/t")),
(states.Working(QuotedAnswer("e", "Which one?", "A."), 3), states.WorkerRan("/t"), states.Judging("/t", 3)),
(states.Judging("/t"), states.Ruled(Interrupt("e", "Which one?")), states.Interrupting("/t", Interrupt("e", "Which one?"))),
(states.Judging("/t", 3), states.Ruled(Interrupt("e", "Which one?")), states.Interrupting("/t", Interrupt("e", "Which one?"), 3)),
(states.Judging("/t"), answered, states.Working(QuotedAnswer("e", "Which one?", "A."), 1)),
(states.Judging("/t", 2), answered, states.Working(QuotedAnswer("e", "Which one?", "A."), 3)),
(states.Judging("/t", 3), answered, states.Interrupting("/t", QuotedAnswer("e", "Which one?", "A."), 3)),
(states.Judging("/t"), states.Ruled(Finished("e")), states.Closing("/t", Finished("e"))),
(states.Judging("/t", 3), states.Ruled(Finished("e")), states.Closing("/t", Finished("e"))),
(states.Judging("/t"), states.Ruled(Unfinished("e")), states.Closing("/t", Unfinished("e"))),
(states.Judging("/t", 3), states.Ruled(Unfinished("e")), states.Closing("/t", Unfinished("e"))),
(states.Interrupting("/t", Interrupt("e", "q")), states.HumanTalked(), states.Judging("/t")),
(states.Interrupting("/t", Interrupt("e", "q"), 3), states.HumanTalked(), states.Judging("/t", 3)),
(states.Interrupting("/t", Interrupt("e", "q")), states.HumanTyped(TYPED), states.Working(TYPED)),
(states.Interrupting("/t", Interrupt("e", "q"), 3), states.HumanTyped(TYPED), states.Working(TYPED, 3)),
(states.Working(TYPED, 3), states.WorkerRan("/t"), states.Judging("/t", 3)),
(states.Closing("/t", Finished("e")), states.Archived(), states.Ended(Finished("e"))),
(states.Closing("/t", Unfinished("e")), states.Archived(), states.Ended(Unfinished("e"))),
)
for state, event, after in legal:
with self.subTest(state=state, event=event):
self.assertEqual(states.transition(state, event), after)
every_state = (states.Working(), states.Working(QuotedAnswer("e", "Which one?", "A."), 3), states.Working(TYPED, 3), states.Judging("/t"), states.Judging("/t", 3), states.Interrupting("/t", Interrupt("e", "q")), states.Interrupting("/t", Interrupt("e", "q"), 3), states.Closing("/t", Finished("e")), states.Ended(Finished("e")))
every_event = (states.WorkerRan("/t"), states.Ruled(Interrupt("e", "Which one?")), answered, states.Ruled(Finished("e")), states.Ruled(Unfinished("e")), states.HumanTalked(), states.HumanTyped(TYPED), states.Archived())
for state in every_state:
for event in every_event:
if (state, event) not in [(state, event) for state, event, _ in legal]:
with self.subTest(state=state, event=event), self.assertRaises(ValueError):
states.transition(state, event)
def test_a_rulings_step_up_reaches_the_worker_it_resumes_through_the_answer_the_human_types_and_no_further(self):
ran_on = Rung("claude-opus-5-5", "xhigh")
stepped = routing.StepUp(Route(ran_on), Route(Rung("claude-fable-5-2", "low"), escalated_at=3))
interrupt, quoted = Interrupt("e", "q", "round 2: 🔴"), QuotedAnswer("e", "q", "A.", "round 2: 🔴")
for state, event, after in (
(states.Judging("/t"), states.Ruled(quoted, stepped), states.Working(quoted, 1, True)),
(states.Judging("/t", 3), states.Ruled(quoted, stepped), states.Interrupting("/t", quoted, 3, ran_on)),
(states.Judging("/t"), states.Ruled(interrupt, stepped), states.Interrupting("/t", interrupt, 0, ran_on)),
(states.Interrupting("/t", interrupt, 0, ran_on), states.HumanTyped(TYPED), states.Working(TYPED, 0, True)),
(states.Interrupting("/t", interrupt, 0, ran_on), states.HumanTalked(), states.Judging("/t")),
(states.Working(TYPED, 0, True), states.WorkerRan("/t"), states.Judging("/t")),
):
with self.subTest(state=state, event=event):
self.assertEqual(states.transition(state, event), after)
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -6,7 +6,7 @@ import pathlib
import tempfile
import unittest
from loop_claude.test_loop_claude import FakeEffects, NOW, ran, Streamed
from loop_effects.doubles import FakeEffects, NOW, ran, Streamed
from model_routing.ladder import Rung
from loop_effects.child_shells import Shell
from agent_cycle import stuck_workers, worker
+91
View File
@@ -0,0 +1,91 @@
import pathlib
import subprocess
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from failure import Failure
from loop_effects.doubles import FakeEffects
from agent_cycle import tmux
from agent_cycle.doubles import WINDOW
class TmuxCommands(unittest.TestCase):
def test_refuses_a_tmux_older_than_needed_naming_both_versions(self):
for found, reason in (("tmux 3.2", None), ("tmux 3.8", None), ("tmux next-3.4", None), ("tmux 10.0", None),
("tmux 3.1c", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 3.1c"),
("tmux 2.9a", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 2.9a"),
("", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found no tmux")):
with self.subTest(found=found):
fake = FakeEffects(tmux_version=found)
self.assertEqual(tmux.unusable(fake.tmux, "loop-claude"), reason)
self.assertEqual(fake.tmux_calls, [["-V"]])
def test_names_the_session_for_the_project(self):
self.assertEqual(tmux.session_name(tmux.LOOP, "claude-config"), "loop-claude-config")
# tmux reads `.` and `:` in a target as separators.
self.assertEqual(tmux.session_name(tmux.LOOP, "my.site:2 x"), "loop-my-site-2-x")
self.assertEqual(tmux.session_name(tmux.LOOP, "Acme/Billing"), "loop-Acme-Billing")
def test_lists_the_windows_without_the_waiting_mark(self):
fake = FakeEffects(windows="item-372 ?\t0\t@1\nitem-380\t1\t@2\nno-item\t0\t@5\n")
self.assertEqual(tmux.listed("loop-proj", fake.tmux), [tmux.Listed("item-372", "@1", False, asks=True), tmux.Listed("item-380", "@2", True), tmux.Listed("no-item", "@5", False)])
self.assertEqual(fake.tmux_calls, [["list-windows", "-t", "=loop-proj", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"]])
self.assertEqual(tmux.listed("loop-proj", FakeEffects().tmux), [])
failed = lambda args: subprocess.CompletedProcess(args, 1, "item-1\t0\t@1\n", "no server")
self.assertEqual(tmux.listed("loop-proj", failed), [])
def test_the_first_window_makes_the_session_and_each_later_one_joins_it(self):
fake = FakeEffects()
window = tmux.Window("loop-proj", "item-7", "")
self.assertIsNone(tmux.start_window(window, "/r", {"LOOP_CLAUDE_X": "1"}, ["/b/loop-claude", "--chunk", "item-7", "ship it;"], True, True, fake.tmux))
self.assertIsNone(tmux.start_window(window, "/r", {}, ["/b/loop-claude"], False, True, fake.tmux))
self.assertEqual(fake.tmux_calls, [
["new-session", "-d", "-s", "loop-proj", "-n", "item-7", "-c", "/r", "-e", "LOOP_CLAUDE_X=1", "-P", "-F", "#{window_id}", "--", "/b/loop-claude", "--chunk", "item-7", "ship it\;"],
["set-option", "-t", "=loop-proj:", "@loop-root", "/r"],
["set-option", "-w", "-t", "@9", "@chunk-root", "/r"],
["set-option", "-w", "-t", "@9", "remain-on-exit", "failed"],
["new-window", "-d", "-t", "=loop-proj:", "-n", "item-7", "-c", "/r", "-P", "-F", "#{window_id}", "--", "/b/loop-claude"],
["set-option", "-w", "-t", "@9", "@chunk-root", "/r"],
["set-option", "-w", "-t", "@9", "remain-on-exit", "failed"],
])
def test_says_why_tmux_started_no_window(self):
failed = lambda args: subprocess.CompletedProcess(args, 1, "", "duplicate session: loop-proj\n")
silent = lambda args: subprocess.CompletedProcess(args, 2, "", "")
window = tmux.Window("loop-proj", "item-7", "")
self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, True, failed), Failure("tmux could not start the window loop-proj:item-7: duplicate session: loop-proj"))
self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, True, silent), Failure("tmux could not start the window loop-proj:item-7: it exited 2"))
def test_fails_where_it_cannot_mark_a_session_it_made_as_the_checkouts_but_not_where_remain_on_exit_fails(self):
def answering(failing: str):
def run(args):
fails = args[:1] == ["set-option"] and failing in args
return subprocess.CompletedProcess(args, 1 if fails else 0, "" if fails else "@9\n", "no such session\n" if fails else "")
return run
window = tmux.Window("loop-proj", "item-7", "")
self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, True, answering(tmux.OWNER)), Failure("tmux started the session loop-proj but could not mark it as this checkout's: no such session"))
self.assertIsNone(tmux.start_window(window, "/r", {}, ["x"], True, True, answering("remain-on-exit")))
self.assertIsNone(tmux.start_window(window, "/r", {}, ["x"], False, True, answering(tmux.OWNER)))
def test_reads_the_checkout_the_session_names(self):
fake = FakeEffects(owner="/c/proj")
self.assertEqual(tmux.owner("loop-proj", fake.tmux), "/c/proj")
self.assertEqual(fake.tmux_calls, [["show-options", "-qv", "-t", "=loop-proj:", "@loop-root"]])
def test_marks_a_waiting_window_by_its_pane_and_clears_the_mark(self):
fake = FakeEffects()
tmux.mark(WINDOW, True, fake.tmux)
tmux.mark(WINDOW, False, fake.tmux)
tmux.close(tmux.Listed("item-7", "@3", True), fake.tmux)
self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"], ["rename-window", "-t", "%4", "item-7"], ["kill-window", "-t", "@3"]])
self.assertEqual(WINDOW.target, "loop-proj:item-7")
def test_shows_the_session_by_attaching_or_inside_tmux_by_switching(self):
self.assertEqual(tmux.show("loop-proj", False), ["attach-session", "-t", "=loop-proj"])
self.assertEqual(tmux.show("loop-proj", True), ["switch-client", "-t", "=loop-proj"])
if __name__ == "__main__":
unittest.main()
+153
View File
@@ -0,0 +1,153 @@
import contextlib
import dataclasses
import io
import json
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from claude_code import entries, session
from failure import Failure
from loop_effects.doubles import FakeEffects, NOW, ran, Streamed
from agent_cycle import judge, lingering_workers, worker
from claude_code.reply import Limited
from machine_load import memory
from model_routing.ladder import Rung
from loop_effects.child_shells import Shell
from text_files.utc_stamp import utc
class Limits(unittest.TestCase):
def test_the_worker_reports_the_latest_reset_its_stream_names(self):
rejected = [{"type": "rate_limit_event", "rate_limit_info": {"status": "rejected", "resetsAt": resets_at}} for resets_at in (NOW + 60, NOW + 600)]
allowed = {"type": "rate_limit_event", "rate_limit_info": {"status": "allowed", "resetsAt": NOW + 9000}}
limited = {"type": "result", "api_error_status": 429, "result": "You've hit your session limit"}
fake = FakeEffects(streams=[Streamed(1, rejected[1], allowed, rejected[0], limited)])
self.assertEqual(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), fake.effects()), Limited(NOW + 600, "You've hit your session limit"))
def test_a_limit_outranks_the_exit_code_and_its_message_fits_one_line(self):
def limited(**said):
return {"type": "result", "api_error_status": 429, **said}
self.assertEqual(worker.worker_outcome(-9, limited(result="limit\n resets 7pm"), []), Limited(None, "limit resets 7pm"))
self.assertEqual(worker.worker_outcome(1, limited(result=" \n"), [NOW]), Limited(NOW, "claude gave no message"))
self.assertEqual(worker.worker_outcome(0, limited(), []), Limited(None, "claude gave no message"))
self.assertEqual(worker.worker_outcome(0, None, [NOW]), Failure("claude exited 0 without a result event"))
self.assertIsNone(worker.worker_outcome(0, {"type": "result"}, [NOW]))
def test_the_judge_reports_a_limit_whatever_its_exit_code(self):
reply = json.dumps({"type": "result", "api_error_status": 429, "is_error": True, "result": "You've hit your session limit · resets 7:40pm (Europe/Stockholm)"})
fake = FakeEffects(claude=[ran(reply, code=1, stderr="limited")])
self.assertEqual(judge.run_judge("/r", [], "", judge.Grounds("", "", ""), fake.effects()), Limited(None, "You've hit your session limit · resets 7:40pm (Europe/Stockholm)"))
def test_reads_only_a_rejecting_events_whole_seconds(self):
for info, want in [({"status": "rejected", "resetsAt": NOW}, NOW), ({"status": "allowed", "resetsAt": NOW}, None), ({"status": "rejected", "resetsAt": True}, None), ({"status": "rejected"}, None), ("rejected", None)]:
with self.subTest(info=info):
self.assertEqual(entries.rejected_until({"type": "rate_limit_event", "rate_limit_info": info}), want)
class LingeringWorkers(unittest.TestCase):
RESULT = {"type": "result"}
def worker_with(self, *events, held=()):
"""Runs a worker whose claude prints events and holds shells `held`; returns the claude, what loop-claude said and the record's text."""
clock = [NOW]
streamed = Streamed(0, *events, clock=clock, held=held)
fake = FakeEffects(streams=[streamed])
effects = dataclasses.replace(fake.effects(), now=lambda: clock[0])
with tempfile.TemporaryDirectory() as folder, contextlib.redirect_stdout(io.StringIO()) as said:
# The archive folder of a project's first stop doesn't exist yet.
archive = pathlib.Path(folder, "transcripts", "project")
worker.run_worker("/r", [], {}, archive, Rung("m"), effects)
record = archive / lingering_workers.RECORD
return streamed, said.getvalue(), record.read_text() if record.exists() else ""
def shell(self, written=NOW - 1000, runs_busy_tool=False, holds_slot=False):
return Shell(pid=7, started=NOW - 1000, command="until false; do sleep 15; done", written=written, runs_busy_tool=runs_busy_tool, holds_slot=holds_slot)
def test_a_worker_that_lingers_after_its_result_loses_its_quiet_shell_and_the_stop_is_said_and_recorded_in_utc(self):
shell = self.shell()
streamed, said, record = self.worker_with(self.RESULT, None, held=[shell])
self.assertEqual(streamed.stopped, [shell])
line = f"stopped shell 7, which held a worker after its final result; it had run 1300 s and written nothing for 1300 s: {shell.command}"
self.assertEqual(said, f"<{NOW + 300}> {line}\n")
self.assertEqual(record, f"- {utc(NOW + 300)} {line}\n")
def test_a_worker_keeps_its_shells_where_it_printed_after_its_result_or_printed_none_or_no_check_came_due(self):
for events in ((self.RESULT, {"type": "assistant", "message": {"content": []}}, None), (self.RESULT,), (None,), ({"type": "assistant", "message": {"content": []}}, None)):
with self.subTest(events=events):
streamed, said, record = self.worker_with(*events, held=[self.shell()])
self.assertEqual((streamed.stopped, said, record), ([], "", ""))
def test_a_worker_keeps_every_shell_while_one_of_them_writes_output_runs_a_busy_tool_or_holds_a_heavy_slot(self):
for busy in (self.shell(written=NOW + 200), self.shell(runs_busy_tool=True), self.shell(holds_slot=True)):
with self.subTest(busy=busy):
streamed, _, _ = self.worker_with(self.RESULT, None, held=[self.shell(), busy])
self.assertEqual(streamed.stopped, [])
def test_a_worker_keeps_a_shell_that_wrote_within_the_quiet_time_and_stops_one_that_wrote_before_it(self):
# The check comes 300 s after the result, so a shell written at NOW + 300 - QUIET is the latest that is still kept.
kept = self.shell(written=NOW + 300 - lingering_workers.QUIET)
stopped = self.shell(written=NOW + 300 - lingering_workers.QUIET - 1)
self.assertEqual(self.worker_with(self.RESULT, None, held=[kept])[0].stopped, [])
self.assertEqual(self.worker_with(self.RESULT, None, held=[stopped])[0].stopped, [stopped])
def test_a_worker_reads_on_past_a_check_that_found_it_still_working(self):
clock = [NOW]
effects = dataclasses.replace(FakeEffects(streams=[Streamed(0, None, self.RESULT, clock=clock)]).effects(), now=lambda: clock[0])
self.assertIsNone(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), effects))
def test_a_stop_that_cannot_be_recorded_is_said_on_stderr_and_every_shell_is_still_stopped(self):
clock = [NOW]
held = [self.shell(), self.shell()]
streamed = Streamed(0, self.RESULT, None, clock=clock, held=held)
effects = dataclasses.replace(FakeEffects(streams=[streamed]).effects(), now=lambda: clock[0])
with tempfile.NamedTemporaryFile() as blocker, contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()) as failed:
worker.run_worker("/r", [], {}, pathlib.Path(blocker.name), Rung("m"), effects)
self.assertEqual(streamed.stopped, held)
self.assertIn("could not record the stop", failed.getvalue())
def test_every_stop_of_a_check_is_recorded_though_the_archive_folder_exists_after_the_first(self):
held = [self.shell(), self.shell()]
_, said, record = self.worker_with(self.RESULT, None, held=held)
self.assertEqual(record.count("stopped shell 7"), 2)
class MemoryWait(unittest.TestCase):
def test_a_worker_starts_only_once_a_quarter_of_memory_is_available(self):
fake = FakeEffects(free_memory=[memory.Memory(available=3, total=16), memory.Memory(available=3, total=16), memory.Memory(available=4, total=16)], streams=[Streamed(0)])
with contextlib.redirect_stderr(io.StringIO()) as said:
self.assertIsNone(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), fake.effects()))
self.assertEqual([seam for seam, _ in fake.calls], ["memory", "sleep", "memory", "sleep", "memory", "stream_claude"])
self.assertEqual(fake.made("sleep"), [worker.MEMORY_POLL] * 2)
self.assertEqual(said.getvalue().count("waiting"), 1, said.getvalue())
def test_a_worker_starts_at_once_where_memory_cannot_be_read(self):
fake = FakeEffects(free_memory=[None], streams=[Streamed(0)])
self.assertIsNone(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), fake.effects()))
self.assertEqual([seam for seam, _ in fake.calls], ["memory", "stream_claude"])
class WorkerArgs(unittest.TestCase):
OPENED = session.session_named("A proj a-driver", pathlib.Path("/h/.claude/projects"))
def test_opens_a_fresh_session_by_name_and_resumes_an_open_one(self):
fresh = worker.worker_args(self.OPENED, Rung("m"), "Do it.", resume=False)
self.assertEqual(fresh[fresh.index("--session-id") + 1], str(self.OPENED.id))
self.assertEqual(fresh[fresh.index("--name") + 1], "A proj a-driver")
resumed = worker.worker_args(self.OPENED, Rung("m"), "Continue the task.", resume=True)
self.assertEqual(resumed[resumed.index("--resume") + 1], str(self.OPENED.id))
self.assertNotIn("--session-id", resumed)
self.assertEqual(resumed[resumed.index("-p") + 1], "Continue the task.")
def test_disallows_a_wakeup_so_a_worker_that_ends_its_turn_exits(self):
for resume in (False, True):
with self.subTest(resume=resume):
args = worker.worker_args(self.OPENED, Rung("m"), "Do it.", resume=resume)
self.assertEqual(args[args.index("--disallowedTools") + 1], "ScheduleWakeup")
if __name__ == "__main__":
unittest.main()
+33
View File
@@ -0,0 +1,33 @@
import dataclasses
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from agent_cycle.doubles import PROMPTS
from model_routing.route import Route
from model_routing.ladder import Rung
from agent_cycle import worker_prompts
class TypedAnswers(unittest.TestCase):
def test_splits_the_prompt_handing_over_the_answer_into_the_answer_on_the_line_after_its_marker_and_the_rest_and_no_other_prompt(self):
route = Route(Rung("worker-model"))
marked = "The human answered your question at a-driver's terminal:\nB"
typed = worker_prompts.typed_answer(PROMPTS.typed("Which one?", "Use B.", route, None, False))
self.assertEqual(typed.answer, "Use B.")
self.assertTrue(typed.loop_words.startswith("Your question was: Which one?\n\nContinue the task."))
self.assertEqual(worker_prompts.typed_answer(PROMPTS.typed(f"Which one?\n{marked}", "Use B.", route, None, False)).answer, "Use B.")
# A ruling that stepped the worker up adds the sentences naming its new rung.
self.assertEqual(worker_prompts.typed_answer(PROMPTS.typed("Which one?", "Use B.", Route(Rung("claude-sonnet-5-5", "high"), 179, 2, 5), None, True)).answer, "Use B.")
self.assertIsNone(worker_prompts.typed_answer(PROMPTS.answered("Which one?", marked, route, None, False)))
self.assertIsNone(worker_prompts.typed_answer(PROMPTS.resumed(marked, route, None)))
def test_the_typed_answers_prompt_names_the_entry_points_answer_destinations(self):
ledger = dataclasses.replace(PROMPTS, answer_destinations="the ledger")
self.assertIn("is recorded in the ledger at the one place it governs.", ledger.typed("Which one?", "Use B.", Route(Rung("worker-model")), None, False))
if __name__ == "__main__":
unittest.main()
+2 -2
View File
@@ -1,4 +1,4 @@
"""The verdicts cycle.judge.verdict_from builds from the judge's reply; cycle.run.rule then stamps every verdict but Finished with the items the chunk takes. They sit outside cycle/, since routing reads them and cycle imports routing."""
"""The verdicts judge.verdict_from builds from the judge's reply; run.rule then stamps every verdict but Finished with the items the chunk takes. They sit in a module of their own, since routing.py reads them and run.py imports routing.py."""
import dataclasses
from transcript_archive.judge_run import JudgeRun
@@ -7,7 +7,7 @@ from claude_code.tokens import Tokens
@dataclasses.dataclass(frozen=True)
class TakenIn:
"""The highest L, and the highest E + I, among the plan items the worker names on its latest `Chunk items:` line, as cycle.chunk_items reads them. The item the chunk was routed on counts only where the worker names it."""
"""The highest L, and the highest E + I, among the plan items the worker names on its latest `Chunk items:` line, as the entry point reads them (run.py's Cycle.taken_in). The item the chunk was routed on counts only where the worker names it."""
# The highest L among them, 0 for none: an escalation steps up from the rung that L picks, and an unfinished verdict also raises the worker to it.
highest_likelihood: int = 0
# The highest E + I among them, 0 for none: after an unfinished verdict, it routes the next cycle's reviewers.
+170
View File
@@ -0,0 +1,170 @@
"""What every reader of Jira Cloud's REST API version 3 shares: a search read page by page, the key, fields, "is blocked by" links and property of each issue it finds, checked for their shape, and the transition that finishes an issue."""
import dataclasses
import re
import typing
from failure import Failure
from project_config.settings import Finish, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION
from atlassian.client import Http
ISSUE_KEY = re.compile(r"([A-Z][A-Z0-9_]*)-([0-9]+)")
PAGE = 100
# The pages one read asks for at most, so an answer that never names a last page ends the read.
MAX_PAGES = 100
# The key of the status category Jira Cloud gives every done status.
DONE = "done"
# How a Failure names an answer in which a field is missing or of another type.
SHAPE = "is not in the shape Jira Cloud's REST API version 3 documents"
# What a failed create ends with: a request Jira never answered, or answered brokenly, may still have filed the issue.
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under {under} before filing it again"
# What a failed move ends with, for the same reason.
PERHAPS_MOVED = "unless Jira refused the request, the issue may have moved: look at {key} before finishing the item again"
@dataclasses.dataclass(frozen=True)
class Link:
"""One "is blocked by" link of an issue."""
# The key of the issue that blocks.
blocker: str
# The link's own id, which Jira deletes it by.
id: str
@dataclasses.dataclass(frozen=True)
class Entry:
"""The key, fields, links and property of one issue a search finds, which every reader checks alike."""
key: str
# The entry's `fields`, from which each reader takes the ones it asked for.
fields: dict[str, typing.Any]
links: tuple[Link, ...]
# The issue property the search asked for; empty where the issue has none. A reader checks that it is a JSON object.
held: object
def entry_of(raw: object, property_: str) -> Entry | Failure:
"""The Entry of one issue a search finds, or which issue is not in the shape Jira documents, and how. property_ names the issue property the search asked for."""
key = raw.get("key") if isinstance(raw, dict) else None
if not isinstance(raw, dict) or not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
return Failure("the search answered an issue without an issue key")
# Each nested field is checked as Jira documents it; one missing or of another type ends the read here, as a Failure, so no later step meets it.
fields, properties = raw.get("fields"), raw.get("properties", {})
if not isinstance(fields, dict):
return Failure(f"{key} {SHAPE}: its fields are no object")
if not isinstance(properties, dict):
return Failure(f"{key} {SHAPE}: its properties are no object")
links = blocked_by_links(fields)
if isinstance(links, Failure):
return Failure(f"{key} {SHAPE}: {links.reason}")
return Entry(key, fields, links, properties.get(property_, {}))
def unshaped(key: str, failure: Exception) -> str:
"""Why the issue named key cannot be read, for a field that is missing or of another type than Jira documents."""
return f"{key} {SHAPE}: {failure!r}"
def offer_of(raw: object) -> tuple[str, str, str, str] | None:
"""One transition Jira offers, as its ID, its name, the name of the status it leads to and the key of that status's category; None where raw holds no such transition."""
to = raw.get("to") if isinstance(raw, dict) else None
category = to.get("statusCategory") if isinstance(to, dict) else None
if not isinstance(raw, dict) or not isinstance(to, dict) or not isinstance(category, dict):
return None
id_, name, status, key = raw.get("id"), raw.get("name"), to.get("name"), category.get("key")
return (id_, name, status, key) if isinstance(id_, str) and isinstance(name, str) and isinstance(status, str) and isinstance(key, str) else None
def status_category(issue: object) -> str | Failure:
"""The key of the category of the status in Jira's answer about one issue, or how the answer is not in the shape Jira documents."""
fields = issue.get("fields") if isinstance(issue, dict) else None
status = fields.get("status") if isinstance(fields, dict) else None
category = status.get("statusCategory") if isinstance(status, dict) else None
key = category.get("key") if isinstance(category, dict) else None
return key if isinstance(key, str) else Failure("its status category has no key that is text")
def finishing_transition(site: str, key: str, finish: Finish, authorization: str, http: Http) -> str | Failure:
"""The ID of the transition finish names, as Jira offers it now for the issue named key, which is a plan item or an epic; or why none of those offered finishes it."""
offered = http("GET", f"{site}/rest/api/3/issue/{key}/transitions", authorization, None)
if isinstance(offered, Failure):
return Failure(f"cannot read {key}'s transitions: {offered.reason}")
return transition_of(offered, key, finish)
def move_refusal(site: str, key: str, transition: str, finish: Finish, authorization: str, http: Http) -> str | None:
"""Moves the issue named key by the transition with the ID transition, which finishing_transition answered for finish. Why Jira did not take the move; None where it did."""
moved = http("POST", f"{site}/rest/api/3/issue/{key}/transitions", authorization, {"transition": {"id": transition}})
return f"cannot move {key} by {finish.transition!r}: {moved.reason}" if isinstance(moved, Failure) else None
def transition_of(offered: object, key: str, finish: Finish) -> str | Failure:
"""The ID of the transition finish names, among those Jira offered for the issue named key, which are the ones its present status has; or why none of them finishes the item."""
raw = offered.get("transitions") if isinstance(offered, dict) else None
found = [offer for each in raw if (offer := offer_of(each))] if isinstance(raw, list) else []
if not isinstance(raw, list) or len(found) != len(raw):
return Failure(f"Jira answered {key}'s transitions in a shape other than the one it documents")
named = [each for each in found if each[1] == finish.transition]
if not named:
others = " and ".join(f"{name!r} to {status!r}" for _, name, status, _ in found)
return Failure(f"{key} has no transition {finish.transition!r} from its status, " + (f"only {others}: move the issue in Jira to a status that has it, or name in `{JIRA_FINISH_TRANSITION}` one that leads to a done status" if found else "nor any other the account may make"))
id_, _, status, category = next((each for each in named if each[2] == finish.status), named[0])
if status != finish.status:
return Failure(f"{key}'s transition {finish.transition!r} leads to the status {status!r}, not to {finish.status!r}, which `{JIRA_FINISH_STATUS}` names")
# `children` reads an issue as finished by its status's category, so a move to any other category would leave the item in the plan.
if category != DONE:
return Failure(f"{key}'s transition {finish.transition!r} leads to the status {status!r}, which Jira doesn't count as done, so the item would stay in the plan: name a transition to a done status")
return id_
def project_of(key: str) -> str:
"""The key of the Jira project an issue key lies in."""
return key.rpartition("-")[0]
def rank_refusal(site: str, key: str, before: str, authorization: str, http: Http) -> str | None:
"""Ranks the issue named key just before the issue named before; why Jira refused, or None where it didn't. Jira answers a rank it made with no body, and one it refused with an error, or with an entry per issue that holds the errors."""
answer = http("PUT", f"{site}/rest/agile/1.0/issue/rank", authorization, {"issues": [key], "rankBeforeIssue": before})
if isinstance(answer, Failure):
return answer.reason
entries = answer.get("entries") if isinstance(answer, dict) else None
refused = [entry for entry in entries if isinstance(entry, dict) and entry.get("status") != 200] if isinstance(entries, list) else []
return "; ".join(str(error) for entry in refused for error in entry.get("errors") or [f"Jira answered {entry.get('status')}"]) or None
def searched(site: str, what: str, jql: str, asked: dict[str, object], authorization: str, http: Http, max_pages: int = MAX_PAGES) -> list[object] | Failure:
"""Every issue a JQL search on site finds, page by page, up to max_pages pages. The dict asked holds the request's `fields` and `properties` lists. The text what names the issues sought, in the Failure of a search that doesn't end."""
body: dict[str, object] = {"jql": jql, "maxResults": PAGE} | asked
found: list[object] = []
for _ in range(max_pages):
page = http("POST", f"{site}/rest/api/3/search/jql", authorization, body)
if isinstance(page, Failure):
return page
if not isinstance(page, dict) or not isinstance(page.get("issues"), list):
return Failure("the search answered something other than a page of issues")
found += page["issues"]
if not page.get("nextPageToken"):
return found
body = body | {"nextPageToken": page["nextPageToken"]}
return Failure(f"the search for {what} did not end within {max_pages} pages")
def blocked_by_links(fields: dict[str, typing.Any]) -> tuple[Link, ...] | Failure:
"""The "is blocked by" links among an issue's `issuelinks` field, or how that field is not in the shape Jira documents."""
listed = fields.get("issuelinks")
if not isinstance(listed, list):
return Failure("its issuelinks are no list")
found = []
for link in listed:
if not isinstance(link, dict):
return Failure("its issuelinks hold a link that is no object")
if "inwardIssue" not in link:
continue
kind = link.get("type")
if not isinstance(kind, dict) or not isinstance(kind.get("inward"), str):
return Failure("its issuelinks hold a link whose type names no inward text")
if kind["inward"] != "is blocked by":
continue
blocker, id_ = link["inwardIssue"], link.get("id")
if not isinstance(blocker, dict) or not isinstance(blocker.get("key"), str) or not isinstance(id_, str):
return Failure("its issuelinks hold a link whose id or whose linked issue's key is no text")
found.append(Link(blocker["key"], id_))
return tuple(found)
+1 -1
View File
@@ -10,7 +10,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from atlassian import client
from rest_client.json_request import ErrorAnswer
from failure import Failure
from rest_client.test_json_request import Served
from rest_client.doubles import Served
AUTHORIZATION = "Basic " + base64.b64encode(b"me@example.com:s3cret").decode()
+74
View File
@@ -0,0 +1,74 @@
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from atlassian.jira import Entry, entry_of, Link
from failure import Failure
from atlassian import jira
BLOCKED = {"id": "9", "type": {"inward": "is blocked by"}, "inwardIssue": {"key": "ABC-2"}}
SHAPE = "ABC-1 is not in the shape Jira Cloud's REST API version 3 documents: "
def raw(*links: object, **more: object) -> dict[str, object]:
return {"key": "ABC-1", "fields": {"issuelinks": list(links)}} | more
class TheHeadOfAnEntry(unittest.TestCase):
def test_holds_the_key_the_fields_the_blocked_by_links_and_the_property_asked_for(self):
outward = {"id": "3", "type": {"inward": "is blocked by"}, "outwardIssue": {"key": "ABC-3"}}
relates = {"id": "4", "type": {"inward": "relates to"}, "inwardIssue": {"key": "ABC-4"}}
found = entry_of(raw(BLOCKED, outward, relates, properties={"plan": {"L": 2}}), "plan")
self.assertEqual(found, Entry("ABC-1", {"issuelinks": [BLOCKED, outward, relates]}, (Link("ABC-2", "9"),), {"L": 2}))
self.assertEqual(entry_of(raw(), "plan"), Entry("ABC-1", {"issuelinks": []}, (), {}))
def test_finds_a_blocked_by_link_listed_after_a_link_of_another_kind(self):
outward = {"id": "3", "type": {"inward": "is blocked by"}, "outwardIssue": {"key": "ABC-3"}}
relates = {"id": "4", "type": {"inward": "relates to"}, "inwardIssue": {"key": "ABC-4"}}
for other in (outward, relates):
with self.subTest(other=other):
self.assertEqual(jira.blocked_by_links({"issuelinks": [other, BLOCKED]}), (Link("ABC-2", "9"),))
def test_an_entry_without_an_issue_key_is_a_failure(self):
for wrong in ([], {}, {"key": 7}, {"key": "abc"}):
with self.subTest(wrong=wrong):
self.assertEqual(entry_of(wrong, "plan"), Failure("the search answered an issue without an issue key"))
def test_an_entry_in_another_shape_is_a_failure_naming_the_issue_and_the_field(self):
cases = {
"its fields are no object": {"key": "ABC-1"},
"its properties are no object": raw(properties=[]),
"its issuelinks are no list": {"key": "ABC-1", "fields": {}},
"its issuelinks hold a link that is no object": raw("link"),
"its issuelinks hold a link whose type names no inward text": raw(BLOCKED | {"type": None}),
"its issuelinks hold a link whose id or whose linked issue's key is no text": raw(BLOCKED | {"inwardIssue": {"key": 7}}),
}
for how, wrong in cases.items():
with self.subTest(how=how):
self.assertEqual(entry_of(wrong, "plan"), Failure(SHAPE + how))
self.assertEqual(entry_of(raw(BLOCKED | {"id": 9}), "plan"), Failure(SHAPE + "its issuelinks hold a link whose id or whose linked issue's key is no text"))
self.assertEqual(entry_of(raw(BLOCKED | {"inwardIssue": None}), "plan"), Failure(SHAPE + "its issuelinks hold a link whose id or whose linked issue's key is no text"))
def test_a_fault_in_reading_the_links_is_not_reported_as_jiras(self):
class Faulty(dict):
def get(self, key, default=None):
raise TypeError("ours")
with self.assertRaisesRegex(TypeError, "ours"):
entry_of({"key": "ABC-1", "fields": Faulty()}, "plan")
class TheCategoryOfAnIssuesStatus(unittest.TestCase):
def test_is_the_key_jira_gives_the_category(self):
self.assertEqual(jira.status_category({"fields": {"status": {"statusCategory": {"key": "done"}}}}), "done")
def test_an_answer_in_another_shape_is_a_failure_naming_the_field(self):
for wrong in (["ABC-1"], {}, {"fields": []}, {"fields": {}}, {"fields": {"status": "Done"}}, {"fields": {"status": {}}}, {"fields": {"status": {"statusCategory": "done"}}}, {"fields": {"status": {"statusCategory": {}}}}, {"fields": {"status": {"statusCategory": {"key": 3}}}}):
with self.subTest(wrong=wrong):
self.assertEqual(jira.status_category(wrong), Failure("its status category has no key that is text"))
if __name__ == "__main__":
unittest.main()
+7 -8
View File
@@ -5,7 +5,6 @@ import subprocess
import sys
import tempfile
import unittest
import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
@@ -13,7 +12,7 @@ from project_config.templates import BARE, Titles
from branch_chunks.fixes import blamed, fixing_changes, removed, removed_work, Traced, with_fixes
from branch_chunks.cut import Branch, chunk_log, Commit, cut_branch, landing, NOT_WORK, shipped, Span
from branch_chunks.read import branch_chunks, traced_to, unsupported
from project_config.test_project_config import configured
from project_config.doubles import configured
from failure import Failure
from git_cli.git_repo import git
from test_env import isolated_env
@@ -207,13 +206,14 @@ class Blamed(unittest.TestCase):
class Unsupported(unittest.TestCase):
def test_says_which_files_the_config_puts_where_no_commits_are_read(self):
checkout = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory())).resolve()
self.assertIsNone(unsupported(str(checkout), checkout), "no git checkout, so no config names it")
run = functools.partial(git, env=isolated_env())
self.assertIsNone(unsupported(str(checkout), checkout, run), "no git checkout, so no config names it")
folder = configured(self, checkout)
home = folder.parents[1]
self.assertIsNone(unsupported(str(checkout), home))
self.assertIsNone(unsupported(str(checkout), home, run))
config = folder / "config.toml"
config.write_text(config.read_text().replace('"checkout:todo.md"', '"folder:todo.md"').replace('"checkout:docs/reading-cost.md"', '"checkout:records/reading-cost.md"'))
self.assertEqual(unsupported(str(checkout), home), f"the project's config puts {folder}/todo.md, {checkout}/records/reading-cost.md elsewhere than todo.md, docs/comprehension-history.md, docs/reading-cost.md in the checkout, the only paths whose commits are read so far")
self.assertEqual(unsupported(str(checkout), home, run), f"the project's config puts {folder}/todo.md, {checkout}/records/reading-cost.md elsewhere than todo.md, docs/comprehension-history.md, docs/reading-cost.md in the checkout, the only paths whose commits are read so far")
class TracedTo(unittest.TestCase):
@@ -257,9 +257,8 @@ class BranchChunks(unittest.TestCase):
return self.git("rev-parse", "--short", "HEAD").strip()
def read(self, run=None, titles=BARE):
"""branch_chunks over main, run by git_repo's git, or else by run."""
with unittest.mock.patch.dict(os.environ, self.env, clear=True):
return branch_chunks("main", functools.partial(run or git, cwd=str(self.checkout)), titles)
"""branch_chunks over main, run by git_repo's git, or else by run, in the test's own environment."""
return branch_chunks("main", functools.partial(run or git, cwd=str(self.checkout), env=self.env), titles)
def fixing_history(self):
"""Merges seven chunks onto main, then checks out a branch off it. Returns each of main's chunks by label, with the chunks that fixed it.
+4
View File
@@ -0,0 +1,4 @@
"""What the entry points that drive chunks share: loop-claude and do-epic.
Each runs a chunk as cycles of bin/agent_cycle/. cycle_start.py starts a cycle and chunk_run.py closes it, on the Loop this_loop.py holds, once claims.py has claimed the checkout. prompts.py words what every worker is told, project/ reads a checkout's premises, records and saved models, and drivers.py names the entry points. No module here imports an entry point's package, and bin/repo_checks/test_package_cycles.py fails on such an import.
"""
+73
View File
@@ -0,0 +1,73 @@
"""One chunk's run across its cycles, for every entry point that drives chunks: its state (Run, ChunkRun), and how a cycle closes: the run the next cycle starts from, the step-up an unfinished verdict makes archived, and the stop once a chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
import dataclasses
import pathlib
import typing
from agent_cycle.run import archived_step, archived_stop, Concluded
from agent_cycle.verdicts import Finished
from chunk_driver.this_loop import Loop
from agent_cycle.stop import needs_human, Stop
from agent_cycle.routing import next_route, step_up
from model_routing.ladder import ProjectRung
from model_routing.route import Route
from claude_code.session import transcript_file
# How many unfinished verdicts running end the loop, so a worker and a judge that disagree about a chunk cannot cycle unattended without end. A policy number, not derived from the routing steps: well above the two unfinished verdicts running that archived chunks have reached at most, and low enough that a stuck pair is stopped after six cycles. It counts every unfinished verdict, including those of cycles where the human answered an interrupt: a rerun is cheap, and a chunk that needs that many answers is worth a look. The human confirmed both the number and what it counts.
UNFINISHED_LIMIT = 6
class Run(typing.Protocol):
"""What closing a cycle reads and replaces in an entry point's run of one chunk. ChunkRun below is one; loop-claude's LoopRun is another, since it hands the same fields on at a reload."""
# Every run is a dataclass, which is what lets closing a cycle replace its fields.
__dataclass_fields__: typing.ClassVar[dict[str, typing.Any]]
@property
def unfinished_evidence(self) -> str: ...
@property
def route(self) -> Route | None: ...
@property
def unfinished_run(self) -> int: ...
@dataclasses.dataclass(frozen=True)
class ChunkRun:
"""The run of one chunk, for an entry point that never hands it to another process, such as do-epic."""
cycle_number: int = 0
# The evidence of the latest unfinished verdict; empty before the chunk's first one, and once it finishes.
unfinished_evidence: str = ""
# Set by each unfinished verdict; None before the chunk's first one, and once it finishes.
route: Route | None = None
# How many unfinished verdicts the chunk has had running.
unfinished_run: int = 0
def closed_cycle[R: Run](run: R, task: str, project: ProjectRung | None, concluded: Concluded, archive_folder: pathlib.Path, driver: str, loop: Loop) -> R | Stop:
"""The run after a cycle of driver, the entry point, concluded on the chunk's task, with the step-up archived; the stop that needs the human, archived, once the chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
after = after_verdict(run, project, concluded)
# A finished chunk drops its route, so it has no step-up, and its run of unfinished verdicts restarts.
if after.route is None:
return after
# The unfinished verdict, archived as the cycle closed, names the route its cycle ended on; the step-up names the one the next cycle runs on.
path = transcript_file(loop.session)
if path and (failure := archived_step(archive_folder, path, task, step_up(concluded.route, after.route), loop.effects)):
return Stop(1, failure.reason)
if limit := reached_limit(after, driver):
# Archived with the route the next cycle would run on, so a rerun resumes there.
return archived_stop(archive_folder, loop.session, task, after.route, limit, limit.reason, loop.effects)
return after
def reached_limit(run: Run, driver: str) -> Stop | None:
"""The stop that needs the human once the run holds UNFINISHED_LIMIT unfinished verdicts running."""
if run.unfinished_run < UNFINISHED_LIMIT:
return None
return needs_human("go on with the chunk", f"the judge ruled it unfinished {run.unfinished_run} cycles running; finish the chunk yourself, or rerun {driver} to give it {UNFINISHED_LIMIT} more cycles")
def after_verdict[R: Run](run: R, project: ProjectRung | None, concluded: Concluded) -> R:
"""The run the next cycle starts from. A finished chunk drops its route and its run of unfinished verdicts. An unfinished one takes the route routing.next_route makes of the route its cycle ended on, which an interrupt's or a quoted answer's escalation may have raised, and the evidence the judge gave for the verdict; and counts the verdict."""
verdict = concluded.verdict
if isinstance(verdict, Finished):
return dataclasses.replace(run, route=None, unfinished_evidence="", unfinished_run=0)
route = next_route(project, concluded.route, verdict)
return dataclasses.replace(run, route=route, unfinished_evidence=verdict.evidence, unfinished_run=run.unfinished_run + 1)
+68
View File
@@ -0,0 +1,68 @@
"""How entry points claim a checkout before they start a chunk in it, so that two never start the same chunk and do-epic never starts one where loop-claude runs.
An entry point holds `starting` on `project_lock` while it lists the windows of the checkout's chunks and starts new ones. Where the lock cannot be taken, `starting` yields a Failure, and the entry point doesn't start any chunk. It starts each chunk in a tmux window with CHUNK_FLAG, and hands that window the variables named with HANDED.
"""
import contextlib
import fcntl
import pathlib
import typing
from failure import Failure
from transcript_archive.project_key import project_key
from agent_cycle import tmux
# Under the home folder, a folder per project key holds LOCK.
CLAIMS = ".local/state/loop-claude"
# The file an entry point keeps locked with flock while it lists the windows of a project's chunks and starts new ones. Two loops whose chunks finish together then don't start a chunk twice, and do-epic, which holds the same lock, never starts a chunk in a checkout while loop-claude does.
LOCK = "starting"
# The flag that tells the entry point in a window its chunk, as `--chunk item-372` for loop-claude and `--chunk ABC-34 ABC-12` for do-epic. Nobody types it, so no panel picked it.
CHUNK_FLAG = "--chunk"
# The prefix of the variables both entry points read. A window gets the tmux server's environment, not the starting terminal's, so they are handed to each window by name.
HANDED = "LOOP_CLAUDE_"
@contextlib.contextmanager
def starting(folder: pathlib.Path) -> typing.Iterator[Failure | None]:
"""Holds the LOCK in folder: project_lock's for a project, or do-epic's for an epic. It yields a Failure, and doesn't hold any lock, where the file cannot be opened or locked."""
with locked(folder / LOCK) as failure:
yield failure
@contextlib.contextmanager
def locked(file: pathlib.Path) -> typing.Iterator[Failure | None]:
"""Holds a flock on file, which it makes with its folder. It yields a Failure, and doesn't hold any lock, where the file cannot be opened or locked."""
try:
file.parent.mkdir(parents=True, exist_ok=True)
lock = open(file, "a")
except OSError as failure:
yield Failure(f"cannot lock {failure.filename or file}: {failure.strerror}")
return
with lock:
try:
fcntl.flock(lock, fcntl.LOCK_EX)
except OSError as failure:
yield Failure(f"cannot lock {file}: {failure.strerror}")
return
yield None
def project_lock(root: str, home: pathlib.Path) -> pathlib.Path:
"""The folder of the LOCK that everything starting a chunk in the checkout root holds."""
return home / CLAIMS / project_key(root, home)
def loop_session(root: str, home: pathlib.Path) -> str:
"""The tmux session that holds the windows of loop-claude's chunks of the checkout root."""
return tmux.session_name(tmux.LOOP, project_key(root, home))
def loop_chunks(root: str, home: pathlib.Path, run: tmux.Tmux) -> list[tmux.Listed]:
"""The windows of loop-claude's chunks that run in the checkout root. A session that runs another checkout's chunks under the same name doesn't hold any."""
session = loop_session(root, home)
running = [window for window in tmux.listed(session, run) if not window.dead]
return running if running and tmux.owner(session, run) == root else []
def epic_chunks(root: str, run: tmux.Tmux) -> list[tmux.Placed]:
"""The windows of do-epic's sessions whose chunks run in the checkout root. A parked chunk doesn't run."""
return [placed for placed in tmux.everywhere(run) if placed.session.startswith(f"{tmux.EPIC}-") and placed.root == root and not placed.window.dead and not placed.window.parked]
+114
View File
@@ -0,0 +1,114 @@
"""How a cycle starts, for every entry point that drives chunks: the models and the ladder it routes on, the route its worker runs on, and the Cycle built from them.
An entry point starts a cycle in this order:
1. It reads what it alone knows, such as the premises, the records and the task.
2. `project_routing` or `routing_on` reads the models and the ladder.
3. `announced_route` picks the route and prints the cycle's opening line.
4. `cycle_of` builds the Cycle.
Each read is made afresh every cycle, so an edit between two cycles takes effect in the next.
"""
import dataclasses
import pathlib
import typing
from agent_cycle.run import Cycle
from claude_code.entries import Entry
from failure import Failure
from agent_cycle.routing import first_route, judge_rung, kept, Scored
from chunk_driver.project.model_choice import get_or_ask_models, Models
from git_cli.git_repo import Git
from model_routing.ladder import Ladder, ProjectRung, Rung
from transcript_archive.read import latest_worker
from chunk_driver.this_loop import Loop
from transcript_archive.project_key import project_key
from model_routing.ladder_file import project_ladder_file, read_ladder
from model_routing.route import Route, route_from
from loop_effects.effects import say
from agent_cycle.stop import Stop
from agent_cycle.verdicts import TakenIn
from claude_code.session import transcript_file
from agent_cycle.worker_prompts import WorkerPrompts
@dataclasses.dataclass(frozen=True)
class Routing:
"""What a cycle routes on, as read when it started."""
# The project's worker and judge models.
models: Models
ladder: Ladder
# The file the ladder was read from: the shipped one, or the project's own.
ladder_file: pathlib.Path
@property
def project(self) -> ProjectRung | None:
"""The project's rung on the ladder; None for a worker model off the ladder."""
return self.ladder.project_rung(self.models.worker)
def project_routing(loop: Loop, driver: str) -> Routing | Stop:
"""The routing of the project checked out at the loop's root: its models, asked of the human where none is saved, and its ladder. The stop where one cannot be read."""
models = get_or_ask_models(project_key(loop.root, loop.paths.home), loop.paths, loop.env, driver, loop.effects)
if isinstance(models, Stop):
return models
ladder_file = project_ladder_file(loop.root, loop.paths.home)
return Stop(1, ladder_file.reason) if isinstance(ladder_file, Failure) else routing_on(models, ladder_file)
def routing_on(models: Models, ladder_file: pathlib.Path) -> Routing | Stop:
"""The routing of models on the ladder ladder_file holds; the stop where it cannot be read."""
ladder = read_ladder(ladder_file)
return Stop(1, ladder.reason) if isinstance(ladder, Failure) else Routing(models, ladder, ladder_file)
def announced_route(cycle_number: int, kept_route: Route | None, routing: Routing, archive_folder: pathlib.Path, item: typing.Callable[[], Scored | None], loop: Loop) -> Route:
"""The route cycle cycle_number runs on, printed with the cycle's opening line. kept_route is the one the chunk's run holds, and item finds what routes a fresh chunk."""
route = routed(kept_route, routing.project, routing.models.worker, lambda: archived_route(archive_folder, loop), item)
say(f"\n=== cycle {cycle_number}, worker on {route.rung} ===", loop.effects, stdout=True)
return route
def routed(route: Route | None, project: ProjectRung | None, model: str, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Scored | None]) -> Route:
"""The route the cycle runs on. project is the project's rung, and model the project's model.
Where project is None, the model is off the ladder, and the worker runs on it. Otherwise the chunk keeps route, or on a rerun the route its archive last names, as routing.kept allows. Else what item finds routes the chunk afresh.
"""
if project is None:
return Route(Rung(model))
route = route or archived()
return route if route and kept(route, project) else first_route(project, item())
def archived_route(archive_folder: pathlib.Path, loop: Loop) -> Route | None:
"""The route named by the latest verdict archived for the loop's open chunk; None for a chunk not yet judged."""
transcript = transcript_file(loop.session)
worker = latest_worker(archive_folder, transcript) if transcript else None
if isinstance(worker, Failure):
say(f"routing the chunk afresh: {worker.reason}", loop.effects)
return None
route = route_from(worker) if worker else None
if worker and route is None:
say(f"routing the chunk afresh: the archive's latest Worker: line names no route that can be read: {worker}", loop.effects)
return route
def cycle_of(loop: Loop, routing: Routing, route: Route, archive_folder: pathlib.Path, *, task: str, facts: str, premises: str, records: str, unfinished_evidence: str, prompts: WorkerPrompts, taken_in: typing.Callable[[list[Entry]], TakenIn]) -> Cycle:
"""The cycle that runs in the loop's window on route. The keyword arguments are what the entry point alone knows; Cycle says what each is."""
return Cycle(config=setup_version(loop.paths.claude_dir, loop.effects.git), effects=loop.effects, facts=facts, judge=judge_rung(routing.ladder, routing.models.judge), premises=premises, project=routing.project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=task, prompts=prompts, taken_in=taken_in, unfinished_evidence=unfinished_evidence, window=loop.window)
def setup_version(claude_dir: pathlib.Path, git: Git) -> str:
"""The release and commit of this setup that the worker's claude reads, as `v0.8.0 (<sha>)`; a commit past the latest release reads as git describe gives it, such as `v0.8.0-3-g1a2b3c4 (<sha>)`. Where git can't tell, `unknown` and why."""
described = git("describe", "--tags", "--match", "v*", "--always", cwd=str(claude_dir))
commit = git("rev-parse", "HEAD", cwd=str(claude_dir))
if isinstance(described, Failure):
return unknown_version(described)
if isinstance(commit, Failure):
return unknown_version(commit)
return f"{described.strip()} ({commit.strip()})"
def unknown_version(failure: Failure) -> str:
# One line, since the archive reads the line after it as the judge's.
return f"unknown: {' '.join(failure.reason.split())}"
+16
View File
@@ -0,0 +1,16 @@
"""Test doubles for the suites that drive a chunk: the loop of an entry point that exists only in tests."""
import pathlib
import typing
from loop_effects.effects import Effects
from chunk_driver.this_loop import Loop
from loop_state import paths
from claude_code import session
from agent_cycle.doubles import WINDOW
def a_loop(effects: Effects, **fields: typing.Any) -> Loop:
"""A loop in /r on effects, with fields in place of its defaults."""
home = pathlib.Path("/h")
defaults = {"root": "/r", "session": session.session_named("A proj a-driver", home / ".claude" / "projects"), "paths": paths.paths_from(home), "env": {}, "launcher": pathlib.Path("/b/a-driver"), "window": WINDOW}
return Loop(**{**defaults, **fields}, effects=effects)
+3
View File
@@ -0,0 +1,3 @@
"""The entry points that drive chunks, each by its name as its workers, its judge and the human read it. A prompt function that takes a driver takes one of these."""
LOOP_CLAUDE = "loop-claude"
DO_EPIC = "do-epic"
@@ -5,11 +5,11 @@ import sys
import typing
from claude_code.model_catalog import catalog_models
from chunk_driver.claims import CLAIMS, locked
from loop_effects.effects import Effects, say
from failure import Failure
from agent_cycle.stop import needs_human, needs_terminal, Stop
from loop_state.paths import Paths
from text_files.file_writes import replace_json
@dataclasses.dataclass(frozen=True)
@@ -18,14 +18,26 @@ class Models:
worker: str
# The environment variables that name the judge's and the worker's model for one run, over the model saved or configured.
JUDGE_MODEL = "LOOP_CLAUDE_JUDGE_MODEL"
WORKER_MODEL = "LOOP_CLAUDE_WORKER_MODEL"
# The file, in CLAIMS under the home folder, that save_model keeps locked with flock while it rereads and replaces the models file. The models file's own folder is a checkout of this setup, which takes no lock file.
MODELS_LOCK = "models.lock"
def overridden(env: typing.Mapping[str, str], default: str) -> Models:
"""The models of a run: default, unless env names another for the judge or the worker."""
return Models(env.get(JUDGE_MODEL) or default, env.get(WORKER_MODEL) or default)
def get_or_ask_models(project_key: str, paths: Paths, env: typing.Mapping[str, str], driver: str, effects: Effects) -> Models | Stop:
judge, worker = env.get("LOOP_CLAUDE_JUDGE_MODEL"), env.get("LOOP_CLAUDE_WORKER_MODEL")
if judge and worker:
return Models(judge, worker)
named = overridden(env, "")
if named.judge and named.worker:
return named
chosen = get_or_ask_model(project_key, paths, driver, effects)
if isinstance(chosen, Stop):
return chosen
return Models(judge or chosen, worker or chosen)
return overridden(env, chosen)
def get_or_ask_model(project_key: str, paths: Paths, driver: str, effects: Effects) -> str | Stop:
@@ -38,18 +50,29 @@ def get_or_ask_model(project_key: str, paths: Paths, driver: str, effects: Effec
chosen = ask_for_model(project_key, saved, paths.model_catalog, driver, effects)
if isinstance(chosen, Stop):
return chosen
# Reread: another loop may have saved its project while this one waited on the human.
models = read_models_file(saved)
if isinstance(models, Failure):
return Stop(1, models.reason)
if failure := replace_json(saved, {**models, project_key: chosen}, indent="\t", sort_keys=True):
return Stop(1, f"could not save the model in {saved}: {failure.reason}")
if failure := save_model(project_key, chosen, paths, effects.replace_file):
return Stop(1, failure.reason)
say(f"{project_key} runs on {chosen}", effects)
return chosen
def save_model(project_key: str, chosen: str, paths: Paths, replace: typing.Callable[[pathlib.Path, bytes], Failure | None]) -> Failure | None:
"""Adds the project's model to the models file, whose only writer this is, through replace, which writes a file as replace_file does. It holds MODELS_LOCK from its read to its write, so two entry points that save at once both keep their pick."""
saved = paths.models_file
with locked(paths.home / CLAIMS / MODELS_LOCK) as unlocked:
if unlocked:
return Failure(f"could not save the model in {saved}: {unlocked.reason}")
# Read under the lock: another entry point may have saved its project while this one waited on the human.
models = read_models_file(saved)
if isinstance(models, Failure):
return models
if failure := replace(saved, json.dumps({**models, project_key: chosen}, indent="\t", sort_keys=True).encode()):
return Failure(f"could not save the model in {saved}: {failure.reason}")
return None
def read_models_file(saved: pathlib.Path) -> dict[str, str] | Failure:
"""Each project's model, as the models file at saved maps them; none where there is no such file. loop-claude and the replay tools read it."""
"""Each project's model, as the models file at saved maps them; none where there is no such file. The entry points, loop-claude and do-epic, and the replay tools read it."""
try:
with open(saved, encoding="utf-8") as handle:
models = json.load(handle)
@@ -2,7 +2,7 @@ import pathlib
from failure import Failure
from project_config.main import located
from loop_claude.project.text_file import read_utf8
from chunk_driver.project.text_file import read_utf8
from record_store.stores import store
# The records the judge reads beside the premises, by the name that heads each, with the setting in the project's config that places it; then the human's rules below.
@@ -0,0 +1,165 @@
import contextlib
import dataclasses
import fcntl
import io
import json
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from failure import Failure
from loop_effects.doubles import FakeEffects
from chunk_driver.drivers import LOOP_CLAUDE
from chunk_driver.project import model_choice
from loop_state.paths import paths_from
from text_files.file_writes import replace_file
from agent_cycle.stop import Stop
class Overridden(unittest.TestCase):
def test_the_environment_names_the_judges_or_the_workers_model_over_the_default(self):
self.assertEqual(model_choice.overridden({}, "saved"), model_choice.Models("saved", "saved"))
self.assertEqual(model_choice.overridden({"LOOP_CLAUDE_JUDGE_MODEL": "j", "LOOP_CLAUDE_WORKER_MODEL": ""}, "saved"), model_choice.Models("j", "saved"))
self.assertEqual(model_choice.overridden({"LOOP_CLAUDE_JUDGE_MODEL": "j", "LOOP_CLAUDE_WORKER_MODEL": "w"}, "saved"), model_choice.Models("j", "w"))
class ModelsOfARun(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.paths = paths_from(pathlib.Path(temp.name))
self.paths.models_file.parent.mkdir()
def test_a_run_that_names_both_models_reads_no_models_file_and_asks_nothing(self):
self.paths.models_file.write_text("not json")
fake = FakeEffects()
env = {"LOOP_CLAUDE_JUDGE_MODEL": "j", "LOOP_CLAUDE_WORKER_MODEL": "w"}
self.assertEqual(model_choice.get_or_ask_models("proj", self.paths, env, LOOP_CLAUDE, fake.effects()), model_choice.Models("j", "w"))
self.assertEqual(fake.calls, [])
def test_a_run_that_names_one_model_takes_the_other_from_the_models_file(self):
self.paths.models_file.write_text(json.dumps({"proj": "saved"}))
fake = FakeEffects()
self.assertEqual(model_choice.get_or_ask_models("proj", self.paths, {"LOOP_CLAUDE_JUDGE_MODEL": "j"}, LOOP_CLAUDE, fake.effects()), model_choice.Models("j", "saved"))
self.assertEqual(model_choice.get_or_ask_models("proj", self.paths, {"LOOP_CLAUDE_WORKER_MODEL": "w"}, LOOP_CLAUDE, fake.effects()), model_choice.Models("saved", "w"))
self.assertEqual(fake.calls, [])
def test_a_run_that_names_one_model_asks_for_the_projects_where_none_is_saved_and_saves_the_pick(self):
fake, said = FakeEffects(inputs=["my-model"]), io.StringIO()
with contextlib.redirect_stderr(said):
models = model_choice.get_or_ask_models("proj", self.paths, {"LOOP_CLAUDE_JUDGE_MODEL": "j"}, LOOP_CLAUDE, fake.effects())
self.assertEqual(models, model_choice.Models("j", "my-model"))
self.assertEqual(json.loads(self.paths.models_file.read_text()), {"proj": "my-model"})
self.assertIn(" proj runs on my-model\n", said.getvalue())
def test_off_a_terminal_it_stops_naming_the_driver_to_rerun_and_the_file_to_merge_the_model_into(self):
fake = FakeEffects(on_terminal=False)
stop = model_choice.get_or_ask_models("proj", self.paths, {}, LOOP_CLAUDE, fake.effects())
assert isinstance(stop, Stop)
self.assertEqual(stop.code, 2)
self.assertIn(f"pick proj's model, or merge {{\"proj\": \"<model>\"}} into {self.paths.models_file}", stop.reason)
self.assertIn(LOOP_CLAUDE, stop.reason)
self.assertFalse(self.paths.models_file.exists())
def test_stops_with_exit_1_where_the_models_file_maps_no_projects_to_model_names(self):
for held in ("[]", '{"proj": 5}', '{"proj": ""}'):
with self.subTest(held=held):
self.paths.models_file.write_text(held)
stop = model_choice.get_or_ask_models("proj", self.paths, {}, LOOP_CLAUDE, FakeEffects().effects())
self.assertEqual(stop, Stop(1, f"could not read {self.paths.models_file}: not an object mapping projects to model names"))
class SavingAModel(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.paths = paths_from(pathlib.Path(temp.name))
self.lock = self.paths.home / ".local/state/loop-claude/models.lock"
def test_adds_the_projects_model_to_those_the_file_holds(self):
self.paths.models_file.parent.mkdir()
self.paths.models_file.write_text(json.dumps({"other": "model-x"}))
self.assertIsNone(model_choice.save_model("proj", "model-b", self.paths, replace_file))
self.assertEqual(json.loads(self.paths.models_file.read_text()), {"other": "model-x", "proj": "model-b"})
def test_writes_the_file_for_a_human_to_merge_into_a_project_per_tab_indented_line_in_the_order_of_their_keys(self):
self.paths.models_file.parent.mkdir()
self.paths.models_file.write_text(json.dumps({"zebra": "model-z", "other": "model-x"}))
self.assertIsNone(model_choice.save_model("proj", "model-b", self.paths, replace_file))
self.assertEqual(self.paths.models_file.read_text(), '{\n\t"other": "model-x",\n\t"proj": "model-b",\n\t"zebra": "model-z"\n}')
def lock_is_held(self) -> bool:
with open(self.lock, "a") as other:
try:
fcntl.flock(other, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
return True
return False
def test_holds_the_models_lock_from_its_read_to_its_write(self):
self.paths.models_file.parent.mkdir()
held = []
class Watched(pathlib.PosixPath):
"""The models file, which notes whether the lock is held each time it is opened."""
def __fspath__(watched) -> str:
held.append(self.lock_is_held())
return super().__fspath__()
def replace(path: pathlib.Path, data: bytes) -> None:
held.append(self.lock_is_held())
paths = dataclasses.replace(self.paths, models_file=Watched(self.paths.models_file))
self.assertIsNone(model_choice.save_model("proj", "model-b", paths, replace))
self.assertEqual(held, [True, True])
self.assertFalse(self.lock_is_held())
def test_doesnt_save_anything_where_it_cannot_take_the_lock(self):
self.paths.models_file.parent.mkdir()
self.lock.parent.parent.mkdir(parents=True)
self.lock.parent.write_text("")
saved = model_choice.save_model("proj", "model-b", self.paths, replace_file)
assert isinstance(saved, Failure)
self.assertRegex(saved.reason, f"^could not save the model in {self.paths.models_file}: cannot lock {self.lock.parent}: \\w")
self.assertFalse(self.paths.models_file.exists())
def test_a_pick_is_saved_through_save_model_alone(self):
self.paths.models_file.parent.mkdir()
fake, held = FakeEffects(inputs=["my-model"]), []
def replace(path: pathlib.Path, data: bytes) -> Failure:
held.append(self.lock_is_held())
return Failure("could not save")
fake.replace_file = replace
self.assertEqual(model_choice.get_or_ask_model("proj", self.paths, LOOP_CLAUDE, fake.effects()), Stop(1, f"could not save the model in {self.paths.models_file}: could not save"))
self.assertEqual(held, [True])
class ModelPrompt(unittest.TestCase):
OFFERED = [("claude-opus-5-5", "Opus 5.5"), ("claude-haiku-4-5", "Haiku 4.5")]
def test_reads_a_number_as_the_offered_model_and_anything_else_as_a_name(self):
self.assertEqual(model_choice.model_picked(" 2 ", self.OFFERED), "claude-haiku-4-5")
self.assertEqual(model_choice.model_picked("1", self.OFFERED), "claude-opus-5-5")
self.assertEqual(model_choice.model_picked("0", self.OFFERED), Failure("pick 1 to 2"))
self.assertEqual(model_choice.model_picked("opus", self.OFFERED), "opus")
self.assertEqual(model_choice.model_picked("3", []), "3")
self.assertEqual(model_choice.model_picked("3", self.OFFERED), Failure("pick 1 to 2"))
self.assertIsNone(model_choice.model_picked(" ", self.OFFERED))
def test_asks_again_until_an_answer_picks_a_model(self):
with tempfile.TemporaryDirectory() as empty:
fake = FakeEffects(inputs=["", "my-model"])
self.assertEqual(model_choice.ask_for_model("proj", pathlib.Path("/m.json"), pathlib.Path(empty), LOOP_CLAUDE, fake.effects()), "my-model")
self.assertEqual(len(fake.made("input")), 2)
fake = FakeEffects(inputs=[EOFError()])
self.assertEqual(model_choice.ask_for_model("proj", pathlib.Path("/m.json"), pathlib.Path(empty), LOOP_CLAUDE, fake.effects()), Stop(2, "the human is needed to pick proj's model, but the input ended"))
if __name__ == "__main__":
unittest.main()
+95
View File
@@ -0,0 +1,95 @@
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from project_config.doubles import configured
from failure import Failure
from agent_cycle import judge
from chunk_driver.project import records
class Records(unittest.TestCase):
def test_heads_each_record_by_its_name_the_humans_rules_last(self):
with tempfile.TemporaryDirectory() as root:
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
config = configured(self, pathlib.Path(root), home=claude_md.parents[1]) / "config.toml"
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), "")
claude_md.parent.mkdir(parents=True)
claude_md.write_text("# Global\n")
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), "From ~/.claude/CLAUDE.md:\n# Global")
pathlib.Path(root, "docs").mkdir()
pathlib.Path(root, "docs", "decisions.md").write_text("# Decisions\n")
pathlib.Path(root, "AGENTS.md").write_text("\n# Rules\n")
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), "From AGENTS.md:\n# Rules\n\nFrom docs/decisions.md:\n# Decisions\n\nFrom ~/.claude/CLAUDE.md:\n# Global")
pathlib.Path(claude_md.parent, "rules").mkdir()
pathlib.Path(claude_md.parent, "rules", "todo-items.md").write_text("# Todo\n")
pathlib.Path(claude_md.parent, "rules", "goals-and-audience.md").write_text("# Goals\n")
pathlib.Path(claude_md.parent, "rules", "notes.txt").write_text("not a rule\n")
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards").mkdir(parents=True)
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards", "comments.md").write_text("# Comments\n")
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).endswith("From ~/.claude/CLAUDE.md:\n# Global\n\nFrom ~/.claude/rules/goals-and-audience.md:\n# Goals\n\nFrom ~/.claude/rules/todo-items.md:\n# Todo\n\nFrom ~/.claude/skills/larv-review/standards/comments.md:\n# Comments"))
config.write_text(config.read_text().replace('"~/.claude/skills/larv-review/standards"', '"~/review-standards"'))
pathlib.Path(claude_md.parents[1], "review-standards").mkdir()
pathlib.Path(claude_md.parents[1], "review-standards", "quality.md").write_text("# Ours\n")
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).endswith("From ~/.claude/rules/todo-items.md:\n# Todo\n\nFrom ~/review-standards/quality.md:\n# Ours"), "the config's standards folder replaces the default")
config.write_text(config.read_text().replace('"~/review-standards"', f'"{root}/standards"'))
pathlib.Path(root, "standards").mkdir()
pathlib.Path(root, "standards", "quality.md").write_text("# Elsewhere\n")
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).endswith(f"From {root}/standards/quality.md:\n# Elsewhere"), "a folder outside home is named in full")
config.write_text(config.read_text().replace('"checkout:AGENTS.md"', '"folder:AGENTS.md"').replace('"checkout:docs/decisions.md"', '"folder:log/decisions.md"'))
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).startswith("From ~/.claude/CLAUDE.md:\n# Global"), "the checkout's AGENTS.md and docs/decisions.md are its owners'")
pathlib.Path(config.parent, "log").mkdir()
pathlib.Path(config.parent, "log", "decisions.md").write_text("# Ours\n")
pathlib.Path(config.parent, "AGENTS.md").write_text("# Our rules\n")
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).startswith("From AGENTS.md:\n# Our rules\n\nFrom docs/decisions.md:\n# Ours\n\nFrom ~/.claude/CLAUDE.md:\n# Global"))
config.unlink()
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"no config names the checkout {root}: write ~/.claude-projects/<key>/config.toml, as `project-config --help` shows - the judge cannot read the records."))
def test_puts_the_records_handed_in_first_and_reads_only_the_humans_rules_without_a_checkout(self):
with tempfile.TemporaryDirectory() as root:
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
configured(self, pathlib.Path(root), home=claude_md.parents[1])
claude_md.parent.mkdir(parents=True)
claude_md.write_text("# Global\n")
pathlib.Path(claude_md.parent, "rules").mkdir()
pathlib.Path(claude_md.parent, "rules", "todo-items.md").write_text("# Todo\n")
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards").mkdir(parents=True)
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards", "comments.md").write_text("# Comments\n")
pathlib.Path(root, "AGENTS.md").write_text("# Rules\n")
first = (("the epic ABC-1's description, ## Decisions", "Squash.\n"), ("the epic ABC-1's description, ## Scoring", " \n"))
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent, first), "From the epic ABC-1's description, ## Decisions:\nSquash.\n\nFrom AGENTS.md:\n# Rules\n\nFrom ~/.claude/CLAUDE.md:\n# Global\n\nFrom ~/.claude/rules/todo-items.md:\n# Todo\n\nFrom ~/.claude/skills/larv-review/standards/comments.md:\n# Comments")
self.assertEqual(records.read_records(None, claude_md.parents[1], claude_md.parent, first), "From the epic ABC-1's description, ## Decisions:\nSquash.\n\nFrom ~/.claude/CLAUDE.md:\n# Global\n\nFrom ~/.claude/rules/todo-items.md:\n# Todo")
def test_stops_on_a_record_it_cannot_read(self):
with tempfile.TemporaryDirectory() as root:
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
configured(self, pathlib.Path(root), home=claude_md.parents[1])
claude_md.mkdir(parents=True)
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"CLAUDE.md in {claude_md.parent}: Is a directory - the judge cannot read the records."))
pathlib.Path(root, "docs", "decisions.md").mkdir(parents=True)
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"cannot read {root}/docs/decisions.md: Is a directory - the judge cannot read the records."))
pathlib.Path(root, "AGENTS.md").write_bytes(b"# Rules\n\xff\n")
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"AGENTS.md in {root} is not UTF-8 on line 2 - the judge cannot read the records."))
def test_stops_on_a_rule_it_cannot_read(self):
with tempfile.TemporaryDirectory() as root:
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
configured(self, pathlib.Path(root), home=claude_md.parents[1])
rules = pathlib.Path(claude_md.parent, "rules")
rules.mkdir(parents=True)
pathlib.Path(rules, "todo-items.md").write_bytes(b"\xff\n")
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"todo-items.md in {rules} is not UTF-8 on line 1 - the judge cannot read the records."))
class JudgeRules(unittest.TestCase):
def test_the_judge_prompt_names_every_rule_the_judge_reads_from_the_human(self):
self.assertEqual([name for name in records.HUMAN_RULE_NAMES if name not in judge.judge_rules("the ledger", "a-driver")], [])
if __name__ == "__main__":
unittest.main()
@@ -1,20 +1,17 @@
"""What loop-claude says to the worker each cycle. model_routing.review_choice chooses where the reviews run. review_launches and reruns put that choice into sentences, and escalation says where escalated subagents run; rung_sentences returns the sentences of all three. In a chunk's first prompt, routed_by places them after its routing sentences where an item routed the chunk, and fresh places them alone otherwise. rungs_held repeats them at the end of a prompt after a step-up or an unfinished verdict."""
"""What every entry point says to the worker each cycle. model_routing.review_choice chooses where the reviews run. review_launches and reruns put that choice into sentences, and escalation says where escalated subagents run; rung_sentences returns the sentences of all three. An entry point places them in a chunk's first prompt, as loop-claude does in bin/loop_claude/chunk/fresh_prompt.py. rungs_held repeats them at the end of a prompt after a step-up or an unfinished verdict."""
import functools
import typing
from agent_cycle.worker_prompts import answers_recorded, WorkerPrompts
from agent_cycle.escalation_mark import asks_higher_rung
from model_routing.review_choice import chosen, ReviewChoice
from loop_claude.cycle.chunk_items import CHUNK_ITEMS
from model_routing.route import ESCALATION_MARKER, Route
from agent_cycle.routing import escalation_rungs
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES, RECORDS_NAMED
from chunk_driver.project.records import HUMAN_RULE_NAMES, RECORD_FILES, RECORDS_NAMED
from review_rules.review_passes import INSTRUCTION_FILES, INSTRUCTION_FOLDERS
from text_files.markdown_text import listed, quoted
from model_routing.ladder import ProjectRung, Rung
# loop-claude's name as its workers, its judge and the human read it. A function below that takes a driver words a sentence for any entry point, such as do-epic.
DRIVER = "loop-claude"
# Where the judge may quote an answer from, named as the human and the worker see them; of the goals file the judge sees only the premises.
QUOTE_SOURCES = listed(("the goals file's Goals and Audience", *RECORD_FILES, *HUMAN_RULE_NAMES), "or")
# Where the worker may write an answer the human typed, so it counts as recorded.
@@ -28,45 +25,8 @@ THE_SECURITY_PASS = "the security pass's subagent"
# Workers only, not CLAUDE.md, which reviewers load too: a reviewer under a like rule missed findings.
READING_RULE = "Read in small pieces: never read more than 80 lines of a file at once, nor take a tool output over 3,000 characters into your context."
# chunk_items reads the latest such line; three fresh readers picked the label (docs/decisions.md → `routing: loop-claude reads the items a chunk takes`).
NAME_THE_ITEMS = f"Name every plan item this chunk takes on one line, as in `{CHUNK_ITEMS} 297 (L 3, E + I 5), 298 (L 3, E + I 7)`, and write the whole line again whenever the set changes."
AFTER_CODE_HEALTH = 'which rank after code health (~/.claude/CLAUDE.md → "Shared goal") unless the goals file voids it'
def fresh(driver: str, task: str, premises: str, route: Route, project: ProjectRung | None, facts: str) -> str:
"""A chunk's first prompt; facts is preflight_facts' or preflight_failed's text."""
routed = f"{routed_by(driver, route, project)} {recount(route)}" if route.item else " ".join(rung_sentences(driver, route, project))
opening = f"{routed}\n\n" if routed else ""
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one. {READING_RULE} {NAME_THE_ITEMS}\n\n{opening}{facts}\n\nThe chunk's plan items, or else the task above, set its scope. The goals below, {AFTER_CODE_HEALTH}, choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file, {premises}"
def routed_by(driver: str, route: Route, project: ProjectRung | None) -> str:
return " ".join((
f"{driver} runs you on {route.rung}, routed by item {route.item}'s L of {route.likelihood}.",
*rung_sentences(driver, route, project),
))
def recount(route: Route) -> str:
"""Has the worker count the routing item's size again once it knows the code, since the L that routed it was scored before anyone read that code."""
return f"After your first read of the code, count item {route.item}'s size again: the lines, files and components you will change. Where that count would change its L, have a subagent that scores plan items re-score it; a re-score whose L picks a rung above yours is escalation trigger 1."
def preflight_facts(printed: str) -> str:
"""What chunk-preflight printed, for the worker to take as found."""
return f"loop-claude ran `~/.claude/bin/chunk-preflight` at the checkout root just before this run. Take what it printed as found rather than finding it again:\n\n{printed}"
def held_by_others(items: list[int]) -> str:
"""Keeps a fresh worker off the plan items the project's other chunks hold, running or stopped."""
return f"\n\nOther chunks of this project, running or stopped, hold {'items ' if len(items) > 1 else 'item '}{', '.join(map(str, items))}. Leave those items out of this chunk, even where the facts above list them as items that may join it."
def preflight_failed(reason: str) -> str:
"""Has the worker find the facts chunk-preflight would have, its worktree among them."""
return f"loop-claude could not run `~/.claude/bin/chunk-preflight`, so find the chunk's facts yourself. Why it could not run: {reason}"
def review_launches(route: Route, project: ProjectRung | None) -> list[str]:
@@ -196,6 +156,3 @@ class RecordNames(typing.NamedTuple):
def worker_prompts(driver: str, fresh: typing.Callable[[str, str, Route, ProjectRung | None, str], str], naming: str, names: RecordNames = RecordNames()) -> WorkerPrompts:
"""The prompts of an entry point's cycles: its own fresh prompt, and the resumed prompt and rung sentences every entry point shares, each naming driver. resumed says what naming is. names is loop-claude's unless the entry point keeps records elsewhere, as do-epic does."""
return WorkerPrompts(driver, fresh, functools.partial(resumed, driver, naming), functools.partial(rungs_held, driver), *names)
WORKER = worker_prompts(DRIVER, functools.partial(fresh, DRIVER), NAME_THE_ITEMS)
+113
View File
@@ -0,0 +1,113 @@
import contextlib
import dataclasses
import io
import json
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from chunk_driver.doubles import a_loop
from chunk_driver import chunk_run, cycle_start
from loop_effects.doubles import FakeEffects, NOW
from agent_cycle.verdicts import Finished, TakenIn, Unfinished
from transcript_archive import outcomes, write
from model_routing.ladder import ProjectRung, Rung
from model_routing.ladder_file import read_ladder, SHIPPED
from text_files.file_writes import replace_file
from model_routing.route import Route
from agent_cycle import run
from claude_code import session
from agent_cycle.stop import Stop
OPUS_PROJECT = ProjectRung(read_ladder(SHIPPED), Rung("claude-opus-5-5", "medium"))
class Steps(unittest.TestCase):
OPUS = Rung("claude-opus-5-5", "medium")
# The rungs an L 3 and an L 4 pick from OPUS.
FABLE_LOW = Rung("claude-fable-5-2", "low")
FABLE_MEDIUM = Rung("claude-fable-5-2", "medium")
def test_a_finished_verdict_clears_the_route_and_the_evidence(self):
run_with = chunk_run.ChunkRun(cycle_number=1, unfinished_evidence="old", route=Route(self.OPUS))
self.assertEqual(chunk_run.after_verdict(run_with, OPUS_PROJECT, run.Concluded(Finished("x"), run_with.route)), dataclasses.replace(run_with, route=None, unfinished_evidence=""))
self.assertEqual(chunk_run.after_verdict(dataclasses.replace(run_with, unfinished_run=3), OPUS_PROJECT, run.Concluded(Finished("x"), run_with.route)).unfinished_run, 0)
self.assertEqual(chunk_run.after_verdict(run_with, OPUS_PROJECT, run.Concluded(Unfinished("new", taken_in=TakenIn(highest_likelihood=4)), run_with.route)), dataclasses.replace(run_with, route=Route(self.FABLE_MEDIUM, unfinished=1), unfinished_evidence="new", unfinished_run=1))
def test_steps_from_the_route_its_cycle_ended_on_which_a_ruling_may_have_raised(self):
run_with = chunk_run.ChunkRun(cycle_number=1, unfinished_evidence="", route=Route(self.OPUS, 3, 3, 4))
raised = Route(self.FABLE_MEDIUM, 3, 3, 4, escalated_at=5)
self.assertEqual(chunk_run.after_verdict(run_with, OPUS_PROJECT, run.Concluded(Unfinished("new"), raised)).route, dataclasses.replace(raised, unfinished=1))
def test_keeps_the_route_its_cycle_ended_on_off_the_ladder(self):
off_ladder = Route(Rung("worker-model"))
run_with = chunk_run.ChunkRun(cycle_number=1, unfinished_evidence="")
self.assertEqual(chunk_run.after_verdict(run_with, None, run.Concluded(Unfinished("new", "round 2: 🔴", TakenIn(highest_likelihood=4), entries_read=9), off_ladder)).route, off_ladder)
def test_an_unfinished_verdicts_escalation_marks_the_transcript_entries_the_judge_read(self):
run_with = chunk_run.ChunkRun(cycle_number=1, unfinished_evidence="", route=Route(self.FABLE_LOW, 3, 3, 4, escalated_at=2))
self.assertEqual(chunk_run.after_verdict(run_with, OPUS_PROJECT, run.Concluded(Unfinished("new", "round 2: 🔴", entries_read=9), run_with.route)).route, Route(self.FABLE_MEDIUM, 3, 3, 4, escalated_at=9))
self.assertEqual(chunk_run.after_verdict(run_with, OPUS_PROJECT, run.Concluded(Unfinished("new", entries_read=9), run_with.route)).route, Route(self.FABLE_LOW, 3, 3, 4, unfinished=1, escalated_at=2))
def test_archives_a_step_up_as_it_happens_so_a_rerun_resumes_on_it(self):
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj a-driver", pathlib.Path(home, "projects"))
archive = pathlib.Path(home, "archive")
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
before = Route(self.FABLE_LOW, 3, 3, 4)
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(before.worker_line(), "working", "j", ""), NOW, replace=replace_file))
after = chunk_run.closed_cycle(chunk_run.ChunkRun(cycle_number=1, unfinished_evidence=""), "Do it.", OPUS_PROJECT, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_likelihood=4)), before), archive, "a-driver", a_loop(FakeEffects().effects(), session=opened))
self.assertEqual(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)), Route(self.FABLE_MEDIUM, 3, 3, 4))
with contextlib.redirect_stderr(io.StringIO()):
chunk_run.closed_cycle(after, "Do it.", OPUS_PROJECT, run.Concluded(Finished("done"), after.route), archive, "a-driver", a_loop(FakeEffects().effects(), session=opened))
self.assertEqual(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)), Route(self.FABLE_MEDIUM, 3, 3, 4))
blocked = pathlib.Path(home, "blocked")
blocked.write_text("")
stop = chunk_run.closed_cycle(chunk_run.ChunkRun(cycle_number=1, unfinished_evidence=""), "Do it.", OPUS_PROJECT, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_likelihood=4)), before), blocked, "a-driver", a_loop(FakeEffects().effects(), session=opened))
self.assertEqual((stop.code, stop.reason.split(":")[0]), (1, "could not archive the transcript"))
def outcomes_after(self, unfinished_run: int, taken_in: TakenIn) -> tuple[object, str]:
"""What closed_cycle answers for a-driver's chunk `Do it.`, after an unfinished verdict that follows unfinished_run others, and the outcomes it archived for a transcript not archived before."""
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj a-driver", pathlib.Path(home, "projects"))
archive = pathlib.Path(home, "archive")
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
before = Route(self.FABLE_LOW, 3, 3, 4)
with contextlib.redirect_stderr(io.StringIO()):
closed = chunk_run.closed_cycle(chunk_run.ChunkRun(cycle_number=1, unfinished_evidence="", unfinished_run=unfinished_run), "Do it.", OPUS_PROJECT, run.Concluded(Unfinished("working", taken_in=taken_in), before), archive, "a-driver", a_loop(FakeEffects().effects(), session=opened))
return closed, "".join(file.read_text() for file in archive.rglob("*") if file.is_file() and file.suffix != ".jsonl")
def test_a_step_up_archived_first_for_its_transcript_names_the_chunks_task(self):
closed, outcomes_kept = self.outcomes_after(0, TakenIn(highest_likelihood=4))
self.assertEqual(closed.route.rung, self.FABLE_MEDIUM)
self.assertIn("Do it.\n", outcomes_kept)
def test_the_sixth_unfinished_verdict_running_stops_for_the_human_naming_the_driver_to_rerun_and_archives_why_with_the_task(self):
reason = "the human is needed to go on with the chunk, but the judge ruled it unfinished 6 cycles running; finish the chunk yourself, or rerun a-driver to give it 6 more cycles"
closed, outcomes_kept = self.outcomes_after(5, TakenIn())
self.assertEqual(closed, Stop(2, reason))
self.assertIn("Do it.\n", outcomes_kept)
self.assertIn(f"Stopped: {reason}\n", outcomes_kept)
closed, outcomes_kept = self.outcomes_after(4, TakenIn())
self.assertEqual((closed.unfinished_run, outcomes_kept), (5, ""))
def test_archives_a_higher_e_plus_i_taken_in_so_a_rerun_keeps_it(self):
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj a-driver", pathlib.Path(home, "projects"))
archive = pathlib.Path(home, "archive")
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
before = Route(self.FABLE_LOW, 3, 3, 4)
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(before.worker_line(), "working", "j", ""), NOW, replace=replace_file))
chunk_run.closed_cycle(chunk_run.ChunkRun(cycle_number=1, unfinished_evidence=""), "Do it.", OPUS_PROJECT, run.Concluded(Unfinished("working", taken_in=TakenIn(highest_escape_and_impact=6)), before), archive, "a-driver", a_loop(FakeEffects().effects(), session=opened))
self.assertEqual(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)), Route(self.FABLE_LOW, 3, 3, 4, escape_and_impact_taken_in=6))
+41
View File
@@ -0,0 +1,41 @@
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from failure import Failure
from chunk_driver.claims import LOCK, starting
class Starting(unittest.TestCase):
def setUp(self) -> None:
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.folder = pathlib.Path(temp.name)
def test_yields_none_while_it_holds_the_lock_in_a_folder_it_makes(self) -> None:
with starting(self.folder / "state" / "proj") as failure:
self.assertIsNone(failure)
self.assertTrue((self.folder / "state" / "proj" / LOCK).is_file())
def test_yields_a_failure_naming_the_path_it_could_not_make(self) -> None:
(self.folder / "state").write_text("", encoding="utf-8")
with starting(self.folder / "state") as failure:
assert isinstance(failure, Failure)
self.assertRegex(failure.reason, f"^cannot lock {self.folder}/state: \\w")
def test_yields_a_failure_naming_the_lock_it_could_not_open(self) -> None:
(self.folder / LOCK).mkdir()
with starting(self.folder) as failure:
assert isinstance(failure, Failure)
self.assertRegex(failure.reason, f"^cannot lock {self.folder}/{LOCK}: \\w")
def test_an_error_raised_while_the_lock_is_held_is_the_callers(self) -> None:
with self.assertRaises(OSError), starting(self.folder):
raise OSError("not the lock's")
if __name__ == "__main__":
unittest.main()
+204
View File
@@ -0,0 +1,204 @@
import contextlib
import dataclasses
import functools
import io
import json
import os
import pathlib
import sys
import tempfile
import types
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from chunk_driver.doubles import a_loop
from project_config.doubles import configured
from agent_cycle.run import Cycle
from chunk_driver import cycle_start
from failure import Failure
from loop_effects.doubles import FakeEffects, NOW, STAMP
from git_cli import git_repo
from chunk_driver.project.model_choice import Models
from transcript_archive import outcomes, write
from loop_state.paths import paths_from
from model_routing.ladder import ProjectRung, Rung
from model_routing.ladder_file import read_ladder, SHIPPED
from text_files.file_writes import replace_file
from model_routing.route import Route, WorkerLine
from claude_code import session
from agent_cycle.stop import Stop
from agent_cycle.verdicts import TakenIn
from agent_cycle.worker_prompts import WorkerPrompts
OPUS_PROJECT = ProjectRung(read_ladder(SHIPPED), Rung("claude-opus-5-5", "medium"))
class SetupVersion(unittest.TestCase):
def test_names_the_release_and_the_commit_or_why_git_cannot_tell(self):
with tempfile.TemporaryDirectory() as root:
git = functools.partial(git_repo.git, env={**os.environ, "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1", "GIT_CEILING_DIRECTORIES": root})
gone = cycle_start.setup_version(pathlib.Path(root), git)
self.assertRegex(gone, r"^unknown: [^\n]*not a git repository")
for step in (["init", "-q"], ["-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "--allow-empty", "-m", "one"]):
self.assertIsInstance(git(*step, cwd=root), str)
head = git("rev-parse", "HEAD", cwd=root).strip()
self.assertEqual(cycle_start.setup_version(pathlib.Path(root), git), f"{head[:7]} ({head})")
git("tag", "v0.1.0", cwd=root)
self.assertEqual(cycle_start.setup_version(pathlib.Path(root), git), f"v0.1.0 ({head})")
git("-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "--allow-empty", "-m", "two", cwd=root)
head = git("rev-parse", "HEAD", cwd=root).strip()
self.assertEqual(cycle_start.setup_version(pathlib.Path(root), git), f"v0.1.0-1-g{head[:7]} ({head})")
def test_names_why_git_cannot_name_the_commit_on_one_line(self):
def fake_git(*args, cwd=None):
return Failure("fatal: bad HEAD\nhint: repair it") if args[0] == "rev-parse" else "v0.1.0\n"
self.assertEqual(cycle_start.setup_version(pathlib.Path("/r"), fake_git), "unknown: fatal: bad HEAD hint: repair it")
class ProjectRouting(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.home = pathlib.Path(temp.name)
self.paths = paths_from(self.home)
self.checkout = self.home / "code" / "acme"
self.checkout.mkdir(parents=True)
self.config = configured(self, self.checkout, home=self.home)
self.paths.models_file.parent.mkdir(exist_ok=True)
def routing(self, fake=None, env=None):
return cycle_start.project_routing(a_loop((fake or FakeEffects()).effects(), root=str(self.checkout), paths=self.paths, env=env or {}), "a-driver")
def test_routes_on_the_projects_saved_model_and_names_the_shipped_ladder_or_the_projects_own_as_its_file(self):
self.paths.models_file.write_text(json.dumps({"acme": "claude-opus-5-5"}))
shipped = read_ladder(SHIPPED)
self.assertEqual(self.routing(), cycle_start.Routing(Models("claude-opus-5-5", "claude-opus-5-5"), shipped, SHIPPED))
self.assertEqual(self.routing(env={"LOOP_CLAUDE_JUDGE_MODEL": "j"}), cycle_start.Routing(Models("j", "claude-opus-5-5"), shipped, SHIPPED))
own = self.config / "routing-ladder.json"
own.write_text('[{"model": "claude-fable-5-2", "effort": "low"}]', encoding="utf-8")
routing = self.routing()
assert isinstance(routing, cycle_start.Routing)
self.assertEqual((routing.ladder_file, [str(rung) for rung in routing.ladder.rungs]), (own, ["claude-fable-5-2 at low effort"]))
def test_stops_with_exit_1_naming_the_ladder_file_that_cannot_be_read(self):
self.paths.models_file.write_text(json.dumps({"acme": "claude-opus-5-5"}))
own = self.config / "routing-ladder.json"
own.write_text("not json", encoding="utf-8")
stop = self.routing()
assert isinstance(stop, Stop)
self.assertEqual(stop.code, 1)
self.assertIn(str(own), stop.reason)
def test_stops_with_exit_1_naming_the_config_that_cannot_be_read(self):
self.paths.models_file.write_text(json.dumps({"acme": "claude-opus-5-5"}))
(self.config / "config.toml").write_text("not toml", encoding="utf-8")
stop = self.routing()
assert isinstance(stop, Stop)
self.assertEqual(stop.code, 1)
self.assertIn(str(self.config / "config.toml"), stop.reason)
def test_asks_for_the_model_where_none_is_saved_and_off_a_terminal_stops_naming_the_driver_to_rerun(self):
stop = self.routing(FakeEffects(on_terminal=False))
assert isinstance(stop, Stop)
self.assertEqual(stop.code, 2)
self.assertIn("pick acme's model", stop.reason)
self.assertIn("rerun a-driver on a terminal", stop.reason)
with contextlib.redirect_stderr(io.StringIO()):
self.assertEqual(self.routing(FakeEffects(inputs=["claude-opus-5-5"])), cycle_start.Routing(Models("claude-opus-5-5", "claude-opus-5-5"), read_ladder(SHIPPED), SHIPPED))
class CycleStart(unittest.TestCase):
ROUTING = cycle_start.Routing(Models("judge-model", "claude-opus-5-5"), OPUS_PROJECT.ladder, SHIPPED)
def test_announces_the_route_the_chunks_run_holds_and_else_the_one_its_archive_last_names(self):
def never():
raise AssertionError("not asked")
kept = Route(Rung("claude-opus-5-5", "xhigh"), 3, 4, 5)
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj a-driver", pathlib.Path(home, "projects"))
loop = a_loop(FakeEffects().effects(), session=opened)
archive = pathlib.Path(home, "archive")
with contextlib.redirect_stdout(io.StringIO()) as said:
self.assertEqual(cycle_start.announced_route(4, kept, self.ROUTING, archive, never, loop), kept)
self.assertEqual(said.getvalue(), f"\n{STAMP} === cycle 4, worker on claude-opus-5-5 at xhigh effort ===\n")
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(kept.worker_line(), "working", "j", ""), NOW, replace=replace_file))
with contextlib.redirect_stdout(io.StringIO()):
self.assertEqual(cycle_start.announced_route(1, None, self.ROUTING, archive, never, loop), kept)
def test_builds_the_cycle_from_the_loop_the_routing_and_what_the_entry_point_knows(self):
asked = []
def git(*args, cwd=None):
asked.append(cwd)
return "v0.1.0\n" if args[0] == "describe" else "abc\n"
effects = dataclasses.replace(FakeEffects().effects(), git=git)
loop, route, archive = a_loop(effects), Route(Rung("claude-opus-5-5", "xhigh"), 3, 4, 5), pathlib.Path("/t/r")
prompts = WorkerPrompts("a-driver", lambda *args: "fresh", lambda *args: "resumed", lambda *args: "", "the sources", "the destinations", "the records named")
def taken_in(entries):
return TakenIn()
cycle = cycle_start.cycle_of(loop, self.ROUTING, route, archive, task="the task", facts="the facts", premises="the premises", records="the records", unfinished_evidence="the evidence", prompts=prompts, taken_in=taken_in)
self.assertEqual(cycle, Cycle(config="v0.1.0 (abc)", effects=effects, facts="the facts", judge=Rung("judge-model"), premises="the premises", project=OPUS_PROJECT, project_archive=archive, records="the records", root="/r",
route=route, session=loop.session, task="the task", prompts=prompts, taken_in=taken_in, unfinished_evidence="the evidence", window=loop.window))
self.assertEqual(asked, ["/h/.claude", "/h/.claude"])
class Routes(unittest.TestCase):
OPUS = Rung("claude-opus-5-5", "medium")
SONNET = Rung("claude-sonnet-5-5", "high")
def test_keeps_a_chunks_route_or_its_archived_one_at_or_above_the_rung_its_item_picks_and_else_routes_afresh(self):
item = types.SimpleNamespace(id=3, likelihood=4, escape_and_impact=5)
archived = Route(Rung("claude-opus-5-5", "xhigh"), 3, 4, 5)
def never():
raise AssertionError("not asked")
self.assertEqual(cycle_start.routed(archived, None, "worker-model", never, never), Route(Rung("worker-model")))
self.assertEqual(cycle_start.routed(archived, OPUS_PROJECT, "claude-opus-5-5", never, never), archived)
self.assertEqual(cycle_start.routed(None, OPUS_PROJECT, "claude-opus-5-5", lambda: archived, never), archived)
self.assertEqual(cycle_start.routed(None, OPUS_PROJECT, "claude-opus-5-5", lambda: None, lambda: item), Route(Rung("claude-fable-5-2", "medium"), 3, 4, 5))
self.assertEqual(cycle_start.routed(Route(self.SONNET), OPUS_PROJECT, "claude-opus-5-5", never, lambda: None), Route(self.OPUS))
cheap = Route(self.SONNET, 4, 2, 3)
self.assertEqual(cycle_start.routed(cheap, OPUS_PROJECT, "claude-opus-5-5", never, never), cheap)
self.assertEqual(cycle_start.routed(Route(Rung("claude-haiku-5-5", "max"), 4, 2, 3), OPUS_PROJECT, "claude-opus-5-5", never, lambda: None), Route(self.OPUS))
def test_resumes_on_the_archived_route_and_says_why_it_routes_afresh(self):
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj a-driver", pathlib.Path(home, "projects"))
archive = pathlib.Path(home, "archive")
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
route = Route(Rung("claude-fable-5-2", "low"), 3, 4, 5)
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)))
transcript.parent.mkdir(parents=True)
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
self.assertIsNone(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)))
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(route.worker_line(), "working", "j", ""), NOW, replace=replace_file))
with contextlib.redirect_stderr(stderr):
self.assertEqual(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)), route)
self.assertEqual(stderr.getvalue(), "")
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.stepped_up(WorkerLine('{"model": "claude-opus-5-5", "rung": 3}')), NOW + 1, replace=replace_file))
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)))
self.assertEqual(stderr.getvalue(), f'{STAMP} routing the chunk afresh: the archive\'s latest Worker: line names no route that can be read: {{"model": "claude-opus-5-5", "rung": 3}}\n')
(chunk,) = archive.iterdir()
chunk.chmod(0)
try:
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(cycle_start.archived_route(archive, a_loop(FakeEffects().effects(), session=opened)))
finally:
chunk.chmod(0o700)
self.assertTrue(stderr.getvalue().startswith(f"{STAMP} routing the chunk afresh: could not read the archive"), stderr.getvalue())
if __name__ == "__main__":
unittest.main()
+120
View File
@@ -0,0 +1,120 @@
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from model_routing.ladder import ProjectRung, Rung
from chunk_driver import prompts
from model_routing.ladder_file import read_ladder, SHIPPED
from model_routing.route import Route
LADDER = read_ladder(SHIPPED)
# The ladder's rungs around the project's: the one below it, its own, and those one to three steps up.
SONNET = Rung("claude-sonnet-5-5", "high")
OPUS = Rung("claude-opus-5-5", "medium")
FABLE_LOW = Rung("claude-fable-5-2", "low")
FABLE_MEDIUM = Rung("claude-fable-5-2", "medium")
OPUS_XHIGH = Rung("claude-opus-5-5", "xhigh")
OPUS_PROJECT = ProjectRung(LADDER, OPUS)
# The sentence with which an entry point that exists only here asks its worker to name the chunk's items.
NAMING = "Name the items."
SCAN = " Run a full scan only from such a subagent, since the scan's agents run on the model that starts them."
SEATS = " Launch each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung."
MARKDOWN = " Where your branch changes only Markdown, none of it `CLAUDE.md`, `AGENTS.md`, `workflow-bench/*-prompt.md` or under `agents/`, `principles/`, `rules/` or `skills/`, launch each reviewer on the project's rung instead, with the Agent tool set to model opus, effort medium."
def held(route, project=OPUS_PROJECT):
"""The rung sentences a-driver's worker reads on route, each after a space."""
return prompts.rungs_held("a-driver", route, project)
def resumed(route, project=OPUS_PROJECT):
"""The prompt that resumes a-driver's worker on route, with no evidence of work remaining."""
return prompts.resumed("a-driver", NAMING, "", route, project)
class Resumed(unittest.TestCase):
def test_resumes_the_worker_saying_when_the_chunk_is_finished(self):
self.assertTrue(resumed(Route(Rung("m")), None).startswith("Continue the task; the previous run ended before the chunk was finished. The chunk is finished once"))
class ReviewLaunches(unittest.TestCase):
def test_keeps_every_review_on_the_projects_rung_while_the_worker_runs_below_it(self):
below = held(Route(SONNET, 3, 2, 4))
self.assertTrue(below.startswith(f" Launch each reviewer, each comprehension seat and the security pass's subagent with the Agent tool set to model opus, effort medium, the project's rung.{SCAN} Your rung is below the project's: launch each subagent that files or scores plan items on the rung `~/.claude/bin/subagent-rung --plan claude-opus-5-5 <L>` prints. Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model fable, effort low` from 5, `model fable, effort medium` from 6, `model opus, effort xhigh` from 7 and `model fable, effort high` from 8. When an escalation trigger"), below)
self.assertNotIn("Launch each", held(Route(OPUS, 3, 2, 4)))
def test_launches_reviewers_on_the_rung_the_items_e_plus_i_picks_above_the_workers(self):
level = held(Route(OPUS, 3, 2, 6))
self.assertTrue(level.startswith(f" Launch each reviewer with the Agent tool set to model fable, effort medium, the rung item 3's E + I of 6 picks. Where an item you take into the chunk has an E + I of 7 or more, launch each reviewer with the Agent tool set to `model opus, effort xhigh` from 7 and `model fable, effort high` from 8.{MARKDOWN} When an escalation trigger"), level)
below = held(Route(SONNET, 3, 2, 5))
self.assertTrue(below.startswith(f" Launch each reviewer with the Agent tool set to model fable, effort low, the rung item 3's E + I of 5 picks.{SEATS}{SCAN} Your rung is below the project's:"), below)
self.assertNotIn("Launch each reviewer", held(Route(FABLE_MEDIUM, 3, 4, 6)))
def test_names_the_reviewers_rung_each_higher_e_plus_i_an_item_taken_in_picks(self):
taking_in = " Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model fable, effort low` from 5, `model fable, effort medium` from 6, `model opus, effort xhigh` from 7 and `model fable, effort high` from 8."
self.assertTrue(held(Route(OPUS, 3, 2, 4)).startswith(f"{taking_in} When an escalation trigger"))
self.assertIn(" Where an item you take into the chunk has an E + I of 7 or more, launch each reviewer with the Agent tool set to `model opus, effort xhigh` from 7 and `model fable, effort high` from 8.", held(Route(OPUS, 3, 2, 6)))
self.assertIn(" Where an item you take into the chunk has an E + I of 7 or more, launch each reviewer with the Agent tool set to `model opus, effort xhigh` from 7 and `model fable, effort high` from 8.", held(Route(FABLE_MEDIUM, 3, 4, 4)))
self.assertNotIn("Where an item you take into", held(Route(OPUS, 3, 2, 8)))
self.assertNotIn("Where an item you take into", held(Route(Rung("m")), None))
def test_launches_reviewers_on_the_rung_the_highest_e_plus_i_taken_in_picks(self):
self.assertIn(f"a-driver runs you on claude-opus-5-5 at medium effort. Launch each reviewer with the Agent tool set to model opus, effort xhigh, the rung picked by an item of E + I 7 taken into the chunk. Where an item you take into the chunk has an E + I of 8 or more, launch each reviewer with the Agent tool set to `model fable, effort high` from 8.{MARKDOWN} When an escalation trigger", resumed(Route(OPUS, 3, 2, 5, escape_and_impact_taken_in=7)))
self.assertIn(" Launch each reviewer with the Agent tool set to model fable, effort low, the rung picked by an item of E + I 5 taken into the chunk.", held(Route(OPUS, escape_and_impact_taken_in=5)))
def test_keeps_seats_and_the_security_pass_on_the_projects_rung_while_the_worker_runs_above_it(self):
above = Route(FABLE_LOW, 3, 3, 4)
sentences = f"{SEATS}{SCAN} Where an item you take into the chunk has an E + I of 6 or more, launch each reviewer with the Agent tool set to `model fable, effort medium` from 6, `model opus, effort xhigh` from 7 and `model fable, effort high` from 8.{MARKDOWN} When an escalation trigger"
self.assertIn(f"{prompts.ANSWERS_RECORDED}. {NAMING} a-driver runs you on claude-fable-5-2 at low effort.{sentences}", resumed(above))
self.assertNotIn("the project's", resumed(Route(OPUS, 3, 2, 4)))
self.assertNotIn("runs you on", resumed(Route(Rung("m")), None))
self.assertTrue(held(above).startswith(sentences), held(above))
def test_lets_a_markdown_only_branch_review_on_the_projects_rung_where_the_reviewers_run_above_it(self):
self.assertIn(MARKDOWN, held(Route(OPUS, 3, 2, 7)))
self.assertIn(MARKDOWN, held(Route(FABLE_LOW, 3, 3, 4)))
self.assertNotIn("only Markdown", held(Route(OPUS, 3, 2, 4)))
class Escalation(unittest.TestCase):
def test_names_the_rungs_an_escalation_trigger_lets_the_worker_hand_a_task_to_at_most_two_above_its_items(self):
routed = held(Route(OPUS, 3, 2, 4))
self.assertTrue(routed.startswith(" Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model fable, effort low` from 5, `model fable, effort medium` from 6, `model opus, effort xhigh` from 7 and `model fable, effort high` from 8. When an escalation trigger in `~/.claude/CLAUDE.md` fires, hand the hard part of the task to a subagent with the Agent tool set to `model fable, effort low`, one rung above yours; above that, ask the human first. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number."), routed)
self.assertIn(" When an escalation trigger in `~/.claude/CLAUDE.md` fires, hand the hard part of the task to a subagent with the Agent tool set to `model fable, effort medium`, one rung above yours, and on each further trigger one rung higher, up to `model opus, effort xhigh`; above that, ask the human first. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number.", held(Route(FABLE_LOW, 3, 3, 4)))
self.assertIn(" When an escalation trigger in `~/.claude/CLAUDE.md` fires, hand the hard part of the task to a subagent with the Agent tool set to `model opus, effort xhigh`, one rung above yours; above that, ask the human first. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number.", held(Route(FABLE_MEDIUM, 3, 3, 4)))
self.assertNotIn("escalation", held(Route(Rung("claude-fable-5-2", "max"), 3, 4, 8)))
self.assertIn(" An escalation trigger in `~/.claude/CLAUDE.md` lets you hand the hard part of a task to a subagent with the Agent tool set to `model fable, effort high`, one rung above yours, only once the human agrees; a-driver will not resume you higher for a trigger either, so ask the human for that yes.", held(Route(OPUS_XHIGH, 3, 3, 4)))
self.assertIn("set to `model fable, effort max`, one rung above yours, only once the human agrees;", held(Route(Rung("claude-fable-5-2", "high"), 3, 4, 5)), "one rung below the top still names the rung above")
self.assertNotIn("rerun", held(Route(Rung("claude-opus-5-5", "low"), 3, 3, 4)), "a worker off the ladder is told no rerun rung")
self.assertIn("hand the hard part of the task to a subagent with the Agent tool set to `model fable, effort high`, one rung above yours;", held(Route(OPUS_XHIGH, 3, 4, 5)))
self.assertNotIn("escalation", held(Route(Rung("m")), None))
self.assertNotIn("escalation", held(Route(Rung("claude-opus-5-5", "low"), 3, 3, 4)))
unrouted = held(Route(OPUS))
self.assertTrue(unrouted.startswith(" Where an item you take into the chunk has an E + I of 5 or more, launch each reviewer with the Agent tool set to `model fable, effort low` from 5, `model fable, effort medium` from 6, `model opus, effort xhigh` from 7 and `model fable, effort high` from 8. When an escalation trigger in `~/.claude/CLAUDE.md` fires, hand the hard part of the task to a subagent with the Agent tool set to `model fable, effort low`, one rung above yours, and on each further trigger one rung higher, up to `model fable, effort medium`; above that, ask the human first. Open the subagent's prompt with `Escalation trigger <n>:`, n the trigger's number. Where the hard part is the rest of the chunk, end your run instead, asking a-driver for a higher rung and naming a trigger that fired since your last step-up, if any: once the judge finds it in your transcript, the next cycle resumes you one rung up, up to `model fable, effort medium`. A reviewer"), unrouted)
def test_offers_the_worker_a_higher_rung_for_the_rest_of_the_chunk_up_to_the_escalation_cap(self):
offer = lambda cap: f" Where the hard part is the rest of the chunk, end your run instead, asking a-driver for a higher rung and naming a trigger that fired since your last step-up, if any: once the judge finds it in your transcript, the next cycle resumes you one rung up, up to `{cap}`."
self.assertIn(f"`Escalation trigger <n>:`, n the trigger's number.{offer('model fable, effort low')}", held(Route(OPUS, 3, 2, 4)))
self.assertIn(f"`Escalation trigger <n>:`, n the trigger's number.{offer('model opus, effort xhigh')}", held(Route(FABLE_LOW, 3, 3, 4)))
self.assertNotIn("Where the hard part", held(Route(OPUS_XHIGH, 3, 3, 4)))
class Reruns(unittest.TestCase):
def test_names_the_rung_a_reviewer_or_seat_reruns_on_one_above_its_own(self):
both = " A reviewer or comprehension seat that asks for a rerun one rung up reruns with the Agent tool set to `model fable, effort low`."
self.assertTrue(resumed(Route(OPUS, 3, 2, 4)).endswith(both))
self.assertTrue(resumed(Route(SONNET, 3, 2, 4)).endswith(both))
self.assertTrue(held(Route(OPUS)).endswith(both))
self.assertTrue(resumed(Route(FABLE_LOW, 3, 3, 4)).endswith(" A reviewer that asks for a rerun one rung up reruns with the Agent tool set to `model fable, effort medium`, and a comprehension seat with `model fable, effort low`."))
self.assertTrue(resumed(Route(OPUS, 3, 2, 7)).endswith(" A reviewer that asks for a rerun one rung up reruns with the Agent tool set to `model fable, effort high`, and a comprehension seat with `model fable, effort low`."))
self.assertTrue(resumed(Route(Rung("claude-fable-5-2", "max"), 3, 4, 8)).endswith(" A comprehension seat that asks for a rerun one rung up reruns with the Agent tool set to `model fable, effort low`."))
self.assertNotIn("rerun", resumed(Route(Rung("m")), None))
self.assertNotIn("rerun", held(Route(Rung("m", "high"), 3, 4, 5), None))
fable = Rung("claude-fable-5-2", "max")
self.assertNotIn("rerun", resumed(Route(fable), ProjectRung(LADDER, fable)))
if __name__ == "__main__":
unittest.main()
@@ -10,8 +10,8 @@ from agent_cycle.tmux import Window
@dataclasses.dataclass(frozen=True)
class Loop:
"""What stays the same through one run of loop-claude; run_loop builds it once per process, so a reload builds it afresh."""
# The checkout the loop runs in. LoopRun.root names it too, since a reload hands it on there.
"""What stays the same while an entry point drives one chunk in one process. loop-claude builds it once per process, so a reload builds it afresh; do-epic builds one per chunk."""
# The checkout the loop runs in. loop-claude's LoopRun.root names it too, since a reload hands it on there.
root: str
# The worker's session.
session: Session
@@ -19,6 +19,6 @@ class Loop:
window: Window
paths: Paths
env: typing.Mapping[str, str]
# The loop-claude script, whose edits reload the loop and beside which chunk-preflight lies.
# The entry point's script, beside which chunk-preflight lies; an edit of loop-claude's reloads its loop.
launcher: pathlib.Path
effects: Effects
+6 -32
View File
@@ -1,9 +1,7 @@
"""The human approval that `chunk-pr merge` waits for where the project's config sets `wait-for-approval`."""
import json
import pathlib
import time
import typing
from forge.reviews import APPROVED, CHANGES_REQUESTED, github_decision, REVIEW_REQUIRED, waiting_threads
from failure import Failure
from forge.commands import Forge, Run
@@ -21,32 +19,21 @@ def awaited(forge: Forge, wait_for_approval: bool) -> bool | Failure:
return wait_for_approval
# Each review thread, with whether it is resolved and who wrote its last comment, beside who opened the pull request. gh fills {owner} and {repo} from the checkout.
THREADS = """query($owner: String!, $repo: String!, $number: Int!) { repository(owner: $owner, name: $repo) { pullRequest(number: $number) {
author { login }
reviewThreads(first: 100) { nodes { isResolved path comments(last: 1) { nodes { author { login } } } } }
} } }"""
def approval(number: str, run: Run) -> bool | Failure:
"""Whether GitHub reports pull request number approved, as its branch protection asks, read once: False while it still wants a review. A Failure where a reviewer requested changes, or where the protection asks for no review."""
viewed = run(["gh", "pr", "view", number, "--json", "reviewDecision"])
try:
decision = viewed if isinstance(viewed, Failure) else json.loads(viewed)["reviewDecision"]
except (ValueError, KeyError, TypeError):
return Failure(f"gh pr view printed no review decision: {str(viewed).strip()[:200]}")
decision = github_decision(number, run)
if isinstance(decision, Failure):
return decision
if decision == "CHANGES_REQUESTED":
if decision == CHANGES_REQUESTED:
return Failure(f"a reviewer requested changes on pull request {number}")
if not decision:
return Failure("wait-for-approval is set, but the branch protection requires no review, so GitHub reports no decision; ask the human to require a review in the protection or to set wait-for-approval to false")
if decision not in ("APPROVED", "REVIEW_REQUIRED"):
if decision not in (APPROVED, REVIEW_REQUIRED):
return Failure(f"GitHub reports the review decision {decision!r} on pull request {number}, and chunk-pr doesn't know that decision")
return decision == "APPROVED"
return decision == APPROVED
def approved(number: str, run: Run, sleep: typing.Callable[[float], None] = time.sleep) -> Failure | None:
def approved(number: str, run: Run, sleep: typing.Callable[[float], None]) -> Failure | None:
"""None once GitHub reports pull request number approved; a Failure where `approval` fails, where a reviewer's comment waits for an answer, or where no approval came within WAIT_SECONDS."""
for waited in range(0, WAIT_SECONDS + 1, POLL_SECONDS):
read = approval(number, run)
@@ -73,16 +60,3 @@ def unanswered(number: str, run: Run) -> Failure | None:
if not waiting:
return None
return Failure(f"a reviewer's comment on pull request {number} waits for an answer, in {', '.join(waiting)}; answer it in its thread, push any fix, and run chunk-pr merge again")
def waiting_threads(number: str, run: Run) -> list[str] | Failure:
"""The files where a reviewer's comment waits for an answer on pull request number, read once: those of each review thread nobody resolved, whose last comment someone other than the pull request's author wrote. So a thread the worker answered, as that author, waits for the reviewer again."""
read = run(["gh", "api", "graphql", "-f", f"query={THREADS}", "-F", "owner={owner}", "-F", "repo={repo}", "-F", f"number={number}"])
if isinstance(read, Failure):
return read
try:
pull = json.loads(read)["data"]["repository"]["pullRequest"]
author = pull["author"]["login"]
return sorted({thread["path"] for thread in pull["reviewThreads"]["nodes"] if not thread["isResolved"] and any(comment["author"]["login"] != author for comment in thread["comments"]["nodes"])})
except (ValueError, KeyError, TypeError):
return Failure(f"gh api graphql printed no review threads: {read.strip()[:200]}")
+2 -2
View File
@@ -139,7 +139,7 @@ def step(argv: list[str], run: Run, config: Config, parks: bool = False) -> str
return Failure("give a title of one line") if title and not one_line(title[0]) else (misfit(title[0], run, config) if title else None) or marked_ready(run, title[0] if title else None)
case ["merge", *given]:
finishes = finishes_of(given)
return Failure(USAGE) if finishes is None else merged(run, config, finishes=finishes, parks=parks)
return Failure(USAGE) if finishes is None else merged(run, config, time.sleep, finishes=finishes, parks=parks)
case ["clean"]:
return cleaned(run)
return Failure(USAGE)
@@ -305,7 +305,7 @@ def finishes_of(given: list[str]) -> list[list[str]] | None:
return finishes
def merged(run: Run, config: Config, sleep: typing.Callable[[float], None] = time.sleep, finishes: typing.Sequence[list[str]] = (), parks: bool = False) -> str | Failure:
def merged(run: Run, config: Config, sleep: typing.Callable[[float], None], finishes: typing.Sequence[list[str]] = (), parks: bool = False) -> str | Failure:
"""Merge the branch's pull request in the project's merge queue. Each pass of the loop is one turn in the queue; a pull request that still wants its approval leaves the queue while it waits, so the next chunk merges meanwhile, and takes a new turn once approved. Where parks, it doesn't wait for the approval: it fails at once, as `left_to_park` says."""
found = pull(run)
if isinstance(found, Failure):
+19 -19
View File
@@ -16,7 +16,7 @@ from failure import Failure
from test_env import isolated_env
from project_config.settings import ProjectConfig
from forge.commands import Run
from chunk_pr.approval import THREADS
from forge.reviews import THREADS
TOOL = pathlib.Path(__file__).resolve().parents[1] / "chunk-pr"
GITHUB = "git@github.com:me/project.git"
@@ -603,7 +603,7 @@ class MergeQueue(unittest.TestCase):
def test_a_branch_behind_the_default_branch_merges_it_in_and_pushes_before_the_checks(self):
checkout = self.moved(Checkout(GITHUB, open=True))
self.assertEqual(merged(checkout.__call__, checkout.config), "Merged pull request #12 (merge).")
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None), "Merged pull request #12 (merge).")
self.assertEqual(changes(checkout), [MERGE, self.PUSHED, ["gh", "pr", "merge", "12", "--merge", "--match-head-commit", "decaf"]])
self.assertLess(checkout.ran.index(self.PUSHED), checkout.ran.index(GH_CHECKS))
@@ -611,7 +611,7 @@ class MergeQueue(unittest.TestCase):
checkout = self.moved(Checkout(GITHUB, open=True))
checkout.answers[tuple(MERGE)] = Failure("git merge --quiet: CONFLICT")
checkout.answers[("git", "diff", "--name-only", "--diff-filter=U")] = "bin/tool.py\n"
failed = merged(checkout.__call__, checkout.config)
failed = merged(checkout.__call__, checkout.config, lambda seconds: None)
self.assertIsInstance(failed, Failure)
self.assertIn("conflicts in bin/tool.py; the merge stays in progress: resolve them, commit, push, and run chunk-pr merge again", failed.reason)
self.assertEqual((changes(checkout), MERGE in checkout.ran), ([], True))
@@ -620,7 +620,7 @@ class MergeQueue(unittest.TestCase):
def test_a_finished_item_is_committed_under_the_configs_title_and_pushed_before_the_merge(self):
checkout, folder = self.with_plan(Checkout(GITHUB, open=True))
files = [f"{folder}/todo.md", f"{folder}/CHANGELOG.md"]
self.assertEqual(merged(checkout.__call__, checkout.config, finishes=[["7", "--changelog", "Exports keep their order."]]), "Merged pull request #12 (merge).")
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7", "--changelog", "Exports keep their order."]]), "Merged pull request #12 (merge).")
self.assertEqual(changes(checkout)[:1], [[self.TODO_ITEM, "finish", "7", "--changelog", "Exports keep their order."]])
self.assertEqual(changes(checkout)[-4:], [
["git", "-C", folder, "add", "--", *files],
@@ -637,7 +637,7 @@ class MergeQueue(unittest.TestCase):
def test_an_item_the_plan_no_longer_holds_is_named_on_the_last_line_and_what_an_earlier_run_left_uncommitted_is_committed(self):
checkout, folder = self.with_plan(Checkout(GITHUB, open=True), held=False)
self.assertEqual(merged(checkout.__call__, checkout.config, finishes=[["7"]]), "Merged pull request #12 (merge). The plan held no item 7 when merge began, so merge left the plan as it was there.")
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7"]]), "Merged pull request #12 (merge). The plan held no item 7 when merge began, so merge left the plan as it was there.")
self.assertNotIn([self.TODO_ITEM, "finish", "7"], checkout.ran)
self.assertEqual([command[3] for command in changes(checkout)[:-1]], ["add", "commit", "origin"])
self.assertEqual(changes(checkout)[-1], ["gh", "pr", "merge", "12", "--merge", "--match-head-commit", "f00d"])
@@ -645,7 +645,7 @@ class MergeQueue(unittest.TestCase):
def test_each_item_the_plan_no_longer_holds_is_named(self):
checkout, _ = self.with_plan(Checkout(GITHUB, open=True), held=False)
checkout.answers[(self.TODO_ITEM, "show", "8")] = Failure("todo-item show 8: todo.md holds no item 8")
self.assertIn("The plan held no item 7, 8 when merge began", str(merged(checkout.__call__, checkout.config, finishes=[["7"], ["8"]])))
self.assertIn("The plan held no item 7, 8 when merge began", str(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7"], ["8"]])))
def test_a_plan_that_turns_unreadable_inside_the_turn_leaves_it_unmerged(self):
checkout, _ = self.with_plan(Checkout(GITHUB, open=True))
@@ -653,7 +653,7 @@ class MergeQueue(unittest.TestCase):
def run_whose_second_show_fails(command: list[str]) -> str | Failure:
shows.extend([command] if command[1:2] == ["show"] else [])
return Failure("todo-item show 7: unreadable") if command[1:2] == ["show"] and len(shows) == 2 else run(command)
self.assertEqual(merged(run_whose_second_show_fails, checkout.config, finishes=[["7"]]), Failure("todo-item show 7: unreadable"))
self.assertEqual(merged(run_whose_second_show_fails, checkout.config, lambda seconds: None, finishes=[["7"]]), Failure("todo-item show 7: unreadable"))
self.assertNotIn([self.TODO_ITEM, "finish", "7"], checkout.ran)
def test_a_finish_that_changes_no_file_commits_and_pushes_nothing(self):
@@ -664,13 +664,13 @@ class MergeQueue(unittest.TestCase):
checkout.answers[("git", "-C", folder, "status", "--porcelain", "--", f"{folder}/todo.md", f"{folder}/CHANGELOG.md")] = "\n"
else:
checkout.answers.update({(self.CONFIG, "plan"): "PAY-1\n", (self.CONFIG, "changelog"): f"{folder}/none.md\n"})
self.assertEqual(merged(checkout.__call__, checkout.config, finishes=[["7", "--no-changelog"]]), "Merged pull request #12 (merge).")
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7", "--no-changelog"]]), "Merged pull request #12 (merge).")
self.assertEqual(changes(checkout), [[self.TODO_ITEM, "finish", "7", "--no-changelog"], ["gh", "pr", "merge", "12", "--merge", "--match-head-commit", HEAD]])
def test_a_branch_outside_the_branch_template_titles_the_finish_with_the_summary_alone(self):
checkout, folder = self.with_plan(Checkout(GITHUB, open=True))
checkout.answers[tuple(BRANCH)] = "feature/PAY-{id}\n"
merged(checkout.__call__, checkout.config, finishes=[["7"]])
merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7"]])
self.assertIn("Finish item 7 in the plan", [command[6] for command in checkout.ran if command[3:4] == ["commit"]])
def test_a_plan_it_cannot_read_or_an_item_it_cannot_finish_leaves_it_unmerged(self):
@@ -678,7 +678,7 @@ class MergeQueue(unittest.TestCase):
with self.subTest(failing):
checkout, _ = self.with_plan(Checkout(GITHUB, open=True))
checkout.answers[(self.TODO_ITEM, failing, "7")] = Failure(f"todo-item {failing} 7: no config names the checkout")
self.assertEqual(merged(checkout.__call__, checkout.config, finishes=[["7"]]), Failure(f"todo-item {failing} 7: no config names the checkout"))
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7"]]), Failure(f"todo-item {failing} 7: no config names the checkout"))
self.assertNotIn(self.PUSHED, checkout.ran)
def test_a_step_of_the_turn_that_fails_leaves_it_unmerged_with_that_failure(self):
@@ -690,23 +690,23 @@ class MergeQueue(unittest.TestCase):
def failing_run(command: list[str]) -> str | Failure:
asked = ["<plan>" if part == folder else part for part in command]
return Failure("that step failed") if tuple(asked[:len(failing)]) == tuple(failing) else run(command)
self.assertEqual(merged(failing_run, checkout.config, finishes=[["7"]]), Failure("that step failed"))
self.assertEqual(merged(failing_run, checkout.config, lambda seconds: None, finishes=[["7"]]), Failure("that step failed"))
self.assertNotIn("merge", [command[2] for command in checkout.ran if command[0] == "gh"])
def test_a_finish_that_fails_after_the_default_branch_was_merged_in_still_pushes_the_branch(self):
checkout, _ = self.with_plan(self.moved(Checkout(GITHUB, open=True)))
checkout.answers[(self.TODO_ITEM, "finish", "7")] = Failure("todo-item finish 7: a decision cites item 7")
self.assertEqual(merged(checkout.__call__, checkout.config, finishes=[["7"]]), Failure("todo-item finish 7: a decision cites item 7"))
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7"]]), Failure("todo-item finish 7: a decision cites item 7"))
self.assertEqual(changes(checkout), [MERGE, self.PUSHED])
def test_the_scoring_run_is_asked_for_once_the_finish_is_committed_and_one_owed_leaves_it_unmerged(self):
checkout, _ = self.with_plan(Checkout(GITHUB, open=True))
commit = lambda: next(index for index, command in enumerate(checkout.ran) if command[3:4] == ["commit"])
self.assertEqual(merged(checkout.__call__, checkout.config, finishes=[["7"]]), "Merged pull request #12 (merge).")
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7"]]), "Merged pull request #12 (merge).")
self.assertLess(commit(), checkout.ran.index(OWED))
checkout, _ = self.with_plan(Checkout(GITHUB, open=True))
checkout.answers[tuple(OWED)] = Failure("the chunk owes 1 scoring run")
self.assertEqual(merged(checkout.__call__, checkout.config, finishes=[["7"]]), Failure("the chunk owes 1 scoring run"))
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None, finishes=[["7"]]), Failure("the chunk owes 1 scoring run"))
self.assertEqual(changes(checkout)[-1], self.PUSHED)
def test_a_merge_holds_the_queue_through_the_checks_and_frees_it_while_it_waits_for_an_approval(self):
@@ -740,7 +740,7 @@ class MergeQueue(unittest.TestCase):
checkout = Checkout(GITHUB, open=True)
checkout.answers[tuple(WAITS)] = "true\n"
checkout.threads = [self.thread("bin/asked.py", "reviewer")]
failed = merged(checkout, checkout.config)
failed = merged(checkout, checkout.config, lambda seconds: None)
self.assertIsInstance(failed, Failure)
self.assertIn("waits for an answer, in bin/asked.py;", failed.reason)
self.assertEqual(changes(checkout), [])
@@ -766,7 +766,7 @@ class MergeQueue(unittest.TestCase):
with self.subTest(name):
checkout = Checkout(GITHUB, open=True)
checkout.answers.update({(self.CONFIG, "comprehension-history"): f"{history}\n", **answers})
done = merged(checkout.__call__, checkout.config)
done = merged(checkout.__call__, checkout.config, lambda seconds: None)
self.assertEqual(isinstance(done, Failure) and "run the paired ruling again" in done.reason, stops, done)
self.assertEqual(changes(checkout), [] if stops else [["gh", "pr", "merge", "12", "--merge", "--match-head-commit", HEAD]])
@@ -774,14 +774,14 @@ class MergeQueue(unittest.TestCase):
ruled = ("git", "log", "-1", "--format=%H", "-G", "^Ruling: ", "origin/main..HEAD", "--", "/checkout/history.md")
checkout = Checkout(GITHUB, open=True)
checkout.answers[(self.CONFIG, "comprehension-history")] = "/elsewhere/history.md\n"
self.assertEqual(merged(checkout.__call__, checkout.config), "Merged pull request #12 (merge).")
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None), "Merged pull request #12 (merge).")
self.assertNotIn("log", [command[1] for command in checkout.ran])
for failing in (ruled, ("git", "rev-list", "--count", "bead..origin/main"), ("git", "rev-parse", "--show-toplevel")):
with self.subTest(failing):
checkout = Checkout(GITHUB, open=True)
checkout.answers.update({(self.CONFIG, "comprehension-history"): "/checkout/history.md\n", ruled: "bead\n"})
checkout.answers[failing] = Failure("git failed")
self.assertEqual(merged(checkout.__call__, checkout.config), Failure("git failed"))
self.assertEqual(merged(checkout.__call__, checkout.config, lambda seconds: None), Failure("git failed"))
def test_a_turn_keeps_every_other_merge_out_until_it_ends_even_by_an_error(self):
with self.assertRaises(ValueError):
@@ -809,7 +809,7 @@ class MergeQueue(unittest.TestCase):
def test_a_queue_it_cannot_open_leaves_it_unmerged(self):
checkout = Checkout(GITHUB, open=True)
checkout.answers[tuple(COMMON)] = "/nonexistent/folder\n"
self.assertEqual(merged(checkout, checkout.config), Failure("cannot take a turn in the merge queue, /nonexistent/folder/chunk-pr-merge-queue: No such file or directory"))
self.assertEqual(merged(checkout, checkout.config, lambda seconds: None), Failure("cannot take a turn in the merge queue, /nonexistent/folder/chunk-pr-merge-queue: No such file or directory"))
self.assertEqual(checkout.acted(), [])
+1 -2
View File
@@ -11,7 +11,6 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from project_config.templates import BRANCH
from release_plan.next_chunk import CarriedOver, NoItem
from chunk_preflight import choices
from failure import Failure
from chunk_preflight.gathered import Gathered, ItemFacts, NamedPath, Ran, Worktree
from test_env import isolated_env
@@ -21,7 +20,7 @@ from release_plan import next_chunk
from chunk_preflight.facts import open_pulls, render
from plan_store.todo_md import plan_of
from chunk_preflight.choices import worktree_add
from project_config.test_project_config import write_config
from project_config.doubles import write_config
BIN = pathlib.Path(__file__).resolve().parents[1]
README = "# Project\n\n## Goals\n\n**Void:** code health, since the plan's scores weigh goal 1 alone.\n\n1. **Achieve:** Work.\n\n## Audience\n\nMe.\n"
+7 -1
View File
@@ -1,9 +1,15 @@
#!/usr/bin/env python3
import functools
import gzip
import os
import pathlib
import sys
import time
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from compress_archives import main
from text_files.compressed import compress
from text_files.file_writes import replace_file
sys.exit(main.main(sys.argv[1:]))
sys.exit(main.main(sys.argv[1:], os.environ, pathlib.Path.home(), time.time, functools.partial(compress, pack=gzip.compress, replace=replace_file, sync=os.fsync)))
+3 -6
View File
@@ -14,31 +14,28 @@ its plain name. README.md → Install has the crontab line that runs it daily.
It prints how many files it compressed. It exits 1, saying why, on any argument but --help or -h,
and at the first file it cannot compress, leaving that file and every one after it plain.
"""
import os
import pathlib
import stat
import sys
import time
import typing
from transcript_archive.location import archive_root
from text_files.compressed import compress, SUFFIX
from failure import Failure
from review_rules.review_folder import REPORTS
from text_files.compressed import SUFFIX
# Why 30 days: docs/decisions.md → `archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them`.
AGE_DAYS = 30
def main(argv: list[str], env: typing.Mapping[str, str] = os.environ, home: pathlib.Path | None = None, now: float | None = None) -> int:
def main(argv: list[str], env: typing.Mapping[str, str], home: pathlib.Path, now: typing.Callable[[], float], compress: typing.Callable[[pathlib.Path], Failure | None]) -> int:
if argv in (["-h"], ["--help"]):
print(__doc__.strip())
return 0
if argv:
print(f"unknown arguments {' '.join(argv)}; usage: compress-archives", file=sys.stderr)
return 1
home = home or pathlib.Path.home()
cutoff = (time.time() if now is None else now) - AGE_DAYS * 24 * 60 * 60
cutoff = now() - AGE_DAYS * 24 * 60 * 60
count = 0
for path in [*old_files(archive_root(home, env), "*.jsonl", cutoff), *old_files(home / REPORTS, "*", cutoff)]:
if failed := compress(path):
@@ -11,6 +11,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from text_files.doubles import compress
from failure import Failure
from compress_archives import main
@@ -43,10 +44,10 @@ class Main(unittest.TestCase):
path.write_bytes(f"{path.name}\n".encode())
os.utime(path, (NOW - seconds_ago, NOW - seconds_ago))
def run_main(self, argv=(), env=None, now=NOW):
def run_main(self, argv=(), env=None, now=NOW, compress=compress):
out, err = io.StringIO(), io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main(list(argv), env or {}, self.home, now)
code = main.main(list(argv), env or {}, self.home, lambda: now, compress)
return code, out.getvalue(), err.getvalue()
def kept(self):
@@ -80,9 +81,8 @@ class Main(unittest.TestCase):
self.assertEqual(self.run_main(now=now), (0, "Compressed 0 files.\n", ""))
self.assertTrue((self.archive / "a.jsonl").is_file())
def test_measures_the_30_days_from_the_clock_by_default(self):
with mock.patch.object(main.time, "time", return_value=NOW):
self.assertEqual(self.run_main(now=None), (0, "Compressed 4 files.\n", ""))
def test_measures_the_30_days_from_the_clock_it_is_given(self):
self.assertEqual(self.run_main(now=NOW), (0, "Compressed 4 files.\n", ""))
def test_walks_on_past_a_gzip_a_link_and_a_file_removed_while_it_walks(self):
gone = mock.Mock(**{"is_symlink.return_value": False, "stat.side_effect": FileNotFoundError})
@@ -109,8 +109,7 @@ class Main(unittest.TestCase):
self.assertEqual(self.run_main(), (0, "Compressed 0 files.\n", ""))
def test_stops_at_the_first_failure_keeping_that_file_and_every_later_one(self):
with mock.patch.object(main, "compress", return_value=Failure("could not compress x: full")):
code, out, err = self.run_main()
code, out, err = self.run_main(compress=lambda path: Failure("could not compress x: full"))
self.assertEqual((code, out, err), (1, "", "could not compress x: full\n"))
self.assertTrue((self.archive / "a.jsonl").is_file())
+5 -4
View File
@@ -6,6 +6,7 @@ import pathlib
import shutil
import sys
import tempfile
import typing
from release_plan.release_standing import chunk_item, Cut, default_plan, Next, ReadyToCut, ReviewDue, Unplanned
from project_config.settings import DEFAULT_FLOOR
@@ -21,7 +22,7 @@ from deploy_release import release_pull
from project_config import templates
def cut(clone: str, checkout: str) -> str | Failure:
def cut(clone: str, checkout: str, sleep: typing.Callable[[float], None]) -> str | Failure:
"""What the cut did, as the tool prints it; a Failure with git's, the forge's or the tool's reason where a step stopped."""
if isinstance(same := same_origin(clone, checkout), Failure):
return same
@@ -39,7 +40,7 @@ def cut(clone: str, checkout: str) -> str | Failure:
title, tag = f"Cut release {named}", f"v{named}"
merged = commit_titled(title, ref, clone)
if merged is None:
landed = land(clone, ref, named, title)
landed = land(clone, ref, named, title, sleep)
if isinstance(landed, Failure):
return landed
if isinstance(fetched := git("fetch", "--quiet", "origin", cwd=clone), Failure):
@@ -149,7 +150,7 @@ def commit_titled(title: str, ref: str, clone: str) -> str | None | Failure:
return next((line.split(" ", 1)[0] for line in log.splitlines() if line.split(" ", 1)[1:] == [title]), None)
def land(clone: str, ref: str, named: str, title: str) -> Failure | None:
def land(clone: str, ref: str, named: str, title: str, sleep: typing.Callable[[float], None]) -> Failure | None:
"""Push the branch release-<named> with the cut commit and open its pull request, where an earlier run hasn't; then wait for its checks and merge it."""
url = git("config", "--get", "remote.origin.url", cwd=clone)
if isinstance(url, Failure):
@@ -182,7 +183,7 @@ def land(clone: str, ref: str, named: str, title: str) -> Failure | None:
if isinstance(required, Failure):
return required
said(f"waiting for the checks of pull request {number}" + (f"; {base} requires {', '.join(sorted(required))}" if required else ""))
if isinstance(passed := pulls.checks_passed(on, number, head, required, run), Failure):
if isinstance(passed := pulls.checks_passed(on, number, head, required, run, sleep), Failure):
return passed
said(f"the checks of pull request {number} passed")
if isinstance(merged := release_pull.merge(on, number, head, run), Failure):
+2 -1
View File
@@ -39,6 +39,7 @@ move stopped, fix what it names and run a bare deploy-release.
import os
import pathlib
import sys
import time
from deploy_release.cut import cut
from deploy_release.deploy import deploy
@@ -55,7 +56,7 @@ def main(argv: list[str]) -> int:
print("deploy-release: usage: deploy-release [--cut] [<checkout>]", file=sys.stderr)
return 1
checkout = rest[0] if rest else str(pathlib.Path.home() / ".claude")
moved = cut(os.getcwd(), checkout) if cutting else deploy(checkout)
moved = cut(os.getcwd(), checkout, time.sleep) if cutting else deploy(checkout)
if isinstance(moved, Failure):
print(f"deploy-release: {moved.reason}", file=sys.stderr)
return 1
+1 -1
View File
@@ -10,4 +10,4 @@ from loop_effects import effects
from do_epic import main
from loop_state import paths
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home()), LAUNCHER, effects.system_effects()))
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home()), LAUNCHER, effects.system_effects(pathlib.Path.home(), os.environ)))
+2 -2
View File
@@ -6,6 +6,6 @@ import sys
LAUNCHER = pathlib.Path(__file__).resolve()
sys.path.insert(0, str(LAUNCHER.parent))
from do_epic.e2e import main
from do_epic.e2e import main, sandbox, scenario
sys.exit(main.main(sys.argv[1:], os.environ, LAUNCHER.parent / "do-epic"))
sys.exit(main.main(sys.argv[1:], os.environ, LAUNCHER.parent / "do-epic", sandbox.epic_made, scenario.run))
+18 -19
View File
@@ -5,16 +5,16 @@ import typing
from adf_codec.convert import adfs_to_markdown, markdown_to_adf, Refused
from atlassian.client import basic_auth, Http, jira_json, ToAdf, ToMarkdown
from plan_store.jira_epic import blocked_by_links, DONE, FILED_BY, finishing_transition, ISSUE_KEY, ISSUE_TYPE, move_refusal, PERHAPS_MOVED, project_of, PROPERTY, rank_refusal, scores, searched
from atlassian.jira import DONE, entry_of, finishing_transition, ISSUE_KEY, MAX_PAGES, move_refusal, PERHAPS_FILED, PERHAPS_MOVED, project_of, rank_refusal, searched, SHAPE, status_category, unshaped
from project_config.settings import Epic, Finish, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION
from do_epic.backend.protocol import EpicIssue, no_child, no_parent, no_words, unfit, unfit_label
from failure import Failure
from plan_store.jira_epic import FILED_BY, ISSUE_TYPE, PROPERTY, scores
FIELDS = ["summary", "description", "labels", "assignee", "status", "parent", "issuelinks"]
# How many child issues one search for sub-tasks names, which keeps its JQL well inside the length Jira takes.
PARENTS_PER_SEARCH = 50
NO_FINISH = "do-epic's settings name no transition to {to} by: set `{transition}` and `{status}` in ~/.claude-projects/do-epic/config.toml, to the names Jira shows, such as \"Mark as done\" and \"Done\""
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under {parent} before filing it again"
@dataclasses.dataclass(frozen=True)
@@ -33,6 +33,8 @@ class JiraEpic:
to_markdown: ToMarkdown = adfs_to_markdown
to_adf: ToAdf = markdown_to_adf
finish: Finish | None = None
# The pages a search may take before the read fails.
max_pages: int = MAX_PAGES
@property
def epic_key(self) -> str:
@@ -74,10 +76,10 @@ class JiraEpic:
fields["description"] = description
created = self.http("POST", f"{self.epic.site}/rest/api/3/issue", authorization, {"fields": fields | {"issuetype": kind}, "properties": properties})
if isinstance(created, Failure):
return Failure(f"cannot file the issue under {parent}: {created.reason}; {PERHAPS_FILED.format(parent=parent)}")
return Failure(f"cannot file the issue under {parent}: {created.reason}; {PERHAPS_FILED.format(under=parent)}")
key = created.get("key") if isinstance(created, dict) else None
if not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
return Failure(f"Jira answered the new issue without an issue key; {PERHAPS_FILED.format(parent=parent)}")
return Failure(f"Jira answered the new issue without an issue key; {PERHAPS_FILED.format(under=parent)}")
return key
def add_label(self, key: str, label: str) -> None | Failure:
@@ -125,11 +127,10 @@ class JiraEpic:
answered = self.http("GET", f"{self.epic.site}/rest/api/3/issue/{self.epic_key}?fields=status", authorization, None)
if isinstance(answered, Failure):
return Failure(f"cannot read {self.epic_key}'s status: {answered.reason}")
try:
category = answered["fields"]["status"]["statusCategory"]["key"] # type: ignore[index]
except (KeyError, TypeError) as failure:
return Failure(f"{self.epic_key}'s status is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}")
return bool(category == "done")
category = status_category(answered)
if isinstance(category, Failure):
return Failure(f"{self.epic_key}'s status {SHAPE}: {category.reason}")
return category == DONE
def close(self, summary: str) -> None | Failure:
if self.finish is None:
@@ -223,7 +224,7 @@ class JiraEpic:
def searched(self, what: str, parents: tuple[str, ...], authorization: str) -> list[Found] | Failure:
"""The issues whose parent is one of parents, in rank order."""
named = ", ".join(f'"{key}"' for key in parents)
answered = searched(self.epic.site, what, f"parent in ({named}) ORDER BY Rank ASC", {"fields": FIELDS, "properties": [PROPERTY]}, authorization, self.http)
answered = searched(self.epic.site, what, f"parent in ({named}) ORDER BY Rank ASC", {"fields": FIELDS, "properties": [PROPERTY]}, authorization, self.http, self.max_pages)
if isinstance(answered, Failure):
return answered
found = [found_of(raw) for raw in answered]
@@ -243,19 +244,17 @@ class JiraEpic:
def found_of(raw: object) -> Found | Failure:
"""The issue in one entry of a search's answer, or which issue is not in the shape Jira documents."""
key = raw.get("key") if isinstance(raw, dict) else None
if not isinstance(raw, dict) or not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
return Failure("the search answered an issue without an issue key")
# The answer's nested fields are read as Jira documents them; one missing or of another type ends the read here.
entry = entry_of(raw, PROPERTY)
if isinstance(entry, Failure):
return entry
key, fields, held = entry.key, entry.fields, entry.held
blockers = tuple(link.blocker for link in entry.links)
try:
fields = raw["fields"]
summary, labels, assignee, status, parent, description = fields["summary"], fields["labels"], fields["assignee"], fields["status"], fields["parent"]["key"], fields["description"]
name = assignee["displayName"] if assignee is not None else None
state, done = status["name"], status["statusCategory"]["key"] == "done"
blockers = tuple(link.blocker for link in blocked_by_links(fields))
held = raw.get("properties", {}).get(PROPERTY, {})
state, done = status["name"], status["statusCategory"]["key"] == DONE
except (AttributeError, KeyError, TypeError) as failure:
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}")
return Failure(unshaped(key, failure))
texts = (summary, *labels, state, parent, *blockers) if isinstance(labels, list) else ()
if not texts or not all(isinstance(text, str) for text in texts) or not isinstance(name, str | None) or not all(ISSUE_KEY.fullmatch(each) for each in (parent, *blockers)) or not isinstance(description, dict | None):
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: its summary, labels, assignee, status, parent, description or a linked issue's key holds another kind of value")
+10 -12
View File
@@ -5,14 +5,12 @@ import re
import sys
import tempfile
import unittest
import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from project_config.settings import Epic, Finish
from do_epic.backend.contract import EpicStoreContract
from failure import Failure
from plan_store import jira_epic
from do_epic.backend.jira import JiraEpic
from adf_codec.convert import Refused
@@ -214,7 +212,7 @@ class Jira(Case):
self.assertEqual(self.new_store().close("Shipped."), Failure("posted the summary on ABC-12, but cannot move ABC-12 by 'Mark as done': Jira answered 503; close the epic in Jira rather than posting the summary again"))
def test_says_why_the_epics_status_cannot_be_read(self) -> None:
for answer, reason in ((Failure("Jira answered 503"), "cannot read ABC-12's status: Jira answered 503"), ({"fields": {}}, "ABC-12's status is not in the shape Jira Cloud's REST API version 3 documents: KeyError('status')"), (["ABC-12"], "ABC-12's status is not in the shape Jira Cloud's REST API version 3 documents: TypeError('list indices must be integers or slices, not str')")):
for answer, reason in ((Failure("Jira answered 503"), "cannot read ABC-12's status: Jira answered 503"), ({"fields": {}}, "ABC-12's status is not in the shape Jira Cloud's REST API version 3 documents: its status category has no key that is text"), (["ABC-12"], "ABC-12's status is not in the shape Jira Cloud's REST API version 3 documents: its status category has no key that is text"), ({"fields": {"status": {"statusCategory": {"key": 3}}}}, "ABC-12's status is not in the shape Jira Cloud's REST API version 3 documents: its status category has no key that is text")):
self.jira.answers[("GET", EPIC_STATUS)] = answer
self.assertEqual(self.new_store().closed(), Failure(reason))
@@ -328,14 +326,15 @@ class Jira(Case):
("ABC-34", "the search answered an issue without an issue key"),
({"fields": valid}, "the search answered an issue without an issue key"),
({"key": "abc-34", "fields": valid}, "the search answered an issue without an issue key"),
({"key": "ABC-34", "fields": {"summary": "Count the runs.", "parent": {"key": "ABC-12"}}}, shape + ": KeyError('labels')"),
({"key": "ABC-34", "fields": {"summary": "Count the runs.", "parent": {"key": "ABC-12"}, "issuelinks": []}}, shape + ": KeyError('labels')"),
*((({"key": "ABC-34", "fields": valid | changed}, shape + ": its summary, labels, assignee, status, parent, description or a linked issue's key holds another kind of value") for changed in (
{"summary": 3}, {"labels": "api"}, {"labels": [3]}, {"assignee": {"displayName": 0}}, {"status": {"name": 0, "statusCategory": {"key": "new"}}}, {"parent": {"key": "ABC"}}, {"parent": {"key": 12}}, {"description": "text"}, {"issuelinks": [linked | {"inwardIssue": {"key": 7}}]}, {"issuelinks": [linked | {"inwardIssue": {"key": "ABC-x"}}]},
{"summary": 3}, {"labels": "api"}, {"labels": [3]}, {"assignee": {"displayName": 0}}, {"status": {"name": 0, "statusCategory": {"key": "new"}}}, {"parent": {"key": "ABC"}}, {"parent": {"key": 12}}, {"description": "text"}, {"issuelinks": [linked | {"inwardIssue": {"key": "ABC-x"}}]},
))),
({"key": "ABC-34", "fields": {key: value for key, value in valid.items() if key != "issuelinks"}}, shape + ": KeyError('issuelinks')"),
({"key": "ABC-34", "fields": {key: value for key, value in valid.items() if key != "issuelinks"}}, shape + ": its issuelinks are no list"),
({"key": "ABC-34", "fields": valid, "properties": {"plan-item": "L 2"}}, "ABC-34's issue property plan-item is not a JSON object"),
({"key": "ABC-34", "fields": valid, "properties": {"plan-item": {"L": "many"}}}, "item ABC-34's L 'many' is not a whole number from 1 to 4; ABC-34 holds it in its issue property plan-item"),
({"key": "ABC-34", "fields": valid, "properties": None}, shape + ": AttributeError(\"'NoneType' object has no attribute 'get'\")"),
({"key": "ABC-34", "fields": valid, "properties": None}, shape + ": its properties are no object"),
({"key": "ABC-34", "fields": valid | {"issuelinks": [linked | {"inwardIssue": {"key": 7}}]}}, shape + ": its issuelinks hold a link whose id or whose linked issue's key is no text"),
):
with self.subTest(raw=raw):
self.jira.answers[("POST", SEARCH)] = {"issues": [raw]}
@@ -348,14 +347,13 @@ class Jira(Case):
self.assertEqual(JiraEpic(EPIC, self.env_file, self.jira, lambda documents: converted, to_adf).read(), Failure(reason))
def test_a_search_that_does_not_end_names_the_issues_it_sought(self) -> None:
store = self.new_store()
store = dataclasses.replace(self.new_store(), max_pages=1)
children = [store.file("ABC-12", "Count the runs.", "", ()) for _ in range(2)]
for _ in range(3):
store.file(str(children[0]), "A task.", "", ())
with unittest.mock.patch.object(jira_epic, "MAX_PAGES", 1):
self.assertEqual(store.read(), Failure("the search for the sub-tasks of ABC-12's child issues did not end within 1 pages"))
store.file("ABC-12", "Count the runs.", "", ())
self.assertEqual(store.read(), Failure("the search for ABC-12's child issues did not end within 1 pages"))
self.assertEqual(store.read(), Failure("the search for the sub-tasks of ABC-12's child issues did not end within 1 pages"))
store.file("ABC-12", "Count the runs.", "", ())
self.assertEqual(store.read(), Failure("the search for ABC-12's child issues did not end within 1 pages"))
def test_refuses_to_file_under_another_epics_child_or_an_issue_it_cannot_read(self) -> None:
self.jira.issues.append({"key": "ABC-5", "fields": {"parent": {"key": "ABC-1"}, "status": TO_DO}})
+4 -3
View File
@@ -1,19 +1,20 @@
"""What a do-epic cycle's worker is told: its task, from the issue, and its first prompt. The resumed prompt and the rung sentences are the ones loop-claude's cycles use, naming do-epic; a chunk that was parked is first told why it resumes, and one whose task follows the epic's state is told that task as it resumes."""
import dataclasses
from chunk_driver.drivers import DO_EPIC
from do_epic.backend.protocol import EpicIssue
from do_epic import findings
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES
from chunk_driver.project.records import HUMAN_RULE_NAMES, RECORD_FILES
from text_files.markdown_text import listed
import pathlib
from project_config.settings import ProjectConfig
from model_routing.ladder import ProjectRung
from loop_claude.cycle import prompts
from chunk_driver import prompts
from model_routing.route import Route
from agent_cycle.worker_prompts import WorkerPrompts
# The entry point's name as the worker, the judge and the human read it.
DRIVER = "do-epic"
DRIVER = DO_EPIC
def shipping(config: ProjectConfig) -> str:
+7 -3
View File
@@ -13,8 +13,12 @@ It reads three environment variables and fails, naming each that is missing:
{chr(10).join(f' {name}: {meaning}' for name, meaning in settings.MEANING.items())}
It prints a line per check, and exits 0 only where every check held."""
EpicMade = typing.Callable[[settings.Sandbox, str], typing.ContextManager[str | Failure]]
Run = typing.Callable[[settings.Sandbox, str, pathlib.Path, typing.Mapping[str, str]], scenario.Report]
def main(argv: list[str], env: typing.Mapping[str, str], launcher: pathlib.Path) -> int:
def main(argv: list[str], env: typing.Mapping[str, str], launcher: pathlib.Path, epic_made: EpicMade, run: Run) -> int:
"""Runs do-epic-e2e: epic_made makes the sandbox epic, as sandbox.epic_made does, and run drives do-epic against it, as scenario.run does."""
if argv:
print(HELP, file=sys.stdout if argv in (["-h"], ["--help"]) else sys.stderr)
return 0 if argv in (["-h"], ["--help"]) else 1
@@ -25,12 +29,12 @@ def main(argv: list[str], env: typing.Mapping[str, str], launcher: pathlib.Path)
report = scenario.Report()
left: sandbox.Leftover | None = None
try:
with sandbox.epic_made(found, "do-epic e2e sandbox epic") as key:
with epic_made(found, "do-epic e2e sandbox epic") as key:
if isinstance(key, Failure):
print(f"do-epic-e2e: {key.reason}", file=sys.stderr)
return 1
print(f"Sandbox epic {key} in {found.jira_url}")
report = scenario.run(found, key, launcher, env)
report = run(found, key, launcher, env)
except sandbox.Leftover as failure:
left = failure
# The checks are printed even where the epic could not be deleted.
+1 -1
View File
@@ -6,7 +6,7 @@ from atlassian.client import basic_auth, Http, jira_json, ToAdf
from failure import Failure
from adf_codec.convert import markdown_to_adf
from do_epic.e2e.settings import Sandbox
from plan_store.jira_epic import searched
from atlassian.jira import searched
EPIC_TYPE = "Epic"
# How a team-managed project's workflow moves an issue to done, as do-epic's `jira-finish-transition` and `jira-finish-status` name it.
+9 -6
View File
@@ -1,6 +1,5 @@
import contextlib
import io
from unittest import mock
import pathlib
import sys
import tempfile
@@ -18,7 +17,7 @@ class MainTest(unittest.TestCase):
def run_main(self, argv: list[str]) -> tuple[int, str, str]:
out, err = io.StringIO(), io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main(argv, {}, pathlib.Path("do-epic"))
code = main.main(argv, {}, pathlib.Path("do-epic"), sandbox.epic_made, scenario.run)
return code, out.getvalue(), err.getvalue()
def test_fails_naming_each_missing_setting_before_it_touches_anything(self) -> None:
@@ -56,10 +55,14 @@ class ReportedTest(unittest.TestCase):
out, err = io.StringIO(), io.StringIO()
report = report or scenario.Report(["a check held"], ["a check broke"])
with mock.patch.object(sandbox, "epic_made", epic_made), mock.patch.object(scenario, "run", return_value=report) as ran:
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main([], self.env, pathlib.Path("do-epic"))
self.ran_with = ran.call_args.args if ran.called else None
self.ran_with = None
def ran(*args: object) -> scenario.Report:
self.ran_with = args
return report
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main([], self.env, pathlib.Path("do-epic"), epic_made, ran)
return code, out.getvalue(), err.getvalue()
def test_prints_each_check_and_fails_where_one_broke(self) -> None:
+7 -52
View File
@@ -2,22 +2,17 @@
A gate is something only time or a person passes: checks that still run, an approval a reviewer has yet to give, or the new review of a reviewer whose requested changes the worker answered. While a chunk's pull request waits on one, its worker doesn't have anything to do, so do-epic parks the chunk (`parked`). Anything else is news the worker acts on."""
import dataclasses
import json
import typing
from forge.reviews import ANSWERED, APPROVED, CHANGES_REQUESTED, review_decision, REVIEW_REQUIRED, waiting_threads
from forge.pulls import CheckFailed, checks_now, open_pull, Pending, required_checks
from failure import Failure
from forge.commands import forge_of, gitea, Run
from chunk_pr.approval import waiting_threads
from forge.commands import forge_of, Run
# What a pull request waits on, as the words after "awaits".
CHECKS = "its checks"
APPROVAL = "an approval"
NEW_REVIEW = "a new review"
# What `review_decision` returns in place of GitHub's `CHANGES_REQUESTED` once the worker answered those changes: GitHub keeps its decision until the reviewer reviews again.
ANSWERED = "CHANGES_ANSWERED"
# What the title of a draft opens with on Gitea, in any case: Gitea's default `WORK_IN_PROGRESS_PREFIXES`.
GITEA_DRAFT = ("WIP:", "[WIP]")
@dataclasses.dataclass(frozen=True)
@@ -41,7 +36,7 @@ class Open:
def gate(branch: str, checkout: str, at: typing.Callable[[str], Run]) -> Gate | Open | Failure:
"""What the chunk on branch waits on, in a worktree of checkout; at gives the runner of commands in a folder. A Failure where git or the forge cannot be read, which doesn't say anything about the gate.
A draft never waits on a gate: its worker still works on it. On Gitea only the checks are a gate, since chunk-pr reads approvals on GitHub only (`chunk_pr.approval.awaited`). Changes a reviewer requested are news until the worker answered them (`answered`); from then on the pull request awaits a new review.
A draft never waits on a gate: its worker still works on it. On Gitea only the checks are a gate, since chunk-pr reads approvals on GitHub only (`chunk_pr.approval.awaited`). Changes a reviewer requested are news until the worker answered them (`forge.reviews.answered`); from then on the pull request awaits a new review.
"""
worktree = worktree_of(branch, at(checkout))
if not isinstance(worktree, str):
@@ -56,7 +51,7 @@ def gate(branch: str, checkout: str, at: typing.Callable[[str], Run]) -> Gate |
pull = open_pull(forge, branch, head, run)
if not isinstance(pull, str):
return pull or Open(f"no pull request from origin's {branch} is open at {head[:12]}; it may be merged or closed already")
decision = review_decision(pull, head, run) if forge == "gh" else gitea_decision(pull, run)
decision = review_decision(forge, pull, head, run)
if not isinstance(decision, str):
return decision or Open(f"pull request #{pull} is a draft")
required = required_checks(forge, base.removeprefix("origin/"), run)
@@ -67,16 +62,16 @@ def gate(branch: str, checkout: str, at: typing.Callable[[str], Run]) -> Gate |
return checks
if isinstance(checks, Pending):
return Gate(worktree, pull, CHECKS)
if decision == "CHANGES_REQUESTED":
if decision == CHANGES_REQUESTED:
return Open(f"a reviewer requested changes on pull request #{pull}")
waiting = waiting_threads(pull, run) if forge == "gh" else []
if isinstance(waiting, Failure):
return waiting
if waiting:
return Open(f"a reviewer's comment on pull request #{pull} waits for an answer, in {', '.join(waiting)}")
if decision in (ANSWERED, "REVIEW_REQUIRED"):
if decision in (ANSWERED, REVIEW_REQUIRED):
return Gate(worktree, pull, NEW_REVIEW if decision == ANSWERED else APPROVAL)
return Open(f"pull request #{pull} passed its checks" + (" and is approved" if decision == "APPROVED" else ""))
return Open(f"pull request #{pull} passed its checks" + (" and is approved" if decision == APPROVED else ""))
def worktree_of(branch: str, run: Run) -> str | None | Failure:
@@ -89,43 +84,3 @@ def worktree_of(branch: str, run: Run) -> str | None | Failure:
if f"branch refs/heads/{branch}" in lines and lines[0].startswith("worktree "):
return lines[0].removeprefix("worktree ")
return None
def review_decision(pull: str, head: str, run: Run) -> str | None | Failure:
"""GitHub's review decision on the pull request, whose head commit is head, read once: `APPROVED`, `REVIEW_REQUIRED`, `CHANGES_REQUESTED`, or empty where its branch protection asks for no review. ANSWERED in place of `CHANGES_REQUESTED` once the worker answered those changes. None where it is a draft."""
viewed = run(["gh", "pr", "view", pull, "--json", "isDraft,reviewDecision,author,reviews,comments"])
if isinstance(viewed, Failure):
return viewed
try:
parsed = json.loads(viewed)
if parsed["isDraft"]:
return None
decision = str(parsed["reviewDecision"] or "")
return ANSWERED if decision == "CHANGES_REQUESTED" and answered(parsed, head) else decision
except (ValueError, KeyError, TypeError):
return Failure(f"gh pr view printed no review decision: {viewed.strip()[:200]}")
def answered(pull: dict[str, typing.Any], head: str) -> bool:
"""Whether the worker answered the newest review that requested changes on pull, as `gh pr view --json author,reviews,comments` prints it, whose head commit is head: the head is no longer the commit that review was made on, or the pull request's author commented or replied in a thread after it. GitHub lists a reply in a thread as a review of its own. False where it lists no review that requested changes. Raises KeyError or TypeError where pull has another shape.
The review names its commit, so no clock tells whether the worker pushed: a commit's date says when it was made, never when it was pushed.
"""
asked = max((review for review in pull["reviews"] if review["state"] == "CHANGES_REQUESTED"), key=lambda review: review["submittedAt"], default=None)
if asked is None:
return False
author = pull["author"]["login"]
# A review the author has yet to submit has no time, and answers nobody.
said = [review["submittedAt"] or "" for review in pull["reviews"] if review["author"]["login"] == author] + [comment["createdAt"] for comment in pull["comments"] if comment["author"]["login"] == author]
return asked["commit"]["oid"] != head or any(at > asked["submittedAt"] for at in said)
def gitea_decision(pull: str, run: Run) -> str | None | Failure:
"""What stands in for `review_decision` on Gitea, read once: empty, since no review is a gate there, and None where the pull request is a draft, which Gitea tells by its title (GITEA_DRAFT)."""
viewed = gitea(run, f"repos/{{owner}}/{{repo}}/pulls/{pull}")
if isinstance(viewed, Failure):
return viewed
try:
return None if viewed["title"].upper().startswith(GITEA_DRAFT) else ""
except (KeyError, TypeError, AttributeError):
return Failure(f"tea api printed no title of pull request {pull}: {str(viewed)[:200]}")
+51 -46
View File
@@ -8,12 +8,13 @@ import pathlib
import sys
import typing
from loop_claude.loop.chunk_start import archived_route, routed
from chunk_driver.cycle_start import announced_route, cycle_of, project_routing, routing_on
from chunk_driver.claims import CHUNK_FLAG
from do_epic import chunk_prompt, findings, finish, gates, human_tasks, parked, planning, scheduler, split, status
from chunk_driver.chunk_run import ChunkRun, closed_cycle
from do_epic.repo_pick import ClaudeWork, HumanTask, recorded, Repo
from loop_claude.loop.close import closed_cycle
from project_config.main import located as config_located
from loop_claude.project.records import CONSEQUENCE, read_records
from chunk_driver.project.records import CONSEQUENCE, read_records
from agent_cycle.run import Cycle, run_cycle
from do_epic.config import DoEpicConfig, load
from loop_effects.effects import Effects, say
@@ -23,21 +24,17 @@ from do_epic.backend.protocol import EpicIssue, EpicStore
from do_epic.epic_record import EpicRecord
from failure import Failure
from agent_cycle.verdicts import Finished, TakenIn
from loop_claude.project.model_choice import get_or_ask_models, Models
from plan_store.jira_epic import ISSUE_KEY
from atlassian.jira import ISSUE_KEY
from do_epic.backend.jira import JiraEpic
from agent_cycle.routing import judge_rung
from claude_code.reply import Limited
from do_epic.repos import locate
from loop_claude.loop.this_loop import Loop
from loop_claude.loop.reload import LoopRun
from chunk_driver.this_loop import Loop
from chunk_driver.project.model_choice import Models, overridden
from loop_state.paths import Paths
from transcript_archive.project_key import project_key
from model_routing.ladder_file import project_ladder_file, read_ladder, SHIPPED
from loop_claude.project.premises import read_premises
from chunk_driver.project.premises import read_premises
from forge.commands import Run, runner
from claude_code.session import session_named
from loop_claude.loop.main import setup_version
from model_routing.ladder_file import SHIPPED
from agent_cycle.stop import Stop
from agent_cycle.direct_message import tell
from agent_cycle import tmux
@@ -110,13 +107,13 @@ def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher:
if stop.reason:
say(stop.reason, effects, stdout=not stop.code)
# A chunk's stop is DMed, as loop-claude's is, so a chunk left overnight never ends unseen.
if stop.reason and argv[:1] == [scheduler.CHUNK_FLAG]:
if stop.reason and argv[:1] == [CHUNK_FLAG]:
tell(f"do-epic {' '.join(argv)} stopped: {stop.reason}", effects)
return stop.code
def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend, stdin: typing.TextIO, forge_at: typing.Callable[[str], Run]) -> Stop:
chunk, words = (argv[1], argv[2:]) if len(argv) >= 2 and argv[0] == scheduler.CHUNK_FLAG else (None, argv)
chunk, words = (argv[1], argv[2:]) if len(argv) >= 2 and argv[0] == CHUNK_FLAG else (None, argv)
status_asked = chunk is None and words[1:] == [STATUS_FLAG]
words = words[:1] if status_asked else words
# What the words after the epic's key write into the epic, where they are a filing or a decision.
@@ -242,7 +239,7 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
"""Runs the cycles of the chunk of the issue called key until one stops the loop, or the judge rules the chunk finished. For a human's task it runs no cycle: it waits until the task no longer holds up an issue (`human_tasks.awaited`), then starts the next chunks. A chunk in a repo parks after each cycle that leaves its pull request waiting on a gate, and before its first where it was parked when its window last ended, as at a reboot."""
pane = epic.env.get("TMUX_PANE")
if not pane:
return Stop(1, f"{scheduler.CHUNK_FLAG} is how do-epic starts itself in a tmux window; run do-epic <EPIC-KEY>")
return Stop(1, f"{CHUNK_FLAG} is how do-epic starts itself in a tmux window; run do-epic <EPIC-KEY>")
# None for the epic's finishing, which is no issue.
issue = None if key == epic.key else issue_read(key, epic)
if isinstance(issue, Stop):
@@ -254,26 +251,30 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
if isinstance(place, Stop):
return place
root, repo = place
if unmade := folder_made(root):
return Stop(1, unmade.reason)
session = session_named(f"A {epic.key} do-epic {key}", epic.paths.claude_code_transcripts)
loop = Loop(root=root, session=session, window=tmux.Window(epic.session, key, pane), paths=epic.paths, env=epic.env, launcher=epic.launcher, effects=epic.effects)
loop_run = LoopRun(0, root, "", "", chunk=key)
run = ChunkRun()
# What a chunk that was parked is told as it resumes; empty for a chunk that wasn't.
news = parked_through(repo, loop, epic, starts=True) if repo is not None else ""
while True:
if isinstance(news, Stop):
return news
loop_run = dataclasses.replace(loop_run, cycle_number=loop_run.cycle_number + 1)
cycle = next_cycle(loop_run, loop, repo is not None, epic, news)
run = dataclasses.replace(run, cycle_number=run.cycle_number + 1)
cycle = next_cycle(run, loop, repo is not None, epic, news)
if isinstance(cycle, Stop):
return cycle
loop_run = dataclasses.replace(loop_run, task=cycle.task)
if isinstance(cycle, EpicReopened):
say(f"the epic {epic.key} holds an open issue again, so its finishing chunk ends until none is open", epic.effects, stdout=True)
return handed_on(key, epic)
concluded = run_cycle(cycle)
if isinstance(concluded, Stop):
return concluded
closed = closed_cycle(loop_run, cycle.project, concluded, cycle.project_archive, chunk_prompt.DRIVER, loop)
closed = closed_cycle(run, cycle.task, cycle.project, concluded, cycle.project_archive, chunk_prompt.DRIVER, loop)
if isinstance(closed, Stop):
return closed
loop_run = closed
run = closed
news = parked_through(repo, loop, epic, starts=False) if repo is not None else ""
# A chunk that was parked has work again, whatever the judge ruled before its pull request passed the gate.
if isinstance(concluded.verdict, Finished) and news == "":
@@ -358,15 +359,22 @@ def chunk_root(key: str, issue: EpicIssue | None, epic: EpicRun) -> tuple[str, s
if isinstance(pick, Failure):
return Stop(1, pick.reason)
if isinstance(pick, ClaudeWork):
folder = epic.paths.home / scheduler.STATE / epic.key
folder.mkdir(parents=True, exist_ok=True)
return str(folder), None
return str(epic.paths.home / scheduler.STATE / epic.key), None
if not isinstance(pick, Repo):
return Stop(1, f"{key} is no chunk: no label names its repo")
found = locate(pick.name, [], epic.config, epic.paths.home, epic.effects.git)
return Stop(1, found.reason) if isinstance(found, Failure) else (str(real(found.checkout)), pick.name)
def folder_made(root: str) -> Failure | None:
"""Makes the folder a chunk works in where it is missing, as the epic's folder is before its first chunk without a repo."""
try:
pathlib.Path(root).mkdir(parents=True, exist_ok=True)
except OSError as failure:
return Failure(f"could not make the chunk's folder {root}: {failure.strerror}")
return None
def handed_on(key: str, epic: EpicRun) -> Stop:
"""The stop of a do-epic whose chunk finished. It first moves the chunk's issue to done (the epic's finishing has none), and its split item too where that was the last of the item's sub-tasks, so that no finished chunk starts again and what they block may start, then starts each chunk that may start now. Its window closes as it exits."""
# The epic's finishing has no issue to move: the epic is closed by `--close-epic`.
@@ -399,8 +407,8 @@ def split_items_closed(epic: EpicRun, issues: list[EpicIssue]) -> list[EpicIssue
return issues
def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun, news: str = "") -> Cycle | Stop:
"""The cycle loop_run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read. news is what `parked_through` tells the worker of a chunk that was parked, and opens its prompt. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
def next_cycle(run: ChunkRun, loop: Loop, in_repo: bool, epic: EpicRun, news: str = "") -> Cycle | EpicReopened | Stop:
"""The cycle run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read, and EpicReopened where the epic's finishing has no cycle to run. news is what `parked_through` tells the worker of a chunk that was parked, and opens its prompt. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
premises = premises_read(loop.root if in_repo else None, epic)
if isinstance(premises, Failure):
return Stop(1, premises.reason)
@@ -411,27 +419,25 @@ def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun, news
if isinstance(records, Failure):
return Stop(1, records.reason)
work = chunk_work(loop.window.name, loop.root if in_repo else None, language or epic.config.language, epic)
if isinstance(work, Stop):
if isinstance(work, (Stop, EpicReopened)):
return work
task, routing = work
models = get_or_ask_models(project_key(loop.root, epic.paths.home), epic.paths, epic.env, chunk_prompt.DRIVER, epic.effects) if in_repo else repoless_models(epic)
if isinstance(models, Stop):
return models
ladder_file = project_ladder_file(loop.root, epic.paths.home) if in_repo else SHIPPED
if isinstance(ladder_file, Failure):
return Stop(1, ladder_file.reason)
ladder = read_ladder(ladder_file)
if isinstance(ladder, Failure):
return Stop(1, ladder.reason)
task, issue = work
routing = project_routing(loop, chunk_prompt.DRIVER) if in_repo else routing_on(repoless_models(epic), SHIPPED)
if isinstance(routing, Stop):
return routing
archive_folder = epic.paths.home / ARCHIVE / epic.key
project = ladder.project_rung(models.worker)
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, loop), lambda: routing)
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", epic.effects, stdout=True)
return Cycle(config=setup_version(epic.paths.claude_dir, epic.effects.git), effects=epic.effects, facts="", judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=task, prompts=chunk_prompt.unparked(news) if news else chunk_prompt.restating(task) if loop_run.chunk == epic.key else chunk_prompt.WORKER, taken_in=lambda entries: TakenIn(), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
route = announced_route(run.cycle_number, run.route, routing, archive_folder, lambda: issue, loop)
prompts = chunk_prompt.unparked(news) if news else chunk_prompt.restating(task) if loop.window.name == epic.key else chunk_prompt.WORKER
return cycle_of(loop, routing, route, archive_folder, task=task, facts="", premises=premises, records=records, unfinished_evidence=run.unfinished_evidence, prompts=prompts, taken_in=lambda entries: TakenIn())
def chunk_work(key: str, checkout: str | None, language: str, epic: EpicRun) -> tuple[str, RoutingIssue | None] | Stop:
"""The task of the chunk called key, and the issue that routes it, or None where that issue has no scores: both from the epic's issues, read afresh. The epic's finishing has no issue, and it ends here once the epic holds an open issue again, since working that issue comes first."""
@dataclasses.dataclass(frozen=True)
class EpicReopened:
"""The epic holds an open issue again, so its finishing chunk has no work until none is open: working that issue comes first."""
def chunk_work(key: str, checkout: str | None, language: str, epic: EpicRun) -> tuple[str, RoutingIssue | None] | EpicReopened | Stop:
"""The task of the chunk called key, and the issue that routes it, or None where that issue has no scores: both from the epic's issues, read afresh. The epic's finishing has no issue, and its work is EpicReopened once the epic holds an open issue again."""
if key != epic.key:
issue = issue_read(key, epic)
if isinstance(issue, Stop):
@@ -444,14 +450,13 @@ def chunk_work(key: str, checkout: str | None, language: str, epic: EpicRun) ->
if isinstance(issues, Failure):
return Stop(1, issues.reason)
if not finish.due(issues):
say(f"the epic {epic.key} holds an open issue again, so its finishing chunk ends until none is open", epic.effects, stdout=True)
return handed_on(key, epic)
return EpicReopened()
return finish.task(epic.key, issues, epic.launcher, language), None
def repoless_models(epic: EpicRun) -> Models:
"""The models of a chunk without a repo: `repoless-model`, unless the environment names another for the run, as it may for loop-claude."""
return Models(epic.env.get("LOOP_CLAUDE_JUDGE_MODEL") or epic.config.repoless_model, epic.env.get("LOOP_CLAUDE_WORKER_MODEL") or epic.config.repoless_model)
return overridden(epic.env, epic.config.repoless_model)
def records_read(root: str | None, epic: EpicRun) -> str | Failure:
+58 -61
View File
@@ -1,28 +1,25 @@
"""Which of an epic's issues run as chunks now, each in its own window of the epic's tmux session (docs/decisions.md → `do-epic: drives one Jira epic …`).
Three limits hold a chunk back: the epic's cap of chunks at once, its repo's `parallel-chunks`, counted across every epic, and a loop-claude that runs chunks in its repo. A parked chunk (`parked`) counts toward none of them: its window only polls the forge, and `resume` gives it a running slot under the same limits once its worker has something to do. do-epic checks a repo and starts a chunk in it while holding the lock loop-claude holds when it starts one (`chunk_windows.starting`), so the two never start chunks in one checkout together."""
Three limits hold a chunk back: the epic's cap of chunks at once, its repo's `parallel-chunks`, counted across every epic, and a loop-claude that runs chunks in its repo. A parked chunk (`parked`) counts toward none of them: its window only polls the forge, and `resume` gives it a running slot under the same limits once its worker has something to do. do-epic checks a repo and starts a chunk in it while holding the lock loop-claude holds when it starts one (`chunk_driver.claims.starting`), so the two never start chunks in one checkout together."""
import dataclasses
import pathlib
import typing
from do_epic.epic_record import CAP
from loop_claude.loop.chunk_windows import epic_chunks, HANDED, LOCK, project_lock, starting
from chunk_driver.claims import CHUNK_FLAG, epic_chunks, HANDED, loop_chunks, project_lock, starting
from do_epic.backend.protocol import EpicIssue
from failure import Failure
from do_epic.repo_pick import HumanTask, recorded, Repo
from project_config.settings import PARALLEL_CHUNKS
from do_epic.parked import parked_chunks
from chunk_pr.approval import PARKS
from transcript_archive.project_key import project_key
from do_epic.split import SPLIT_LABEL
from agent_cycle import tmux
# Under the home folder, a folder per epic key holds the epic's LOCK. A chunk without a repo works there.
# Under the home folder, a folder per epic key holds the epic's lock file, which `claims.LOCK` names. A chunk without a repo works there.
STATE = ".local/state/do-epic"
# Why a human's task with a live window waits, as the epic's status says.
HUMAN_WAIT = "it is a human's task; do-epic polls Jira until you close it"
# The flag that tells the do-epic in a window its chunk, as `--chunk ABC-34 ABC-12`. Nobody types it, so no panel picked it.
CHUNK_FLAG = "--chunk"
@dataclasses.dataclass(frozen=True)
@@ -143,45 +140,44 @@ def start_chunks(epic: Epic, issues: list[EpicIssue], finishing: bool, own: str
parked = parked_chunks(epic.folder)
if isinstance(parked, Failure):
return parked
try:
with starting(epic.folder):
windows = tmux.listed(session, run)
live = {window.name: window for window in windows if not window.dead and window.name != own}
running = {window.name for window in windows if not window.dead and not window.parked} - {own}
stopped = {window.name for window in windows if window.dead} - set(live)
# The windows this call started that take a running slot.
taken = 0
for chunk in found:
if chunk.issue in running or chunk.issue == own:
continue
slot = chunk.issue not in parked and not chunk.human
if chunk.human and chunk.issue in live:
waits.append(Waits(chunk.issue, HUMAN_WAIT))
elif chunk.issue in parked and chunk.issue in live:
waits.append(Waits(chunk.issue, parked[chunk.issue].waits(live[chunk.issue].awaits)))
elif own and (chunk.issue in stopped or chunk.issue in parked):
waits.append(Waits(chunk.issue, f"its window stopped; run `do-epic {epic.key}` to start it again"))
elif slot and len(running) + taken >= epic.cap:
waits.append(Waits(chunk.issue, f"the epic runs as many chunks as its `{CAP}` line allows: {epic.cap}"))
with starting(epic.folder) as unlocked:
if unlocked:
return unlocked
windows = tmux.listed(session, run)
live = {window.name: window for window in windows if not window.dead and window.name != own}
running = {window.name for window in windows if not window.dead and not window.parked} - {own}
stopped = {window.name for window in windows if window.dead} - set(live)
# The windows this call started that take a running slot.
taken = 0
for chunk in found:
if chunk.issue in running or chunk.issue == own:
continue
slot = chunk.issue not in parked and not chunk.human
if chunk.human and chunk.issue in live:
waits.append(Waits(chunk.issue, HUMAN_WAIT))
elif chunk.issue in parked and chunk.issue in live:
waits.append(Waits(chunk.issue, parked[chunk.issue].waits(live[chunk.issue].awaits)))
elif own and (chunk.issue in stopped or chunk.issue in parked):
waits.append(Waits(chunk.issue, f"its window stopped; run `do-epic {epic.key}` to start it again"))
elif slot and len(running) + taken >= epic.cap:
waits.append(Waits(chunk.issue, f"the epic runs as many chunks as its `{CAP}` line allows: {epic.cap}"))
else:
now = start_chunk(epic, chunk if chunk.issue not in parked else Chunk(chunk.issue, None), own, windows, run)
if isinstance(now, Failure):
return now
if isinstance(now, Waits):
waits.append(now)
else:
now = start_chunk(epic, chunk if chunk.issue not in parked else Chunk(chunk.issue, None), own, windows, run)
if isinstance(now, Failure):
return now
if isinstance(now, str):
waits.append(Waits(chunk.issue, now))
else:
windows = now
started.append(chunk.issue)
taken += slot
except OSError as failure:
return Failure(f"cannot lock {failure.filename or epic.folder / LOCK}: {failure.strerror}")
windows = now
started.append(chunk.issue)
taken += slot
# The epic's finishing is no issue: its wait is listed after every issue's.
order = [*(issue.key for issue in issues), epic.key]
return Started(started, sorted(waits, key=lambda wait: order.index(wait.issue)))
def start_chunk(epic: Epic, chunk: Chunk, own: str | None, windows: list[tmux.Listed], run: tmux.Tmux) -> list[tmux.Listed] | str | Failure:
"""Starts chunk's window beside windows, the epic's session's, and returns the session's windows afterwards; the reason where its repo takes no chunk now, and a Failure where tmux started none. It holds the repo's lock from the check to the start. Raises OSError where that lock cannot be taken. A chunk without a repo starts in the epic's folder. So does a parked chunk, since it takes no room in its repo."""
def start_chunk(epic: Epic, chunk: Chunk, own: str | None, windows: list[tmux.Listed], run: tmux.Tmux) -> list[tmux.Listed] | Waits | Failure:
"""Starts chunk's window beside windows, the epic's session's, and returns the session's windows afterwards; a Waits with the reason where its repo takes no chunk now, and a Failure where tmux started none, or where the repo's lock cannot be taken. It holds that lock from the check to the start. A chunk without a repo starts in the epic's folder. So does a parked chunk, since it takes no room in its repo."""
session = tmux.session_name(tmux.EPIC, epic.key)
handed = {**{name: value for name, value in epic.env.items() if name.startswith(HANDED)}, PARKS: "1"}
command = [str(epic.launcher), CHUNK_FLAG, chunk.issue, epic.key]
@@ -189,37 +185,38 @@ def start_chunk(epic: Epic, chunk: Chunk, own: str | None, windows: list[tmux.Li
return tmux.start_chunk(tmux.Window(session, chunk.issue, ""), str(epic.folder), handed, command, windows, False, run)
located = epic.locate(chunk.repo)
if isinstance(located, Failure):
return located.reason
return Waits(chunk.issue, located.reason)
checkout, limit = located
with starting(project_lock(str(checkout), epic.home)):
with starting(project_lock(str(checkout), epic.home)) as unlocked:
if unlocked:
return unlocked
if reason := refusal(chunk.repo, room_in(str(checkout), limit, (session, own), epic.home, run)):
return reason
return Waits(chunk.issue, reason)
return tmux.start_chunk(tmux.Window(session, chunk.issue, ""), str(checkout), handed, command, windows, False, run)
def room_in(checkout: str, limit: int, leaving: tuple[str, str | None], home: pathlib.Path, run: tmux.Tmux) -> RepoRoom:
"""The room in a repo's checkout under its limit. leaving is the session and name of the window that asks, which doesn't count: its chunk finished or is parked."""
theirs = [placed for placed in epic_chunks(checkout, run) if (placed.session, placed.window.name) != leaving]
loop_session = tmux.session_name(tmux.LOOP, project_key(checkout, home))
loops = any(not window.dead for window in tmux.listed(loop_session, run)) and tmux.owner(loop_session, run) == checkout
return RepoRoom(limit, len(theirs), loops)
return RepoRoom(limit, len(theirs), bool(loop_chunks(checkout, home, run)))
def resume(epic: Epic, repo: str, window: tmux.Window, run: tmux.Tmux) -> str | None:
"""Takes a running slot for the parked chunk in window, whose worker has something to do again in the repo called repo: None once the window counts as a running chunk, else why the chunk still waits. The limits are those a starting chunk meets, checked under the same locks. The window itself never counts against them: one started again for a parked chunk, as after a reboot, isn't named as parked until its first read of the gate."""
try:
with starting(epic.folder):
running = [listed for listed in tmux.listed(window.session, run) if not listed.dead and not listed.parked and listed.name != window.name]
if len(running) >= epic.cap:
return f"the epic runs as many chunks as its `{CAP}` line allows: {epic.cap}"
located = epic.locate(repo)
if isinstance(located, Failure):
return located.reason
checkout, limit = located
with starting(project_lock(str(checkout), epic.home)):
if reason := refusal(repo, room_in(str(checkout), limit, (window.session, window.name), epic.home, run)):
return reason
tmux.unpark(window, str(checkout), run)
return None
except OSError as failure:
return f"cannot lock {failure.filename or epic.folder / LOCK}: {failure.strerror}"
with starting(epic.folder) as unlocked:
if unlocked:
return unlocked.reason
running = [listed for listed in tmux.listed(window.session, run) if not listed.dead and not listed.parked and listed.name != window.name]
if len(running) >= epic.cap:
return f"the epic runs as many chunks as its `{CAP}` line allows: {epic.cap}"
located = epic.locate(repo)
if isinstance(located, Failure):
return located.reason
checkout, limit = located
with starting(project_lock(str(checkout), epic.home)) as unlocked:
if unlocked:
return unlocked.reason
if reason := refusal(repo, room_in(str(checkout), limit, (window.session, window.name), epic.home, run)):
return reason
tmux.unpark(window, str(checkout), run)
return None
+1 -1
View File
@@ -10,7 +10,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from do_epic.test_main import DESCRIPTION, DoEpicTest
from do_epic.epic_record import EpicRecord
from failure import Failure
from loop_claude.test_loop_claude import FakeEffects
from loop_effects.doubles import FakeEffects
from record_store.held import Held
from do_epic import main
from do_epic.backend.fake import MemoryEpic
+12 -3
View File
@@ -9,10 +9,11 @@ from do_epic.test_main import Description, DESCRIPTION, DoEpicTest, MODELS, UNRE
from do_epic.config import DoEpicConfig, load
from do_epic.epic_record import EpicRecord
from failure import Failure
from loop_claude.test_loop_claude import FakeEffects, judged, Worker
from do_epic import findings, finish, main, scheduler
from loop_effects.doubles import FakeEffects, judged, Worker
from do_epic import findings, finish, main
from project_config.settings import Finish
from do_epic.test_scheduler import issue
from chunk_driver.claims import LOCK
from do_epic.backend.fake import MemoryEpic
from loop_state.paths import paths_from
from claude_code.session import session_named
@@ -215,7 +216,7 @@ class TheFinishingChunk(DoEpicTest):
def test_stops_where_the_epics_issues_cannot_be_read_or_its_lock_cannot_be_taken(self) -> None:
self.issues = [issue("ABC-20", "do-epic-claude", done=True)]
(self.home / ".local/state/do-epic/ABC-12" / scheduler.LOCK).mkdir(parents=True)
(self.home / ".local/state/do-epic/ABC-12" / LOCK).mkdir(parents=True)
code, _, err = self.run_main(["ABC-12"], FakeEffects(on_terminal=False))
self.assertEqual(code, 1)
self.assertIn("cannot lock ", err)
@@ -233,3 +234,11 @@ class TheFinishingChunk(DoEpicTest):
self.assertIn("the epic ABC-12 holds an open issue again, so its finishing chunk ends until none is open\n", out)
self.assertIn("the chunk finished; started ABC-31\n", out)
self.assertEqual([seam for seam, _ in fake.calls if seam == "stream_claude"], [])
def test_asking_for_the_finishing_chunks_work_names_a_reopened_epic_and_doesnt_start_any_window(self) -> None:
self.issues = [*REVIEWED, issue("ABC-31", "do-epic-claude")]
fake = FakeEffects(windows="ABC-12\t0\t@9\n")
out = io.StringIO()
with contextlib.redirect_stdout(out):
self.assertEqual(main.chunk_work("ABC-12", None, "English", self.epic_run(fake)), main.EpicReopened())
self.assertEqual((out.getvalue(), fake.tmux_calls), ("", []))
+2 -2
View File
@@ -6,11 +6,11 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from project_config.test_project_config import configured
from project_config.doubles import configured
from do_epic.test_main import DESCRIPTION, Description, DoEpicTest, MODELS, UNREAD
from do_epic.epic_record import EpicRecord
from failure import Failure
from loop_claude.test_loop_claude import FakeEffects
from loop_effects.doubles import FakeEffects
from do_epic.scheduler import HUMAN_WAIT
from do_epic.test_scheduler import issue
from do_epic.backend.fake import MemoryEpic
+66 -7
View File
@@ -12,21 +12,21 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from do_epic import chunk_prompt, main
from project_config.test_project_config import configured
from chunk_driver.chunk_run import ChunkRun
from project_config.doubles import configured
from do_epic.config import DoEpicConfig
from project_config.settings import Epic, Finish
from do_epic.backend.protocol import EpicIssue
from do_epic.epic_record import EpicRecord
from agent_cycle import escalation_mark, judge, tmux
from failure import Failure
from loop_claude.test_loop_claude import FakeEffects, judged, Worker
from loop_effects.doubles import FakeEffects, judged, Worker
from do_epic.test_gates import Forge
from do_epic.test_scheduler import issue
from loop_claude.loop.this_loop import Loop
from loop_claude.loop.reload import LoopRun
from chunk_driver.this_loop import Loop
from do_epic.backend.fake import MemoryEpic
from do_epic import parked
from loop_state.paths import paths_from
from text_files.file_writes import replace_file
from model_routing.route import Route
from model_routing.ladder import Rung
from do_epic.test_do_epic import SETTINGS
@@ -180,16 +180,31 @@ class TheCommand(DoEpicTest):
class ACycleOfAChunkWithoutARepo(DoEpicTest):
def cycle(self, env: dict[str, str] | None = None) -> main.Cycle:
def test_naming_the_chunks_folder_doesnt_make_it(self) -> None:
folder = self.home / ".local/state/do-epic/ABC-12"
self.assertEqual(main.chunk_root("ABC-21", self.issues[1], self.epic_run(FakeEffects())), (str(folder), None))
self.assertFalse(folder.exists())
self.assertIsNone(main.folder_made(str(folder)))
self.assertTrue(folder.is_dir())
def test_a_chunk_whose_folder_cannot_be_made_stops_with_the_reason(self) -> None:
(self.home / ".local/state").mkdir(parents=True, exist_ok=True)
(self.home / ".local/state/do-epic").write_text("", encoding="utf-8")
code, _, err = self.run_main(["--chunk", "ABC-21", "ABC-12"], FakeEffects(), {"TMUX_PANE": "%1"})
self.assertEqual(code, 1)
self.assertRegex(err, f"could not make the chunk's folder {self.home}/.local/state/do-epic/ABC-12: \\w")
def cycle(self, env: dict[str, str] | None = None, run: ChunkRun = ChunkRun(1)) -> main.Cycle:
fake = FakeEffects(git=[Failure("no git"), Failure("no git")])
paths = paths_from(self.home)
epic = self.epic_run(fake, env)
place = main.chunk_root("ABC-21", self.issues[1], epic)
assert isinstance(place, tuple)
root, repo = place
self.assertIsNone(main.folder_made(root))
loop = Loop(root=root, session=main.session_named("A ABC-12 do-epic ABC-21", paths.claude_code_transcripts), window=tmux.Window("epic-ABC-12", "ABC-21", "%1"), paths=paths, env=env or {}, launcher=pathlib.Path("/b/do-epic"), effects=fake.effects())
with contextlib.redirect_stdout(io.StringIO()):
found = main.next_cycle(LoopRun(1, root, "", "", chunk="ABC-21"), loop, repo is not None, epic)
found = main.next_cycle(run, loop, repo is not None, epic)
assert isinstance(found, main.Cycle), found
return found
@@ -202,6 +217,16 @@ class ACycleOfAChunkWithoutARepo(DoEpicTest):
self.assertEqual(self.cycle({"LOOP_CLAUDE_WORKER_MODEL": "off-the-ladder"}).route, Route(Rung("off-the-ladder")))
self.assertEqual(self.cycle({"LOOP_CLAUDE_JUDGE_MODEL": "claude-fable-5-2"}).judge.model, "claude-fable-5-2")
def test_keeps_the_route_and_the_evidence_its_run_holds(self) -> None:
kept = Route(Rung("claude-opus-5-5", "xhigh"), 21, 3, 5)
cycle = self.cycle(run=ChunkRun(2, unfinished_evidence="the evidence", route=kept))
self.assertEqual((cycle.route, cycle.unfinished_evidence), (kept, "the evidence"))
def test_a_chunk_that_was_not_parked_starts_on_the_drivers_own_prompts_with_no_facts(self) -> None:
cycle = self.cycle()
self.assertIs(cycle.prompts, chunk_prompt.WORKER)
self.assertEqual(cycle.facts, "")
def test_works_in_the_epics_folder_and_archives_apart_from_loop_claude(self) -> None:
cycle = self.cycle()
self.assertEqual((cycle.root, cycle.project_archive, cycle.records), (str(self.home / ".local/state/do-epic/ABC-12"), self.home / ".claude/do-epic-transcripts/ABC-12", ""))
@@ -378,6 +403,33 @@ class AChunksRun(DoEpicTest):
self.assertEqual(code, 2, out)
self.assertIn("the judge ruled it unfinished 6 cycles running; finish the chunk yourself, or rerun do-epic to give it 6 more cycles", err + out)
def test_every_archived_transcript_opens_with_the_chunks_task_through_step_ups_and_the_stop_for_the_human(self) -> None:
fake = FakeEffects(streams=[self.worker()] * 6, claude=[judged("unfinished", escalation="working")] + [judged("unfinished")] * 5, git=UNREAD)
code, out, _ = self.run_main(["--chunk", "ABC-21", "ABC-12"], fake, MODELS)
self.assertEqual(code, 2, out)
outcomes = [path.read_text(encoding="utf-8") for path in (self.home / ".claude/do-epic-transcripts/ABC-12").glob("*/*.outcomes.md")]
self.assertTrue(any("Z Stepped up\n" in text for text in outcomes) and any("Z Stopped: " in text for text in outcomes), outcomes)
self.assertEqual([text for text in outcomes if not text.startswith("Task: Work the Jira issue ABC-21 of the epic ABC-12 as one chunk")], [])
def test_a_step_up_archived_beside_a_new_copy_of_the_transcript_opens_with_the_chunks_task(self) -> None:
fake = FakeEffects(streams=[self.worker()] * 6, claude=[judged("unfinished", escalation="working")] + [judged("unfinished")] * 5, git=UNREAD)
def grown_after_each_copy(path: pathlib.Path, data: bytes) -> Failure | None:
"""Writes the file, and adds a line to the worker's transcript after each copy of it, as a worker that still writes does."""
unwritten = replace_file(path, data)
if path.suffix == ".jsonl":
for transcript in (paths_from(self.home).claude_code_transcripts.parent / "projects" / "-fake").glob("*.jsonl"):
with open(transcript, "a", encoding="utf-8") as handle:
handle.write(json.dumps({"type": "queue-operation", "timestamp": str(path)}) + "\n")
return unwritten
fake.replace_file = grown_after_each_copy
code, out, _ = self.run_main(["--chunk", "ABC-21", "ABC-12"], fake, MODELS)
self.assertEqual(code, 2, out)
outcomes = [path.read_text(encoding="utf-8") for path in (self.home / ".claude/do-epic-transcripts/ABC-12").glob("*/*.outcomes.md")]
self.assertTrue(any("Z Stepped up\n" in text for text in outcomes) and any("Z Stopped: " in text for text in outcomes), outcomes)
self.assertEqual([text for text in outcomes if not text.startswith("Task: Work the Jira issue ABC-21 of the epic ABC-12 as one chunk")], [])
def test_a_finished_chunk_moves_its_issue_to_done_before_the_chunk_it_blocks_starts(self) -> None:
store = MemoryEpic("ABC-12", [self.issues[0], self.issues[1], issue("ABC-22", "do-epic-claude", blockers=("ABC-21",))])
self.backend = lambda config, key: (store, EpicRecord(Description(self.description))) # type: ignore[method-assign,arg-type]
@@ -493,6 +545,13 @@ class AChunksRun(DoEpicTest):
self.assertEqual(code, 0, err)
self.assertIn("=== cycle 1, worker on claude-fable-5-2 at low effort ===\n", out)
def test_a_chunk_in_a_repo_without_a_saved_model_stops_off_a_terminal_naming_do_epic_as_what_to_rerun(self) -> None:
self.repo_chunk()
code, out, err = self.run_main(["--chunk", "ABC-20", "ABC-12"], FakeEffects(git=UNREAD, on_terminal=False), {"TMUX_PANE": "%1"})
self.assertEqual((code, out), (2, ""))
self.assertIn("pick api's model", err)
self.assertIn("rerun do-epic on a terminal", err)
def test_a_chunk_in_a_repo_stops_where_the_repos_ladder_or_records_cannot_be_read(self) -> None:
folder = self.repo_chunk()
(folder / "routing-ladder.json").write_text("[", encoding="utf-8")
+2 -2
View File
@@ -9,12 +9,12 @@ import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from project_config.test_project_config import configured
from project_config.doubles import configured
from do_epic.test_main import Description, DoEpicTest, structured as says
from do_epic.backend.protocol import EpicIssue
from do_epic.epic_record import EpicRecord
from failure import Failure
from loop_claude.test_loop_claude import FakeEffects
from loop_effects.doubles import FakeEffects
from do_epic.test_scheduler import issue
from claude_code.reply import Limited
from do_epic.config import load
+12 -7
View File
@@ -2,21 +2,21 @@ import pathlib
import sys
import tempfile
import unittest
import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from test_env import assert_not_root
from do_epic.config import DoEpicConfig
from failure import Failure
from do_epic.repos import locate, Located, Unsure
from project_config.settings import ProjectConfig
from project_config.test_project_config import write_config
from project_config.doubles import write_config
class FakeGit:
"""git as locate calls it: origins maps a checkout to its origin URL, cloned holds each `clone` that worked, and refused names a URL whose clone fails after it made the checkout's `.git`, as a stopped git does."""
def __init__(self, origins: dict[str, str | Failure], refused: str | None = None) -> None:
self.origins, self.refused = origins, refused
"""git as locate calls it: origins maps a checkout to its origin URL, cloned holds each `clone` that worked, and refused names a URL whose clone fails after it made the checkout's `.git`, as a stopped git does. Where locked, that clone also takes away the right to remove what it made."""
def __init__(self, origins: dict[str, str | Failure], refused: str | None = None, locked: bool = False) -> None:
self.origins, self.refused, self.locked = origins, refused, locked
self.cloned: list[tuple[str, str]] = []
self.clone_timeouts: list[float | None] = []
self.clone_options: list[tuple[str, ...]] = []
@@ -30,6 +30,8 @@ class FakeGit:
url, destination = args[clone + 2:clone + 4]
pathlib.Path(destination, ".git").mkdir(parents=True)
if url == self.refused:
if self.locked:
pathlib.Path(destination).chmod(0o500)
return Failure("git clone: repository not found")
self.cloned.append((url, destination))
self.clone_timeouts.append(timeout)
@@ -110,9 +112,12 @@ class LocateTest(unittest.TestCase):
self.assertIn(f"{self.billing} has an origin, billing, with no host and owner to put another repo under", self.unsure(FakeGit(self.origins)))
def test_asks_all_the_same_where_the_leftovers_of_a_failed_clone_cannot_be_removed(self) -> None:
assert_not_root(self)
self.configure()
with unittest.mock.patch("do_epic.repos.shutil.rmtree", side_effect=lambda path, ignore_errors: None if ignore_errors is True else self.fail("rmtree must ignore errors")):
self.assertIn("cannot clone", self.unsure(FakeGit(self.origins, refused="git@github.com:acme/ledger")))
left = self.home / "code" / "ledger"
self.addCleanup(lambda: left.chmod(0o700))
self.assertIn("cannot clone", self.unsure(FakeGit(self.origins, refused="git@github.com:acme/ledger", locked=True)))
self.assertTrue((left / ".git").is_dir())
def test_asks_where_the_clone_fails_naming_the_url_and_the_reason(self) -> None:
self.configure()
+23 -2
View File
@@ -6,13 +6,14 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from do_epic.scheduler import Chunk, CHUNK_FLAG, chunks, Epic, HUMAN_WAIT, refusal, RepoRoom, resume, start_chunks, Started, Waits
from do_epic.scheduler import Chunk, chunks, Epic, HUMAN_WAIT, refusal, RepoRoom, resume, start_chunks, Started, Waits
from chunk_driver.claims import CHUNK_FLAG
from do_epic import chunk_prompt
from do_epic.backend.protocol import EpicIssue
from failure import Failure
from do_epic.parked import file_of, park, Parked
from do_epic.epic_record import parallel_chunks_in
from loop_claude.cycle import prompts
from chunk_driver import prompts
from model_routing.route import Route
from model_routing.ladder import Rung
from project_config import settings
@@ -295,6 +296,10 @@ class StartingAnEpicsChunks(unittest.TestCase):
self.assertTrue((self.home / ".local/state/loop-claude/api/starting").is_file())
self.assertTrue((self.home / ".local/state/do-epic/ABC-12/starting").is_file())
def test_a_parked_chunk_resumes_under_the_starting_lock_of_its_repos_checkout(self) -> None:
self.assertIsNone(self.resumed(self.epic(1, api=1), Tmux(("epic-ABC-12", "ABC-20 awaits an approval", False, ""))))
self.assertEqual([lock.parent.name for lock in (self.home / ".local/state/loop-claude").glob("*/starting")], ["api"])
def test_the_waits_keep_the_trackers_rank_order(self) -> None:
run = Tmux(("epic-ABC-12", "ABC-20", True, str(self.home / "code/api")), ("epic-ABC-12", "ABC-22", False, str(self.home / "code/api")))
found = self.start(self.epic(1, api=1), [issue("ABC-20", "repo:api"), issue("ABC-21"), issue("ABC-22", "repo:api")], run, own="ABC-22")
@@ -314,6 +319,22 @@ class StartingAnEpicsChunks(unittest.TestCase):
assert isinstance(found, Failure)
self.assertRegex(found.reason, f"^cannot lock {self.home}/.local/state/do-epic/ABC-12: \\w")
def test_says_which_repos_lock_it_could_not_take(self) -> None:
(self.home / ".local/state").mkdir(parents=True)
(self.home / ".local/state/loop-claude").write_text("", encoding="utf-8")
found = start_chunks(self.epic(1, api=1), [issue("ABC-20", "repo:api")], False, None, Tmux())
assert isinstance(found, Failure)
self.assertRegex(found.reason, f"^cannot lock {self.home}/.local/state/loop-claude/\\S+: \\w")
self.assertRegex(self.resumed(self.epic(1, api=1), Tmux()) or "", f"^cannot lock {self.home}/.local/state/loop-claude/\\S+: \\w")
def test_an_error_raised_under_the_lock_is_not_reported_as_the_locks(self) -> None:
def failing(args: list[str]) -> subprocess.CompletedProcess[str]:
raise OSError("tmux is gone")
with self.assertRaisesRegex(OSError, "tmux is gone"):
start_chunks(self.epic(1, api=1), [issue("ABC-20", "repo:api")], False, None, failing)
with self.assertRaisesRegex(OSError, "tmux is gone"):
resume(self.epic(1, api=1), "api", tmux.Window("epic-ABC-12", "ABC-20", "%0"), failing)
CONFIG = settings.ProjectConfig("api", pathlib.Path("/c/api"), pathlib.Path("/code/api"), {"changelog": pathlib.Path("/code/api/CHANGELOG.md")}, epic=settings.Epic("https://acme.atlassian.net", "API-1"))
+1 -2
View File
@@ -2,7 +2,6 @@
import dataclasses
import fnmatch
import json
import time
import typing
from failure import Failure
@@ -104,7 +103,7 @@ def checks_now(forge: Forge, number: str, head: str, required: set[str], run: Ru
return None if (count == 0 and settled or state == "success") and not missing else Pending(missing)
def checks_passed(forge: Forge, number: str, head: str, required: set[str], run: Run, sleep: typing.Callable[[float], None] = time.sleep, settle: int = 0) -> Failure | None:
def checks_passed(forge: Forge, number: str, head: str, required: set[str], run: Run, sleep: typing.Callable[[float], None], settle: int = 0) -> Failure | None:
"""None once every check on pull request number, whose head commit is head, has passed, or where it has none for settle seconds; a Failure where one fails, or where they have not all passed within WAIT_SECONDS. checks_now says what counts as passed on Gitea."""
if forge == "gh":
for waited in range(0, settle + 1, POLL_SECONDS):
+187
View File
@@ -0,0 +1,187 @@
"""A pull request's review state, read once from its forge: the review decision, whether it is a draft, whether the worker answered the changes a reviewer requested, and the review threads that wait for an answer. chunk-pr and do-epic decide on what this reads, and run no review query of their own."""
import dataclasses
import json
from failure import Failure
from forge.commands import Forge, gitea, Run
# GitHub's review decisions on a pull request. CHANGES_REQUESTED is also the state of the review that asked for them.
APPROVED = "APPROVED"
REVIEW_REQUIRED = "REVIEW_REQUIRED"
CHANGES_REQUESTED = "CHANGES_REQUESTED"
# What `review_decision` returns in place of CHANGES_REQUESTED once the worker answered those changes: GitHub keeps its decision until the reviewer reviews again.
ANSWERED = "CHANGES_ANSWERED"
# What the title of a draft opens with on Gitea, in any case: Gitea's default `WORK_IN_PROGRESS_PREFIXES`.
GITEA_DRAFT = ("WIP:", "[WIP]")
# Each review thread, with whether it is resolved and who wrote its last comment, beside who opened the pull request. gh fills {owner} and {repo} from the checkout.
THREADS = """query($owner: String!, $repo: String!, $number: Int!) { repository(owner: $owner, name: $repo) { pullRequest(number: $number) {
author { login }
reviewThreads(first: 100) { nodes { isResolved path comments(last: 1) { nodes { author { login } } } } }
} } }"""
def review_decision(forge: Forge, pull: str, head: str, run: Run) -> str | None | Failure:
"""The review decision on the pull request, whose head commit is head: `answered_decision` on GitHub, `gitea_decision` on Gitea. None where it is a draft."""
return answered_decision(pull, head, run) if forge == "gh" else gitea_decision(pull, run)
def github_decision(pull: str, run: Run) -> str | Failure:
"""GitHub's review decision on the pull request as it stands, read once: APPROVED, REVIEW_REQUIRED, CHANGES_REQUESTED, or empty where its branch protection asks for no review."""
viewed = run(["gh", "pr", "view", pull, "--json", "reviewDecision"])
if isinstance(viewed, Failure):
return viewed
try:
decision: str = json.loads(viewed)["reviewDecision"] or ""
return decision
except (ValueError, KeyError, TypeError):
return Failure(f"gh pr view printed no review decision: {viewed.strip()[:200]}")
def answered_decision(pull: str, head: str, run: Run) -> str | None | Failure:
"""GitHub's review decision on the pull request, whose head commit is head, read once: APPROVED, REVIEW_REQUIRED, CHANGES_REQUESTED, or empty where its branch protection asks for no review. ANSWERED in place of CHANGES_REQUESTED once the worker answered those changes. None where it is a draft."""
viewed = run(["gh", "pr", "view", pull, "--json", "isDraft,reviewDecision,author,reviews,comments"])
if isinstance(viewed, Failure):
return viewed
view = pull_view(viewed)
if isinstance(view, Failure):
return view
if view.draft:
return None
return ANSWERED if view.decision == CHANGES_REQUESTED and answered(view, head) else view.decision
@dataclasses.dataclass(frozen=True)
class Review:
"""One review of a pull request. GitHub lists a reply in a thread as a review of its own."""
author: str
state: str
# Empty for a review its author has yet to submit.
submitted_at: str
# The commit a review that requested changes was made on; empty for any other review.
commit: str
@dataclasses.dataclass(frozen=True)
class PullView:
"""A pull request as `gh pr view --json isDraft,reviewDecision,author,reviews,comments` prints it."""
draft: bool
# Empty where its branch protection asks for no review.
decision: str
reviews: tuple[Review, ...]
# When its author submitted each review and made each comment. A review the author has yet to submit has an empty time, and doesn't answer anyone.
said: tuple[str, ...]
# What `held` gives where the answer lacks the key asked for: of no type a field may have, so each field's check refuses it.
MISSING = object()
def held(value: object, *path: str) -> object:
"""What a forge's answer holds along path, one key per level; MISSING where a level is no object or lacks its key."""
for key in path:
value = value.get(key, MISSING) if isinstance(value, dict) else MISSING
return value
def pull_view(printed: str) -> PullView | Failure:
"""The pull request that gh printed, with each field checked here, so what decides on it meets no other shape."""
unshaped = Failure(f"gh pr view printed no review decision: {printed.strip()[:200]}")
try:
pull = json.loads(printed)
except ValueError:
return unshaped
author, draft, decision, listed, comments = held(pull, "author", "login"), held(pull, "isDraft"), held(pull, "reviewDecision") or "", held(pull, "reviews"), held(pull, "comments")
if not isinstance(author, str) or draft is MISSING or not isinstance(decision, str) or not isinstance(listed, list) or not isinstance(comments, list):
return unshaped
reviews = []
for review in listed:
reviewer, state, submitted_at = held(review, "author", "login"), held(review, "state"), held(review, "submittedAt") or ""
commit = held(review, "commit", "oid") if state == CHANGES_REQUESTED else ""
if not isinstance(reviewer, str) or not isinstance(state, str) or not isinstance(submitted_at, str) or not isinstance(commit, str):
return unshaped
reviews.append(Review(reviewer, state, submitted_at, commit))
commented = []
for comment in comments:
commenter, created_at = held(comment, "author", "login"), held(comment, "createdAt")
if not isinstance(commenter, str):
return unshaped
if commenter != author:
continue
if not isinstance(created_at, str):
return unshaped
commented.append(created_at)
return PullView(bool(draft), decision, tuple(reviews), tuple(review.submitted_at for review in reviews if review.author == author) + tuple(commented))
def answered(pull: PullView, head: str) -> bool:
"""Whether the worker answered the newest review that requested changes on pull, whose head commit is head: the head is no longer the commit that review was made on, or the pull request's author commented or replied in a thread after it. False where pull lists no review that requested changes.
The review names the commit it was made on, so a head that differs shows the worker pushed. A commit's date cannot show that: it says when the commit was made, never when it was pushed.
"""
asked = max((review for review in pull.reviews if review.state == CHANGES_REQUESTED), key=lambda review: review.submitted_at, default=None)
if asked is None:
return False
return asked.commit != head or any(at > asked.submitted_at for at in pull.said)
def gitea_decision(pull: str, run: Run) -> str | None | Failure:
"""What stands in for `answered_decision` on Gitea, read once: empty, since no review is a gate there, and None where the pull request is a draft, which Gitea tells by its title (GITEA_DRAFT)."""
viewed = gitea(run, f"repos/{{owner}}/{{repo}}/pulls/{pull}")
if isinstance(viewed, Failure):
return viewed
title = viewed.get("title") if isinstance(viewed, dict) else None
if not isinstance(title, str):
return Failure(f"tea api printed no title of pull request {pull}: {str(viewed)[:200]}")
return None if title.upper().startswith(GITEA_DRAFT) else ""
def waiting_threads(number: str, run: Run) -> list[str] | Failure:
"""The files where a reviewer's comment waits for an answer on pull request number, read once: those of each review thread nobody resolved, whose last comment someone other than the pull request's author wrote. So a thread the worker answered, as that author, waits for the reviewer again."""
read = run(["gh", "api", "graphql", "-f", f"query={THREADS}", "-F", "owner={owner}", "-F", "repo={repo}", "-F", f"number={number}"])
if isinstance(read, Failure):
return read
threads = threads_read(read)
if isinstance(threads, Failure):
return threads
return waits(*threads)
@dataclasses.dataclass(frozen=True)
class Thread:
"""One review thread of a pull request that nobody resolved."""
path: str
# Who wrote its last comment; empty for a thread without one.
last: tuple[str, ...]
def threads_read(printed: str) -> tuple[str, tuple[Thread, ...]] | Failure:
"""Who opened the pull request and its review threads that nobody resolved, as gh printed THREADS' answer, with each field checked here."""
unshaped = Failure(f"gh api graphql printed no review threads: {printed.strip()[:200]}")
try:
pull = held(json.loads(printed), "data", "repository", "pullRequest")
except ValueError:
return unshaped
author, nodes = held(pull, "author", "login"), held(pull, "reviewThreads", "nodes")
if not isinstance(author, str) or not isinstance(nodes, list):
return unshaped
threads = []
for thread in nodes:
resolved = held(thread, "isResolved")
if resolved is MISSING:
return unshaped
# A resolved thread is skipped unread: its last comment may be a deleted account's, whose author gh prints as null.
if resolved:
continue
path, comments = held(thread, "path"), held(thread, "comments", "nodes")
if not isinstance(path, str) or not isinstance(comments, list):
return unshaped
last = tuple(login for comment in comments if isinstance(login := held(comment, "author", "login"), str))
if len(last) != len(comments):
return unshaped
threads.append(Thread(path, last))
return author, tuple(threads)
def waits(author: str, threads: tuple[Thread, ...]) -> list[str]:
"""The files of the threads that wait for an answer from author, who opened the pull request."""
return sorted({thread.path for thread in threads if any(last != author for last in thread.last)})
+139
View File
@@ -0,0 +1,139 @@
import json
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from forge.reviews import ANSWERED, CHANGES_REQUESTED, PullView, Review, Thread
from failure import Failure
from forge import reviews
ASKED = {"author": {"login": "rev"}, "state": CHANGES_REQUESTED, "submittedAt": "2026-11-02T10:00:00Z", "commit": {"oid": "abc"}}
PULL = {"isDraft": False, "reviewDecision": CHANGES_REQUESTED, "author": {"login": "me"}, "reviews": [ASKED], "comments": []}
def printing(answer: object):
"""A run whose command prints answer as JSON."""
return lambda args: json.dumps(answer)
class Faulty:
"""A pull request's number or head whose use in a comparison or in a command raises a fault of our own."""
def __ne__(self, other: object) -> bool:
raise TypeError("ours")
def __format__(self, spec: str) -> str:
raise TypeError("ours")
class ThePullRequestAsViewed(unittest.TestCase):
def test_parses_what_gh_prints_into_one_value(self):
pending = {"author": {"login": "me"}, "state": "PENDING", "submittedAt": None, "commit": None}
printed = json.dumps(PULL | {"reviews": [ASKED, pending], "comments": [{"author": {"login": "me"}, "createdAt": "2026-11-02T11:00:00Z"}, {"author": {"login": "rev"}, "createdAt": "2026-11-02T12:00:00Z"}]})
self.assertEqual(reviews.pull_view(printed), PullView(False, CHANGES_REQUESTED, (Review("rev", CHANGES_REQUESTED, "2026-11-02T10:00:00Z", "abc"), Review("me", "PENDING", "", "")), ("", "2026-11-02T11:00:00Z")))
def test_keeps_each_comment_of_the_authors_whoever_commented_before_it(self):
printed = json.dumps(PULL | {"comments": [{"author": {"login": "rev"}, "createdAt": "2026-11-02T12:00:00Z"}, {"author": {"login": "me"}, "createdAt": "2026-11-02T13:00:00Z"}]})
viewed = reviews.pull_view(printed)
assert isinstance(viewed, PullView)
self.assertEqual(viewed.said, ("2026-11-02T13:00:00Z",))
def test_an_answer_in_another_shape_is_a_failure_naming_what_gh_printed(self):
unshaped = ("not json", "x" * 300, "[]", json.dumps(PULL | {"reviews": [ASKED | {"author": {"login": 7}}]}), json.dumps(PULL | {"reviews": None}), json.dumps(PULL | {"author": None}), json.dumps(PULL | {"reviews": [ASKED | {"commit": None}]}), json.dumps(PULL | {"reviews": [ASKED | {"submittedAt": 7}]}), json.dumps(PULL | {"comments": [{"author": {"login": 7}, "createdAt": "2026-11-02T11:00:00Z"}]}), json.dumps(PULL | {"comments": [{"author": {"login": PULL["author"]["login"]}, "createdAt": None}]}), json.dumps(PULL | {"reviews": [{key: value for key, value in ASKED.items() if key != "submittedAt"}]}), json.dumps({key: value for key, value in PULL.items() if key != "isDraft"}))
for printed in unshaped:
with self.subTest(printed=printed):
self.assertEqual(reviews.pull_view(printed), Failure(f"gh pr view printed no review decision: {printed[:200]}"))
self.assertEqual(reviews.answered_decision("7", "abc", lambda args: printed), Failure(f"gh pr view printed no review decision: {printed[:200]}"))
class WhetherTheWorkerAnswered(unittest.TestCase):
def view(self, *others: Review, said: tuple[str, ...] = ()) -> PullView:
return PullView(False, CHANGES_REQUESTED, (Review("rev", CHANGES_REQUESTED, "2026-11-02T10:00:00Z", "abc"), *others), tuple(review.submitted_at for review in others if review.author == "me") + said)
def test_a_new_head_or_a_later_word_of_the_authors_answers_the_newest_request(self):
self.assertFalse(reviews.answered(self.view(), "abc"))
self.assertTrue(reviews.answered(self.view(), "def"))
self.assertTrue(reviews.answered(self.view(said=("2026-11-02T10:30:00Z",)), "abc"))
self.assertFalse(reviews.answered(self.view(said=("2026-11-02T09:00:00Z", "")), "abc"))
self.assertFalse(reviews.answered(self.view(said=("2026-11-02T10:00:00Z",)), "abc"), "a word at the review's own instant is not after it")
self.assertTrue(reviews.answered(self.view(Review("me", "COMMENTED", "2026-11-02T10:30:00Z", "")), "abc"))
self.assertFalse(reviews.answered(PullView(False, "", (), ()), "abc"))
def test_the_decision_reads_answered_once_the_worker_answered(self):
self.assertEqual(reviews.answered_decision("7", "abc", printing(PULL)), CHANGES_REQUESTED)
self.assertEqual(reviews.answered_decision("7", "def", printing(PULL)), ANSWERED)
self.assertIsNone(reviews.answered_decision("7", "abc", printing(PULL | {"isDraft": True})))
self.assertEqual(reviews.answered_decision("7", "abc", printing(PULL | {"reviewDecision": None, "reviews": []})), "")
def test_a_fault_in_the_decision_is_not_reported_as_the_forges(self):
with self.assertRaisesRegex(TypeError, "ours"):
reviews.answered_decision("7", Faulty(), printing(PULL))
class TheThreadsThatWait(unittest.TestCase):
def printed(self, *threads: tuple[str, bool, str]) -> str:
nodes = [{"isResolved": resolved, "path": path, "comments": {"nodes": [{"author": {"login": last}}]}} for path, resolved, last in threads]
return json.dumps({"data": {"repository": {"pullRequest": {"author": {"login": "me"}, "reviewThreads": {"nodes": nodes}}}}})
def test_parses_what_gh_prints_into_the_author_and_each_thread_nobody_resolved(self):
self.assertEqual(reviews.threads_read(self.printed(("a.py", False, "rev"), ("b.py", True, "me"), ("c.py", False, "me"))), ("me", (Thread("a.py", ("rev",)), Thread("c.py", ("me",)))))
def pull_with(self, *nodes: dict[str, object]) -> str:
return json.dumps({"data": {"repository": {"pullRequest": {"author": {"login": "me"}, "reviewThreads": {"nodes": nodes}}}}})
def test_skips_a_resolved_thread_unread_so_a_deleted_accounts_comment_there_doesnt_fail_the_read(self):
deleted = {"comments": {"nodes": [{"author": None}]}}
printed = self.pull_with({"isResolved": True, "path": "b.py", **deleted}, {"isResolved": True}, {"isResolved": False, "path": "a.py", "comments": {"nodes": [{"author": {"login": "rev"}}]}})
self.assertEqual(reviews.waiting_threads("7", lambda args: printed), ["a.py"])
def test_a_deleted_accounts_last_word_in_a_thread_nobody_resolved_is_a_failure(self):
printed = self.pull_with({"isResolved": False, "path": "a.py", "comments": {"nodes": [{"author": None}]}})
self.assertEqual(reviews.waiting_threads("7", lambda args: printed), Failure(f"gh api graphql printed no review threads: {printed[:200]}"))
def test_a_thread_waits_while_unresolved_with_a_reviewers_last_word(self):
printed = self.printed(("b.py", False, "rev"), ("a.py", False, "rev"), ("a.py", False, "rev"), ("c.py", True, "rev"), ("d.py", False, "me"))
self.assertEqual(reviews.waiting_threads("7", lambda args: printed), ["a.py", "b.py"])
def test_an_answer_in_another_shape_is_a_failure_naming_what_gh_printed(self):
for printed in ("not json", "x" * 300, "{}", json.dumps({"data": {"repository": {"pullRequest": {"author": {"login": 7}, "reviewThreads": {"nodes": []}}}}}), json.dumps({"data": {"repository": {"pullRequest": {"author": {"login": "me"}, "reviewThreads": {"nodes": [{"isResolved": False}]}}}}}), json.dumps({"data": {"repository": {"pullRequest": {"author": {"login": "me"}, "reviewThreads": {"nodes": [{"isResolved": False, "path": "a.py", "comments": {"nodes": [{"author": {"login": 7}}]}}]}}}}}), json.dumps({"data": {"repository": {"pullRequest": {"author": {"login": "me"}, "reviewThreads": {"nodes": [{"path": "a.py", "comments": {"nodes": []}}]}}}}})):
with self.subTest(printed=printed):
self.assertEqual(reviews.waiting_threads("7", lambda args: printed), Failure(f"gh api graphql printed no review threads: {printed[:200]}"))
def test_a_fault_in_the_choice_is_not_reported_as_the_forges(self):
with self.assertRaisesRegex(TypeError, "ours"):
reviews.waiting_threads(Faulty(), lambda args: self.printed(("a.py", False, "rev")))
class TheDecisionOnGitea(unittest.TestCase):
def decision(self, answer: object):
return reviews.gitea_decision("7", printing(answer))
def test_a_title_tells_a_draft_and_an_answer_without_one_is_a_failure(self):
self.assertIsNone(self.decision({"title": "wip: later"}))
self.assertEqual(self.decision({"title": "Fix it"}), "")
self.assertEqual(self.decision({"title": 7}), Failure("tea api printed no title of pull request 7: {'title': 7}"))
self.assertEqual(self.decision([]), Failure("tea api printed no title of pull request 7: []"))
self.assertEqual(self.decision("y" * 300), Failure(f"tea api printed no title of pull request 7: {'y' * 200}"))
class TheDecisionOnGitHub(unittest.TestCase):
def test_reads_the_review_decision_and_empty_where_no_review_is_asked_for(self):
asked = []
self.assertEqual(reviews.github_decision("7", lambda args: asked.append(args) or '{"reviewDecision": "APPROVED"}'), "APPROVED")
self.assertEqual(asked, [["gh", "pr", "view", "7", "--json", "reviewDecision"]])
self.assertEqual(reviews.github_decision("7", printing({"reviewDecision": None})), "")
self.assertEqual(reviews.github_decision("7", printing({"reviewDecision": ""})), "")
def test_an_answer_without_a_decision_is_a_failure_naming_the_first_200_characters_gh_printed(self):
for printed in ("not json", "{}", "[]", " " + "x" * 300):
with self.subTest(printed=printed):
self.assertEqual(reviews.github_decision("7", lambda args: printed), Failure(f"gh pr view printed no review decision: {printed.strip()[:200]}"))
def test_a_command_that_fails_is_that_failure(self):
self.assertEqual(reviews.github_decision("7", lambda args: Failure("gh cannot start")), Failure("gh cannot start"))
if __name__ == "__main__":
unittest.main()
+6 -5
View File
@@ -23,19 +23,20 @@ GUARD_SETTINGS = [
]
def git(*args: str, cwd: str | None = None, timeout: float | None = None, guarded: bool = False, errors: str = "strict") -> str | Failure:
def git(*args: str, cwd: str | None = None, timeout: float | None = None, guarded: bool = False, errors: str = "strict", env: typing.Mapping[str, str] | None = None) -> str | Failure:
"""What git printed, run in cwd, else in the current directory.
A Failure where git cannot start, exits non-zero, prints text that is not UTF-8, or still runs after timeout seconds, which stops it.
errors "replace" reads each byte that is not UTF-8 as U+FFFD instead of failing. It is for output, such as a diff, whose file text the caller skips.
env is the environment git runs in, else this process's.
guarded turns off the fsmonitor, the signature check, each transport GUARD_SETTINGS names and a partial clone's fetch of its missing objects. It is for a checkout whose config may name a program.
"""
command = " ".join(args)
guards = [option for setting in GUARD_SETTINGS for option in ("-c", setting)] if guarded else []
# GIT_NO_LAZY_FETCH needs git 2.44.
env = {**os.environ, "GIT_NO_LAZY_FETCH": "1"} if guarded else None
child_env = {**(os.environ if env is None else env), "GIT_NO_LAZY_FETCH": "1"} if guarded else env
try:
ran = subprocess.run(["git", *guards, *args], capture_output=True, cwd=cwd, encoding="utf-8", errors=errors, env=env, timeout=timeout)
ran = subprocess.run(["git", *guards, *args], capture_output=True, cwd=cwd, encoding="utf-8", errors=errors, env=child_env, timeout=timeout)
except UnicodeDecodeError as failure:
return Failure(f"git {command} printed text that is not UTF-8: {failure}")
except (OSError, ValueError) as failure:
@@ -52,8 +53,8 @@ class Git(typing.Protocol):
def top() -> str | Failure:
found = git("rev-parse", "--show-toplevel")
def top(run: Git = git) -> str | Failure:
found = run("rev-parse", "--show-toplevel")
return found if isinstance(found, Failure) else found.strip()
+9
View File
@@ -1,4 +1,5 @@
import contextlib
import os
import pathlib
import subprocess
import sys
@@ -35,6 +36,14 @@ class Git(unittest.TestCase):
self.assertTrue(done.reason.startswith("git log could not start: "), done.reason)
self.assertIn("/code/gone", done.reason)
def test_runs_git_in_the_environment_given_and_only_guarded_stops_the_fetch_of_missing_objects(self):
shown = ("-c", "alias.shown=!echo \"$SHOWN_TO_GIT/$GIT_NO_LAZY_FETCH\"", "shown")
given = {**{name: value for name, value in os.environ.items() if name != "GIT_NO_LAZY_FETCH"}, "SHOWN_TO_GIT": "given"}
self.assertEqual(git_repo.git(*shown, env=given), "given/\n")
self.assertEqual(git_repo.git(*shown, env=given, guarded=True), "given/1\n")
self.assertNotIn("SHOWN_TO_GIT", os.environ)
self.assertEqual(git_repo.git(*shown, guarded=True), "/1\n")
class Guarded(unittest.TestCase):
"""A checkout whose own config may name a program for git to run."""
+1 -1
View File
@@ -8,4 +8,4 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from judge_replay import main
from loop_effects.effects import system_effects
sys.exit(main.main(sys.argv[1:], os.environ, pathlib.Path.home(), system_effects()))
sys.exit(main.main(sys.argv[1:], os.environ, pathlib.Path.home(), system_effects(pathlib.Path.home(), os.environ)))
+4 -3
View File
@@ -1,17 +1,18 @@
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, DRIVER
from chunk_driver.prompts import ANSWER_DESTINATIONS
from claude_code.entries import Entry
from agent_cycle import escalation_mark
from failure import Failure
from agent_cycle.judge import Grounds, judge_rules, TASK_HEADING, transcript_heading
from claude_code.transcript import human_answers
from chunk_driver.drivers import LOOP_CLAUDE
from agent_cycle.worker_prompts import typed_answer
def grounds_of(prompt: str, entries: list[Entry]) -> Grounds | Failure:
"""What the quotes of a judge run on prompt are checked against, as judge_prompt would have made it for the transcript's entries: the premises and the records before the task are quotable, the transcript the judge was shown comes before the rules. The prompt is split on the headings judge_prompt writes, at the first of each; a Failure where it lacks one. Rules of an earlier wording stay part of the transcript, a slack that lets an evidence quote them pass."""
before_task, task_heading, after_task = prompt.partition(f"\n\n{TASK_HEADING}")
heading = transcript_heading(DRIVER)
heading = transcript_heading(LOOP_CLAUDE)
_, found_heading, shown = after_task.partition(heading)
if not (task_heading and found_heading):
return Failure(f"the archived prompt lacks the heading {TASK_HEADING.strip()!r} or {heading.strip()!r}, which judge_prompt writes")
return Grounds(shown.removesuffix(f"\n\n{judge_rules(ANSWER_DESTINATIONS, DRIVER)}"), escalation_mark.after_mark(entries, 0), before_task, tuple(human_answers(entries, typed_answer)))
return Grounds(shown.removesuffix(f"\n\n{judge_rules(ANSWER_DESTINATIONS, LOOP_CLAUDE)}"), escalation_mark.after_mark(entries, 0), before_task, tuple(human_answers(entries, typed_answer)))
+11 -10
View File
@@ -53,25 +53,26 @@ import pathlib
import sys
import typing
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, DRIVER
from chunk_driver.prompts import ANSWER_DESTINATIONS
from transcript_archive.location import archive_root, transcript_of
from transcript_archive.read import archived_outcomes
from rung_replay.archive import archiving, print_report, replay_folder
from rung_replay.command import arguments_or_exit, down_from, failed, limited_exit, projects_on
from rung_replay.archive import archiving, replay_folder
from rung_replay.command import arguments_or_exit, down_from, failed, limited_exit, print_report, projects_on, say
from rung_replay.search import AT_ONCE, first_reply, replayed
from agent_cycle.judge import Brief, judge_archived, judge_once
from judge_replay.replay import Case, case_from, Kind, report, reproduced, sample, verdict_kind
from transcript_archive.project_key import checkout_of
from loop_effects.effects import Effects
from failure import Failure
from judge_replay.offline import final_message, LEARNERS, offline_report, RULES, Sample
from rung_replay.search import first_reply, replayed
from judge_replay.archived_prompt import grounds_of
from transcript_archive.judge_run import JudgeRun
from model_routing.ladder import Ladder, ProjectRung, REVIEW_STEPS, Rung
from claude_code.reply import Limited
from chunk_driver.drivers import LOOP_CLAUDE
from model_routing.ladder_file import read_ladder, SHIPPED
from loop_claude.project.premises import read_premises
from loop_claude.project.records import read_records, RECORDS_NAMED
from chunk_driver.project.premises import read_premises
from chunk_driver.project.records import read_records, RECORDS_NAMED
from claude_code.transcript import read_transcript
from agent_cycle.verdicts import Verdict
@@ -100,10 +101,10 @@ def main(argv: list[str], env: typing.Mapping[str, str], home: pathlib.Path, eff
return offline(project.rung.model, cases, archive_root(home, env), effects)
cases = sample(cases, PER_KIND)
folder = replay_folder(archive_root(home, env), TOOL, argv, effects.now())
effects = archiving(effects, folder)
effects = archiving(effects, folder, say)
reference = Rung(project.rung.model, REFERENCE_EFFORT)
# The candidates stop at the lowest rung a worker routes to: no route uses a rung below it for the judge.
replay = replayed(cases, reference, down_from(ladder, project.rung, project.lowest_worker_rung()), lambda found, rung: rule(found, rung, home, effects), reproduced, "verdicts", climbs(project, reference))
replay = replayed(cases, reference, down_from(ladder, project.rung, project.lowest_worker_rung()), lambda found, rung: rule(found, rung, home, effects), reproduced, "verdicts", say, AT_ONCE, climbs(project, reference))
if isinstance(replay, Limited):
return limited_exit(TOOL, replay)
print_report(folder, report(project.rung.model, reference, cases, replay))
@@ -164,7 +165,7 @@ def rule(case: Case, rung: Rung, home: pathlib.Path, effects: Effects) -> Kind |
grounds = grounds_of(kept.prompt, entries)
if isinstance(grounds, Failure):
return verdict_kind(grounds)
verdict: Verdict | Failure | Limited = first_reply(lambda: judge_archived(case.root, kept.prompt, grounds, len(entries), rung, DRIVER, effects))
verdict: Verdict | Failure | Limited = first_reply(lambda: judge_archived(case.root, kept.prompt, grounds, len(entries), rung, LOOP_CLAUDE, effects))
return verdict if isinstance(verdict, Limited) else verdict_kind(verdict)
premises, records = read_premises(case.root, home), read_records(case.root, home, home / ".claude")
if isinstance(premises, Failure):
@@ -172,5 +173,5 @@ def rule(case: Case, rung: Rung, home: pathlib.Path, effects: Effects) -> Kind |
if isinstance(records, Failure):
return verdict_kind(records)
# 0 for the route's mark, as before any step-up: the replay compares verdict kinds, and a mark changes only which escalation quotes count.
verdict = first_reply(lambda: judge_once(case.root, Brief(task=case.task, premises=premises, records=records, records_named=RECORDS_NAMED, answer_destinations=ANSWER_DESTINATIONS, entries=entries, escalated_at=0, driver=DRIVER), rung, effects))
verdict = first_reply(lambda: judge_once(case.root, Brief(task=case.task, premises=premises, records=records, records_named=RECORDS_NAMED, answer_destinations=ANSWER_DESTINATIONS, entries=entries, escalated_at=0, driver=LOOP_CLAUDE), rung, effects))
return verdict if isinstance(verdict, Limited) else verdict_kind(verdict)
+55 -10
View File
@@ -9,20 +9,22 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, DRIVER
from chunk_driver.prompts import ANSWER_DESTINATIONS
from transcript_archive.outcomes import ANSWERED, FINISHED, INTERRUPTED, UNFINISHED
from claude_code.entries import AssistantTurn, HumanTurn, LoopPrompt, Text
from judge_replay.replay import Case, case_from, report, reproduced, sample, verdict_kind
from text_files import compressed
from project_config.test_project_config import configured
from text_files.doubles import compress
from project_config.doubles import configured
from loop_effects.effects import Effects
from failure import Failure
from agent_cycle.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished
from judge_replay.archived_prompt import grounds_of
from agent_cycle.judge import judge_archived, judge_rules, Rejected, TASK_HEADING, transcript_heading
from chunk_driver.drivers import LOOP_CLAUDE
from judge_replay import main
from rung_replay.search import Miss, Replay, Tried
from model_routing.ladder import Rung
from rung_replay.doubles import Together
TOOL = pathlib.Path(__file__).resolve().parents[1] / "judge-replay"
TRANSCRIPT = pathlib.Path("/archive/p/c/a.jsonl")
@@ -133,15 +135,17 @@ class Main(unittest.TestCase):
(chunk / f"t{n}.outcomes.md").write_text(f"Task: Do it.\n\n2026-10-14T10:00:0{n}Z {verdict}\n\nEvidence:\n> e\n")
self.calls = []
self.prompts = []
self.schemas = []
def run_main(self, argv, reply):
def claude(args, cwd, prompt):
self.calls.append((args[args.index("--model") + 1], args[args.index("--effort") + 1], cwd))
self.prompts.append(prompt)
self.schemas.append(args[args.index("--json-schema") + 1])
result = reply(Rung(args[args.index("--model") + 1], args[args.index("--effort") + 1]), "said Verdict: interrupt" in prompt.rsplit("latest last:", 1)[-1])
return subprocess.CompletedProcess(args, 0, json.dumps(result), "")
effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=unused, input=unused, local_time=unused, memory=unused, now=lambda: 1790000000.0, on_terminal=unused, python=unused, sleep=unused, stream_claude=unused, timer=unused, tmux=unused, tmux_on_terminal=unused)
effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=unused, input=unused, local_time=unused, memory=unused, now=lambda: 1790000000.0, on_terminal=unused, python=unused, replace_file=unused, sleep=unused, stream_claude=unused, timer=unused, tmux=unused, tmux_on_terminal=unused)
out, err = io.StringIO(), io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main(argv, {}, self.home, effects)
@@ -223,6 +227,45 @@ class Main(unittest.TestCase):
runs = [json.loads(path.read_text(encoding="utf-8").splitlines()[0]) for path in folder.glob("*.jsonl")]
self.assertEqual({run["prompt"] for run in runs}, set(self.prompts))
def test_says_where_it_could_not_keep_a_run_and_still_prints_the_report(self):
replays = self.home / ".claude" / "loop-claude-transcripts" / "replays"
replays.write_text("")
code, out, err = self.run_main(["claude-opus-5-5"], lambda rung, interrupted: self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}"))
self.assertEqual(code, 0, err)
self.assertIn(f"could not keep the run in {replays}/judge-replay-claude-opus-5-5-20260921T141320Z: ", err)
self.assertIn("The reference judge, claude-opus-5-5 at xhigh effort, reproduced 2.", out)
def test_says_how_many_verdicts_it_replays_on_each_rung(self):
code, _, err = self.run_main(["claude-opus-5-5"], lambda rung, interrupted: self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}"))
self.assertEqual(code, 0, err)
self.assertEqual(err.splitlines()[:2], ["replaying 2 verdicts on claude-opus-5-5 at xhigh effort", "replaying 2 verdicts on claude-fable-5-2 at low effort"])
def test_replays_four_verdicts_at_once(self):
chunk = self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1"
for n in range(2, 8):
(chunk / f"t{n}.jsonl").write_text(json.dumps({"type": "assistant", "message": {"content": [{"type": "text", "text": f"said {FINISHED} {n}"}]}}) + "\n")
(chunk / f"t{n}.outcomes.md").write_text(f"Task: Do it.\n\n2026-10-14T10:00:0{n}Z {FINISHED}\n\nEvidence:\n> e\n")
together = Together(4)
def reply(rung, interrupted):
together(rung)
return self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}")
code, _, err = self.run_main(["claude-opus-5-5"], reply)
self.assertEqual(code, 0, err)
self.assertIn("replaying 8 verdicts on ", err)
self.assertEqual(set(together.most.values()), {4})
def test_the_prompt_and_the_reply_schema_of_a_rebuilt_run_name_loop_claude_and_the_records_and_show_the_transcript_once(self):
code, _, err = self.run_main(["claude-opus-5-5"], lambda rung, interrupted: self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}"))
self.assertEqual(code, 0, err)
for prompt, schema in zip(self.prompts, self.schemas, strict=True):
self.assertIn("\n\nThe records: the project's working rules and decisions, then the human's rules for every project:\n", prompt)
self.assertIn("what loop-claude told it on [loop] lines, latest last:\n", prompt)
self.assertEqual(prompt.count("said Verdict: "), 1)
self.assertIn("loop-claude", schema)
self.assertNotIn("None", schema)
def test_hands_the_judge_the_prompt_the_archive_kept_and_rebuilds_the_prompt_of_a_verdict_with_none(self):
kept = self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1"
prompt = "From the goals file, OLD premises\n\nThe records: none\n\nThe task the worker was given:\nDo it.\n\nWhat the worker did and said, what the human typed on [human] lines and what loop-claude told it on [loop] lines, latest last:\nsaid Verdict: finished\n\nOLD rules"
@@ -234,6 +277,8 @@ class Main(unittest.TestCase):
self.assertIn("The reference judge, claude-opus-5-5 at xhigh effort, reproduced 2.", out)
# The first replay, on the finished verdict, reads the archived prompt; the second, on the interrupt, is rebuilt from the records of today.
self.assertEqual(self.prompts[0], prompt)
# The reply's schema names loop-claude to the judge of either prompt.
self.assertEqual([schema for schema in self.schemas if "loop-claude" not in schema or "None" in schema], [])
self.assertEqual(self.calls[0][2], self.calls[1][2])
self.assertTrue(self.calls[0][2].endswith("/code/proj"))
self.assertIn("The task the worker was given:\nDo it.", self.prompts[1])
@@ -275,7 +320,7 @@ class Main(unittest.TestCase):
self.assertIn("replayed failed: could not read the transcript", out)
def test_replays_a_verdict_whose_transcript_is_compressed(self):
self.assertIsNone(compressed.compress(self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1" / "t0.jsonl"))
self.assertIsNone(compress(self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1" / "t0.jsonl"))
code, out, err = self.run_main(["claude-opus-5-5"], lambda rung, interrupted: self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}"))
self.assertEqual(code, 0, err)
self.assertNotIn("replayed failed", out)
@@ -330,7 +375,7 @@ class Main(unittest.TestCase):
class JudgeArchivedRuns(unittest.TestCase):
"""judge_archived on a prompt as judge_prompt writes it: the premises, the task, the transcript the judge was shown, and the rules."""
PROMPT = f"HEAD: Ship fast.\n\n{TASK_HEADING}Do it.\n\n{transcript_heading(DRIVER)}Worker chose option B.\n[human] Use Postgres.\n\n{judge_rules(ANSWER_DESTINATIONS, DRIVER)}"
PROMPT = f"HEAD: Ship fast.\n\n{TASK_HEADING}Do it.\n\n{transcript_heading(LOOP_CLAUDE)}Worker chose option B.\n[human] Use Postgres.\n\n{judge_rules(ANSWER_DESTINATIONS, LOOP_CLAUDE)}"
ENTRIES = [AssistantTurn(parts=(Text("Worker chose option B."),)), HumanTurn(text="Use Postgres."), LoopPrompt(text="Keep going.")]
def judged(self, prompt=PROMPT, **fields):
@@ -342,9 +387,9 @@ class JudgeArchivedRuns(unittest.TestCase):
cwds.append(cwd)
return subprocess.CompletedProcess(args, 0, json.dumps({"type": "result", "structured_output": output, "total_cost_usd": 0.25}), "")
effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=unused, input=unused, local_time=unused, memory=unused, now=lambda: 1790000000.0, on_terminal=unused, python=unused, sleep=unused, stream_claude=unused, timer=unused, tmux=unused, tmux_on_terminal=unused)
effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=unused, input=unused, local_time=unused, memory=unused, now=lambda: 1790000000.0, on_terminal=unused, python=unused, replace_file=unused, sleep=unused, stream_claude=unused, timer=unused, tmux=unused, tmux_on_terminal=unused)
grounds = grounds_of(prompt, self.ENTRIES)
return (grounds if isinstance(grounds, Failure) else judge_archived("/code/p", prompt, grounds, len(self.ENTRIES), REFERENCE, DRIVER, effects)), cwds
return (grounds if isinstance(grounds, Failure) else judge_archived("/code/p", prompt, grounds, len(self.ENTRIES), REFERENCE, LOOP_CLAUDE, effects)), cwds
def test_runs_in_the_projects_folder_and_stamps_the_verdict_with_the_entries_read(self):
verdict, cwds = self.judged()
@@ -363,11 +408,11 @@ class JudgeArchivedRuns(unittest.TestCase):
self.assertEqual((type(rejected), rejected.cost), (Rejected, 0.25))
def test_checks_neither_the_task_nor_the_rules_as_the_transcript_or_the_premises(self):
self.assertIsInstance(self.judged(evidence=judge_rules(ANSWER_DESTINATIONS, DRIVER))[0], Rejected)
self.assertIsInstance(self.judged(evidence=judge_rules(ANSWER_DESTINATIONS, LOOP_CLAUDE))[0], Rejected)
self.assertIsInstance(self.judged(answer="Do it.", question="Which?")[0], Interrupt)
def test_splits_the_prompt_at_the_first_transcript_heading(self):
quoting = self.PROMPT.replace("Worker chose", f"Quoted: {transcript_heading(DRIVER)}Worker chose", 1)
quoting = self.PROMPT.replace("Worker chose", f"Quoted: {transcript_heading(LOOP_CLAUDE)}Worker chose", 1)
self.assertIsInstance(self.judged(quoting, evidence="Quoted:")[0], Unfinished)
def test_splits_the_prompt_at_the_first_task_heading(self):
+11 -11
View File
@@ -6,7 +6,7 @@ import shutil
from review_rules.comprehension_history import DIMENSIONS
from failure import Failure
from text_files.markdown_text import FENCE, listed, section
from git_cli.git_repo import git
from git_cli.git_repo import git, Git
from project_config.main import located
from review_rules.review_folder import made_review_folder, review_folder
from project_config.settings import ProjectConfig
@@ -127,14 +127,14 @@ def scoring_folder(worktree: str, home: pathlib.Path) -> pathlib.Path:
return pathlib.Path(review_folder(str(home), worktree)) / "scoring"
def prepare_scoring(worktree: str, home: pathlib.Path) -> str | Failure:
def prepare_scoring(worktree: str, home: pathlib.Path, run: Git = git) -> str | Failure:
"""Makes the scoring run's worktree at worktree's HEAD and writes each seat's prompt beside it, in the chunk's review folder; what it made, as the tool prints it."""
folder = scoring_folder(worktree, home)
# A setup record-scoring never removed, as where the seats failed, would otherwise block every later run.
replaced = folder.exists()
if replaced and isinstance(removed := removed_setup(worktree, home), Failure):
if replaced and isinstance(removed := removed_setup(worktree, home, run), Failure):
return removed
config = located(worktree, home)
config = located(worktree, home, run)
if isinstance(config, Failure):
return config
# Where AGENTS.md is missing, this repo keeps the values in seat-prompts.md: .claude/rules/this-repo-keeps-no-agents-md.md.
@@ -156,9 +156,9 @@ def prepare_scoring(worktree: str, home: pathlib.Path) -> str | Failure:
score = score_bullets(skill)
if isinstance(score, Failure):
return score
listing = git("worktree", "list", "--porcelain", cwd=worktree)
branch = git("symbolic-ref", "--short", "-q", "HEAD", cwd=worktree)
head = git("rev-parse", "--short", "HEAD", cwd=worktree)
listing = run("worktree", "list", "--porcelain", cwd=worktree)
branch = run("symbolic-ref", "--short", "-q", "HEAD", cwd=worktree)
head = run("rev-parse", "--short", "HEAD", cwd=worktree)
if isinstance(listing, Failure):
return listing
if isinstance(head, Failure):
@@ -169,8 +169,8 @@ def prepare_scoring(worktree: str, home: pathlib.Path) -> str | Failure:
return made
checkout = folder / "checkout"
# The seats score the chunk's head, which holds origin's default branch since larv-review §4 step 1, as larv-review → "Scoring run" says.
added = git("worktree", "add", "--quiet", "--detach", str(checkout), "HEAD", cwd=worktree)
tracked = added if isinstance(added, Failure) else git("ls-files", cwd=str(checkout))
added = run("worktree", "add", "--quiet", "--detach", str(checkout), "HEAD", cwd=worktree)
tracked = added if isinstance(added, Failure) else run("ls-files", cwd=str(checkout))
if isinstance(tracked, Failure):
shutil.rmtree(folder, ignore_errors=True)
return tracked
@@ -200,12 +200,12 @@ def prepare_scoring(worktree: str, home: pathlib.Path) -> str | Failure:
return before + f"Scoring worktree, at {head.strip()}: {checkout}\nPrompts, each for one {SEAT_AGENT} agent:\n" + "\n".join(written)
def removed_setup(worktree: str, home: pathlib.Path) -> Failure | None:
def removed_setup(worktree: str, home: pathlib.Path, run: Git = git) -> Failure | None:
"""Removes the scoring worktree and prompts prepare_scoring made for the chunk at worktree, where they exist."""
folder = scoring_folder(worktree, home)
if not folder.exists():
return None
if (folder / "checkout").exists() and isinstance(removed := git("worktree", "remove", "--force", str(folder / "checkout"), cwd=worktree), Failure):
if (folder / "checkout").exists() and isinstance(removed := run("worktree", "remove", "--force", str(folder / "checkout"), cwd=worktree), Failure):
return Failure(f"could not remove the scoring worktree {folder / 'checkout'}: {removed.reason}")
shutil.rmtree(folder, ignore_errors=True)
return None
+14 -15
View File
@@ -9,7 +9,6 @@ import sys
import tempfile
import textwrap
import unittest
import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
@@ -18,7 +17,7 @@ from review_rules.comprehension_history import appended, entries, HistoryChange,
from larv_review.history_commands import appended_to
from larv_review.plan import checkout_plan, Emptied, EMPTIED_WORDS, landed_towards, plan_reasons, scoring_run_due
from review_rules import comprehension_history
from project_config.test_project_config import configured
from project_config.doubles import configured
from larv_review.ruling import counted, ruling, split, surplus, votes
from larv_review.history_commands import decided, rerank
from project_config.settings import DEFAULT_FLOOR
@@ -614,18 +613,18 @@ class LarvReviewTool(unittest.TestCase):
(self.config.parent / "todo.md").write_text(todo(row, "| 2 | 0.2.0 | | **Reach the comprehension floor: do it.** | 1 | 1 | 3 | 1 | 1 | 1 | 9.0 |"))
self.addCleanup(os.chdir, os.getcwd())
os.chdir(self.checkout)
with unittest.mock.patch.dict(os.environ, {"HOME": str(self.config.parents[2])}):
self.assertEqual(plan_reasons("origin/main", [2], pathlib.Path.home()), ["ships item 2, a floor item", "leaves release 0.2.0 with no plan item"])
(self.config.parent / "decisions.md").write_text("# Decisions\n\n## review opt-out: the comprehension check\n\nServes goal 1.\n")
self.assertEqual(plan_reasons("origin/main", [2], pathlib.Path.home()), [])
(self.config.parent / "decisions.md").unlink()
gone = plan_reasons("origin/main", [1, 7], pathlib.Path.home())
assert isinstance(gone, Failure), gone
self.assertEqual(gone.reason, f"the plan as it stands holds no item 7, and larv-review doesn't support a plan outside the checkout, so it cannot read the plan as it stood before the item was finished: {self.config.parent}/todo.md")
(self.config.parent / "todo.md").write_bytes(b"\xff")
unreadable = plan_reasons("origin/main", [2], pathlib.Path.home())
assert isinstance(unreadable, Failure), unreadable
self.assertEqual(unreadable.reason, f"the project's plan: {self.config.parent}/todo.md: not UTF-8")
home = self.config.parents[2]
self.assertEqual(plan_reasons("origin/main", [2], home), ["ships item 2, a floor item", "leaves release 0.2.0 with no plan item"])
(self.config.parent / "decisions.md").write_text("# Decisions\n\n## review opt-out: the comprehension check\n\nServes goal 1.\n")
self.assertEqual(plan_reasons("origin/main", [2], home), [])
(self.config.parent / "decisions.md").unlink()
gone = plan_reasons("origin/main", [1, 7], home)
assert isinstance(gone, Failure), gone
self.assertEqual(gone.reason, f"the plan as it stands holds no item 7, and larv-review doesn't support a plan outside the checkout, so it cannot read the plan as it stood before the item was finished: {self.config.parent}/todo.md")
(self.config.parent / "todo.md").write_bytes(b"\xff")
unreadable = plan_reasons("origin/main", [2], home)
assert isinstance(unreadable, Failure), unreadable
self.assertEqual(unreadable.reason, f"the project's plan: {self.config.parent}/todo.md: not UTF-8")
ROW = "| 3 | 0.2.0 | | **And this.** | 1 | 1 | 3 | 1 | 1 | 1 | 9.0 |"
@@ -992,7 +991,7 @@ class LarvReviewTool(unittest.TestCase):
(checkout / "todo.md").write_text("the owners' own notes\n")
self.enterContext(contextlib.chdir(checkout))
said: list[str] = []
self.assertIsNone(rerank(pathlib.Path.home(), said.append))
self.assertIsNone(rerank(config.parents[2], said.append))
self.assertEqual((said, (checkout / "todo.md").read_text()), ([], "the owners' own notes\n"))
def test_reranks_todo_md_by_the_overall_it_records(self):
+5 -6
View File
@@ -1,4 +1,4 @@
from unittest import mock
import functools
import os
import pathlib
import re
@@ -11,8 +11,9 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from larv_review.scoring_setup import bullets, file_globs, prepare_scoring, project_values, removed_setup, score_bullets
from failure import Failure
from git_cli.git_repo import git
from test_env import isolated_env
from project_config.test_project_config import write_config
from project_config.doubles import write_config
TOOL = pathlib.Path(__file__).resolve().parents[1] / "larv-review"
SKILL = pathlib.Path(__file__).resolve().parents[2] / "skills" / "comprehension-panel" / "SKILL.md"
@@ -103,8 +104,7 @@ class PrepareScoring(unittest.TestCase):
def prepare(self):
"""What prepare-scoring returns in the checkout, run in this process so mutation tests see the lines it runs."""
with mock.patch.dict(os.environ, self.env, clear=True):
return prepare_scoring(str(self.checkout), pathlib.Path.home())
return prepare_scoring(str(self.checkout), self.root, functools.partial(git, env=self.env))
def run_tool(self, *args):
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=self.checkout, env=self.env, text=True)
@@ -139,8 +139,7 @@ class PrepareScoring(unittest.TestCase):
self.commit()
checkout = self.root / ".local" / "state" / "larv-review" / "proj" / "scoring" / "checkout"
checkout.mkdir(parents=True)
with mock.patch.dict(os.environ, self.env, clear=True):
removed = removed_setup(str(self.checkout), pathlib.Path.home())
removed = removed_setup(str(self.checkout), self.root, functools.partial(git, env=self.env))
self.assertIsInstance(removed, Failure)
self.assertTrue(removed.reason.startswith(f"could not remove the scoring worktree {checkout}: "), removed.reason)
self.assertTrue(checkout.exists())
+1 -1
View File
@@ -10,4 +10,4 @@ from loop_effects import effects
from loop_claude.loop import main
from loop_state import paths
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home()), LAUNCHER, pathlib.Path.cwd(), effects.system_effects()))
sys.exit(main.main(sys.argv[1:], os.environ, paths.paths_from(pathlib.Path.home()), LAUNCHER, pathlib.Path.cwd(), effects.system_effects(pathlib.Path.home(), os.environ)))
+1 -1
View File
@@ -1,4 +1,4 @@
"""loop-claude: headless claude cycles, judged, that interrupt the human only for a decision that is theirs (loop/usage.py's help).
Its packages hold one concern each: cycle/ words the worker's prompts and reads the items it names, loop/ runs the cycles, and project/ reads a checkout's files. loop-claude's paths live in bin/loop_state/, and its effects in bin/loop_effects/. The cycle itself, its judge, interrupts and routing live in bin/agent_cycle/, which doesn't import this package, so that a second entry point can drive it. routed_item.py, at this level, reads the plan for the item a chunk starts on.
Its packages hold one concern each: chunk/ words a fresh chunk's first prompt and reads the items a worker names, and loop/ runs the cycles. What it shares with do-epic lives in bin/chunk_driver/. loop-claude's paths live in bin/loop_state/, and its effects in bin/loop_effects/. The cycle itself, its judge, interrupts and routing live in bin/agent_cycle/, which doesn't import this package, so that a second entry point can drive it. routed_item.py, at this level, reads the plan for the item a chunk starts on.
"""
@@ -7,7 +7,7 @@ from agent_cycle.quotes import plain
from plan_store.plans import Plan
from agent_cycle.verdicts import TakenIn
# The label opening the line; prompts.NAME_THE_ITEMS asks the worker for it.
# The label opening the line; fresh_prompt.NAME_THE_ITEMS asks the worker for it.
CHUNK_ITEMS = "Chunk items:"
# A line the worker opens with the label, after any `-` or `>` marker or a backtick; what follows names the items.
LINE = re.compile(rf"[\s>`-]*{re.escape(CHUNK_ITEMS)}(.*)")
+52
View File
@@ -0,0 +1,52 @@
"""What loop-claude alone says to its worker: the first prompt of a chunk, with what chunk-preflight found. The resumed prompt and the rung sentences are chunk_driver.prompts', which do-epic's cycles use too."""
import functools
from loop_claude.chunk.chunk_items import CHUNK_ITEMS
from chunk_driver.drivers import LOOP_CLAUDE
from model_routing.ladder import ProjectRung
from chunk_driver.prompts import READING_RULE, rung_sentences, worker_prompts
from model_routing.route import Route
# loop-claude's name as its workers, its judge and the human read it. A function below that takes a driver words a sentence for any entry point, such as do-epic.
DRIVER = LOOP_CLAUDE
# chunk_items reads the latest such line; three fresh readers picked the label (docs/decisions.md → `routing: loop-claude reads the items a chunk takes`).
NAME_THE_ITEMS = f"Name every plan item this chunk takes on one line, as in `{CHUNK_ITEMS} 297 (L 3, E + I 5), 298 (L 3, E + I 7)`, and write the whole line again whenever the set changes."
AFTER_CODE_HEALTH = 'which rank after code health (~/.claude/CLAUDE.md → "Shared goal") unless the goals file voids it'
def fresh(driver: str, task: str, premises: str, route: Route, project: ProjectRung | None, facts: str) -> str:
"""A chunk's first prompt; facts is preflight_facts' or preflight_failed's text."""
routed = f"{routed_by(driver, route, project)} {recount(route)}" if route.item else " ".join(rung_sentences(driver, route, project))
opening = f"{routed}\n\n" if routed else ""
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one. {READING_RULE} {NAME_THE_ITEMS}\n\n{opening}{facts}\n\nThe chunk's plan items, or else the task above, set its scope. The goals below, {AFTER_CODE_HEALTH}, choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file, {premises}"
def routed_by(driver: str, route: Route, project: ProjectRung | None) -> str:
return " ".join((
f"{driver} runs you on {route.rung}, routed by item {route.item}'s L of {route.likelihood}.",
*rung_sentences(driver, route, project),
))
def recount(route: Route) -> str:
"""Has the worker count the routing item's size again once it knows the code, since the L that routed it was scored before anyone read that code."""
return f"After your first read of the code, count item {route.item}'s size again: the lines, files and components you will change. Where that count would change its L, have a subagent that scores plan items re-score it; a re-score whose L picks a rung above yours is escalation trigger 1."
def preflight_facts(printed: str) -> str:
"""What chunk-preflight printed, for the worker to take as found."""
return f"loop-claude ran `~/.claude/bin/chunk-preflight` at the checkout root just before this run. Take what it printed as found rather than finding it again:\n\n{printed}"
def held_by_others(items: list[int]) -> str:
"""Keeps a fresh worker off the plan items the project's other chunks hold, running or stopped."""
return f"\n\nOther chunks of this project, running or stopped, hold {'items ' if len(items) > 1 else 'item '}{', '.join(map(str, items))}. Leave those items out of this chunk, even where the facts above list them as items that may join it."
def preflight_failed(reason: str) -> str:
"""Has the worker find the facts chunk-preflight would have, its worktree among them."""
return f"loop-claude could not run `~/.claude/bin/chunk-preflight`, so find the chunk's facts yourself. Why it could not run: {reason}"
WORKER = worker_prompts(DRIVER, functools.partial(fresh, DRIVER), NAME_THE_ITEMS)
+56
View File
@@ -0,0 +1,56 @@
import pathlib
import re
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from loop_claude.chunk import chunk_items
from claude_code import entries
from failure import Failure
from loop_claude.chunk import fresh_prompt
from plan_store.todo_md import plan_of
from model_routing.route import Route
from model_routing.ladder import Rung
from loop_effects.doubles import said
from agent_cycle.verdicts import TakenIn
# Item 297 scores the higher L, item 298 the higher E + I.
SCORED = "## Items\n\n| ID | Release | Exempt | Item | L | E | I | R | S | Moves | Score |\n|---|---|---|---|---|---|---|---|---|---|---|\n| 297 | 0.8.0 | | **Do it.** | 4 | 2 | 3 | 7 | 2 | 1 by 9 | 25 |\n| 298 | 0.8.0 | | **Do more.** | 3 | 3 | 4 | 8 | 2 | 1 by 9 | 20 |\n"
class ChunkItems(unittest.TestCase):
def test_names_the_items_on_the_workers_latest_chunk_items_line(self):
earlier = said("Chunk items: 297 (L 3, E + I 5)")
latest = said("Starting.\n**Chunk items:** 297 (L 4, E + I 5), item 298 (L 3, E + I 7), #12, 298\nOn to the tests.")
self.assertEqual(chunk_items.named([earlier, latest]), (297, 298, 12))
self.assertEqual(chunk_items.named([latest, earlier]), (297,))
self.assertEqual(chunk_items.named([earlier, said("Chunk items: none")]), ())
self.assertEqual(chunk_items.named([said("Chunk items: 297 and 298 (L 3, E + I 7)")]), (297, 298))
def test_reads_the_example_line_the_prompt_gives_the_worker(self):
example = re.search(r"`(Chunk items:[^`]*)`", fresh_prompt.NAME_THE_ITEMS)[1]
self.assertEqual(chunk_items.named([said(example)]), (297, 298))
def test_every_prompt_that_starts_or_resumes_a_chunk_asks_for_the_line(self):
self.assertIn("on one line, as in `Chunk items: ", fresh_prompt.WORKER.resumed("", Route(Rung("m")), None))
def test_reads_only_a_line_the_worker_opens_with_the_label_in_its_own_text(self):
prompt = entries.LoopPrompt(text="Chunk items: 297 (L 3, E + I 5)")
written = entries.AssistantTurn(parts=(entries.ToolCall("t", "Write", {"content": "Chunk items: 9"}),))
mentioned = said("Once I know them, I write `Chunk items: 5`.")
self.assertEqual(chunk_items.named([prompt, written, mentioned]), ())
def test_takes_the_highest_l_and_e_plus_i_todo_md_scores_among_the_named_items(self):
self.assertEqual(chunk_items.taken_in([said("Chunk items: 297 (L 1, E + I 2), 298, 400")], plan_of(SCORED)), TakenIn(4, 7))
self.assertEqual(chunk_items.taken_in([said("Chunk items: 400")], plan_of(SCORED)), TakenIn())
self.assertEqual(chunk_items.taken_in([], plan_of(SCORED)), TakenIn())
def test_names_the_items_at_no_level_where_todo_md_cannot_be_read(self):
self.assertEqual(chunk_items.taken_in([said("Chunk items: 297")], Failure("no todo.md")), TakenIn())
self.assertEqual(chunk_items.taken_in([said("Chunk items: 297")], plan_of("no items table")), TakenIn())
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,57 @@
import pathlib
import sys
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from loop_claude.chunk import fresh_prompt
from loop_claude.doubles import OPUS_PROJECT
from chunk_driver import prompts
from model_routing.route import Route
from model_routing.ladder import Rung
class FreshPrompt(unittest.TestCase):
"""loop-claude's first prompt of a chunk."""
# What the preflight found, as the fresh prompt quotes it.
FACTS = "- A preflight fact."
def test_names_the_route_in_a_fresh_prompt_only_where_an_item_routed_it(self):
routed = fresh_prompt.WORKER.fresh("Do it.", "## Goals", Route(Rung("m", "high"), 3, 4, 5), None, self.FACTS)
self.assertIn(f"Continue any open draft PR or worktree for the chunk before creating a new one. {prompts.READING_RULE} {fresh_prompt.NAME_THE_ITEMS}\n\nloop-claude runs you on m at high effort, routed by item 3's L of 4.", routed)
unrouted = fresh_prompt.WORKER.fresh("Do it.", "## Goals", Route(Rung("m")), None, self.FACTS)
self.assertIn(f"before creating a new one. {prompts.READING_RULE} {fresh_prompt.NAME_THE_ITEMS}\n\n{self.FACTS}\n\nThe chunk's", unrouted)
self.assertNotIn("loop-claude runs you on", unrouted)
def test_has_the_routed_worker_count_its_items_size_again_and_puts_the_facts_before_the_scope(self):
routed = fresh_prompt.WORKER.fresh("Do it.", "## Goals", Route(Rung("m", "high"), 3, 4, 5), None, self.FACTS)
self.assertIn("item 3's size again", routed)
self.assertIn("escalation trigger 1", routed)
self.assertIn(f"\n\n{self.FACTS}\n\nThe chunk's", routed)
self.assertNotIn("size again", fresh_prompt.WORKER.fresh("Do it.", "## Goals", Route(Rung("m")), None, self.FACTS))
def test_gives_each_fresh_worker_the_reading_rule_once(self):
rule = "never read more than 80 lines of a file at once, nor take a tool output over 3,000 characters into your context"
for route in (Route(Rung("m", "high"), 3, 4, 5), Route(Rung("m"))):
with self.subTest(route=route):
self.assertEqual(fresh_prompt.WORKER.fresh("Do it.", "## Goals", route, None, self.FACTS).count(rule), 1)
def test_quotes_what_the_preflight_printed_or_has_the_worker_find_the_facts_itself(self):
self.assertIn("\n\n- A fact.", fresh_prompt.preflight_facts("- A fact."))
failed = fresh_prompt.preflight_failed("git could not start")
self.assertIn("yourself", failed)
self.assertIn("git could not start", failed)
def test_has_each_fresh_worker_continue_an_open_draft_pr_or_worktree_once_whatever_the_preflight_found(self):
for facts in (fresh_prompt.preflight_facts("- A fact."), fresh_prompt.preflight_failed("git could not start")):
with self.subTest(facts=facts):
self.assertEqual(fresh_prompt.WORKER.fresh("Do it.", "## Goals", Route(Rung("m")), None, facts).count("Continue any open draft PR or worktree"), 1)
def test_puts_the_rung_sentences_after_the_route_or_else_before_the_facts(self):
routed, unrouted = Route(OPUS_PROJECT.rung, 3, 2, 4), Route(OPUS_PROJECT.rung)
self.assertIn(f"routed by item 3's L of 2.{prompts.rungs_held(fresh_prompt.DRIVER, routed, OPUS_PROJECT)} After your first read", fresh_prompt.WORKER.fresh("Do it.", "## Goals", routed, OPUS_PROJECT, self.FACTS))
self.assertIn(f"{fresh_prompt.NAME_THE_ITEMS}\n\n{prompts.rungs_held(fresh_prompt.DRIVER, unrouted, OPUS_PROJECT).lstrip()}\n\n{self.FACTS}", fresh_prompt.WORKER.fresh("Do it.", "## Goals", unrouted, OPUS_PROJECT, self.FACTS))
if __name__ == "__main__":
unittest.main()
+22
View File
@@ -0,0 +1,22 @@
"""Test doubles for loop-claude's suites: the shipped ladder's opus project, a plan and its release standing."""
from model_routing.ladder import Ladder, ProjectRung, Rung
from release_plan.release_standing import LastCut
from model_routing.ladder_file import read_ladder, SHIPPED
LADDER = read_ladder(SHIPPED)
assert isinstance(LADDER, Ladder), LADDER
OPUS_PROJECT = ProjectRung(LADDER, Rung("claude-opus-5-5", "medium"))
# A plan read with no release cut yet.
NO_CUT = LastCut(None, "", 0, [])
TODO = """# todo
## Items
| ID | Release | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|
| 7 | 0.2.0 | question | **Ask first.** | 4 | 3 | 5 | 10 | 1 | 1 by 9 | 30 |
| 3 | 0.2.0 | | **Do the next thing.** | 4 | 2 | 3 | 7 | 3 | 1 by 9 | 25 |
| 1 | 0.3.0 | | **Do a later thing.** | 1 | 1 | 1 | 1 | 1 | 1 by 9 | 30 |
"""
+6 -36
View File
@@ -1,24 +1,18 @@
"""How a cycle starts: the route it runs on, from the archive or the plan, and the facts chunk-preflight finds for a fresh chunk."""
import pathlib
import typing
"""How loop-claude's cycle starts: the plan it fetches, the item it takes, and the facts chunk-preflight finds for a fresh chunk. The route is `chunk_driver.cycle_start`'s."""
from git_cli.git_repo import default_ref
from release_plan.release_standing import DefaultPlan, project_plan, UNSEEN_RELEASES
from loop_effects.effects import Effects, say
from failure import Failure
from agent_cycle.routing import first_route, kept, Scored
from loop_claude.chunk import fresh_prompt
from loop_claude.loop.chunk_windows import held_beside, item_of
from chunk_preflight.main import ITEM
from plan.todo_table import Item
from transcript_archive.read import latest_worker
from project_config.main import located
from loop_claude.loop.this_loop import Loop
from chunk_driver.this_loop import Loop
from plan_store.plans import Plan
from model_routing.ladder import ProjectRung, Rung
from loop_claude.cycle import prompts
from model_routing.route import Route, route_from
from loop_claude.routed_item import routed_item
from claude_code.session import session_open, transcript_file
from claude_code.session import session_open
# Seconds routing waits for origin to name its default branch, asked only where the checkout records none; past them, the chunk runs on the project's rung.
ORIGIN_TIMEOUT = 60
@@ -32,30 +26,6 @@ def scored_plan(default: DefaultPlan | Failure, effects: Effects) -> Plan | Fail
return plan
def routed(route: Route | None, project: ProjectRung | None, model: str, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Scored | None]) -> Route:
"""The route the cycle runs on. project is the project's rung, and model the project's model.
Where project is None, the model is off the ladder, and the worker runs on it. Otherwise the chunk keeps route, or on a rerun the route its archive last names, as routing.kept allows. Else the plan item routes the chunk afresh.
"""
if project is None:
return Route(Rung(model))
route = route or archived()
return route if route and kept(route, project) else first_route(project, item())
def archived_route(archive_folder: pathlib.Path, loop: Loop) -> Route | None:
"""The route named by the latest verdict archived for the loop's open chunk; None for a chunk not yet judged."""
transcript = transcript_file(loop.session)
worker = latest_worker(archive_folder, transcript) if transcript else None
if isinstance(worker, Failure):
say(f"routing the chunk afresh: {worker.reason}", loop.effects)
return None
route = route_from(worker) if worker else None
if worker and route is None:
say(f"routing the chunk afresh: the archive's latest Worker: line names no route that can be read: {worker}", loop.effects)
return route
def chunk_start(loop: Loop) -> tuple[str, DefaultPlan | Failure]:
"""The facts for a fresh worker's prompt, empty where the session is open, and what the default branch holds, read once for the cycle: it routes the chunk, through plan_item, and scores the items the worker names, through scored_plan. It reads the default branch as last fetched: for a fresh chunk, by the preflight that runs first; for an open session, which runs none, by whatever fetched last."""
fresh = not session_open(loop.session)
@@ -66,9 +36,9 @@ def chunk_facts(loop: Loop) -> str:
"""What `chunk-preflight --worktree` printed at the checkout root, for a fresh worker's prompt, or why it could not run. The chunk of a window named for an item starts from that item, and either ends with the items the project's other chunks hold."""
item = item_of(loop.window.name)
done = loop.effects.python([str(loop.launcher.with_name("chunk-preflight")), "--worktree", *([] if item is None else [ITEM, str(item)])], loop.root)
facts = prompts.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}") if done.returncode else prompts.preflight_facts(done.stdout.strip())
facts = fresh_prompt.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}") if done.returncode else fresh_prompt.preflight_facts(done.stdout.strip())
held = held_beside(loop.window.name, loop.root, loop.paths, loop.effects)
return facts + (prompts.held_by_others(held) if held else "")
return facts + (fresh_prompt.held_by_others(held) if held else "")
def fetched_plan(loop: Loop, announce: bool = True) -> DefaultPlan | Failure:
+22 -53
View File
@@ -1,13 +1,12 @@
"""The chunks of one project that run at once, each in its own tmux window: which may start now, and how each starts (docs/decisions.md → `loop-claude: each chunk runs in its own tmux window`)."""
import contextlib
import dataclasses
import fcntl
import pathlib
import re
import typing
from touch_predictions.beside import Beside, chunks_beside
from loop_claude.cycle import chunk_items
from chunk_driver.claims import CHUNK_FLAG, epic_chunks, HANDED, loop_session, project_lock, starting
from loop_claude.chunk import chunk_items
from git_cli.git_repo import default_ref
from loop_effects.effects import Effects, say
from failure import Failure
@@ -22,18 +21,10 @@ from claude_code.transcript import read_transcript
from claude_code.session import Session, session_named, transcript_file
from agent_cycle import tmux
# Under the home folder, a folder per project key holds LOCK.
STATE = ".local/state/loop-claude"
# The file an entry point keeps locked with flock while it lists the windows of a project's chunks and starts new ones. Two loops whose chunks finish together then don't start a chunk twice, and do-epic, which holds the same lock, never starts a chunk in a checkout while loop-claude does.
LOCK = "starting"
# The flag that tells the loop in a window its chunk, as `--chunk item-372`. Nobody types it, so no panel picked it.
CHUNK_FLAG = "--chunk"
# The chunk of a plan that names no item for the next chunk: a release to plan, a release review, or a task outside the plan.
NO_ITEM = "no-item"
# Seconds a fetch of origin may take before a chunk starts; past them, the plan reads as last fetched.
FETCH_TIMEOUT = 60
# The prefix of the variables loop-claude reads. A window gets the tmux server's environment, not the starting terminal's, so they are handed to each window by name.
HANDED = "LOOP_CLAUDE_"
def window_name(item_id: int) -> str:
@@ -51,25 +42,6 @@ def chunk_session(key: str, chunk: str, claude_code_transcripts: pathlib.Path) -
return session_named(f"A {key} loop-claude {chunk}", claude_code_transcripts)
@contextlib.contextmanager
def starting(folder: pathlib.Path) -> typing.Iterator[None]:
"""Holds the LOCK in folder: project_lock's for a project, or do-epic's for an epic. Raises OSError where the file cannot be opened or locked."""
folder.mkdir(parents=True, exist_ok=True)
with open(folder / LOCK, "a") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
yield
def project_lock(root: str, home: pathlib.Path) -> pathlib.Path:
"""The folder of the LOCK that everything starting a chunk in the checkout root holds."""
return home / STATE / project_key(root, home)
def epic_chunks(root: str, run: tmux.Tmux) -> list[tmux.Placed]:
"""The windows of do-epic's sessions whose chunks run in the checkout root. A parked chunk doesn't run."""
return [placed for placed in tmux.everywhere(run) if placed.session.startswith(f"{tmux.EPIC}-") and placed.root == root and not placed.window.dead and not placed.window.parked]
def held_items(chunks: typing.Iterable[str], key: str, claude_code_transcripts: pathlib.Path) -> set[int]:
"""The plan items these chunks hold: the one each started from, and those its worker names on its latest `Chunk items:` line so far. A chunk whose transcript cannot be read holds the item it started from."""
held: set[int] = set()
@@ -141,23 +113,21 @@ def start_chunks(loop_run: LoopRun, leaving: bool, paths: Paths, env: typing.Map
config = located(loop_run.root, paths.home)
if isinstance(config, Failure):
return config
session = tmux.session_name(tmux.LOOP, project_key(loop_run.root, paths.home))
session = loop_session(loop_run.root, paths.home)
handed = {name: value for name, value in env.items() if name.startswith(HANDED)}
lock = project_lock(loop_run.root, paths.home)
try:
with starting(lock):
windows = tmux.listed(session, effects.tmux)
if refused := refusal(session, windows, loop_run.root, effects.tmux):
return refused
chunks = chunks_to_start(windows, loop_run, leaving, config.parallel_chunks, paths, effects)
for chunk in chunks:
started = tmux.start_chunk(tmux.Window(session, chunk, ""), loop_run.root, handed, [str(launcher), CHUNK_FLAG, chunk, loop_run.task], windows, True, effects.tmux)
if isinstance(started, Failure):
return started
windows = started
return chunks
except OSError as failure:
return Failure(f"cannot lock {lock / LOCK}: {failure.strerror}")
with starting(project_lock(loop_run.root, paths.home)) as unlocked:
if unlocked:
return unlocked
windows = tmux.listed(session, effects.tmux)
if refused := refusal(session, windows, loop_run.root, effects.tmux):
return refused
chunks = chunks_to_start(windows, loop_run, leaving, config.parallel_chunks, paths, effects)
for chunk in chunks:
started = tmux.start_chunk(tmux.Window(session, chunk, ""), loop_run.root, handed, [str(launcher), CHUNK_FLAG, chunk, loop_run.task], windows, True, effects.tmux)
if isinstance(started, Failure):
return started
windows = started
return chunks
def refusal(session: str, windows: list[tmux.Listed], root: str, run: tmux.Tmux) -> Failure | None:
@@ -185,13 +155,12 @@ def chunks_to_start(windows: list[tmux.Listed], loop_run: LoopRun, leaving: bool
def held_beside(chunk: str, root: str, paths: Paths, effects: Effects) -> list[int]:
"""The plan items that the project's other chunks hold now, running or stopped, which a chunk starting as `chunk` leaves to them. It waits for a loop that is starting windows, so chunks started together see each other. Where the lock cannot be taken, it reads the windows without it."""
key = project_key(root, paths.home)
wait_for_starts(paths.home, key)
others = [window.name for window in tmux.listed(tmux.session_name(tmux.LOOP, key), effects.tmux) if window.name != chunk]
return sorted(held_items(others, key, paths.claude_code_transcripts))
wait_for_starts(root, paths.home)
others = [window.name for window in tmux.listed(loop_session(root, paths.home), effects.tmux) if window.name != chunk]
return sorted(held_items(others, project_key(root, paths.home), paths.claude_code_transcripts))
def wait_for_starts(home: pathlib.Path, key: str) -> None:
"""Returns once no loop of the project is starting windows: start_chunks holds LOCK while it starts them. Where the lock cannot be taken, it returns at once."""
with contextlib.suppress(OSError), starting(home / STATE / key):
def wait_for_starts(root: str, home: pathlib.Path) -> None:
"""Returns once no loop of the project is starting windows: start_chunks holds the lock file `claims.LOCK` names while it starts them. Where the lock cannot be taken, it returns at once."""
with starting(project_lock(root, home)):
pass
-47
View File
@@ -1,47 +0,0 @@
"""How a cycle closes: the loop run the next cycle starts from, the step-up an unfinished verdict makes archived, and the stop once a chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
import dataclasses
import pathlib
from agent_cycle.run import archived_step, archived_stop, Concluded
from agent_cycle.verdicts import Finished
from loop_claude.loop.this_loop import Loop
from loop_claude.loop.reload import LoopRun
from agent_cycle.stop import needs_human, Stop
from agent_cycle.routing import next_route, step_up
from model_routing.ladder import ProjectRung
from claude_code.session import transcript_file
# How many unfinished verdicts running end the loop, so a worker and a judge that disagree about a chunk cannot cycle unattended without end. A policy number, not derived from the routing steps: well above the two unfinished verdicts running that archived chunks have reached at most, and low enough that a stuck pair is stopped after six cycles. It counts every unfinished verdict, including those of cycles where the human answered an interrupt: a rerun is cheap, and a chunk that needs that many answers is worth a look. The human confirmed both the number and what it counts.
UNFINISHED_LIMIT = 6
def closed_cycle(loop_run: LoopRun, project: ProjectRung | None, concluded: Concluded, archive_folder: pathlib.Path, driver: str, loop: Loop) -> LoopRun | Stop:
"""The loop run after a cycle of driver, the entry point, concluded, with the step-up archived; the stop that needs the human, archived, once the chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
after = after_verdict(loop_run, project, concluded)
# A finished chunk drops its route, so it has no step-up, and its run of unfinished verdicts restarts.
if after.route is None:
return after
# The unfinished verdict, archived as the cycle closed, names the route its cycle ended on; the step-up names the one the next cycle runs on.
path = transcript_file(loop.session)
if path and (failure := archived_step(archive_folder, path, after.task, step_up(concluded.route, after.route), loop.effects)):
return Stop(1, failure.reason)
if limit := reached_limit(after, driver):
# Archived with the route the next cycle would run on, so a rerun resumes there.
return archived_stop(archive_folder, loop.session, after.task, after.route, limit, limit.reason, loop.effects)
return after
def reached_limit(loop_run: LoopRun, driver: str) -> Stop | None:
"""The stop that needs the human once the loop run holds UNFINISHED_LIMIT unfinished verdicts running."""
if loop_run.unfinished_run < UNFINISHED_LIMIT:
return None
return needs_human("go on with the chunk", f"the judge ruled it unfinished {loop_run.unfinished_run} cycles running; finish the chunk yourself, or rerun {driver} to give it {UNFINISHED_LIMIT} more cycles")
def after_verdict(loop_run: LoopRun, project: ProjectRung | None, concluded: Concluded) -> LoopRun:
"""The loop run the next cycle starts from. A finished chunk drops its route and its run of unfinished verdicts. An unfinished one takes the route routing.next_route makes of the route its cycle ended on, which an interrupt's or a quoted answer's escalation may have raised, and the evidence the judge gave for the verdict; and counts the verdict."""
verdict = concluded.verdict
if isinstance(verdict, Finished):
return dataclasses.replace(loop_run, route=None, unfinished_evidence="", unfinished_run=0)
route = next_route(project, concluded.route, verdict)
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence, unfinished_run=loop_run.unfinished_run + 1)
+61 -63
View File
@@ -2,36 +2,58 @@ import dataclasses
import pathlib
import typing
from chunk_driver.cycle_start import announced_route, cycle_of, project_routing
from transcript_archive.location import archive_root, project_archive
from loop_claude.loop.chunk_start import archived_route, chunk_start, plan_item, routed, scored_plan
from loop_claude.loop.routing_check import check_routing
from loop_claude.project.checkout import checkout_root
from loop_claude.loop.chunk_windows import CHUNK_FLAG, chunk_session, item_of, start_chunks
from loop_claude.cycle import chunk_items, prompts
from loop_claude.loop.close import closed_cycle
from agent_cycle.run import Cycle, run_cycle
from loop_claude.loop.checkout import checkout_root
from chunk_driver.claims import CHUNK_FLAG, loop_session
from loop_claude.chunk import chunk_items, fresh_prompt
from loop_claude.loop.chunk_windows import chunk_session, item_of, start_chunks
from loop_claude.loop.chunk_start import chunk_start, plan_item, scored_plan
from chunk_driver.chunk_run import closed_cycle
from agent_cycle.run import Concluded, Cycle, run_cycle
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
from loop_effects.effects import Effects, say
from failure import Failure
from agent_cycle.verdicts import Finished
from loop_claude.project.model_choice import get_or_ask_models
from git_cli.git_repo import Git
from loop_claude.loop.usage import HELP
from agent_cycle.routing import judge_rung
from loop_claude.loop.this_loop import Loop
from chunk_driver.this_loop import Loop
from loop_state.paths import Paths
from transcript_archive.project_key import project_key
from model_routing.ladder_file import project_ladder_file, read_ladder, SHIPPED
from loop_claude.project.premises import read_premises
from loop_claude.project.records import read_records
from model_routing.ladder import ProjectRung, Rung
from chunk_driver.project.premises import read_premises
from chunk_driver.project.records import read_records
from model_routing.ladder_file import SHIPPED
from agent_cycle.stop import Stop
from agent_cycle.direct_message import tell
from agent_cycle import tmux
# Starts the chunks that may start now, as chunk_windows.start_chunks does.
StartChunks = typing.Callable[[LoopRun, bool, Paths, typing.Mapping[str, str], pathlib.Path, Effects], list[str] | Failure]
@dataclasses.dataclass(frozen=True)
class Steps:
"""The steps a loop takes, each the function of its name, which main binds."""
start_chunks: StartChunks
fingerprint: typing.Callable[[pathlib.Path], str]
exec_if_edited: typing.Callable[[Loop, str, LoopRun], Stop | None]
next_cycle: typing.Callable[[LoopRun, Loop], Cycle | Stop]
run_cycle: typing.Callable[[Cycle], Stop | Concluded]
closed_cycle: typing.Callable[[LoopRun, str, ProjectRung | None, Concluded, pathlib.Path, str, Loop], LoopRun | Stop]
check_routing: typing.Callable[[pathlib.Path, Rung, Loop], Stop | None]
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
steps = Steps(start_chunks, fingerprint, exec_if_edited, next_cycle, run_cycle, closed_cycle, check_routing)
return exit_code(lambda: run_loop(argv, env, paths, launcher, cwd, effects, steps), cwd, effects)
def exit_code(run: typing.Callable[[], Stop], cwd: pathlib.Path, effects: Effects) -> int:
"""The exit code of the loop that run runs in cwd, once its stop is said and sent as a DM."""
try:
stop = run_loop(argv, env, paths, launcher, cwd, effects)
stop = run()
except KeyboardInterrupt:
stop = Stop(130, "")
except Exception as crash:
@@ -45,49 +67,49 @@ def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher:
return stop.code
def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> Stop:
def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects, steps: Steps) -> Stop:
loop_run = loop_run_from(argv, cwd, effects)
if isinstance(loop_run, Stop):
return loop_run
# Before anything starts: a worktree, a claude session or a tmux window.
if reason := tmux.unusable(effects.tmux, prompts.DRIVER):
if reason := tmux.unusable(effects.tmux, fresh_prompt.DRIVER):
return Stop(1, reason)
if loop_run.chunk is None:
return windows_opened(loop_run, env, paths, launcher, effects)
return windows_opened(loop_run, env, paths, launcher, effects, steps.start_chunks)
pane = env.get("TMUX_PANE")
if not pane:
return Stop(1, f"{CHUNK_FLAG} is how loop-claude starts itself in a tmux window; run loop-claude <task>")
baseline = fingerprint(launcher)
baseline = steps.fingerprint(launcher)
key = project_key(loop_run.root, paths.home)
window = tmux.Window(tmux.session_name(tmux.LOOP, key), loop_run.chunk, pane)
window = tmux.Window(loop_session(loop_run.root, paths.home), loop_run.chunk, pane)
loop = Loop(root=loop_run.root, session=chunk_session(key, loop_run.chunk, paths.claude_code_transcripts), window=window, paths=paths, env=env, launcher=launcher, effects=effects)
while True:
if stop := exec_if_edited(loop, baseline, loop_run):
if stop := steps.exec_if_edited(loop, baseline, loop_run):
return stop
loop_run = dataclasses.replace(loop_run, cycle_number=loop_run.cycle_number + 1)
cycle = next_cycle(loop_run, loop)
cycle = steps.next_cycle(loop_run, loop)
if isinstance(cycle, Stop):
return cycle
concluded = run_cycle(cycle)
concluded = steps.run_cycle(cycle)
if isinstance(concluded, Stop):
return concluded
closed = closed_cycle(loop_run, cycle.project, concluded, cycle.project_archive, prompts.DRIVER, loop)
closed = steps.closed_cycle(loop_run, cycle.task, cycle.project, concluded, cycle.project_archive, fresh_prompt.DRIVER, loop)
if isinstance(closed, Stop):
return closed
loop_run = closed
# Off the ladder the worker runs on the project's model itself, which its route names.
if stop := check_routing(archive_root(paths.home, env), cycle.project.rung if cycle.project else cycle.route.rung, loop):
if stop := steps.check_routing(archive_root(paths.home, env), cycle.project.rung if cycle.project else cycle.route.rung, loop):
return stop
if isinstance(concluded.verdict, Finished):
return handed_on(loop_run, loop)
return handed_on(loop_run, loop, steps.start_chunks)
def windows_opened(loop_run: LoopRun, env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects) -> Stop:
"""What the loop-claude the human runs does: starts each chunk that may start now in a window of the project's tmux session, then puts that session on the terminal until the human detaches."""
started = start_chunks(loop_run, False, paths, env, launcher, effects)
def windows_opened(loop_run: LoopRun, env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, start: StartChunks) -> Stop:
"""What the loop-claude the human runs does: starts, through start, each chunk that may start now in a window of the project's tmux session, then puts that session on the terminal until the human detaches."""
started = start(loop_run, False, paths, env, launcher, effects)
if isinstance(started, Failure):
return Stop(1, started.reason)
session = tmux.session_name(tmux.LOOP, project_key(loop_run.root, paths.home))
session = loop_session(loop_run.root, paths.home)
say(f"started {', '.join(started)} in the tmux session {session}" if started else f"the tmux session {session} runs every chunk that may run now, so `{loop_run.task}` started in no new window", effects, stdout=True)
if not effects.on_terminal():
say(f"not on a terminal; see the chunks with `tmux attach -t ={session}`", effects, stdout=True)
@@ -96,9 +118,9 @@ def windows_opened(loop_run: LoopRun, env: typing.Mapping[str, str], paths: Path
return Stop(1, f"`tmux {' '.join(shown)}` failed; the chunks go on in the tmux session") if effects.tmux_on_terminal(shown) else Stop(0, "")
def handed_on(loop_run: LoopRun, loop: Loop) -> Stop:
"""The stop of a loop whose chunk finished, once it has started each chunk that may start now. Its window closes as it exits."""
started = start_chunks(loop_run, True, loop.paths, loop.env, loop.launcher, loop.effects)
def handed_on(loop_run: LoopRun, loop: Loop, start: StartChunks) -> Stop:
"""The stop of a loop whose chunk finished, once it has started, through start, each chunk that may start now. Its window closes as it exits."""
started = start(loop_run, True, loop.paths, loop.env, loop.launcher, loop.effects)
if isinstance(started, Failure):
return Stop(1, f"the chunk finished, but the next could not start: {started.reason}")
say(f"the chunk finished; started {', '.join(started) or 'no other, since the chunks running fill what may run now'}", loop.effects, stdout=True)
@@ -113,25 +135,17 @@ def next_cycle(loop_run: LoopRun, loop: Loop) -> Cycle | Stop:
records = read_records(loop.root, loop.paths.home, loop.paths.claude_dir)
if isinstance(records, Failure):
return Stop(1, records.reason)
models = get_or_ask_models(project_key(loop.root, loop.paths.home), loop.paths, loop.env, prompts.DRIVER, loop.effects)
if isinstance(models, Stop):
return models
ladder_file = project_ladder_file(loop.root, loop.paths.home)
if isinstance(ladder_file, Failure):
return Stop(1, ladder_file.reason)
ladder = read_ladder(ladder_file)
if isinstance(ladder, Failure):
return Stop(1, ladder.reason)
routing = project_routing(loop, fresh_prompt.DRIVER)
if isinstance(routing, Stop):
return routing
archive_folder = project_archive(loop.root, loop.paths.home, loop.env)
# After the reads above, so a loop that stops on one makes no worktree.
facts, default = chunk_start(loop)
project = ladder.project_rung(models.worker)
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, loop), lambda: plan_item(default, item_of(loop.window.name), loop.effects))
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", loop.effects, stdout=True)
if ladder_file != SHIPPED:
say(own_ladder(ladder_file, models.worker, project is None), loop.effects)
route = announced_route(loop_run.cycle_number, loop_run.route, routing, archive_folder, lambda: plan_item(default, item_of(loop.window.name), loop.effects), loop)
if routing.ladder_file != SHIPPED:
say(own_ladder(routing.ladder_file, routing.models.worker, routing.project is None), loop.effects)
plan = scored_plan(default, loop.effects)
return Cycle(config=setup_version(loop.paths.claude_dir, loop.effects.git), effects=loop.effects, facts=facts, judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=loop_run.task, prompts=prompts.WORKER, taken_in=lambda entries: chunk_items.taken_in(entries, plan), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
return cycle_of(loop, routing, route, archive_folder, task=loop_run.task, facts=facts, premises=premises, records=records, unfinished_evidence=loop_run.unfinished_evidence, prompts=fresh_prompt.WORKER, taken_in=lambda entries: chunk_items.taken_in(entries, plan))
def own_ladder(file: pathlib.Path, model: str, unrouted: bool) -> str:
@@ -152,19 +166,3 @@ def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopR
if not (loop_run and loop_run.root and loop_run.task):
return Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}")
return loop_run
def setup_version(claude_dir: pathlib.Path, git: Git) -> str:
"""The release and commit of this setup that the worker's claude reads, as `v0.8.0 (<sha>)`; a commit past the latest release reads as git describe gives it, such as `v0.8.0-3-g1a2b3c4 (<sha>)`. Where git can't tell, `unknown` and why."""
described = git("describe", "--tags", "--match", "v*", "--always", cwd=str(claude_dir))
commit = git("rev-parse", "HEAD", cwd=str(claude_dir))
if isinstance(described, Failure):
return unknown_version(described)
if isinstance(commit, Failure):
return unknown_version(commit)
return f"{described.strip()} ({commit.strip()})"
def unknown_version(failure: Failure) -> str:
# One line, since the archive reads the line after it as the judge's.
return f"unknown: {' '.join(failure.reason.split())}"
+1 -1
View File
@@ -5,7 +5,7 @@ import pathlib
import sys
import typing
from loop_claude.loop.this_loop import Loop
from chunk_driver.this_loop import Loop
from model_routing.route import Route, route_of
from loop_effects.effects import say
from agent_cycle.stop import Stop
+2 -2
View File
@@ -6,7 +6,7 @@ from routing_figures.calls import Call, calls, totals
from agent_cycle.hand_over import claude_args, ClaudeCommand, hand_over
from loop_effects.effects import Effects, say
from failure import Failure
from loop_claude.loop.this_loop import Loop
from chunk_driver.this_loop import Loop
from loop_state.paths import Paths
from model_routing.ladder_file import read_ladder, SHIPPED
from routing_figures.render import render
@@ -67,6 +67,6 @@ def raise_calls(root: pathlib.Path, new: list[Call], rung: Rung, loop: Loop) ->
if unstarted:
return Stop(1, f"could not start claude for the routing calls in {loop.paths.config_clone}, the setup's development clone: {unstarted.reason}")
# Raised once the human has seen them, answered or not: an unanswered call reaches the plan through docs/decisions.md → "Working rules", so it is not asked twice.
if unmarked := state.mark_raised(root, new, utc(loop.effects.now())):
if unmarked := state.mark_raised(root, new, utc(loop.effects.now()), loop.effects.replace_file):
say(f"routing figures: {unmarked.reason}", loop.effects)
return None
+160
View File
@@ -0,0 +1,160 @@
import contextlib
import dataclasses
import io
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from chunk_driver.doubles import a_loop
from loop_claude.loop import chunk_start
from project_config.doubles import configured
from release_plan.release_standing import DefaultPlan, LastCut, UNSEEN_RELEASES
from failure import Failure
from loop_effects.doubles import FakeEffects, ran, STAMP
from loop_claude.chunk import fresh_prompt
from loop_claude.doubles import NO_CUT, TODO
from loop_state import paths
from plan_store.todo_md import plan_of
from claude_code import session
from agent_cycle import tmux
class RouteSources(unittest.TestCase):
def test_routes_on_the_plan_item_and_says_why_it_routes_on_the_projects_rung(self):
self.assertIsNone(chunk_start.plan_item(DefaultPlan(plan_of(""), NO_CUT), None, FakeEffects().effects()))
self.assertEqual(chunk_start.plan_item(DefaultPlan(plan_of(TODO), LastCut("v0.1.0", "", 3, [TODO])), None, FakeEffects().effects()).id, 3)
self.assertIsNone(chunk_start.plan_item(DefaultPlan(plan_of(TODO), LastCut("v0.3.0", "", 3, [TODO])), None, FakeEffects().effects()))
def test_routes_a_window_named_for_an_item_on_that_item_whatever_the_plan_takes_next(self):
plan = DefaultPlan(plan_of(TODO), LastCut("v0.1.0", "", 3, [TODO]))
later = next(item.id for item in plan.plan.items if item.id != 3)
self.assertEqual(chunk_start.plan_item(plan, later, FakeEffects().effects()).id, later)
# An item that left the plan, as once it shipped.
self.assertIsNone(chunk_start.plan_item(plan, 999, FakeEffects().effects()))
# A chunk start reads the default branch once, for routing and for scoring the items the worker names.
with tempfile.TemporaryDirectory() as root:
folder = configured(self, pathlib.Path(root))
config = folder / "config.toml"
config.write_text(config.read_text().replace('"checkout:todo.md"', '"folder:todo.md"'))
(folder / "todo.md").write_text(TODO)
opened = session.session_named("A proj loop-claude", pathlib.Path(root, "projects"))
transcript = pathlib.Path(root, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text("")
unread = Failure("no git")
fake = FakeEffects(git=["origin\n", "origin/main\n", unread, unread, unread, unread])
with contextlib.redirect_stderr(io.StringIO()) as stderr:
facts, held = chunk_start.chunk_start(a_loop(fake.effects(), root=root, session=opened, paths=paths.paths_from(folder.parents[1])))
self.assertEqual((facts, held.plan), ("", plan_of(TODO)))
self.assertEqual([args for args in fake.made("git") if args[0] == "remote"], [["remote"]])
# An open session is no fresh chunk, so the human isn't told again of releases the plan cannot show.
self.assertEqual(stderr.getvalue(), "")
unreadable = Failure("git show origin/main:todo.md printed text that is not UTF-8")
for plan, said in ((DefaultPlan(plan_of(TODO.replace("| 4 | 2 | 3 |", "| x | 2 | 3 |")), NO_CUT), "routing on the project's rung: item 3's L 'x' is not a whole number from 1 to 4\n"), (DefaultPlan(unreadable, NO_CUT), f"routing on the project's rung: {unreadable.reason}\n"), (Failure("no origin remote"), "routing on the project's rung: no origin remote\n")):
with self.subTest(said=said):
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIsNone(chunk_start.plan_item(plan, None, FakeEffects().effects()))
self.assertTrue(stderr.getvalue().startswith(f"{STAMP} {said}"), stderr.getvalue())
def test_says_why_the_items_the_worker_names_lift_no_rung(self):
unscored = plan_of(TODO.replace("| 4 | 2 | 3 |", "| x | 2 | 3 |"))
for default, said in ((DefaultPlan(unscored, NO_CUT), "the items the worker names lift no rung: item 3's L 'x' is not a whole number from 1 to 4\n"), (Failure("no git"), "the items the worker names lift no rung: no git\n"), (DefaultPlan(plan_of(TODO), NO_CUT), "")):
with self.subTest(said=said):
with contextlib.redirect_stderr(io.StringIO()) as stderr:
scored = chunk_start.scored_plan(default, FakeEffects().effects())
self.assertEqual(stderr.getvalue(), f"{STAMP} {said}" if said else "")
self.assertEqual(scored, default.plan if isinstance(default, DefaultPlan) else default)
def test_reads_the_changelog_the_default_branch_holds_through_the_loops_git(self):
def git(*args, **options):
answers = {"remote": "origin\n", "symbolic-ref": "origin/main\n", "show": f"what {args[-1]} holds\n"}
return answers.get(args[0], "1\n")
with tempfile.TemporaryDirectory() as root:
folder = configured(self, pathlib.Path(root))
effects = dataclasses.replace(FakeEffects().effects(), git=git)
held = chunk_start.fetched_plan(a_loop(effects, root=root, paths=paths.paths_from(folder.parents[1])))
self.assertEqual(held.cut.changelog, "what origin/main:CHANGELOG.md holds\n")
def test_reads_a_plan_outside_the_checkout_as_it_stands_and_stops_without_a_config(self):
with tempfile.TemporaryDirectory() as root:
folder = configured(self, pathlib.Path(root))
loop = a_loop(FakeEffects(git=["origin\n", "origin/main\n"] * 2).effects(), root=root, paths=paths.paths_from(folder.parents[1]))
config = folder / "config.toml"
tracked = a_loop(FakeEffects(git=["origin\n", "origin/main\n", *[Failure("no git")] * 4]).effects(), root=root, paths=paths.paths_from(folder.parents[1]))
with contextlib.redirect_stderr(io.StringIO()) as stderr:
chunk_start.fetched_plan(tracked)
self.assertEqual(stderr.getvalue(), "", "a plan git tracks hides no release")
config.write_text(config.read_text().replace('"checkout:todo.md"', '"folder:todo.md"'))
(folder / "todo.md").write_text(TODO)
unread = Failure("no git")
standing = a_loop(FakeEffects(git=["origin\n", "origin/main\n", unread, unread, unread, unread]).effects(), root=root, paths=paths.paths_from(folder.parents[1]))
with contextlib.redirect_stderr(io.StringIO()) as stderr:
held = chunk_start.fetched_plan(standing)
self.assertEqual((held.plan, held.cut.todos_since, held.cut.changelog), (plan_of(TODO), [], ""), "the plan as it stands, whatever git answers")
self.assertEqual(stderr.getvalue(), f"{STAMP} {UNSEEN_RELEASES}\n", "a release review loop-claude cannot call due is never skipped silently")
quiet = a_loop(FakeEffects(git=["origin\n", "origin/main\n", unread, unread, unread, unread]).effects(), root=root, paths=paths.paths_from(folder.parents[1]))
with contextlib.redirect_stderr(io.StringIO()) as stderr:
chunk_start.fetched_plan(quiet, announce=False)
self.assertEqual(stderr.getvalue(), "", "said once per chunk, as it starts")
config.unlink()
with contextlib.redirect_stderr(io.StringIO()) as stderr:
self.assertIn("no config names the checkout", chunk_start.scored_plan(chunk_start.fetched_plan(loop), loop.effects).reason)
self.assertIn("the items the worker names lift no rung: no config names the checkout", stderr.getvalue())
self.assertIn("no config names the checkout", chunk_start.fetched_plan(loop).reason)
class ChunkFacts(unittest.TestCase):
# The launcher beside which chunk_start finds /b/chunk-preflight.
LAUNCHER = pathlib.Path("/b/loop-claude")
def test_runs_the_preflight_at_the_checkout_root_and_quotes_what_it_printed(self):
fake = FakeEffects(python=[ran("- A fact.\n")])
said = chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER))
self.assertEqual(fake.made("python"), [(["/b/chunk-preflight", "--worktree", "--item", "7"], "/r")])
self.assertTrue(said.endswith("\n\n- A fact."), said)
self.assertNotIn("yourself", said)
def test_a_chunk_without_an_item_runs_the_preflight_without_naming_one(self):
fake = FakeEffects(python=[ran("- A fact.\n")])
chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER, window=tmux.Window("loop-proj", "no-item", "%4")))
self.assertEqual(fake.made("python"), [(["/b/chunk-preflight", "--worktree"], "/r")])
def test_the_facts_end_with_the_items_the_projects_other_running_chunks_hold(self):
# The loop's own window, a stopped one and two that run, one of them waiting for the human.
fake = FakeEffects(python=[ran("- A fact.\n")], windows="item-7\t0\t@1\nitem-5\t1\t@2\nitem-12 ?\t0\t@3\nitem-9\t0\t@4\n")
said = chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER))
self.assertTrue(said.endswith("- A fact.\n\nOther chunks of this project, running or stopped, hold items 5, 9, 12. Leave those items out of this chunk, even where the facts above list them as items that may join it."), said)
self.assertEqual(fresh_prompt.held_by_others([9]), "\n\nOther chunks of this project, running or stopped, hold item 9. Leave those items out of this chunk, even where the facts above list them as items that may join it.")
self.assertTrue(fresh_prompt.held_by_others([4, 9]).startswith("\n\nOther chunks of this project, running or stopped, hold items 4, 9."))
# A worker whose preflight failed is told them too.
fake = FakeEffects(python=[ran(code=1, stderr="git could not start\n")], windows="item-9\t0\t@4\n")
said = chunk_start.chunk_facts(a_loop(fake.effects(), launcher=self.LAUNCHER))
self.assertTrue(said.endswith("Other chunks of this project, running or stopped, hold item 9. Leave those items out of this chunk, even where the facts above list them as items that may join it."), said)
def test_has_the_worker_find_the_facts_itself_where_the_preflight_failed(self):
for failed, why in ((ran(code=1, stderr="git could not start\n"), "git could not start"), (ran(code=1), "exited 1")):
with self.subTest(why=why):
said = chunk_start.chunk_facts(a_loop(FakeEffects(python=[failed]).effects(), launcher=self.LAUNCHER))
self.assertIn("yourself", said)
self.assertIn(why, said)
def test_runs_the_preflight_only_where_the_session_is_not_open(self):
with tempfile.TemporaryDirectory() as home:
opened = session.session_named("A proj loop-claude", pathlib.Path(home, "projects"))
fresh = FakeEffects(python=[ran("- A fact.\n")])
facts, _ = chunk_start.chunk_start(a_loop(fresh.effects(), session=opened, launcher=self.LAUNCHER))
self.assertEqual((fresh.made("python"), "- A fact." in facts), ([(["/b/chunk-preflight", "--worktree", "--item", "7"], "/r")], True))
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
transcript.parent.mkdir(parents=True)
transcript.write_text("")
resumed = FakeEffects()
facts, _ = chunk_start.chunk_start(a_loop(resumed.effects(), session=opened, launcher=self.LAUNCHER))
self.assertEqual((resumed.made("python"), facts), ([], ""))
if __name__ == "__main__":
unittest.main()
+287
View File
@@ -0,0 +1,287 @@
import contextlib
import dataclasses
import io
import json
import pathlib
import subprocess
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from loop_effects.doubles import assistant, FakeEffects
from loop_claude.loop import chunk_windows
from project_config.doubles import configured
from failure import Failure
from git_cli import git_repo
from test_env import isolated_env
from loop_claude.loop import main
from release_plan import next_chunk
from loop_state import paths
from agent_cycle import tmux
from touch_predictions.doubles import todo, plan as touch_plan
def plan_text(*touched):
"""A todo.md of items 1, 2 and on, each predicted to touch the path given for it."""
return todo(*((number, "1.0.0", "", f"- `{path}`") for number, path in enumerate(touched, 1)))
class ChunkWindows(unittest.TestCase):
def plan(self, *touched):
"""A plan of items 1, 2 and on, each predicted to touch the path given for it."""
items = touch_plan(*((number, "1.0.0", "", f"- `{path}`") for number, path in enumerate(touched, 1)))
return next_chunk.NextChunk(items=items, first=items[0], text="", branch_template="")
def test_starts_the_next_item_and_those_picked_beside_it_up_to_the_limit(self):
# Item 2 overlaps item 1, so it waits for it; items 3 and 4 run beside it.
plan = self.plan("a.py", "a.py", "c.py", "d.py")
self.assertEqual(chunk_windows.may_start(plan, [], set(), 1), ["item-1"])
self.assertEqual(chunk_windows.may_start(plan, [], set(), 2), ["item-1", "item-3"])
self.assertEqual(chunk_windows.may_start(plan, [], set(), 9), ["item-1", "item-3", "item-4"])
def test_leaves_the_items_running_chunks_hold_and_starts_only_as_many_as_the_room_left(self):
plan = self.plan("a.py", "a.py", "c.py", "d.py")
self.assertEqual(chunk_windows.may_start(plan, ["item-1"], {1, 3}, 1), ["item-4"])
self.assertEqual(chunk_windows.may_start(plan, ["item-1"], {1}, 2), ["item-3", "item-4"])
self.assertEqual(chunk_windows.may_start(plan, ["item-1"], {1}, 0), [])
def test_counts_the_running_chunks_against_the_limit_and_restarts_stopped_ones_first(self):
windows = [tmux.Listed("item-1", "", False), tmux.Listed("item-3", "", True), tmux.Listed("item-4", "", True)]
self.assertEqual(chunk_windows.room_of(windows, "item-1", False, 3), chunk_windows.Room(["item-1"], ["item-3", "item-4"], ["item-3", "item-4"], 0))
self.assertEqual(chunk_windows.room_of(windows, "item-1", False, 2), chunk_windows.Room(["item-1"], ["item-3", "item-4"], ["item-3"], 0))
self.assertEqual(chunk_windows.room_of(windows, "item-1", False, 5), chunk_windows.Room(["item-1"], ["item-3", "item-4"], ["item-3", "item-4"], 2))
def test_a_loop_whose_chunk_finished_leaves_its_room_and_restarts_no_stopped_chunk(self):
windows = [tmux.Listed("item-1", "", False), tmux.Listed("item-3", "", True)]
self.assertEqual(chunk_windows.room_of(windows, "item-1", True, 2), chunk_windows.Room([], ["item-3"], [], 2))
def test_leaves_no_room_where_more_chunks_run_than_the_limit(self):
windows = [tmux.Listed(f"item-{n}", "", False) for n in (1, 2, 3)]
self.assertEqual(chunk_windows.room_of(windows, "item-1", False, 2).left, 0)
def test_starts_no_item_that_overlaps_a_running_chunks_items(self):
# Item 1 merged, so 2 is the plan's next item; it overlaps item 3, which still runs, at c.py.
items = touch_plan((2, "1.0.0", "", "- `a.py`\n- `c.py`"), (3, "1.0.0", "", "- `c.py`"), (4, "1.0.0", "", "- `d.py`"))
plan = next_chunk.NextChunk(items=items, first=items[0], text="", branch_template="")
self.assertEqual(chunk_windows.may_start(plan, ["item-3"], {3}, 2), ["item-4"])
# A running chunk's item without a prediction leaves no other item free to start beside it.
unpredicted = touch_plan((2, "1.0.0", "", "- `a.py`"), (3, "1.0.0", "", None))
self.assertEqual(chunk_windows.may_start(next_chunk.NextChunk(items=unpredicted, first=unpredicted[0], text="", branch_template=""), ["item-3"], {3}, 2), [])
# A held item that left the plan, as once it merged, overlaps nothing.
self.assertEqual(chunk_windows.may_start(plan, ["item-9"], {9}, 2), ["item-2", "item-4"])
def test_a_plan_without_an_item_starts_one_chunk_and_only_alone(self):
none = next_chunk.NoItem("release 1.0.0 has no items")
self.assertEqual(chunk_windows.may_start(none, [], set(), 3), ["no-item"])
self.assertEqual(chunk_windows.may_start(none, ["item-1"], {1}, 3), [])
def test_names_a_window_for_its_item_and_reads_the_item_back(self):
self.assertEqual(chunk_windows.window_name(372), "item-372")
self.assertEqual([chunk_windows.item_of(chunk) for chunk in ("item-372", "no-item", "item-", "item-3x", "xitem-3")], [372, None, None, None, None])
# Only ASCII digits name an item, as window_name writes them.
self.assertIsNone(chunk_windows.item_of("item-\u0663"))
def test_starts_only_the_next_item_where_the_touch_predictions_cannot_be_read(self):
items = touch_plan((1, "1.0.0", "", "- a.py"), (2, "1.0.0", "", "- `c.py`"))
self.assertEqual(chunk_windows.may_start(next_chunk.NextChunk(items=items, first=items[0], text="", branch_template=""), [], set(), 3), ["item-1"])
def test_a_chunk_whose_transcript_cannot_be_read_holds_the_item_it_started_from(self):
with tempfile.TemporaryDirectory() as temp:
transcripts = pathlib.Path(temp)
folder = transcripts / "-r"
folder.mkdir()
# A folder in place of the transcript file, so reading it fails.
(folder / f"{chunk_windows.chunk_session('proj', 'item-7', transcripts).id}.jsonl").mkdir()
self.assertEqual(chunk_windows.held_items(["item-7"], "proj", transcripts), {7})
def test_tells_a_chunk_the_items_the_projects_other_live_chunks_hold(self):
with tempfile.TemporaryDirectory() as temp:
home = pathlib.Path(temp)
held = paths.paths_from(home)
folder = held.claude_code_transcripts / "-r"
folder.mkdir(parents=True)
lines = [assistant({"type": "text", "text": "Chunk items: 8 (L 1, E + I 2), 11 (L 2, E + I 3)"})]
(folder / f"{chunk_windows.chunk_session('r', 'item-8', held.claude_code_transcripts).id}.jsonl").write_text("".join(json.dumps(line) + "\n" for line in lines))
# Its own window, item-7, holds nothing for it; item-9's stopped chunk holds its item.
fake = FakeEffects(windows="item-7\t0\t@1\nitem-8 ?\t0\t@2\nitem-9\t1\t@3\nitem-10\t0\t@4\n")
self.assertEqual(chunk_windows.held_beside("item-7", "/r", held, fake.effects()), [8, 9, 10, 11])
self.assertEqual(fake.tmux_calls, [["list-windows", "-t", "=loop-r", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"]])
class StartChunks(unittest.TestCase):
"""start_chunks: the windows a loop opens for the chunks that may start now."""
LAUNCHER = pathlib.Path("/b/loop-claude")
def setUp(self):
made = tempfile.TemporaryDirectory()
self.addCleanup(made.cleanup)
self.root = pathlib.Path(made.name, "r")
self.home = configured(self, self.root).parents[1]
self.session = "loop-" + str(self.root).lstrip("/").replace("/", "-").replace(".", "-")
# Items 1 and 2 overlap, so 2 waits; 3 runs beside 1.
self.plan = plan_text("a.py", "a.py", "c.py")
def limit(self, chunks):
config = self.home / ".claude-projects" / self.root.name / "config.toml"
config.write_text(config.read_text() + f"parallel-chunks = {chunks}\n")
def start(self, fake, chunk=None, leaving=False, plan=None, env=None, owner=None):
"""start_chunks' answer, for a session whose @loop-root names owner, by default this checkout."""
fake.owner = str(self.root) if owner is None else owner
run = main.LoopRun(0, str(self.root), "ship it", "", chunk=chunk)
ran = []
effects = dataclasses.replace(fake.effects(), git=self.git_reading(self.plan if plan is None else plan, ran))
with contextlib.redirect_stderr(io.StringIO()) as said:
started = chunk_windows.start_chunks(run, leaving, paths.paths_from(self.home), env or {}, self.LAUNCHER, effects)
self.said = said.getvalue()
fetches = [args for args in ran if args[0] == "fetch"]
self.planned = len(fetches)
self.assertIn(fetches, ([], [["fetch", "--quiet", "origin"]]))
return started
def git_reading(self, plan, ran):
"""A git that notes each call in ran, and reads plan, a todo.md's text, on the checkout's default branch; a Failure answers every call instead, and an error is raised by every call."""
if isinstance(plan, str):
(self.root / "todo.md").write_text(plan)
for command in (["add", "todo.md"], ["-c", "user.name=t", "-c", "user.email=t@example.com", "commit", "--quiet", "--allow-empty", "-m", "plan"]):
subprocess.run(["git", "-C", str(self.root), *command], check=True, env=isolated_env())
def git(*args, **options):
ran.append(list(args))
if isinstance(plan, BaseException):
raise plan
return plan if isinstance(plan, Failure) else git_repo.git(*args, **options, env=isolated_env())
return git
def starts(self, fake):
"""The window starts of fake's tmux calls, each as (command, window name, what it runs)."""
return [(call[0], call[call.index("-n") + 1], call[call.index("--") + 1:]) for call in fake.tmux_calls if call[0] in ("new-session", "new-window")]
def test_opens_a_window_for_each_chunk_that_may_start_the_first_making_the_session(self):
self.limit(2)
fake = FakeEffects()
self.assertEqual(self.start(fake, env={"LOOP_CLAUDE_WORKER_MODEL": "m", "PATH": "/bin"}), ["item-1", "item-3"])
self.assertEqual(self.starts(fake), [("new-session", "item-1", [str(self.LAUNCHER), "--chunk", "item-1", "ship it"]), ("new-window", "item-3", [str(self.LAUNCHER), "--chunk", "item-3", "ship it"])])
first = next(call for call in fake.tmux_calls if call[0] == "new-session")
self.assertEqual((first[first.index("-s") + 1], first[first.index("-c") + 1]), (self.session, str(self.root)))
# Only loop-claude's own variables are handed on, since a window gets the tmux server's environment.
self.assertEqual([word for word in first if "=" in word], ["LOOP_CLAUDE_WORKER_MODEL=m"])
self.assertEqual(fake.tmux_calls[0], ["list-windows", "-t", f"={self.session}", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"])
# The session names the checkout whose chunks it runs.
self.assertIn(["set-option", "-t", f"={self.session}:", "@loop-root", str(self.root)], fake.tmux_calls)
def test_refuses_a_session_that_runs_another_checkouts_chunks(self):
for owner, named in (("/c/acme.api", "/c/acme.api"), ("", "another checkout")):
with self.subTest(owner=owner):
fake = FakeEffects(windows="item-5\t0\t@1\n")
self.assertEqual(self.start(fake, owner=owner), Failure(f"the tmux session {self.session} runs the chunks of {named}, whose key maps to the same session name; rename one of the two checkouts' folders"))
self.assertEqual(self.starts(fake), [])
self.assertIn(["show-options", "-qv", "-t", f"={self.session}:", "@loop-root"], fake.tmux_calls)
def test_refuses_a_checkout_do_epic_runs_chunks_in(self):
fake = FakeEffects(windows=f"epic-ABC-12\tABC-34 ?\t0\t@4\t{self.root}\nepic-ABC-12\tABC-35\t0\t@5\t/c/other\nepic-ABC-12\tABC-36\t1\t@6\t{self.root}\nloop-x\titem-1\t0\t@7\t{self.root}\n")
self.assertEqual(self.start(fake), Failure(f"do-epic runs chunks in {self.root}: ABC-34 of epic-ABC-12; loop-claude doesn't start a chunk there until those chunks finish"))
self.assertEqual(self.starts(fake), [])
def test_leaves_out_the_items_a_running_chunk_names_on_its_chunk_items_line(self):
self.limit(2)
transcripts = paths.paths_from(self.home).claude_code_transcripts
folder = transcripts / "-r"
folder.mkdir(parents=True)
opened = chunk_windows.chunk_session(str(self.root).lstrip("/"), "item-5", transcripts)
(folder / f"{opened.id}.jsonl").write_text(json.dumps(assistant({"type": "text", "text": "Chunk items: 5 (L 1, E + I 2), 1 (L 1, E + I 2)"})) + "\n")
self.assertEqual(self.start(FakeEffects(windows="item-5\t0\t@1\n")), ["item-3"])
def test_starts_one_chunk_without_parallel_chunks(self):
fake = FakeEffects()
self.assertEqual(self.start(fake), ["item-1"])
def test_starts_none_and_reads_no_plan_while_the_running_chunks_fill_the_limit(self):
self.limit(2)
fake = FakeEffects(windows="item-5\t0\t@1\nitem-6 ?\t0\t@2\n")
self.assertEqual((self.start(fake), self.planned, self.starts(fake)), ([], 0, []))
# More chunks run than the limit allows, as after it was lowered; a stopped chunk still waits.
fake = FakeEffects(windows="item-5\t0\t@1\nitem-6\t0\t@2\nitem-7\t0\t@3\nitem-8\t1\t@4\nitem-9\t1\t@5\n")
self.assertEqual((self.start(fake), self.planned), ([], 0))
def test_a_finishing_chunks_own_window_counts_as_no_longer_running(self):
fake = FakeEffects(windows="item-5\t0\t@1\n")
self.assertEqual(self.start(fake, chunk="item-5"), [])
fake = FakeEffects(windows="item-5\t0\t@1\n")
self.assertEqual(self.start(fake, chunk="item-5", leaving=True), ["item-1"])
# The session exists, since the finishing loop's window is still in it.
self.assertEqual(self.starts(fake)[0][0], "new-window")
def test_the_humans_loop_claude_starts_a_stopped_chunk_again_first_closing_its_dead_window(self):
fake = FakeEffects(windows="item-5\t1\t@3\n")
self.assertEqual((self.start(fake), self.planned), (["item-5"], 0))
self.assertIn(["kill-window", "-t", "@3"], fake.tmux_calls)
# Its dead window was the session's last, so the session is made again.
self.assertEqual(self.starts(fake)[0][0], "new-session")
self.limit(2)
fake = FakeEffects(windows="item-5\t1\t@3\n")
self.assertEqual(self.start(fake), ["item-5", "item-1"])
def test_a_plan_with_no_item_starts_nothing_beside_a_restarted_chunk(self):
self.limit(2)
fake = FakeEffects(windows="item-5\t1\t@3\n")
self.assertEqual(self.start(fake, plan=plan_text()), ["item-5"])
def test_a_finishing_chunk_leaves_a_stopped_chunk_and_its_items_to_the_human(self):
self.limit(2)
transcripts = paths.paths_from(self.home).claude_code_transcripts
folder = transcripts / "-r"
folder.mkdir(parents=True)
opened = chunk_windows.chunk_session(str(self.root).lstrip("/"), "item-5", transcripts)
(folder / f"{opened.id}.jsonl").write_text(json.dumps(assistant({"type": "text", "text": "Chunk items: 5 (L 1, E + I 2), 1 (L 1, E + I 2)"})) + "\n")
fake = FakeEffects(windows="item-7\t0\t@1\nitem-5\t1\t@3\n")
# Item 1 is the stopped chunk's, so item 3 starts; the stopped one doesn't count against parallel-chunks.
self.assertEqual(self.start(fake, chunk="item-7", leaving=True), ["item-3"])
self.assertEqual([call for call in fake.tmux_calls if call[0] == "kill-window"], [])
fake = FakeEffects(windows="item-7\t0\t@1\nno-item\t1\t@3\n")
self.assertEqual(self.start(fake, chunk="item-7", leaving=True, plan=plan_text()), [])
def test_starts_the_chunk_without_a_plan_item_where_the_plan_cannot_be_read(self):
fake = FakeEffects()
self.assertEqual(self.start(fake, plan=Failure("no origin/main")), ["no-item"])
self.assertIn("starting a chunk without a plan item, since the plan cannot be read: no origin/main", self.said)
def test_says_why_a_window_could_not_start(self):
fake = FakeEffects()
fake.tmux = lambda args: subprocess.CompletedProcess(args, 1, "", "no room") if args[0] == "new-session" else FakeEffects.tmux(fake, args)
self.assertEqual(self.start(fake), Failure(f"tmux could not start the window {self.session}:item-1: no room"))
def test_says_why_it_cannot_take_the_lock_on_starting_windows(self):
(self.home / ".local" / "state").mkdir(parents=True)
(self.home / ".local" / "state" / "loop-claude").write_text("")
started = self.start(FakeEffects())
self.assertIsInstance(started, Failure)
self.assertTrue(started.reason.startswith(f"cannot lock {self.home}/.local/state/loop-claude/"), started.reason)
def test_an_error_raised_under_the_lock_is_not_reported_as_the_locks(self):
with self.assertRaisesRegex(OSError, "the plan's disk is gone"):
self.start(FakeEffects(), plan=OSError("the plan's disk is gone"))
def test_a_chunk_reads_the_other_windows_where_it_cannot_take_the_lock(self):
(self.home / ".local" / "state").mkdir(parents=True)
(self.home / ".local" / "state" / "loop-claude").write_text("")
fake = FakeEffects(windows="item-7\t0\t@1\nitem-9\t1\t@3\n")
self.assertEqual(chunk_windows.held_beside("item-7", str(self.root), paths.paths_from(self.home), fake.effects()), [9])
def test_a_chunk_holds_its_own_item_and_those_on_its_latest_chunk_items_line(self):
with tempfile.TemporaryDirectory() as temp:
transcripts = pathlib.Path(temp)
opened = chunk_windows.chunk_session("proj", "item-7", transcripts)
self.assertEqual(opened.name, "A proj loop-claude item-7")
folder = transcripts / "-r"
folder.mkdir()
lines = [assistant({"type": "text", "text": "Chunk items: 7 (L 1, E + I 2)"}), assistant({"type": "text", "text": "Chunk items: 7 (L 1, E + I 2), 9 (L 2, E + I 3)"})]
(folder / f"{opened.id}.jsonl").write_text("".join(json.dumps(line) + "\n" for line in lines))
self.assertEqual(chunk_windows.held_items(["item-7", "item-8", "no-item"], "proj", transcripts), {7, 8, 9})
if __name__ == "__main__":
unittest.main()
+332
View File
@@ -0,0 +1,332 @@
import contextlib
import dataclasses
import io
import json
import pathlib
import sys
import tempfile
import unittest
import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from chunk_driver.doubles import a_loop
from loop_effects.doubles import CWD, FakeEffects, ran, STAMP
from failure import Failure
from agent_cycle.verdicts import Finished, Unfinished
from loop_claude.loop import main, reload, usage
from loop_claude.doubles import OPUS_PROJECT
from loop_state import paths
from model_routing.route import Route
from model_routing.ladder import Rung
from claude_code import session
from agent_cycle.stop import Stop
from chunk_driver import this_loop
from agent_cycle import tmux
class LoopRuns(unittest.TestCase):
def test_stops_naming_the_reload_when_it_hands_over_anything_but_the_state_with_a_task(self):
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "root": "/r", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "", "unfinished_evidence": ""})], [json.dumps({"cycle_number": "3", "root": "/r", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "", "task": "do it", "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "do it", "turns": 1, "unfinished_evidence": ""})], [json.dumps({"cycle_number": 3, "failures": 0, "root": "/r", "task": "do it", "unfinished_evidence": ""})]):
with self.subTest(handed=handed):
self.assertEqual(main.loop_run_from([main.RELOAD_FLAG, *handed], CWD, FakeEffects().effects()), Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(handed)}"))
def test_decodes_every_field_of_the_state_and_its_route(self):
state = main.LoopRun(cycle_number=2, root="/r", task="do it", unfinished_evidence="old", route=Route(Rung("claude-fable-5-2", "low"), 3, 4, 5, 1, 7, 9), unfinished_run=2)
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(state))), state)
self.assertEqual(set(reload.LOOP_RUN_FIELDS), {field.name for field in dataclasses.fields(main.LoopRun)})
def test_rejects_a_field_of_another_type_or_an_unknown_field(self):
state = dataclasses.asdict(main.LoopRun(cycle_number=2, root="/r", task="do it", unfinished_evidence="", route=Route(Rung("claude-fable-5-2", "low"), 3, 4, 5, 1, 7, 9), unfinished_run=2))
for path, wrong in ((("unfinished_run",), "2"), (("unfinished_run",), True), (("route", "unfinished"), 1.0), (("route", "escalated_at"), "9"), (("route", "item"), True), (("route", "rung"), "claude-opus-5-5"), (("route", "rung", "model"), None), (("route",), "r"), (("route", "extra"), 1), (("route", "model"), "claude-opus-5-5"), (("route", "effort"), "high")):
with self.subTest(path=path, wrong=wrong):
handed = json.loads(json.dumps(state))
target = handed
for key in path[:-1]:
target = target[key]
target[path[-1]] = wrong
self.assertIsNone(reload.decode_handed(json.dumps(handed)))
def test_rejects_a_state_or_rung_missing_a_required_field(self):
state = dataclasses.asdict(main.LoopRun(cycle_number=2, root="/r", task="do it", unfinished_evidence="", route=Route(Rung("claude-fable-5-2", "low"))))
for path in (("cycle_number",), ("root",), ("task",), ("unfinished_evidence",), ("route", "rung"), ("route", "rung", "model")):
with self.subTest(path=path):
handed = json.loads(json.dumps(state))
target = handed
for key in path[:-1]:
target = target[key]
del target[path[-1]]
self.assertIsNone(reload.decode_handed(json.dumps(handed)))
def test_decodes_a_rung_without_an_effort(self):
state = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=Route(Rung("some-model")))
self.assertEqual(reload.decode_handed(json.dumps({**dataclasses.asdict(state), "route": {"rung": {"model": "some-model"}}})), state)
def test_rejects_text_that_is_not_json(self):
self.assertIsNone(reload.decode_handed("not json"))
def test_decodes_a_route_inside_the_state_or_none(self):
state = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=Route(Rung("claude-fable-5-2", "low"), 3, 4, 5, 1))
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(state))), state)
unrouted = dataclasses.replace(state, route=None)
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(unrouted))), unrouted)
handed = dataclasses.asdict(state)
handed["route"]["rung"]["effort"] = 1
self.assertIsNone(reload.decode_handed(json.dumps(handed)))
del handed["route"]["rung"]
self.assertIsNone(reload.decode_handed(json.dumps(handed)))
off_scale = dataclasses.asdict(state)
off_scale["route"]["likelihood"] = 9
self.assertIsNone(reload.decode_handed(json.dumps(off_scale)))
older = dataclasses.asdict(state)
del older["route"]["escape_and_impact_taken_in"]
self.assertEqual(reload.decode_handed(json.dumps(older)), state)
taken = dataclasses.replace(state, route=dataclasses.replace(state.route, escape_and_impact_taken_in=7))
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(taken))), taken)
for taken_in in (9, 5):
with self.subTest(taken_in=taken_in):
handed = dataclasses.asdict(state)
handed["route"]["escape_and_impact_taken_in"] = taken_in
self.assertIsNone(reload.decode_handed(json.dumps(handed)))
def test_decodes_a_state_from_before_a_field_with_a_default_joined_it(self):
self.assertEqual(reload.decode_handed(json.dumps({"cycle_number": 3, "root": "/r", "task": "do it", "unfinished_evidence": ""})), main.LoopRun(3, "/r", "do it", ""))
def test_drops_the_items_taken_in_that_a_loop_started_before_0_11_0_hands_on(self):
state = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="")
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(state) | {"items_taken_in": [12, 14]})), state)
def test_drops_the_slot_that_a_loop_started_before_chunks_ran_in_windows_hands_on(self):
state = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="")
self.assertEqual(reload.decode_handed(json.dumps({key: value for key, value in dataclasses.asdict(state).items() if key != "chunk"} | {"slot": 1})), state)
def test_stops_when_the_reload_cannot_start_python(self):
with tempfile.TemporaryDirectory() as root:
launcher = pathlib.Path(root, "loop-claude")
launcher.write_text("")
fake = FakeEffects(python=[ran()])
self.assertEqual(reload.exec_if_edited(a_loop(fake.effects(), launcher=launcher), "stale", main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="")), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
def steps(start_chunks, **others):
"""The steps of a loop that starts chunks through start_chunks, and takes each other step through the double given for it, else fails."""
def unused(*args):
raise AssertionError(args)
return main.Steps(**{"fingerprint": unused, "exec_if_edited": unused, "next_cycle": unused, "run_cycle": unused, "closed_cycle": unused, "check_routing": unused, **others}, start_chunks=start_chunks)
class WindowStarts(unittest.TestCase):
"""The loop-claude the human runs, which starts the windows, and the stop of a window's loop whose chunk finished."""
HOME = pathlib.Path("/h")
def run_loop(self, argv, fake, env=None, started=("item-3", "item-4")):
"""run_loop's stop and what it said on stdout, with start_chunks answering `started`; self.asked holds what start_chunks was asked."""
self.asked = []
def start(*args):
self.asked.append(args[:2])
return list(started) if isinstance(started, tuple) else started
with contextlib.redirect_stdout(io.StringIO()) as said:
stop = main.run_loop(argv, env or {}, paths.paths_from(self.HOME), pathlib.Path("/b/loop-claude"), CWD, fake.effects(), steps(start))
return stop, said.getvalue()
def test_refuses_to_start_anything_on_a_tmux_older_than_needed(self):
fake = FakeEffects(git=["/r\n"], tmux_version="tmux 3.1c")
self.assertEqual(self.run_loop(["do it"], fake), (Stop(1, "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 3.1c"), ""))
self.assertEqual((fake.tmux_calls, self.asked), ([["-V"]], []))
def test_prints_the_help_without_tmux(self):
fake = FakeEffects(tmux_version="")
self.assertEqual(self.run_loop(["--help"], fake)[0], Stop(0, usage.HELP))
self.assertEqual(fake.tmux_calls, [])
def test_starts_the_chunks_then_attaches_to_the_session_or_inside_tmux_switches_to_it(self):
for env, shown in (({}, ["attach-session", "-t", "=loop-r"]), ({"TMUX": "/tmp/tmux-1000/default,1,0"}, ["switch-client", "-t", "=loop-r"])):
with self.subTest(shown=shown):
fake = FakeEffects(git=["/r\n"])
stop, said = self.run_loop(["do", "it"], fake, env)
self.assertEqual((stop, said), (Stop(0, ""), f"{STAMP} started item-3, item-4 in the tmux session loop-r\n"))
self.assertEqual(fake.tmux_calls, [["-V"], shown])
self.assertEqual(self.asked, [(main.LoopRun(0, "/r", "do it", ""), False)])
def test_hands_the_window_starts_its_paths_environment_launcher_and_effects(self):
fake = FakeEffects(git=["/r\n"], on_terminal=False)
effects, env, launcher = fake.effects(), {"LOOP_CLAUDE_X": "1"}, pathlib.Path("/b/loop-claude")
asked = []
with contextlib.redirect_stdout(io.StringIO()):
main.run_loop(["do it"], env, paths.paths_from(self.HOME), launcher, CWD, effects, steps(lambda *args: asked.append(args) or ["item-3"]))
self.assertEqual(asked, [(main.LoopRun(0, "/r", "do it", ""), False, paths.paths_from(self.HOME), env, launcher, effects)])
def test_a_window_loop_runs_its_chunk_in_its_own_session_and_window_and_hands_on_once_it_finished(self):
fake = FakeEffects(git=["/r\n"])
effects, env = fake.effects(), {"TMUX_PANE": "%4"}
launcher = pathlib.Path("/b/loop-claude")
cycle = unittest.mock.Mock(project=OPUS_PROJECT)
unfinished = unittest.mock.Mock(verdict=unittest.mock.Mock(spec=Unfinished))
finished = unittest.mock.Mock(verdict=unittest.mock.Mock(spec=Finished))
closed = [main.LoopRun(1, "/r", "do it", "", chunk="item-3"), main.LoopRun(2, "/r", "do it", "", chunk="item-3")]
concluded, cycles, started = [unfinished, finished], [], []
taken = steps(lambda *args: started.append(args) or [], fingerprint=lambda launcher: "", exec_if_edited=lambda *args: None, next_cycle=lambda *args: cycles.append(args) or cycle,
run_cycle=lambda cycle: concluded.pop(0), closed_cycle=lambda *args: closed.pop(0), check_routing=lambda *args: None)
with contextlib.redirect_stdout(io.StringIO()):
stop = main.run_loop(["--chunk", "item-3", "do it"], env, paths.paths_from(self.HOME), launcher, CWD, effects, taken)
loop = this_loop.Loop(root="/r", session=session.session_named("A r loop-claude item-3", self.HOME / ".claude" / "projects"), window=tmux.Window("loop-r", "item-3", "%4"), paths=paths.paths_from(self.HOME), env=env, launcher=launcher, effects=effects)
self.assertEqual(stop, Stop(0, ""))
self.assertEqual(cycles, [(main.LoopRun(1, "/r", "do it", "", chunk="item-3"), loop), (main.LoopRun(2, "/r", "do it", "", chunk="item-3"), loop)])
self.assertEqual(started, [(main.LoopRun(2, "/r", "do it", "", chunk="item-3"), True, loop.paths, env, launcher, effects)])
def window_loop(self, cycles, env, **doubles):
"""run_loop's stop for item-3's window loop, and each step it took with what it handed that step. Each step answers as its double in `doubles` does, else as a cycle without a stop: next_cycle answers the next of `cycles`, run_cycle an unfinished verdict, and closed_cycle the run with one more unfinished verdict."""
fake, calls, cycles = FakeEffects(git=["/r\n"]), [], list(cycles)
effects, launcher, home = fake.effects(), pathlib.Path("/b/loop-claude"), paths.paths_from(self.HOME)
self.unfinished = unittest.mock.Mock(verdict=unittest.mock.Mock(spec=Unfinished))
answers = {"fingerprint": lambda *args: "print-1", "exec_if_edited": lambda *args: None, "next_cycle": lambda *args: cycles.pop(0), "run_cycle": lambda *args: self.unfinished,
"closed_cycle": lambda run, *args: dataclasses.replace(run, unfinished_run=run.unfinished_run + 1), "check_routing": lambda *args: None, **doubles}
def step(name):
return lambda *args: calls.append((name, *args)) or answers[name](*args)
stop = main.run_loop(["--chunk", "item-3", "do it"], env, home, launcher, CWD, effects, steps(lambda *args: self.fail(args), **{name: step(name) for name in answers}))
self.loop = this_loop.Loop(root="/r", session=session.session_named("A r loop-claude item-3", self.HOME / ".claude" / "projects"), window=tmux.Window("loop-r", "item-3", "%4"), paths=home, env=env, launcher=launcher, effects=effects)
return stop, calls
def test_a_window_loop_hands_each_step_of_a_cycle_what_the_steps_before_it_made(self):
env = {"TMUX_PANE": "%4", "LOOP_CLAUDE_TRANSCRIPTS": "/t"}
on_ladder = unittest.mock.Mock(task="the cycle's task", project=OPUS_PROJECT, project_archive=pathlib.Path("/t/r"))
off_ladder = unittest.mock.Mock(task="the cycle's task", project=None, project_archive=pathlib.Path("/t/r"), route=unittest.mock.Mock(rung=Rung("my-model")))
stop, calls = self.window_loop([on_ladder, off_ladder, Stop(2, "the human is needed")], env)
self.assertEqual(stop, Stop(2, "the human is needed"))
before, first, second, third = (main.LoopRun(number, "/r", "do it", "", unfinished_run=max(number - 1, 0), chunk="item-3") for number in range(4))
self.assertEqual(calls, [
("fingerprint", pathlib.Path("/b/loop-claude")),
("exec_if_edited", self.loop, "print-1", before),
("next_cycle", first, self.loop),
("run_cycle", on_ladder),
("closed_cycle", first, "the cycle's task", OPUS_PROJECT, self.unfinished, pathlib.Path("/t/r"), "loop-claude", self.loop),
("check_routing", pathlib.Path("/t"), Rung("claude-opus-5-5", "medium"), self.loop),
("exec_if_edited", self.loop, "print-1", dataclasses.replace(first, unfinished_run=1)),
("next_cycle", second, self.loop),
("run_cycle", off_ladder),
("closed_cycle", second, "the cycle's task", None, self.unfinished, pathlib.Path("/t/r"), "loop-claude", self.loop),
("check_routing", pathlib.Path("/t"), Rung("my-model"), self.loop),
("exec_if_edited", self.loop, "print-1", dataclasses.replace(second, unfinished_run=2)),
("next_cycle", third, self.loop),
])
def test_a_window_loop_stops_with_the_stop_of_the_step_that_made_one(self):
cycle = unittest.mock.Mock(project=OPUS_PROJECT, project_archive=pathlib.Path("/t/r"))
for name, taken in (("exec_if_edited", 2), ("run_cycle", 4), ("closed_cycle", 5), ("check_routing", 6)):
with self.subTest(step=name):
stop, calls = self.window_loop([cycle], {"TMUX_PANE": "%4"}, **{name: lambda *args: Stop(1, f"{name} stopped")})
self.assertEqual((stop, len(calls), calls[-1][0]), (Stop(1, f"{name} stopped"), taken, name))
def test_a_window_loop_given_no_task_prints_the_help(self):
self.assertEqual(main.loop_run_from(["--chunk", "item-3"], CWD, FakeEffects(git=["/r\n"]).effects()), Stop(1, usage.HELP))
def test_a_window_loop_outside_a_checkout_stops(self):
self.assertIsInstance(main.loop_run_from(["--chunk", "item-3", "do it"], CWD, FakeEffects(git=[Failure("git cannot start")]).effects()), Stop)
def test_says_that_nothing_started_where_the_session_already_runs_every_chunk_that_may(self):
self.assertEqual(self.run_loop(["do it"], FakeEffects(git=["/r\n"]), started=())[1], f"{STAMP} the tmux session loop-r runs every chunk that may run now, so `do it` started in no new window\n")
def test_off_a_terminal_it_prints_how_to_attach(self):
fake = FakeEffects(git=["/r\n"], on_terminal=False)
stop, said = self.run_loop(["do it"], fake)
self.assertEqual(stop, Stop(0, ""))
self.assertTrue(said.endswith(f"{STAMP} not on a terminal; see the chunks with `tmux attach -t =loop-r`\n"), said)
self.assertEqual(fake.tmux_calls, [["-V"]])
def test_stops_where_no_chunk_could_start(self):
self.assertEqual(self.run_loop(["do it"], FakeEffects(git=["/r\n"]), started=Failure("no plan"))[0], Stop(1, "no plan"))
def test_stops_where_tmux_cannot_show_the_session(self):
fake = FakeEffects(git=["/r\n"])
effects = dataclasses.replace(fake.effects(), tmux_on_terminal=lambda args: 1)
with contextlib.redirect_stdout(io.StringIO()):
stop = main.run_loop(["do it"], {}, paths.paths_from(self.HOME), pathlib.Path("/b/loop-claude"), CWD, effects, steps(lambda *args: ["item-3"]))
self.assertEqual(stop, Stop(1, "`tmux attach-session -t =loop-r` failed; the chunks go on in the tmux session"))
def test_a_window_loop_outside_tmux_stops(self):
fake = FakeEffects(git=["/r\n"])
self.assertEqual(self.run_loop(["--chunk", "item-3", "do it"], fake)[0], Stop(1, "--chunk is how loop-claude starts itself in a tmux window; run loop-claude <task>"))
self.assertEqual(self.asked, [])
def test_reads_the_chunk_a_window_loop_is_started_with(self):
self.assertEqual(main.loop_run_from(["--chunk", "item-3", "do", "it"], CWD, FakeEffects(git=["/r\n"]).effects()), main.LoopRun(0, "/r", "do it", "", chunk="item-3"))
self.assertEqual(main.loop_run_from(["do", "it"], CWD, FakeEffects(git=["/r\n"]).effects()), main.LoopRun(0, "/r", "do it", ""))
def test_a_finished_chunk_starts_each_chunk_that_may_start_and_stops_without_a_reason(self):
loop_run = main.LoopRun(4, "/r", "do it", "", chunk="item-7")
for started, said, stop in ((["item-9", "item-12"], "the chunk finished; started item-9, item-12", Stop(0, "")), ([], "the chunk finished; started no other, since the chunks running fill what may run now", Stop(0, "")),
(Failure("no plan"), "", Stop(1, "the chunk finished, but the next could not start: no plan"))):
with self.subTest(started=started), contextlib.redirect_stdout(io.StringIO()) as printed:
loop, asked = a_loop(FakeEffects().effects(), env={"LOOP_CLAUDE_X": "1"}), []
self.assertEqual(main.handed_on(loop_run, loop, lambda *args: asked.append(args) or started), stop)
self.assertEqual(printed.getvalue(), f"{STAMP} {said}\n" if said else "")
self.assertEqual(asked, [(loop_run, True, loop.paths, loop.env, loop.launcher, loop.effects)])
class StopDms(unittest.TestCase):
def run_main(self, argv, fake):
with tempfile.TemporaryDirectory() as home, contextlib.redirect_stderr(io.StringIO()) as said, contextlib.redirect_stdout(io.StringIO()):
code = main.main(argv, {}, paths.paths_from(pathlib.Path(home)), pathlib.Path(home, "loop-claude"), CWD, fake.effects())
return code, said.getvalue()
def exit_code(self, run, fake):
with contextlib.redirect_stderr(io.StringIO()) as said, contextlib.redirect_stdout(io.StringIO()):
code = main.exit_code(run, CWD, fake.effects())
return code, said.getvalue()
def raising(self, error):
raise error
def test_dms_the_reason_of_a_stop_with_exit_2_such_as_a_chunk_left_unfinished_too_often(self):
fake = FakeEffects()
self.assertEqual(self.exit_code(lambda: Stop(2, "the human is needed to look at the chunk, but it stayed unfinished"), fake)[0], 2)
self.assertEqual(fake.dms, [f"loop-claude in {CWD} stopped: the human is needed to look at the chunk, but it stayed unfinished"])
def test_skips_the_dm_of_a_stop_without_a_reason_as_once_the_windows_started(self):
fake = FakeEffects()
self.assertEqual(self.exit_code(lambda: Stop(0, ""), fake), (0, ""))
self.assertEqual(fake.dms, [])
def test_skips_the_dm_where_ctrl_c_stopped_the_loop(self):
fake = FakeEffects()
self.assertEqual(self.exit_code(lambda: self.raising(KeyboardInterrupt()), fake)[0], 130)
self.assertEqual(fake.dms, [])
def test_dms_a_crash_before_it_reaches_the_terminal(self):
fake = FakeEffects()
with self.assertRaises(RuntimeError):
self.exit_code(lambda: self.raising(RuntimeError("boom")), fake)
self.assertEqual(fake.dms, [f"loop-claude in {CWD} crashed: RuntimeError('boom')"])
def test_dms_the_reason_where_a_loop_given_a_task_stops_with_exit_1(self):
fake = FakeEffects()
code, said = self.run_main([main.RELOAD_FLAG, "3"], fake)
self.assertEqual(code, 1)
self.assertEqual(fake.dms, [f"loop-claude in {CWD} stopped: the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: 3"])
def test_says_why_the_dm_of_a_stop_failed_after_the_reason(self):
fake = FakeEffects(dm_failures=[Failure("cannot reach https://chat.example: timed out")])
self.assertTrue(self.run_main([main.RELOAD_FLAG, "3"], fake)[1].endswith(f"Handed: 3\n{STAMP} could not DM you: cannot reach https://chat.example: timed out\n"))
def test_skips_the_dm_where_it_prints_its_help(self):
for argv, code in (([], 1), (["--help"], 0)):
with self.subTest(argv=argv):
fake = FakeEffects()
self.assertEqual(self.run_main(argv, fake)[0], code)
self.assertEqual(fake.dms, [])
class OwnLadder(unittest.TestCase):
def test_a_cycle_on_a_projects_own_ladder_names_its_file_and_says_where_the_workers_model_is_off_it(self):
file = pathlib.Path("/h/.claude-projects/acme/routing-ladder.json")
self.assertEqual(main.own_ladder(file, "claude-fable-5-2", False), f"routing on the project's own ladder, {file}.")
self.assertEqual(main.own_ladder(file, "claude-opus-5-5", True), f"routing on the project's own ladder, {file}. claude-opus-5-5 is not on it, so the worker runs on claude-opus-5-5 unrouted; LOOP_CLAUDE_WORKER_MODEL and LOOP_CLAUDE_JUDGE_MODEL name other models for a run.")
if __name__ == "__main__":
unittest.main()
+155
View File
@@ -0,0 +1,155 @@
import contextlib
import io
import json
import pathlib
import shutil
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from chunk_driver.doubles import a_loop
from routing_figures import archive, state
from test_env import assert_not_root
from routing_figures.calls import Call
from failure import Failure
from loop_effects.doubles import FakeEffects, NOW
from transcript_archive import outcomes
from loop_state import paths
from loop_claude.loop import routing_check
from model_routing.ladder import Rung
from claude_code import session
from agent_cycle.stop import Stop
from text_files.utc_stamp import utc
# How git answers in a project's checkout it cannot read, such as one that is gone.
UNREAD = Failure("not a git repository")
class RoutingCheck(unittest.TestCase):
def setUp(self):
self.home = pathlib.Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.home)
self.root = self.home / "archive"
self.paths = paths.paths_from(self.home)
self.session = session.session_named("A log loop-claude", self.paths.claude_code_transcripts)
def loop(self, fake):
"""The loop in /code/log."""
return a_loop(fake.effects(), root="/code/log", session=self.session, paths=self.paths)
def chunks(self, count, verdicts, project="log"):
"""count finished chunks, each closed by verdicts verdicts, each with one Explore subagent whose report outgrew its searches."""
for number in range(count):
folder = self.root / project / f"c{number}"
subagents = folder / "session" / "s" / "subagents"
subagents.mkdir(parents=True)
kept = "".join(f"2026-10-14T10:0{verdict}:00Z {outcomes.unfinished('w', 'e', 'j', '')}\n" for verdict in range(verdicts - 1))
(folder / "copy.outcomes.md").write_text(f"Task: t\n\n{kept}2026-10-14T11:00:00Z {outcomes.finished('w', 'm', 'j')}\n")
launch = {"type": "assistant", "timestamp": "2026-10-14T09:00:00Z", "message": {"id": "m", "content": [{"type": "tool_use", "id": "a", "name": "Agent", "input": {"subagent_type": "Explore"}}]}}
report = {"type": "user", "message": {"content": [{"type": "tool_result", "tool_use_id": "a", "content": "long report"}]}}
(folder / "copy.jsonl").write_text(json.dumps(launch) + "\n" + json.dumps(report) + "\n")
(subagents / "agent-a.jsonl").write_text("")
(subagents / "agent-a.meta.json").write_text(json.dumps({"agentType": "Explore", "toolUseId": "a"}))
def check(self, fake):
"""What check_routing returns and says for a loop in /code/log."""
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()) as said:
stop = routing_check.check_routing(self.root, Rung("m", "high"), self.loop(fake))
return stop, said.getvalue()
def test_measures_once_20_cycles_have_closed_since_the_last_measurement(self):
self.chunks(5, 3)
self.assertEqual(self.check(FakeEffects()), (None, ""))
self.assertFalse((self.root / "routing-figures.md").exists())
self.chunks(6, 1, "other")
self.assertEqual(self.check(FakeEffects(on_terminal=False, git=[UNREAD] * 6))[0], None)
self.assertIn("at cycle 21 across", (self.root / "routing-figures.md").read_text())
def test_reads_each_projects_history_through_the_loops_git_and_names_each_it_cannot_read(self):
self.chunks(5, 4)
fake = FakeEffects(on_terminal=False, git=[UNREAD] * 3)
self.check(fake)
located = ["rev-parse", "--path-format=absolute", "--show-toplevel", "--git-common-dir"]
self.assertEqual(fake.made("git"), [located, located, ["remote"]], "the config is located for its title templates, then for where it keeps the plan")
self.assertIn("## Default branches unread\n\n- log: not a git repository\n", (self.root / "routing-figures.md").read_text())
def test_hands_the_terminal_over_once_for_each_new_call(self):
self.chunks(5, 4)
fake = FakeEffects(on_terminal_claude=[0], git=[UNREAD] * 3)
stop, said = self.check(fake)
self.assertIsNone(stop)
call = "Explore subagents' reports took more of the worker's context than their searches would have."
self.assertIn(f"- {call}", said)
self.assertIn(f"- {call}", (self.root / "routing-figures.md").read_text())
((args),) = fake.made("claude_on_terminal")
self.assertEqual(args[:6], ["--model", "m", "--effort", "high", "--permission-mode", "bypassPermissions"])
self.assertIn(f"don't change anything until they answer:\n- {call}", args[6])
self.assertEqual(fake.terminal_environments, [{"CLAUDE_CODE_SANDBOXED": "1"}])
self.assertEqual(fake.terminal_folders, [str(self.home / "code" / "claude-config")], "the development clone, since ~/.claude holds only a release")
self.assertIn("Record what they decide in this checkout, the development clone", args[6])
self.assertIsNone(state.mark_measured(self.root, 0))
self.assertEqual(self.check(FakeEffects(git=[UNREAD] * 3))[0], None)
self.assertNotIn(call, self.check(FakeEffects())[1])
def test_stamps_the_figures_and_the_calls_raised_when_it_measures(self):
self.chunks(5, 4)
(self.root / "routing-calls-raised.md").write_text("# Routing calls raised\n\n- 2026-10-14T10:02:00Z Reset\n")
self.check(FakeEffects(on_terminal_claude=[0], git=[UNREAD] * 3))
figures = (self.root / "routing-figures.md").read_text()
self.assertIn(f"Measured {utc(NOW)}, at cycle 20 across", figures)
self.assertIn("that finished after the reset of 2026-10-14T10:02:00Z", figures)
self.assertIn(f"- {utc(NOW)} Explore subagents'", (self.root / "routing-calls-raised.md").read_text())
def test_says_a_figure_it_cannot_read(self):
self.chunks(5, 4)
(self.root / "routing-state.json").mkdir()
self.assertIn("routing figures: could not read", self.check(FakeEffects())[1])
def test_goes_on_off_a_terminal_and_raises_the_call_again_next_time(self):
self.chunks(5, 4)
stop, said = self.check(FakeEffects(on_terminal=False, git=[UNREAD] * 3))
self.assertIsNone(stop)
self.assertIn("Not on a terminal: the calls are raised again at the next measurement.", said)
self.assertIsNone(state.mark_measured(self.root, 0))
self.assertIn("Not on a terminal", self.check(FakeEffects(on_terminal=False, git=[UNREAD] * 3))[1])
def test_leaves_the_measuring_to_a_loop_holding_the_lock(self):
self.chunks(5, 4)
with archive.locked(self.root):
self.assertEqual(self.check(FakeEffects()), (None, ""))
def test_stops_when_claude_cannot_start_for_the_calls(self):
self.chunks(5, 4)
stop, _ = self.check(FakeEffects(on_terminal_claude=[OSError(2, "No such file or directory")], git=[UNREAD] * 3))
self.assertEqual(stop, Stop(1, f"could not start claude for the routing calls in {self.home / 'code' / 'claude-config'}, the setup's development clone: No such file or directory"))
def test_says_a_figure_it_cannot_write_and_goes_on(self):
assert_not_root(self)
self.chunks(5, 4)
(self.root / "routing-figures.lock").touch()
self.root.chmod(0o555)
self.addCleanup(self.root.chmod, 0o755)
stop, said = self.check(FakeEffects(git=[UNREAD] * 3))
self.assertIsNone(stop)
self.assertIn("routing figures: could not write", said)
def test_puts_every_new_call_to_the_human_and_says_one_it_cannot_keep(self):
new = [Call("reports", ("Explore",)), Call("rounds", ("x", "y"))]
fake = FakeEffects(on_terminal_claude=[0])
self.root.mkdir()
fake.replace_file = lambda path, data: Failure("full")
with contextlib.redirect_stderr(io.StringIO()) as said:
self.assertIsNone(routing_check.raise_calls(self.root, new, Rung("m", "high"), self.loop(fake)))
((args),) = fake.made("claude_on_terminal")
self.assertIn(f"don't change anything until they answer:\n- {new[0].sentence()}\n- {new[1].sentence()}\nThe figures", args[6])
self.assertIn(f"call for a routing change:\n- {new[0].sentence()}\n- {new[1].sentence()}\n", said.getvalue())
self.assertIn(f"routing figures: could not write {self.root / state.RAISED}: full", said.getvalue())
[sent] = fake.dms
self.assertTrue(sent.startswith(f"loop-claude in {self.loop(fake).root} asks:\n\nloop-claude's routing figures call for a routing change:\n- {new[0].sentence()}\n"), sent)
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More