Park a chunk whose pull request waits on a gate, resume it when the gate opens, and print each chunk's state with do-epic --status
CI / tests (pull_request) Failing after 4m4s

This commit is contained in:
2026-11-18 23:17:07 +01:00
parent 1df19f4ac8
commit f0c8a26688
7 changed files with 344 additions and 30 deletions
+3 -3
View File
@@ -178,7 +178,7 @@ It renames `## Unreleased` in `CHANGELOG.md` to the version and merges that thro
Install puts only `loop-claude` on `PATH`; call the others by their path, such as `~/.claude/bin/metric-bench`, from the project's checkout. Those under `~/.claude` are the release's; test a change to one from `~/code/claude-config/bin`. The tools are listed in alphabetical order, and the packages under `bin/` follow under their own heading.
- `chunk-pr open|sync|ready|merge|clean` opens a chunk's pull request as a draft, brings its branch level with origin's default branch, marks it ready and merges it once its checks pass, with `gh` on a GitHub origin and `tea` elsewhere. The chunks of one project merge one at a time: in its turn, `merge` brings the branch level again, finishes the plan items its `--finish` flags name, and waits for the checks; once the pull request is merged, `clean` removes the chunk's worktree and branch and fast-forwards the main checkout. `chunk-pr --help` states the steps.
- `chunk-pr open|sync|ready|merge|clean` opens a chunk's pull request as a draft, brings its branch level with origin's default branch, marks it ready and merges it once its checks pass, with `gh` on a GitHub origin and `tea` elsewhere. The chunks of one project merge one at a time: in its turn, `merge` brings the branch level again, finishes the plan items its `--finish` flags name, and waits for the checks; under do-epic it fails at once where the pull request then awaits an approval, since do-epic parks the chunk; once the pull request is merged, `clean` removes the chunk's worktree and branch and fast-forwards the main checkout. `chunk-pr --help` states the steps.
- `chunk-preflight` finds the mechanical facts a chunk starts from, and with `--worktree` makes its worktree; `loop-claude` runs it with `--worktree` before it routes each chunk, and `chunk-preflight --help` lists the facts.
- `command-patterns [<version>] [--calls <file>]` prints a table of the shell command patterns a release's archived chunks ran, such as `tea pr create`, for an audit of the release, and with `--calls <file>` saves each call as a JSON line; `command-patterns --help` states the columns and the line's keys.
- `compress-archives` gzips each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them; Install's crontab line runs it daily.
@@ -216,8 +216,8 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `branch_chunks/` cuts a default branch into chunks and finds, for each, the items it shipped and the later chunks that fixed its code, for `score_outcomes` and `routing_figures`. It imports `reading_cost`'s record path.
- `command_lines/` reads a shell command: its simple commands, and a docker call's subcommand and memory limits, for `command_patterns`, `reading_cost`, `routing_figures`, `rule_hook` and `run_heavy`.
- `deploy_release` imports `release_plan`'s `release_standing`.
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. Beside it, `chunk_prompt.py` words what a chunk's worker reads. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, and posts comments.
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and waits for its checks (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. `gates.py` reads once whether a chunk's pull request waits on its checks or an approval, and `parked.py` keeps each chunk that does in a state file, so it takes no running slot while it waits and goes on waiting after a reboot. Beside it, `chunk_prompt.py` words what a chunk's worker reads. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, and posts comments.
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and reads its checks, once or until they settle (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
- `git_cli/` runs git for every tool (`git_repo.py`), and reads the git calls in a shell command (`git_command.py`).
- `judge_replay` imports `agent_cycle`'s judge and verdicts, and `loop_claude`'s premises, records and `ANSWER_DESTINATIONS`.
- `loop_claude` runs `chunk-preflight` as a process of its own.
+13 -1
View File
@@ -1,4 +1,7 @@
"""What a do-epic cycle's worker is told: its task, from the issue, and its first prompt. The resumed prompt and the rung sentences are the ones loop-claude's cycles use, naming do-epic."""
"""What a do-epic cycle's worker is told: its task, from the issue, and its first prompt. The resumed prompt and the rung sentences are the ones loop-claude's cycles use, naming do-epic; a chunk that was parked is first told why it resumes."""
import dataclasses
from agent_cycle.worker_prompts import WorkerPrompts
from do_epic.backend.protocol import EpicIssue
from model_routing.ladder import ProjectRung
from loop_claude.cycle import prompts
@@ -26,3 +29,12 @@ def fresh(task: str, premises: str, route: Route, project: ProjectRung | None, f
# The worker names no plan items: the chunk's one item is its issue.
WORKER = prompts.worker_prompts(DRIVER, fresh, "")
# What a worker does with a reviewer's feedback, as the decision asks (docs/decisions.md → `do-epic: drives one Jira epic …`, "Long gates").
FEEDBACK = 'Where a reviewer left feedback, fix what is reasonable and push it, answer the rest in its thread, and change the setup so review would have caught it (~/.claude/CLAUDE.md → "Working loop", "Change the setup for each finding an outside reviewer raises"). Then merge the pull request as before.'
def unparked(news: str) -> WorkerPrompts:
"""The prompts of the first cycle after a parked chunk resumes: WORKER's, with each prompt that starts a worker opening with news, which says what the chunk waited on and what changed. The fresh prompt opens with it too, since a chunk the judge ruled finished before it parked has no session left to resume."""
opening = f"{news} {FEEDBACK}\n\n"
return dataclasses.replace(WORKER, fresh=lambda *task: opening + WORKER.fresh(*task), resumed=lambda *evidence: opening + WORKER.resumed(*evidence))
+115 -14
View File
@@ -1,12 +1,15 @@
"""The do-epic command (docs/decisions.md → `do-epic: drives one Jira epic …`).
`do-epic <EPIC-KEY>` starts each of the epic's chunks that may start now, each in a window of the epic's tmux session, as `scheduler` picks them, and puts that session on the terminal. Each window runs `do-epic --chunk <ISSUE-KEY> <EPIC-KEY>`: the cycles of one issue's chunk, run by the code that runs loop-claude's (`agent_cycle`), until the judge rules it finished. Then it starts the chunks that may start now, and its window closes."""
`do-epic <EPIC-KEY>` starts each of the epic's chunks that may start now, each in a window of the epic's tmux session, as `scheduler` picks them, and puts that session on the terminal. Each window runs `do-epic --chunk <ISSUE-KEY> <EPIC-KEY>`: the cycles of one issue's chunk, run by the code that runs loop-claude's (`agent_cycle`), until the judge rules it finished. Then it starts the chunks that may start now, and its window closes.
After each cycle of a chunk in a repo, the window reads what the chunk's pull request waits on (`gates`). While that is a gate only time or a person passes, the chunk is parked (`parked`): the window frees its running slot and polls the forge, and the chunk's next cycle starts once the worker has something to do and a slot is free. `do-epic <EPIC-KEY> --status` prints what each open issue does."""
import dataclasses
import pathlib
import typing
from loop_claude.loop.chunk_start import archived_route, routed
from do_epic import chunk_prompt, scheduler
from do_epic import chunk_prompt, gates, parked, scheduler
from forge.commands import Run, runner
from do_epic.repo_pick import ClaudeWork, HumanTask, recorded, Repo
from loop_claude.loop.close import closed_cycle
from agent_cycle.run import Cycle, run_cycle
@@ -36,16 +39,24 @@ from agent_cycle.stop import Stop
from agent_cycle.direct_message import tell
from agent_cycle import tmux
HELP = f"""usage: do-epic <EPIC-KEY>
HELP = f"""usage: do-epic <EPIC-KEY> [--status]
Runs the chunks of one Jira epic, such as ABC-12, each in its own window of the tmux session epic-<EPIC-KEY>.
An open issue of the epic runs as a chunk once it has a label and no issue blocks it.
The label names the issue's repo, or says the work changes no repo.
As many run at once as the epic's description allows with a line such as `**Max parallel chunks:** 2`, one where it has none.
A repo takes as many chunks as its `parallel-chunks`, and none while loop-claude runs chunks in it.
A chunk whose pull request waits on its checks or on an approval is parked: it frees its place for the next
chunk, its window's name says what it awaits, and it goes on once the checks pass, the approval comes or a
reviewer comments. A parked chunk survives a reboot: run do-epic <EPIC-KEY> again to go on polling.
--status starts no chunk: it prints each open issue with whether its chunk runs, or why it waits.
The Jira site and its credentials are set in ~/.claude-projects/do-epic/config.toml."""
# Under the home folder, where a chunk's cycles are archived, a folder per epic. It is not loop-claude's archive, since that archive's readers, such as judge-replay and the routing figures, read loop-claude's prompts.
ARCHIVE = ".claude/do-epic-transcripts"
# The flag that prints what each open issue does. lilleman named it when planning release 0.12.0 (the decision's "Long gates").
STATUS_FLAG = "--status"
# How often a parked chunk reads its pull request's gate: a reviewer answers in hours, and each read is three or four calls to the forge.
POLL_SECONDS = 60
# The store of an epic's issues and its record, for the epic with this key.
Backend = typing.Callable[[DoEpicConfig, str], tuple[EpicStore, EpicRecord]]
@@ -67,15 +78,17 @@ class EpicRun:
env: typing.Mapping[str, str]
launcher: pathlib.Path
effects: Effects
# The runner of git's and the forge's commands in a folder.
forge_at: typing.Callable[[str], Run] = runner
@property
def session(self) -> str:
return tmux.session_name(tmux.EPIC, self.key)
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend = jira_backend) -> int:
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend = jira_backend, forge_at: typing.Callable[[str], Run] = runner) -> int:
try:
stop = run(argv, env, paths, launcher, effects, backend)
stop = run(argv, env, paths, launcher, effects, backend, forge_at)
except KeyboardInterrupt:
stop = Stop(130, "")
except Exception as crash:
@@ -89,8 +102,10 @@ def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher:
return stop.code
def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend) -> Stop:
def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend, forge_at: typing.Callable[[str], Run]) -> Stop:
chunk, words = (argv[1], argv[2:]) if len(argv) >= 2 and argv[0] == scheduler.CHUNK_FLAG else (None, argv)
status = chunk is None and words[1:] == [STATUS_FLAG]
words = words[:1] if status else words
if len(words) != 1 or not ISSUE_KEY.fullmatch(words[0]) or (chunk is not None and not ISSUE_KEY.fullmatch(chunk)):
return Stop(0 if words in (["-h"], ["--help"]) else 1, HELP)
# Before anything starts: a clone, a claude session or a tmux window.
@@ -100,10 +115,19 @@ def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher:
if isinstance(config, Failure):
return Stop(1, config.reason)
store, record = backend(config, words[0])
epic = EpicRun(words[0], config, store, record, paths, env, launcher, effects)
epic = EpicRun(words[0], config, store, record, paths, env, launcher, effects, forge_at)
if status:
return status_printed(epic)
return windows_opened(epic) if chunk is None else chunk_run(chunk, epic)
def status_printed(epic: EpicRun) -> Stop:
"""What `do-epic <EPIC-KEY> --status` does: prints a line per open issue, and starts nothing."""
issues = epic.store.read()
lines = issues if isinstance(issues, Failure) else scheduler.status(epic.key, epic.paths.home, issues, epic.effects.tmux)
return Stop(1, lines.reason) if isinstance(lines, Failure) else Stop(0, "\n".join(lines) or f"the epic {epic.key} has no open issue")
def windows_opened(epic: EpicRun) -> Stop:
"""What the do-epic the human runs does: starts each chunk that may start now, says why each other open issue waits, then puts the epic's session on the terminal until the human detaches."""
started = chunks_started(epic, None)
@@ -126,6 +150,12 @@ def chunks_started(epic: EpicRun, own: str | None) -> scheduler.Started | Failur
issues = epic.store.read()
if isinstance(issues, Failure):
return issues
scheduled = scheduled_epic(epic)
return scheduled if isinstance(scheduled, Failure) else scheduler.start_chunks(scheduled, issues, own, epic.effects.tmux)
def scheduled_epic(epic: EpicRun) -> scheduler.Epic | Failure:
"""The epic as scheduler limits its chunks: with its cap, read afresh, and each repo's checkout."""
cap = epic.record.parallel_chunks()
if isinstance(cap, Failure):
return cap
@@ -139,7 +169,7 @@ def chunks_started(epic: EpicRun, own: str | None) -> scheduler.Started | Failur
seen.append(found.checkout)
return real(found.checkout), found.config.parallel_chunks
return scheduler.start_chunks(scheduler.Epic(epic.key, cap, located, epic.paths.home, epic.launcher, epic.env), issues, own, epic.effects.tmux)
return scheduler.Epic(epic.key, cap, located, epic.paths.home, epic.launcher, epic.env)
@dataclasses.dataclass(frozen=True)
@@ -167,7 +197,7 @@ def real(checkout: pathlib.Path) -> pathlib.Path:
def chunk_run(key: str, epic: EpicRun) -> Stop:
"""Runs the cycles of the chunk of the issue called key until one stops the loop, or the judge rules the chunk finished."""
"""Runs the cycles of the chunk of the issue called key until one stops the loop, or the judge rules the chunk finished. A chunk in a repo parks after each cycle that leaves its pull request waiting on a gate, and before its first where it was parked when its window last ended, as at a reboot."""
pane = epic.env.get("TMUX_PANE")
if not pane:
return Stop(1, f"{scheduler.CHUNK_FLAG} is how do-epic starts itself in a tmux window; run do-epic <EPIC-KEY>")
@@ -181,9 +211,13 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
session = session_named(f"A {epic.key} do-epic {key}", epic.paths.claude_code_transcripts)
loop = Loop(root=root, session=session, window=tmux.Window(epic.session, key, pane), paths=epic.paths, env=epic.env, launcher=epic.launcher, effects=epic.effects)
loop_run = LoopRun(0, root, "", "", chunk=key)
# What a chunk that was parked is told as it resumes; empty for a chunk that wasn't.
news = parked_through(issue, loop, epic, starts=True) if in_repo else ""
while True:
if isinstance(news, Stop):
return news
loop_run = dataclasses.replace(loop_run, cycle_number=loop_run.cycle_number + 1)
cycle = next_cycle(loop_run, loop, in_repo, epic)
cycle = next_cycle(loop_run, loop, in_repo, epic, news)
if isinstance(cycle, Stop):
return cycle
loop_run = dataclasses.replace(loop_run, task=cycle.task)
@@ -194,10 +228,77 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
if isinstance(closed, Stop):
return closed
loop_run = closed
if isinstance(concluded.verdict, Finished):
news = parked_through(issue, loop, epic, starts=False) if in_repo else ""
# A chunk that was parked has work again, whatever the judge ruled before its pull request passed the gate.
if isinstance(concluded.verdict, Finished) and news == "":
return handed_on(key, epic)
def parked_through(issue: EpicIssue, loop: Loop, epic: EpicRun, starts: bool) -> str | Stop:
"""Keeps the chunk of issue parked for as long as its pull request waits on a gate, and returns once its worker has something to do: what to tell that worker, or empty where the chunk was never parked. starts says the window just started, where the chunk is parked only if its file says so: nothing is read from the forge for a chunk that starts afresh.
A chunk parks in three steps: its file is written, its window is shown as parked, and the epic's next chunks start in the slot it frees. It resumes in the reverse order: `scheduler.resume` takes it a slot, then its file is removed. So a window that ends in between finds the chunk parked when it starts again.
"""
key, folder = issue.key, epic.paths.home / scheduler.STATE / epic.key
recorded_as = parked.parked_chunks(folder)
if isinstance(recorded_as, Failure):
return Stop(1, recorded_as.reason)
if starts and key not in recorded_as:
return ""
# What the chunk is parked on; None while it isn't parked.
on = recorded_as[key].on if key in recorded_as else None
# What the window's name says the chunk awaits, and the last reason said for staying parked.
shown = said = None
while True:
now = gates.gate(key, loop.root, epic.forge_at)
if on is None and not isinstance(now, gates.Gate):
if isinstance(now, Failure):
say(f"cannot tell whether the chunk's pull request waits on a gate, so the chunk isn't parked: {now.reason}", epic.effects, stdout=True)
return ""
stays = None
if isinstance(now, gates.Gate) and now.on != on:
failed = parked.park(folder, parked.Parked(key, now.worktree, now.pull, str(loop.session.id), now.on))
if failed is not None:
return Stop(1, failed.reason)
say(f"parked: pull request #{now.pull} awaits {now.on}", epic.effects, stdout=True)
elif isinstance(now, Failure):
stays = f"cannot read the gate: {now.reason}"
elif isinstance(now, gates.Open):
full = resumed(issue, loop, epic)
if full is None:
failed = parked.unpark(folder, key)
return Stop(1, failed.reason) if failed is not None else f"{DRIVER_PARKED} its pull request awaited {on}. It resumes now: {now.news}."
stays = f"{now.news}, but the chunk stays parked: {full}"
awaits = SLOT if isinstance(now, gates.Open) else now.on if isinstance(now, gates.Gate) else shown or on
if awaits != shown:
tmux.park(loop.window, str(awaits), epic.effects.tmux)
shown = awaits
if on is None:
# Once the window shows as parked, so the chunks that start count it as running nowhere.
started = chunks_started(epic, key)
say(f"could not start the next chunk: {started.reason}" if isinstance(started, Failure) else f"started {', '.join(started.started) or 'no other'}", epic.effects, stdout=True)
on = now.on if isinstance(now, gates.Gate) else on
if stays is not None and stays != said:
say(f"{stays}; trying again every {POLL_SECONDS} seconds", epic.effects, stdout=True)
said = stays
epic.effects.sleep(POLL_SECONDS)
# How the news for a chunk that was parked opens.
DRIVER_PARKED = f"{chunk_prompt.DRIVER} parked the chunk while"
# What a parked chunk awaits, in its window's name, once its worker has something to do but no running slot is free.
SLOT = "a slot"
def resumed(issue: EpicIssue, loop: Loop, epic: EpicRun) -> str | None:
"""scheduler.resume for the parked chunk of issue, on the epic's cap and repos read afresh: None once it holds a running slot, else why it stays parked."""
pick = recorded(issue)
scheduled = scheduled_epic(epic)
if isinstance(scheduled, Failure):
return scheduled.reason
return scheduler.resume(scheduled, pick.name, loop.window, epic.effects.tmux) if isinstance(pick, Repo) else f"no label of {issue.key} names its repo any longer"
def issue_read(key: str, epic: EpicRun) -> EpicIssue | Stop:
"""The epic's issue called key, read afresh."""
issues = epic.store.read()
@@ -231,8 +332,8 @@ def handed_on(key: str, epic: EpicRun) -> Stop:
return Stop(0, "")
def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun) -> Cycle | Stop:
"""The cycle loop_run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun, news: str = "") -> Cycle | Stop:
"""The cycle loop_run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read. news is what `parked_through` tells the worker of a chunk that was parked, and opens its prompt. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
issue = issue_read(loop.window.name, epic)
if isinstance(issue, Stop):
return issue
@@ -256,7 +357,7 @@ def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun) -> C
project = ladder.project_rung(models.worker)
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, loop), lambda: RoutingIssue(issue) if "L" in issue.scores else None)
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", epic.effects, stdout=True)
return Cycle(config=setup_version(epic.paths.claude_dir, epic.effects.git), effects=epic.effects, facts="", judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=chunk_prompt.task(issue, epic.key, loop.root if in_repo else None), prompts=chunk_prompt.WORKER, taken_in=lambda entries: TakenIn(), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
return Cycle(config=setup_version(epic.paths.claude_dir, epic.effects.git), effects=epic.effects, facts="", judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=chunk_prompt.task(issue, epic.key, loop.root if in_repo else None), prompts=chunk_prompt.unparked(news) if news else chunk_prompt.WORKER, taken_in=lambda entries: TakenIn(), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
def repoless_models(epic: EpicRun) -> Models:
+18
View File
@@ -192,3 +192,21 @@ def resume(epic: Epic, repo: str, window: tmux.Window, run: tmux.Tmux) -> str |
return None
except OSError as failure:
return f"cannot lock {failure.filename or epic.folder / LOCK}: {failure.strerror}"
def status(key: str, home: pathlib.Path, issues: list[EpicIssue], run: tmux.Tmux) -> list[str] | Failure:
"""A line per open issue of the epic called key, in the tracker's rank order: that its chunk runs, or why it waits. It starts nothing and checks no limit, so a chunk that has yet to start is told apart only from one that stopped."""
found, waits = chunks(issues)
parked = parked_chunks(home / STATE / key)
if isinstance(parked, Failure):
return parked
windows = tmux.listed(tmux.session_name(tmux.EPIC, key), run)
live = {window.name for window in windows if not window.dead}
again = f"run `do-epic {key}` to start it again"
reasons = {wait.issue: wait.reason for wait in waits}
for chunk in found:
if chunk.issue in parked:
reasons[chunk.issue] = parked[chunk.issue].waits + ("" if chunk.issue in live else f", and its window stopped; {again}")
elif chunk.issue not in live:
reasons[chunk.issue] = f"its window stopped; {again}" if chunk.issue in {window.name for window in windows} else f"it has not started; `do-epic {key}` starts it once the epic and its repo have room"
return [f"{issue.key} waits: {reasons[issue.key]}" if issue.key in reasons else f"{issue.key} runs" for issue in issues if issue.key in reasons or issue.key in live]
+172 -11
View File
@@ -19,6 +19,8 @@ from agent_cycle import escalation_mark, judge, tmux
from failure import Failure
from loop_claude.test_loop_claude import FakeEffects, judged, Worker
from do_epic.test_scheduler import issue
from do_epic.test_gates import Forge
from do_epic import parked
from loop_claude.loop.this_loop import Loop
from loop_claude.loop.reload import LoopRun
from do_epic.backend.fake import MemoryEpic
@@ -50,14 +52,35 @@ class DoEpicTest(unittest.TestCase):
(folder / "repo-rule.md").write_text("A label that equals a repo's name names that repo.\n", encoding="utf-8")
self.issues = [issue("ABC-20"), dataclasses.replace(issue("ABC-21", "do-epic-claude"), detail="Tidy the docs.", scores={"L": 3, "E": 2, "I": 3})]
self.description: str | Failure = DESCRIPTION
# What the forge answers each read of a chunk's gate, in turn; the last answers every later read. No worktree has the chunk's branch, so no chunk parks.
self.forges = [Forge(worktrees="worktree /code/api\nbranch refs/heads/main\n")]
self.gate_reads = 0
def forge_at(self, folder: str):
def run(command: list[str]) -> str | Failure:
# A read of a gate opens with the checkout's worktrees.
self.gate_reads += command[:2] == ["git", "worktree"]
return self.forges[min(self.gate_reads, len(self.forges)) - 1].at(folder)(command)
return run
def backend(self, config: object, key: str) -> tuple[MemoryEpic, EpicRecord]:
return MemoryEpic(key, list(self.issues)), EpicRecord(Description(self.description)) # type: ignore[arg-type]
def worker(self) -> Worker:
return Worker(paths_from(self.home).claude_code_transcripts.parent)
def repo_chunk(self) -> pathlib.Path:
"""Makes ABC-20 the one issue, a chunk in the repo api whose checkout holds its premises; the folder of the repo's config."""
checkout = self.home / "code" / "api"
folder = configured(self, checkout, home=self.home)
(checkout / "README.md").write_text("## Goals\nServe the API.\n\n## Audience\nIts callers.\n", encoding="utf-8")
self.issues = [issue("ABC-20", "repo:api")]
return folder
def run_main(self, argv: list[str], fake: FakeEffects, env: dict[str, str] | None = None) -> tuple[int, str, str]:
out, err = io.StringIO(), io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main(argv, env or {}, paths_from(self.home), pathlib.Path("/b/do-epic"), fake.effects(), self.backend)
code = main.main(argv, env or {}, paths_from(self.home), pathlib.Path("/b/do-epic"), fake.effects(), self.backend, self.forge_at)
return code, out.getvalue(), err.getvalue()
@@ -237,8 +260,6 @@ class TheHumansRun(DoEpicTest):
class AChunksRun(DoEpicTest):
def worker(self) -> Worker:
return Worker(paths_from(self.home).claude_code_transcripts.parent)
def test_runs_cycles_until_the_judge_rules_it_finished_then_starts_the_next_chunk(self) -> None:
self.issues.append(issue("ABC-22", "do-epic-claude"))
@@ -307,16 +328,13 @@ class AChunksRun(DoEpicTest):
self.assertIn("Ship the import.", prompt)
self.assertIn("The repo's goals, which rank after the epic's and after code health, are from", prompt)
self.assertIn("Serve the API.", prompt)
self.assertIn("the chunk finished; started no other\n", out)
self.assertIn("the chunk finished; ", out)
self.assertIn("Squash every merge.", [what[1] for seam, what in fake.calls if seam == "claude"][0])
def repo_chunk(self) -> pathlib.Path:
"""Makes ABC-20 the one issue, a chunk in the repo api whose checkout holds its premises; the folder of the repo's config."""
checkout = self.home / "code" / "api"
folder = configured(self, checkout, home=self.home)
(checkout / "README.md").write_text("## Goals\nServe the API.\n\n## Audience\nIts callers.\n", encoding="utf-8")
self.issues = [issue("ABC-20", "repo:api")]
return folder
def test_a_chunk_without_a_repo_reads_no_forge(self) -> None:
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD)
self.assertEqual(self.run_main(["--chunk", "ABC-21", "ABC-12"], fake, MODELS)[0], 0)
self.assertEqual(self.gate_reads, 0)
def test_a_chunk_in_a_repo_routes_on_the_repos_model(self) -> None:
folder = self.repo_chunk()
@@ -421,5 +439,148 @@ class WhatAWorkerJudgeAndHumanRead(DoEpicTest):
self.assertEqual(escalation_mark.asks_higher_rung(chunk_prompt.DRIVER), "asking do-epic for a higher rung")
WORKTREE = "worktree /code/api\nbranch refs/heads/main\n\nworktree /code/api-ABC-20\nbranch refs/heads/ABC-20\n"
PARKED_WINDOW = "ABC-20 awaits an approval\t0\t@9\n"
def awaiting(**answers: object) -> Forge:
"""A forge where ABC-20's pull request, #7, passed its checks and still wants its review, but for answers."""
return Forge(worktrees=WORKTREE, **answers) # type: ignore[arg-type]
class AChunkWhosePullRequestWaitsOnAGate(DoEpicTest):
def setUp(self) -> None:
super().setUp()
self.repo_chunk()
self.issues.append(issue("ABC-22", "do-epic-claude"))
self.file = parked.file_of(self.home / ".local/state/do-epic/ABC-12", "ABC-20")
self.session = main.session_named("A ABC-12 do-epic ABC-20", paths_from(self.home).claude_code_transcripts)
def parked_run(self, fake: FakeEffects) -> tuple[int, str, str, list[str]]:
"""The chunk's run, as its exit code, what it printed, and each prompt its workers got."""
code, out, err = self.run_main(["--chunk", "ABC-20", "ABC-12"], fake, MODELS)
return code, out.replace("<1790000000> ", ""), err, [what[what.index("-p") + 1] for seam, what in fake.calls if seam == "stream_claude"]
def test_it_parks_frees_its_slot_and_works_again_once_approved(self) -> None:
self.forges = [awaiting(), awaiting(), awaiting(decision="APPROVED")]
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("finished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
states: list[object] = []
fake.effects = lambda made=fake.effects(): dataclasses.replace(made, sleep=lambda seconds: states.append((seconds, parked.parked_chunks(self.file.parent.parent)))) # type: ignore[method-assign]
code, out, err, prompts = self.parked_run(fake)
self.assertEqual(code, 0, err)
self.assertEqual(states, [(60, {"ABC-20": parked.Parked("ABC-20", "/code/api-ABC-20", "7", str(self.session.id), "an approval")})] * 2)
self.assertFalse(self.file.exists())
self.assertIn("parked: pull request #7 awaits an approval\nstarted ABC-22\n", out)
self.assertEqual(len(prompts), 2)
self.assertTrue(prompts[1].startswith(f"do-epic parked the chunk while its pull request awaited an approval. It resumes now: pull request #7 passed its checks and is approved. {chunk_prompt.FEEDBACK}\n\nWork the Jira issue ABC-20 "), prompts[1])
self.assertIn("the chunk finished; ", out)
checkout = str(self.home / "code/api")
shown = [args[3:] if args[0] == "rename-window" else args[1:] for args in fake.tmux_calls if args[0] == "rename-window" or tmux.ROOT in args and "%1" in args]
self.assertEqual(shown, [["ABC-20 awaits an approval"], ["-w", "-u", "-t", "%1", tmux.ROOT], ["ABC-20"], ["-w", "-t", "%1", tmux.ROOT, checkout]])
def test_an_unfinished_chunk_parks_too_and_a_reviewers_comment_resumes_it(self) -> None:
self.forges = [awaiting(buckets=("pending",)), awaiting(), awaiting(threads=({"isResolved": False, "path": "a.py", "comments": {"nodes": [{"author": {"login": "reviewer"}}]}},))]
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("unfinished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
code, out, err, prompts = self.parked_run(fake)
self.assertEqual(code, 0, err)
self.assertIn("parked: pull request #7 awaits its checks\nstarted ABC-22\nparked: pull request #7 awaits an approval\n", out)
self.assertIn("its pull request awaited an approval. It resumes now: a reviewer's comment on pull request #7 waits for an answer, in a.py. Where a reviewer left feedback, fix what is reasonable", prompts[1])
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits its checks", "ABC-20 awaits an approval", "ABC-20"])
def test_a_chunk_parked_when_its_window_ended_polls_before_any_cycle_and_resumes_its_session(self) -> None:
parked.park(self.file.parent.parent, parked.Parked("ABC-20", "/code/api-ABC-20", "7", str(self.session.id), "an approval"))
self.worker()(["--session-id", str(self.session.id)])
self.forges = [awaiting(), awaiting(decision="CHANGES_REQUESTED")]
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
code, out, err, prompts = self.parked_run(fake)
self.assertEqual(code, 0, err)
self.assertNotIn("parked: ", out)
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits an approval", "ABC-20"])
self.assertEqual(len(prompts), 1)
self.assertTrue(prompts[0].startswith("do-epic parked the chunk while its pull request awaited an approval. It resumes now: a reviewer requested changes on pull request #7. "), prompts[0])
self.assertFalse(self.file.exists())
def test_a_chunk_with_work_again_stays_parked_until_a_slot_is_free(self) -> None:
self.description = DESCRIPTION.replace("2", "1")
self.forges = [awaiting(), awaiting(decision="APPROVED")]
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("finished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW + "ABC-22\t0\t@10\n")
slept: list[bool] = []
def sleep(seconds: float) -> None:
slept.append(self.file.exists())
if len(slept) == 3:
fake.windows = PARKED_WINDOW
fake.effects = lambda made=fake.effects(): dataclasses.replace(made, sleep=sleep) # type: ignore[method-assign]
code, out, err, prompts = self.parked_run(fake)
self.assertEqual(code, 0, err)
self.assertEqual(slept, [True] * 3)
self.assertEqual(out.count("pull request #7 passed its checks and is approved, but the chunk stays parked: the epic runs as many chunks as its `**Max parallel chunks:**` line allows: 1; trying again every 60 seconds\n"), 1)
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits an approval", "ABC-20 awaits a slot", "ABC-20"])
self.assertEqual(len(prompts), 2)
def test_a_forge_that_cannot_be_read_parks_no_chunk_and_unparks_none(self) -> None:
self.forges = [awaiting(fails="gh pr checks")]
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD)
code, out, err, _ = self.parked_run(fake)
self.assertEqual(code, 0, err)
self.assertIn("cannot tell whether the chunk's pull request waits on a gate, so the chunk isn't parked: gh pr checks failed\nthe chunk finished; ", out)
self.assertFalse(self.file.exists())
self.forges = [awaiting(), awaiting(fails="gh pr list"), awaiting(fails="gh pr list"), awaiting(decision="APPROVED")]
self.gate_reads = 0
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("finished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
code, out, err, prompts = self.parked_run(fake)
self.assertEqual(code, 0, err)
self.assertEqual(out.count("cannot read the gate: gh pr list failed; trying again every 60 seconds\n"), 1)
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits an approval", "ABC-20"])
self.assertEqual(len(prompts), 2)
def test_stops_where_the_chunk_cannot_be_recorded_as_parked_or_the_record_cannot_be_read(self) -> None:
self.forges = [awaiting()]
self.file.parent.parent.mkdir(parents=True)
self.file.parent.write_text("a file")
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD)
code, _, err, _ = self.parked_run(fake)
self.assertEqual(code, 1)
self.assertIn(f"cannot record the parked chunk in {self.file}: ", err)
self.file.parent.unlink()
parked.park(self.file.parent.parent, parked.Parked("ABC-20", "/w", "7", "s", "an approval"))
self.file.write_text("{")
code, _, err, prompts = self.parked_run(FakeEffects(git=UNREAD))
self.assertEqual((code, prompts), (1, []))
self.assertIn(f"cannot read the parked chunk in {self.file}: ", err)
class TheStatus(DoEpicTest):
def test_prints_what_each_open_issue_does_and_starts_nothing(self) -> None:
self.issues = [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api"), issue("ABC-22", "repo:api"), issue("ABC-23", "repo:api", blockers=("ABC-20",)), issue("ABC-24", "repo:api"), issue("ABC-25", "repo:api", done=True), issue("ABC-26", "repo:api"), issue("ABC-27")]
folder = self.home / ".local/state/do-epic/ABC-12"
parked.park(folder, parked.Parked("ABC-20", "/code/api-ABC-20", "7", "s", "an approval"))
parked.park(folder, parked.Parked("ABC-26", "/code/api-ABC-26", "9", "s", "its checks"))
fake = FakeEffects(windows="ABC-20 awaits an approval\t0\t@1\nABC-21 ?\t0\t@2\nABC-22\t1\t@3\n")
code, out, err = self.run_main(["ABC-12", "--status"], fake)
self.assertEqual((code, err), (0, ""))
self.assertEqual(out.replace("<1790000000> ", "").splitlines(), [
"ABC-20 waits: it is parked: its pull request #7 awaits an approval",
"ABC-21 runs",
"ABC-22 waits: its window stopped; run `do-epic ABC-12` to start it again",
"ABC-23 waits: ABC-20 blocks it",
"ABC-24 waits: it has not started; `do-epic ABC-12` starts it once the epic and its repo have room",
"ABC-26 waits: it is parked: its pull request #9 awaits its checks, and its window stopped; run `do-epic ABC-12` to start it again",
"ABC-27 waits: no label names its repo",
])
self.assertEqual({args[0] for args in fake.tmux_calls}, {"-V", "list-windows"})
def test_says_so_where_no_issue_is_open_or_the_epic_cannot_be_read(self) -> None:
self.issues = [issue("ABC-20", "repo:api", done=True)]
self.assertIn("the epic ABC-12 has no open issue\n", self.run_main(["ABC-12", "--status"], FakeEffects())[1])
store = MemoryEpic("ABC-12", [])
store.read = lambda: Failure("Jira answered 503") # type: ignore[method-assign]
self.backend = lambda config, key: (store, EpicRecord(Description(self.description))) # type: ignore[method-assign,arg-type]
code, _, err = self.run_main(["ABC-12", "--status"], FakeEffects())
self.assertEqual(code, 1)
self.assertIn("Jira answered 503\n", err)
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -153,7 +153,7 @@ Valid while `~/.claude` is a checkout of this repo.
- **Releases.** An epic item has no release. It ships into its repo's unreleased work: a bullet under `CHANGELOG.md` → `## Unreleased`, or nothing where the repo's plan is in Jira.
- **Repos.** A repo's name is its main checkout's folder name, and a project config names that checkout. A missing checkout is cloned from the host of the epic's other repos; where unsure, do-epic asks. Unsure means the other repos don't all sit under one host and owner (the same host spelt as https and as ssh counts as two), an origin can't be read or isn't an https, ssh or scp-style address, the folder to clone into holds something else, or the clone fails. A repo no config names is cloned into the clone folder, and the human is asked to write its config. Each chunk uses its repo's model, routing, and language for commits and PRs. Jira text is in the epic's language, or else the machine's default.
- **Running.** The epic caps the chunks running at once, and each repo's `parallel-chunks` still applies. do-epic and loop-claude never run chunks in the same repo at once. The cap is a line of the epic's description, `**Max parallel chunks:** 2`, and 1 without it, as `parallel-chunks` is. A repo's `parallel-chunks` counts the chunks of every epic in its checkout. Each window is named for its issue's key, and a tmux window option, `@chunk-root`, names the folder its chunk works in, which is how loop-claude sees do-epic's chunks in a checkout. do-epic and loop-claude each hold the checkout's `starting` lock from checking the checkout's chunks to starting a window there. A chunk without a repo works in `~/.local/state/do-epic/<EPIC-KEY>/`. do-epic archives its cycles under `~/.claude/do-epic-transcripts/`, not in loop-claude's archive, whose readers parse loop-claude's prompts. An open issue runs once a label records its repo; until do-epic picks repos itself, an issue without one waits, and `do-epic <EPIC-KEY>` says so.
- **Long gates.** A chunk whose PR waits on a gate, such as a human approval, parks. It keeps its worktree, PR and session, and its state is in a file that survives a reboot. A parked chunk takes no running slot, and the items it blocks wait. do-epic polls the forge and resumes the chunk's session when the gate passes or a reviewer comments. It fixes reasonable feedback, replies to the rest, and changes the setup so review would have caught it. `do-epic <key> --status` and the tmux window names show what each chunk waits on.
- **Long gates.** A chunk whose PR waits on a gate, such as a human approval, parks. It keeps its worktree, PR and session, and its state is in a file that survives a reboot. A parked chunk takes no running slot, and the items it blocks wait. do-epic polls the forge and resumes the chunk's session when the gate passes or a reviewer comments. It fixes reasonable feedback, replies to the rest, and changes the setup so review would have caught it. `do-epic <key> --status` and the tmux window names show what each chunk waits on. A gate is a pull request's pending checks or, on GitHub, the approval its branch protection requires; a draft waits on neither, since its worker has yet to mark it ready. The chunk's window reads the forge once a minute, which no forge's rate limit notices. A chunk resumes only into a free slot, under the limits a starting chunk meets, and stays parked until then. `do-epic <key>` starts a parked chunk's window again whatever the limits, since it goes on parked. do-epic sets `CHUNK_PR_PARKS=1` for each worker, so `chunk-pr merge` fails at once where the pull request awaits an approval and doesn't hold the slot while it waits.
- **Human tasks.** do-epic DMs the human when one blocks the next item, and polls until its issue closes.
- **Findings.** A finding about the epic's goals is filed in the epic. A finding about a repo's goals follows that repo's rules. Where unsure which, do-epic asks. A set of new items toward an epic goal needs the human's approval before work starts; a finding filed during a chunk doesn't.
- **Finishing.** When the epic holds no open item: release-review passes on each repo the epic changed, then the items they file, then the human confirms each achieve goal and that no maintain goal has a defect. For a goal that isn't met, do-epic proposes items for approval. Then a summary comment on the epic, and closing the epic in Jira, as the very last step.
+22
View File
@@ -2888,3 +2888,25 @@ Read most (opens and searches):
- `README.md` 1
- `bin/rule_hook/endless_wait.py` 1
- `bin/rule_hook/env_file.py` 1
## Reading cost, chunk c7c9393c2307, finished 2026-11-18T21:50:41Z
50171 new input tokens before the first commit.
Read most (opens and searches):
- `bin/chunk_preflight/test_chunk_preflight.py` 3
- `bin/chunk_preflight/main.py` 2
## Reading cost, chunk b5056eb2a505, finished 2026-11-18T22:01:58Z
98259 new input tokens before the first commit.
Read most (opens and searches):
- `rules/todo-items.md` 3
- `bin/loop_claude/loop/chunk_windows.py` 2
- `bin/loop_claude/loop/main.py` 2
- `docs/decisions.md` 2
- `bin/agent_cycle/tmux.py` 1
- `bin/loop-claude` 1