Park a chunk whose pull request waits on a long gate, and resume it when the gate opens #624
@@ -2,6 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- do-epic parks a chunk whose pull request waits on its checks, an approval or a new review: the chunk frees its place for the next one, its tmux window says what it awaits, and its worker goes on when the gate opens or a reviewer comments, even after a reboot. `do-epic <EPIC-KEY> --status` prints what each open issue does.
|
||||
- A chunk that `do-epic` runs files a finding about an epic goal as an issue of the epic, with `do-epic <EPIC-KEY> --file`, and records a decision in the epic's description, with `--record-decision`; its judge reads the epic's Scoring and Decisions before the repo's records.
|
||||
- Added `do-epic <EPIC-KEY>`: it runs the chunks of one Jira epic, each in its own window of the tmux session `epic-<EPIC-KEY>`. As many run at once as the line `**Max parallel chunks:** 2` in the epic's description allows, one without it; a repo takes as many as its `parallel-chunks`, and none while loop-claude runs chunks in it. loop-claude in turn starts no chunk in a checkout where do-epic runs one.
|
||||
- Refuse in rule-hook a Bash or Monitor command that waits forever: a sleeping loop with no time or count cap, and a `pgrep -f` or `pkill -f` whose pattern its own command line matches.
|
||||
|
||||
@@ -216,8 +216,8 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `branch_chunks/` cuts a default branch into chunks and finds, for each, the items it shipped and the later chunks that fixed its code, for `score_outcomes` and `routing_figures`. It imports `reading_cost`'s record path.
|
||||
- `command_lines/` reads a shell command: its simple commands, and a docker call's subcommand and memory limits, for `command_patterns`, `reading_cost`, `routing_figures`, `rule_hook` and `run_heavy`.
|
||||
- `deploy_release` imports `release_plan`'s `release_standing`.
|
||||
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. Beside it, `chunk_prompt.py` words what a chunk's worker reads. Through `findings.py` that worker files a finding as an issue of the epic, with `--file`, and appends a decision to its description, with `--record-decision`. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, and posts comments.
|
||||
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and waits for its checks (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
|
||||
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. `gates.py` reads once whether a chunk's pull request waits on its checks, an approval or a new review, and `parked.py` keeps each chunk that does in a state file, so it takes no running slot while it waits and goes on waiting after a reboot. `chunk_prompt.py` words what a chunk's worker reads. Through `findings.py` that worker files a finding as an issue of the epic, with `--file`, and appends a decision to its description, with `--record-decision`. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, and posts comments.
|
||||
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and reads its checks, once or until they settle (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
|
||||
- `git_cli/` runs git for every tool (`git_repo.py`), and reads the git calls in a shell command (`git_command.py`).
|
||||
- `judge_replay` imports `agent_cycle`'s judge and verdicts, and `loop_claude`'s premises, records and `ANSWER_DESTINATIONS`.
|
||||
- `loop_claude` runs `chunk-preflight` as a process of its own.
|
||||
|
||||
+28
-3
@@ -14,6 +14,8 @@ NEEDED = (3, 2)
|
||||
# What a window's name ends in while its chunk waits for the human. tmux's status bar shows the name, and `!` after it once the bell rang.
|
||||
WAITS = " ?"
|
||||
BELL = "\a"
|
||||
# What follows a window's name while its chunk is parked, before what its pull request waits on, as in `ABC-34 awaits an approval`. A parked chunk takes no running slot (docs/decisions.md → `do-epic: drives one Jira epic …`, "Long gates").
|
||||
PARKED = " awaits "
|
||||
# What a session's name opens with: LOOP for a project's chunks under loop-claude, EPIC for an epic's chunks under do-epic.
|
||||
LOOP = "loop"
|
||||
EPIC = "epic"
|
||||
@@ -41,12 +43,18 @@ class Window:
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Listed:
|
||||
"""A window of the project's session, as tmux lists it."""
|
||||
# Without WAITS.
|
||||
# Without WAITS, and without PARKED and what follows it.
|
||||
name: str
|
||||
# tmux's ID of the window, such as `@3`.
|
||||
id: str
|
||||
# Whether its loop has exited: `remain-on-exit failed` keeps the window of a loop that failed, so its last lines stay readable.
|
||||
dead: bool
|
||||
# What its name says its parked chunk awaits, as the words after PARKED; empty for a chunk that isn't parked.
|
||||
awaits: str = ""
|
||||
|
||||
@property
|
||||
def parked(self) -> bool:
|
||||
return bool(self.awaits)
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -76,14 +84,20 @@ def listed(session: str, run: Tmux) -> list[Listed]:
|
||||
"""The windows of session; none where tmux lists none, as while the session doesn't exist or no tmux server runs."""
|
||||
done = run(["list-windows", "-t", f"={session}", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"])
|
||||
rows = [line.split("\t") for line in done.stdout.splitlines()] if done.returncode == 0 else []
|
||||
return [Listed(row[0].removesuffix(WAITS), row[2], row[1] == "1") for row in rows if len(row) == 3]
|
||||
return [as_listed(row[0], row[2], row[1]) for row in rows if len(row) == 3]
|
||||
|
||||
|
||||
def as_listed(shown: str, id: str, pane_dead: str) -> Listed:
|
||||
"""The window tmux shows under the name shown, with the ID id and `#{pane_dead}` as pane_dead."""
|
||||
name, _, awaits = shown.removesuffix(WAITS).partition(PARKED)
|
||||
return Listed(name, id, pane_dead == "1", awaits)
|
||||
|
||||
|
||||
def everywhere(run: Tmux) -> list[Placed]:
|
||||
"""The windows of every session; none where no tmux server runs."""
|
||||
done = run(["list-windows", "-a", "-F", f"#{{session_name}}\t#{{window_name}}\t#{{pane_dead}}\t#{{window_id}}\t#{{{ROOT}}}"])
|
||||
rows = [line.split("\t") for line in done.stdout.splitlines()] if done.returncode == 0 else []
|
||||
return [Placed(row[0], Listed(row[1].removesuffix(WAITS), row[3], row[2] == "1"), row[4]) for row in rows if len(row) == 5]
|
||||
return [Placed(row[0], as_listed(row[1], row[3], row[2]), row[4]) for row in rows if len(row) == 5]
|
||||
|
||||
|
||||
def start_window(window: Window, cwd: str, env: typing.Mapping[str, str], command: list[str], first: bool, owned: bool, run: Tmux) -> Failure | None:
|
||||
@@ -131,6 +145,17 @@ def mark(window: Window, waits: bool, run: Tmux) -> None:
|
||||
run(["rename-window", "-t", window.pane, window.name + (WAITS if waits else "")])
|
||||
|
||||
|
||||
def park(window: Window, on: str, run: Tmux) -> None:
|
||||
"""Shows window's chunk as parked. on is what the chunk awaits, such as `an approval` or `a slot`: the name says so, and no checkout counts a window so named as a running chunk."""
|
||||
run(["rename-window", "-t", window.pane, window.name + PARKED + on])
|
||||
|
||||
|
||||
def unpark(window: Window, root: str, run: Tmux) -> None:
|
||||
"""Shows window's chunk as running again, in the folder root. The window may name another folder: one started again for a parked chunk starts in the epic's folder."""
|
||||
run(["rename-window", "-t", window.pane, window.name])
|
||||
run(["set-option", "-w", "-t", window.pane, ROOT, root])
|
||||
|
||||
|
||||
def show(session: str, inside: bool) -> list[str]:
|
||||
"""The tmux arguments that put session on the human's terminal: inside tmux the client switches to it, and outside a new client attaches."""
|
||||
return ["switch-client" if inside else "attach-session", "-t", f"={session}"]
|
||||
|
||||
+24
-10
@@ -10,6 +10,8 @@ from forge.commands import Forge, Run
|
||||
# A human answers in hours, so merge polls once a minute and gives up after 90 minutes. That leaves half an hour for the queue and the checks inside the 2 hours a background command may run; a run that needs more is killed before the give-up message prints, and its next run goes on from what it pushed.
|
||||
POLL_SECONDS = 60
|
||||
WAIT_SECONDS = 90 * 60
|
||||
# The environment variable do-epic sets to 1 for its chunks' workers. With it, merge doesn't wait for an approval: do-epic parks the chunk and polls the forge itself, so the chunk holds no running slot meanwhile (docs/decisions.md → `do-epic: drives one Jira epic …`, "Long gates").
|
||||
PARKS = "CHUNK_PR_PARKS"
|
||||
|
||||
|
||||
def awaited(forge: Forge, wait_for_approval: bool) -> bool | Failure:
|
||||
@@ -58,17 +60,29 @@ def approved(number: str, run: Run, sleep: typing.Callable[[float], None] = time
|
||||
return Failure(f"pull request {number} wasn't approved within {WAIT_SECONDS // 60} minutes; run chunk-pr merge again once it is")
|
||||
|
||||
|
||||
def left_to_park(number: str, run: Run) -> Failure:
|
||||
"""Why a merge under PARKS ends at once where pull request number passed its checks and awaits an approval, in place of waiting as `approved` does: a reviewer's comment waits for an answer, which the worker gives first, or else do-epic parks the chunk, so the worker ends its run."""
|
||||
return unanswered(number, run) or Failure(f"pull request {number} passed its checks and awaits an approval; end your run now, since do-epic parks the chunk and resumes this session once it is approved or a reviewer comments")
|
||||
|
||||
|
||||
def unanswered(number: str, run: Run) -> Failure | None:
|
||||
"""A Failure naming the files where a reviewer's comment waits for an answer on pull request number: a review thread nobody resolved, whose last comment someone other than the pull request's author wrote. So a thread the worker answered, as that author, waits for the reviewer again. None where no thread waits."""
|
||||
read = run(["gh", "api", "graphql", "-f", f"query={THREADS}", "-F", "owner={owner}", "-F", "repo={repo}", "-F", f"number={number}"])
|
||||
try:
|
||||
pull = read if isinstance(read, Failure) else json.loads(read)["data"]["repository"]["pullRequest"]
|
||||
if isinstance(pull, Failure):
|
||||
return pull
|
||||
author = pull["author"]["login"]
|
||||
waiting = sorted({thread["path"] for thread in pull["reviewThreads"]["nodes"] if not thread["isResolved"] and any(comment["author"]["login"] != author for comment in thread["comments"]["nodes"])})
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh api graphql printed no review threads: {str(read).strip()[:200]}")
|
||||
"""A Failure naming the files where a reviewer's comment waits for an answer on pull request number, as `waiting_threads` finds them, or saying why they cannot be read. None where no thread waits."""
|
||||
waiting = waiting_threads(number, run)
|
||||
if isinstance(waiting, Failure):
|
||||
return waiting
|
||||
if not waiting:
|
||||
return None
|
||||
return Failure(f"a reviewer's comment on pull request {number} waits for an answer, in {', '.join(waiting)}; answer it in its thread, push any fix, and run chunk-pr merge again")
|
||||
|
||||
|
||||
def waiting_threads(number: str, run: Run) -> list[str] | Failure:
|
||||
"""The files where a reviewer's comment waits for an answer on pull request number, read once: those of each review thread nobody resolved, whose last comment someone other than the pull request's author wrote. So a thread the worker answered, as that author, waits for the reviewer again."""
|
||||
read = run(["gh", "api", "graphql", "-f", f"query={THREADS}", "-F", "owner={owner}", "-F", "repo={repo}", "-F", f"number={number}"])
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
try:
|
||||
pull = json.loads(read)["data"]["repository"]["pullRequest"]
|
||||
author = pull["author"]["login"]
|
||||
return sorted({thread["path"] for thread in pull["reviewThreads"]["nodes"] if not thread["isResolved"] and any(comment["author"]["login"] != author for comment in thread["comments"]["nodes"])})
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh api graphql printed no review threads: {read.strip()[:200]}")
|
||||
|
||||
+11
-8
@@ -45,7 +45,8 @@ merge merges the pull request in its turn: the chunks of one project on this
|
||||
Where the project's config sets `wait-for-approval = true` (`project-config --help`), it
|
||||
merges only once GitHub reports the pull request approved as its branch protection
|
||||
requires. It waits for that outside its turn, so other chunks merge meanwhile, and takes
|
||||
a new turn once approved.
|
||||
a new turn once approved. In a chunk do-epic runs, which sets CHUNK_PR_PARKS=1, it
|
||||
fails at once instead of waiting: do-epic parks the chunk until the approval comes.
|
||||
It stops, without merging, and says what to do before running it again:
|
||||
- on a merge conflict in step 1, naming the conflicted files, with the merge in progress;
|
||||
- where `larv-review scoring-run-owed` exits 1 after step 3, because the chunk owes a
|
||||
@@ -85,7 +86,7 @@ import sys
|
||||
import time
|
||||
import typing
|
||||
|
||||
from chunk_pr.approval import approval, approved, awaited, unanswered
|
||||
from chunk_pr.approval import approval, approved, awaited, left_to_park, PARKS, unanswered
|
||||
from forge.pulls import checks_passed, merged_pull, open_pull, required_checks
|
||||
from failure import Failure
|
||||
from forge.commands import Forge, forge_of, gitea, Run, runner
|
||||
@@ -119,7 +120,7 @@ def main(argv: list[str]) -> int:
|
||||
if argv[:1] in (["-h"], ["--help"]):
|
||||
print(__doc__.strip())
|
||||
return 0
|
||||
done = step(argv, runner(os.getcwd()), functools.cache(lambda: located(os.getcwd(), pathlib.Path.home())))
|
||||
done = step(argv, runner(os.getcwd()), functools.cache(lambda: located(os.getcwd(), pathlib.Path.home())), os.environ.get(PARKS) == "1")
|
||||
if isinstance(done, Failure):
|
||||
print(f"chunk-pr: {done.reason}", file=sys.stderr)
|
||||
return 1
|
||||
@@ -127,8 +128,8 @@ def main(argv: list[str]) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def step(argv: list[str], run: Run, config: Config) -> str | Failure:
|
||||
"""Run the step argv names: the line to print, or a Failure. A malformed argv doesn't run anything."""
|
||||
def step(argv: list[str], run: Run, config: Config, parks: bool = False) -> str | Failure:
|
||||
"""Run the step argv names: the line to print, or a Failure. A malformed argv doesn't run anything. parks is whether a merge fails at once where it would wait for an approval (`merged`)."""
|
||||
match argv:
|
||||
case ["open", title, *body] if len(body) <= 1 and not title.startswith("-"):
|
||||
return Failure("give a title of one line") if not one_line(title) else misfit(title, run, config) or opened(title, body[0] if body else "", run)
|
||||
@@ -138,7 +139,7 @@ def step(argv: list[str], run: Run, config: Config) -> str | Failure:
|
||||
return Failure("give a title of one line") if title and not one_line(title[0]) else (misfit(title[0], run, config) if title else None) or marked_ready(run, title[0] if title else None)
|
||||
case ["merge", *given]:
|
||||
finishes = finishes_of(given)
|
||||
return Failure(USAGE) if finishes is None else merged(run, config, finishes=finishes)
|
||||
return Failure(USAGE) if finishes is None else merged(run, config, finishes=finishes, parks=parks)
|
||||
case ["clean"]:
|
||||
return cleaned(run)
|
||||
return Failure(USAGE)
|
||||
@@ -304,8 +305,8 @@ def finishes_of(given: list[str]) -> list[list[str]] | None:
|
||||
return finishes
|
||||
|
||||
|
||||
def merged(run: Run, config: Config, sleep: typing.Callable[[float], None] = time.sleep, finishes: typing.Sequence[list[str]] = ()) -> str | Failure:
|
||||
"""Merge the branch's pull request in the project's merge queue. Each pass of the loop is one turn in the queue; a pull request that still wants its approval leaves the queue while it waits, so the next chunk merges meanwhile, and takes a new turn once approved."""
|
||||
def merged(run: Run, config: Config, sleep: typing.Callable[[float], None] = time.sleep, finishes: typing.Sequence[list[str]] = (), parks: bool = False) -> str | Failure:
|
||||
"""Merge the branch's pull request in the project's merge queue. Each pass of the loop is one turn in the queue; a pull request that still wants its approval leaves the queue while it waits, so the next chunk merges meanwhile, and takes a new turn once approved. Where parks, it doesn't wait for the approval: it fails at once, as `left_to_park` says."""
|
||||
found = pull(run)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
@@ -336,6 +337,8 @@ def merged(run: Run, config: Config, sleep: typing.Callable[[float], None] = tim
|
||||
taken = untaken or merge_turn(pull_request, finishes, loaded, run, sleep)
|
||||
if not isinstance(taken, AwaitsApproval):
|
||||
return taken if isinstance(taken, Failure) else taken + absent
|
||||
if parks:
|
||||
return left_to_park(number, run)
|
||||
# Out of the queue, so the project's other chunks merge while this one waits.
|
||||
waited = approved(number, run, sleep)
|
||||
if isinstance(waited, Failure):
|
||||
|
||||
@@ -377,6 +377,26 @@ class Merge(unittest.TestCase):
|
||||
self.assertEqual(merged(checkout, checkout.config, slept.append), "Merged pull request #12 (merge).")
|
||||
self.assertEqual(slept, [60])
|
||||
|
||||
def test_a_merge_that_parks_returns_at_once_while_the_branch_protection_still_wants_a_review(self):
|
||||
checkout = self.requiring_approval(GITHUB, "REVIEW_REQUIRED")
|
||||
failed = merged(checkout, checkout.config, self.fail, parks=True)
|
||||
self.assertEqual(failed.reason, "pull request 12 passed its checks and awaits an approval; end your run now, since do-epic parks the chunk and resumes this session once it is approved or a reviewer comments")
|
||||
self.assertEqual(checkout.ran.count(GH_DECISION), 1)
|
||||
self.assertEqual(step(["merge"], self.requiring_approval(GITHUB, "REVIEW_REQUIRED"), checkout.config, parks=True), failed)
|
||||
|
||||
def test_a_merge_that_parks_reports_a_reviewers_unanswered_comment_before_the_approval_it_awaits(self):
|
||||
checkout = self.requiring_approval(GITHUB, "REVIEW_REQUIRED")
|
||||
checkout.threads = [MergeQueue.thread("bin/asked.py", "reviewer")]
|
||||
failed = merged(checkout, checkout.config, self.fail, parks=True)
|
||||
self.assertEqual(failed.reason, "a reviewer's comment on pull request 12 waits for an answer, in bin/asked.py; answer it in its thread, push any fix, and run chunk-pr merge again")
|
||||
checkout = self.requiring_approval(GITHUB, "REVIEW_REQUIRED")
|
||||
checkout.answers[tuple(GH_THREADS)] = Failure("gh api graphql failed")
|
||||
self.assertEqual(merged(checkout, checkout.config, self.fail, parks=True), Failure("gh api graphql failed"))
|
||||
|
||||
def test_a_merge_that_parks_merges_an_approved_pull_request(self):
|
||||
checkout = self.requiring_approval()
|
||||
self.assertEqual(merged(checkout, checkout.config, self.fail, parks=True), "Merged pull request #12 (merge).")
|
||||
|
||||
def test_an_approval_that_never_comes_leaves_it_unmerged_after_90_minutes(self):
|
||||
checkout = self.requiring_approval(GITHUB, "REVIEW_REQUIRED")
|
||||
slept: list[float] = []
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
"""What a do-epic cycle's worker is told: its task, from the issue, and its first prompt. The resumed prompt and the rung sentences are the ones loop-claude's cycles use, naming do-epic."""
|
||||
"""What a do-epic cycle's worker is told: its task, from the issue, and its first prompt. The resumed prompt and the rung sentences are the ones loop-claude's cycles use, naming do-epic; a chunk that was parked is first told why it resumes."""
|
||||
import dataclasses
|
||||
|
||||
from do_epic.backend.protocol import EpicIssue
|
||||
from do_epic import findings
|
||||
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES
|
||||
@@ -7,6 +9,7 @@ import pathlib
|
||||
from model_routing.ladder import ProjectRung
|
||||
from loop_claude.cycle import prompts
|
||||
from model_routing.route import Route
|
||||
from agent_cycle.worker_prompts import WorkerPrompts
|
||||
|
||||
# The entry point's name as the worker, the judge and the human read it.
|
||||
DRIVER = "do-epic"
|
||||
@@ -36,3 +39,12 @@ NAMES = prompts.RecordNames(
|
||||
records_named="the epic's scoring and decisions, then the repo's working rules and decisions, then the human's rules for every project",
|
||||
)
|
||||
WORKER = prompts.worker_prompts(DRIVER, fresh, "", NAMES)
|
||||
|
||||
# What a worker does with a reviewer's feedback, as the decision asks (docs/decisions.md → `do-epic: drives one Jira epic …`, "Long gates").
|
||||
FEEDBACK = 'Where a reviewer left feedback, fix what is reasonable and push the fix, answer each other comment in its thread, and change the setup so our review would have caught the finding (~/.claude/CLAUDE.md → "Working loop", "Change the setup for each finding an outside reviewer raises on a PR, deterministic tools first"). Then merge the pull request as before.'
|
||||
|
||||
|
||||
def unparked(news: str) -> WorkerPrompts:
|
||||
"""The prompts of the first cycle after a parked chunk resumes: WORKER's, with each prompt that starts a worker opening with news, which says what the chunk waited on and what changed. The fresh prompt opens with it too, since a chunk the judge ruled finished before it parked has no session left to resume."""
|
||||
opening = f"{news} {FEEDBACK}\n\n"
|
||||
return dataclasses.replace(WORKER, fresh=lambda *task: opening + WORKER.fresh(*task), resumed=lambda *evidence: opening + WORKER.resumed(*evidence))
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
"""What the pull request of a do-epic chunk waits on, read once from its forge (docs/decisions.md → `do-epic: drives one Jira epic …`, "Long gates").
|
||||
|
||||
A gate is something only time or a person passes: checks that still run, an approval a reviewer has yet to give, or the new review of a reviewer whose requested changes the worker answered. While a chunk's pull request waits on one, its worker doesn't have anything to do, so do-epic parks the chunk (`parked`). Anything else is news the worker acts on."""
|
||||
import dataclasses
|
||||
import json
|
||||
import typing
|
||||
|
||||
from forge.pulls import CheckFailed, checks_now, open_pull, Pending, required_checks
|
||||
from failure import Failure
|
||||
from forge.commands import forge_of, gitea, Run
|
||||
from chunk_pr.approval import waiting_threads
|
||||
|
||||
# What a pull request waits on, as the words after "awaits".
|
||||
CHECKS = "its checks"
|
||||
APPROVAL = "an approval"
|
||||
NEW_REVIEW = "a new review"
|
||||
# What `review_decision` returns in place of GitHub's `CHANGES_REQUESTED` once the worker answered those changes: GitHub keeps its decision until the reviewer reviews again.
|
||||
ANSWERED = "CHANGES_ANSWERED"
|
||||
# What the title of a draft opens with on Gitea, in any case: Gitea's default `WORK_IN_PROGRESS_PREFIXES`.
|
||||
GITEA_DRAFT = ("WIP:", "[WIP]")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Gate:
|
||||
"""The gate a chunk's open pull request waits on."""
|
||||
# The chunk's worktree, where the pull request's branch is checked out.
|
||||
worktree: str
|
||||
# The pull request's number.
|
||||
pull: str
|
||||
# CHECKS, APPROVAL or NEW_REVIEW.
|
||||
on: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Open:
|
||||
"""A chunk that waits on no gate, so its worker goes on."""
|
||||
# Why, as a sentence without its full stop, for the worker of a chunk that was parked.
|
||||
news: str
|
||||
|
||||
|
||||
def gate(branch: str, checkout: str, at: typing.Callable[[str], Run]) -> Gate | Open | Failure:
|
||||
"""What the chunk on branch waits on, in a worktree of checkout; at gives the runner of commands in a folder. A Failure where git or the forge cannot be read, which doesn't say anything about the gate.
|
||||
|
||||
A draft never waits on a gate: its worker still works on it. On Gitea only the checks are a gate, since chunk-pr reads approvals on GitHub only (`chunk_pr.approval.awaited`). Changes a reviewer requested are news until the worker answered them (`answered`); from then on the pull request awaits a new review.
|
||||
"""
|
||||
worktree = worktree_of(branch, at(checkout))
|
||||
if not isinstance(worktree, str):
|
||||
return worktree or Open(f"no worktree of {checkout} has the branch {branch} checked out")
|
||||
run = at(worktree)
|
||||
read = [run(["git", *args]) for args in (["remote", "get-url", "origin"], ["rev-parse", "HEAD"], ["symbolic-ref", "--short", "refs/remotes/origin/HEAD"])]
|
||||
failed = next((answer for answer in read if isinstance(answer, Failure)), None)
|
||||
if failed is not None:
|
||||
return failed
|
||||
origin, head, base = (str(answer).strip() for answer in read)
|
||||
forge = forge_of(origin)
|
||||
pull = open_pull(forge, branch, head, run)
|
||||
if not isinstance(pull, str):
|
||||
return pull or Open(f"no pull request from origin's {branch} is open at {head[:12]}; it may be merged or closed already")
|
||||
decision = review_decision(pull, head, run) if forge == "gh" else gitea_decision(pull, run)
|
||||
if not isinstance(decision, str):
|
||||
return decision or Open(f"pull request #{pull} is a draft")
|
||||
required = required_checks(forge, base.removeprefix("origin/"), run)
|
||||
checks = required if isinstance(required, Failure) else checks_now(forge, pull, head, required, run)
|
||||
if isinstance(checks, CheckFailed):
|
||||
return Open(f"{checks.reason}; fix it and push")
|
||||
if isinstance(checks, Failure):
|
||||
return checks
|
||||
if isinstance(checks, Pending):
|
||||
return Gate(worktree, pull, CHECKS)
|
||||
if decision == "CHANGES_REQUESTED":
|
||||
return Open(f"a reviewer requested changes on pull request #{pull}")
|
||||
waiting = waiting_threads(pull, run) if forge == "gh" else []
|
||||
if isinstance(waiting, Failure):
|
||||
return waiting
|
||||
if waiting:
|
||||
return Open(f"a reviewer's comment on pull request #{pull} waits for an answer, in {', '.join(waiting)}")
|
||||
if decision in (ANSWERED, "REVIEW_REQUIRED"):
|
||||
return Gate(worktree, pull, NEW_REVIEW if decision == ANSWERED else APPROVAL)
|
||||
return Open(f"pull request #{pull} passed its checks" + (" and is approved" if decision == "APPROVED" else ""))
|
||||
|
||||
|
||||
def worktree_of(branch: str, run: Run) -> str | None | Failure:
|
||||
"""The worktree, of the checkout run works in, that has branch checked out; None where none has."""
|
||||
listed = run(["git", "worktree", "list", "--porcelain"])
|
||||
if isinstance(listed, Failure):
|
||||
return listed
|
||||
for block in listed.split("\n\n"):
|
||||
lines = block.splitlines()
|
||||
if f"branch refs/heads/{branch}" in lines and lines[0].startswith("worktree "):
|
||||
return lines[0].removeprefix("worktree ")
|
||||
return None
|
||||
|
||||
|
||||
def review_decision(pull: str, head: str, run: Run) -> str | None | Failure:
|
||||
"""GitHub's review decision on the pull request, whose head commit is head, read once: `APPROVED`, `REVIEW_REQUIRED`, `CHANGES_REQUESTED`, or empty where its branch protection asks for no review. ANSWERED in place of `CHANGES_REQUESTED` once the worker answered those changes. None where it is a draft."""
|
||||
viewed = run(["gh", "pr", "view", pull, "--json", "isDraft,reviewDecision,author,reviews,comments"])
|
||||
if isinstance(viewed, Failure):
|
||||
return viewed
|
||||
try:
|
||||
parsed = json.loads(viewed)
|
||||
if parsed["isDraft"]:
|
||||
return None
|
||||
decision = str(parsed["reviewDecision"] or "")
|
||||
return ANSWERED if decision == "CHANGES_REQUESTED" and answered(parsed, head) else decision
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh pr view printed no review decision: {viewed.strip()[:200]}")
|
||||
|
||||
|
||||
def answered(pull: dict[str, typing.Any], head: str) -> bool:
|
||||
"""Whether the worker answered the newest review that requested changes on pull, as `gh pr view --json author,reviews,comments` prints it, whose head commit is head: the head is no longer the commit that review was made on, or the pull request's author commented or replied in a thread after it. GitHub lists a reply in a thread as a review of its own. False where it lists no review that requested changes. Raises KeyError or TypeError where pull has another shape.
|
||||
|
||||
The review names its commit, so no clock tells whether the worker pushed: a commit's date says when it was made, never when it was pushed.
|
||||
"""
|
||||
asked = max((review for review in pull["reviews"] if review["state"] == "CHANGES_REQUESTED"), key=lambda review: review["submittedAt"], default=None)
|
||||
if asked is None:
|
||||
return False
|
||||
author = pull["author"]["login"]
|
||||
# A review the author has yet to submit has no time, and answers nobody.
|
||||
said = [review["submittedAt"] or "" for review in pull["reviews"] if review["author"]["login"] == author] + [comment["createdAt"] for comment in pull["comments"] if comment["author"]["login"] == author]
|
||||
return asked["commit"]["oid"] != head or any(at > asked["submittedAt"] for at in said)
|
||||
|
||||
|
||||
def gitea_decision(pull: str, run: Run) -> str | None | Failure:
|
||||
"""What stands in for `review_decision` on Gitea, read once: empty, since no review is a gate there, and None where the pull request is a draft, which Gitea tells by its title (GITEA_DRAFT)."""
|
||||
viewed = gitea(run, f"repos/{{owner}}/{{repo}}/pulls/{pull}")
|
||||
if isinstance(viewed, Failure):
|
||||
return viewed
|
||||
try:
|
||||
return None if viewed["title"].upper().startswith(GITEA_DRAFT) else ""
|
||||
except (KeyError, TypeError, AttributeError):
|
||||
return Failure(f"tea api printed no title of pull request {pull}: {str(viewed)[:200]}")
|
||||
+118
-19
@@ -1,13 +1,15 @@
|
||||
"""The do-epic command (docs/decisions.md → `do-epic: drives one Jira epic …`).
|
||||
|
||||
`do-epic <EPIC-KEY>` starts each of the epic's chunks that may start now, each in a window of the epic's tmux session, as `scheduler` picks them, and puts that session on the terminal. Each window runs `do-epic --chunk <ISSUE-KEY> <EPIC-KEY>`: the cycles of one issue's chunk, run by the code that runs loop-claude's (`agent_cycle`), until the judge rules it finished. Then it starts the chunks that may start now, and its window closes."""
|
||||
`do-epic <EPIC-KEY>` starts each of the epic's chunks that may start now, each in a window of the epic's tmux session, as `scheduler` picks them, and puts that session on the terminal. Each window runs `do-epic --chunk <ISSUE-KEY> <EPIC-KEY>`: the cycles of one issue's chunk, run by the code that runs loop-claude's (`agent_cycle`), until the judge rules it finished. Then it starts the chunks that may start now, and its window closes.
|
||||
|
||||
After each cycle of a chunk in a repo, the window reads what the chunk's pull request waits on (`gates`). While that is a gate only time or a person passes, the chunk is parked (`parked`): the window frees its running slot and polls the forge, and the chunk's next cycle starts once the worker has something to do and a slot is free. `do-epic <EPIC-KEY> --status` prints what each open issue does."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from loop_claude.loop.chunk_start import archived_route, routed
|
||||
from do_epic import chunk_prompt, findings, scheduler
|
||||
from do_epic import chunk_prompt, findings, gates, parked, scheduler
|
||||
from do_epic.repo_pick import ClaudeWork, HumanTask, recorded, Repo
|
||||
from loop_claude.loop.close import closed_cycle
|
||||
from loop_claude.project.records import CONSEQUENCE, read_records
|
||||
@@ -31,13 +33,14 @@ from loop_state.paths import Paths
|
||||
from transcript_archive.project_key import project_key
|
||||
from model_routing.ladder_file import project_ladder_file, read_ladder, SHIPPED
|
||||
from loop_claude.project.premises import read_premises
|
||||
from forge.commands import Run, runner
|
||||
from claude_code.session import session_named
|
||||
from loop_claude.loop.main import setup_version
|
||||
from agent_cycle.stop import Stop
|
||||
from agent_cycle.direct_message import tell
|
||||
from agent_cycle import tmux
|
||||
|
||||
HELP = f"""usage: do-epic <EPIC-KEY>
|
||||
HELP = f"""usage: do-epic <EPIC-KEY> [--status]
|
||||
{findings.USAGE}
|
||||
|
||||
Runs the chunks of one Jira epic, such as ABC-12, each in its own window of the tmux session epic-<EPIC-KEY>.
|
||||
@@ -45,10 +48,19 @@ An open issue of the epic runs as a chunk once it has a label and no issue block
|
||||
The label names the issue's repo, or says the work changes no repo.
|
||||
As many run at once as the epic's description allows with a line such as `**Max parallel chunks:** 2`, one where it has none.
|
||||
A repo takes as many chunks as its `parallel-chunks`, and none while loop-claude runs chunks in it.
|
||||
A chunk whose worker ends its run while its pull request waits on its checks, an approval or a new review
|
||||
is parked: it frees its place for the next chunk, and its window's name says what it awaits. Its worker goes
|
||||
on once the checks pass, the approval comes or a reviewer comments, and a place is free; until one is, the
|
||||
name says `awaits a slot`. A parked chunk survives a reboot: run do-epic <EPIC-KEY> again to go on polling.
|
||||
--status starts no chunk: it prints each open issue with whether its chunk runs, or why it waits.
|
||||
{findings.HELP}
|
||||
The Jira site and its credentials are set in ~/.claude-projects/do-epic/config.toml."""
|
||||
# Under the home folder, where a chunk's cycles are archived, a folder per epic. It is not loop-claude's archive, since that archive's readers, such as judge-replay and the routing figures, read loop-claude's prompts.
|
||||
ARCHIVE = ".claude/do-epic-transcripts"
|
||||
# The flag that prints what each open issue does.
|
||||
STATUS_FLAG = "--status"
|
||||
# How often a parked chunk reads its pull request's gate: a reviewer answers in hours, and each read is three or four calls to the forge.
|
||||
POLL_SECONDS = 60
|
||||
|
||||
# The store of an epic's issues and its record, for the epic with this key.
|
||||
Backend = typing.Callable[[DoEpicConfig, str], tuple[EpicStore, EpicRecord]]
|
||||
@@ -70,15 +82,17 @@ class EpicRun:
|
||||
env: typing.Mapping[str, str]
|
||||
launcher: pathlib.Path
|
||||
effects: Effects
|
||||
# The runner of git's and the forge's commands in a folder.
|
||||
forge_at: typing.Callable[[str], Run] = runner
|
||||
|
||||
@property
|
||||
def session(self) -> str:
|
||||
return tmux.session_name(tmux.EPIC, self.key)
|
||||
|
||||
|
||||
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend = jira_backend, stdin: typing.TextIO = sys.stdin) -> int:
|
||||
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend = jira_backend, stdin: typing.TextIO = sys.stdin, forge_at: typing.Callable[[str], Run] = runner) -> int:
|
||||
try:
|
||||
stop = run(argv, env, paths, launcher, effects, backend, stdin)
|
||||
stop = run(argv, env, paths, launcher, effects, backend, stdin, forge_at)
|
||||
except KeyboardInterrupt:
|
||||
stop = Stop(130, "")
|
||||
except Exception as crash:
|
||||
@@ -92,8 +106,10 @@ def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher:
|
||||
return stop.code
|
||||
|
||||
|
||||
def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend, stdin: typing.TextIO) -> Stop:
|
||||
def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend, stdin: typing.TextIO, forge_at: typing.Callable[[str], Run]) -> Stop:
|
||||
chunk, words = (argv[1], argv[2:]) if len(argv) >= 2 and argv[0] == scheduler.CHUNK_FLAG else (None, argv)
|
||||
status = chunk is None and words[1:] == [STATUS_FLAG]
|
||||
words = words[:1] if status else words
|
||||
# What the words after the epic's key write into the epic, where they are a filing or a decision.
|
||||
writes = findings.parsed(words[1:]) if chunk is None else None
|
||||
if (len(words) != 1 and not writes) or not ISSUE_KEY.fullmatch(words[0]) or (chunk is not None and not ISSUE_KEY.fullmatch(chunk)):
|
||||
@@ -107,10 +123,19 @@ def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher:
|
||||
if isinstance(config, Failure):
|
||||
return Stop(1, config.reason)
|
||||
store, record = backend(config, words[0])
|
||||
epic = EpicRun(words[0], config, store, record, paths, env, launcher, effects)
|
||||
epic = EpicRun(words[0], config, store, record, paths, env, launcher, effects, forge_at)
|
||||
if status:
|
||||
return status_printed(epic)
|
||||
return windows_opened(epic) if chunk is None else chunk_run(chunk, epic)
|
||||
|
||||
|
||||
def status_printed(epic: EpicRun) -> Stop:
|
||||
"""What `do-epic <EPIC-KEY> --status` does: prints a line per open issue, and doesn't start anything."""
|
||||
issues = epic.store.read()
|
||||
lines = issues if isinstance(issues, Failure) else scheduler.status(epic.key, epic.paths.home, issues, epic.effects.tmux)
|
||||
return Stop(1, lines.reason) if isinstance(lines, Failure) else Stop(0, "\n".join(lines) or f"the epic {epic.key} has no open issue")
|
||||
|
||||
|
||||
def epic_written(key: str, writes: tuple[str, str, str | None], paths: Paths, backend: Backend, stdin: typing.TextIO) -> Stop:
|
||||
"""Files a finding in the epic, or records a decision in its description. It needs no tmux, so a chunk's worker may run it anywhere."""
|
||||
config = load(paths.home)
|
||||
@@ -142,6 +167,12 @@ def chunks_started(epic: EpicRun, own: str | None) -> scheduler.Started | Failur
|
||||
issues = epic.store.read()
|
||||
if isinstance(issues, Failure):
|
||||
return issues
|
||||
scheduled = scheduled_epic(epic)
|
||||
return scheduled if isinstance(scheduled, Failure) else scheduler.start_chunks(scheduled, issues, own, epic.effects.tmux)
|
||||
|
||||
|
||||
def scheduled_epic(epic: EpicRun) -> scheduler.Epic | Failure:
|
||||
"""The epic as scheduler limits its chunks: with its cap, read afresh, and each repo's checkout."""
|
||||
cap = epic.record.parallel_chunks()
|
||||
if isinstance(cap, Failure):
|
||||
return cap
|
||||
@@ -155,7 +186,7 @@ def chunks_started(epic: EpicRun, own: str | None) -> scheduler.Started | Failur
|
||||
seen.append(found.checkout)
|
||||
return real(found.checkout), found.config.parallel_chunks
|
||||
|
||||
return scheduler.start_chunks(scheduler.Epic(epic.key, cap, located, epic.paths.home, epic.launcher, epic.env), issues, own, epic.effects.tmux)
|
||||
return scheduler.Epic(epic.key, cap, located, epic.paths.home, epic.launcher, epic.env)
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -183,7 +214,7 @@ def real(checkout: pathlib.Path) -> pathlib.Path:
|
||||
|
||||
|
||||
def chunk_run(key: str, epic: EpicRun) -> Stop:
|
||||
"""Runs the cycles of the chunk of the issue called key until one stops the loop, or the judge rules the chunk finished."""
|
||||
"""Runs the cycles of the chunk of the issue called key until one stops the loop, or the judge rules the chunk finished. A chunk in a repo parks after each cycle that leaves its pull request waiting on a gate, and before its first where it was parked when its window last ended, as at a reboot."""
|
||||
pane = epic.env.get("TMUX_PANE")
|
||||
if not pane:
|
||||
return Stop(1, f"{scheduler.CHUNK_FLAG} is how do-epic starts itself in a tmux window; run do-epic <EPIC-KEY>")
|
||||
@@ -193,13 +224,17 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
|
||||
place = chunk_root(issue, epic)
|
||||
if isinstance(place, Stop):
|
||||
return place
|
||||
root, in_repo = place
|
||||
root, repo = place
|
||||
session = session_named(f"A {epic.key} do-epic {key}", epic.paths.claude_code_transcripts)
|
||||
loop = Loop(root=root, session=session, window=tmux.Window(epic.session, key, pane), paths=epic.paths, env=epic.env, launcher=epic.launcher, effects=epic.effects)
|
||||
loop_run = LoopRun(0, root, "", "", chunk=key)
|
||||
# What a chunk that was parked is told as it resumes; empty for a chunk that wasn't.
|
||||
news = parked_through(repo, loop, epic, starts=True) if repo is not None else ""
|
||||
while True:
|
||||
if isinstance(news, Stop):
|
||||
return news
|
||||
loop_run = dataclasses.replace(loop_run, cycle_number=loop_run.cycle_number + 1)
|
||||
cycle = next_cycle(loop_run, loop, in_repo, epic)
|
||||
cycle = next_cycle(loop_run, loop, repo is not None, epic, news)
|
||||
if isinstance(cycle, Stop):
|
||||
return cycle
|
||||
loop_run = dataclasses.replace(loop_run, task=cycle.task)
|
||||
@@ -210,10 +245,74 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
|
||||
if isinstance(closed, Stop):
|
||||
return closed
|
||||
loop_run = closed
|
||||
if isinstance(concluded.verdict, Finished):
|
||||
news = parked_through(repo, loop, epic, starts=False) if repo is not None else ""
|
||||
# A chunk that was parked has work again, whatever the judge ruled before its pull request passed the gate.
|
||||
if isinstance(concluded.verdict, Finished) and news == "":
|
||||
return handed_on(key, epic)
|
||||
|
||||
|
||||
def parked_through(repo: str, loop: Loop, epic: EpicRun, starts: bool) -> str | Stop:
|
||||
"""Keeps the chunk in loop's window parked. Its work is in the repo called repo. It stays parked for as long as its pull request waits on a gate, and returns once its worker has something to do: what to tell that worker, or empty where the chunk was never parked. starts says the window just started, where the chunk is parked only if its file says so: the forge isn't read for a chunk that starts afresh.
|
||||
|
||||
A chunk parks in three steps: its file is written, its window is shown as parked, and the epic's next chunks start in the slot it frees. It resumes in the reverse order: `scheduler.resume` gives it a running slot, then its file is removed. So a window that ends in between finds the chunk parked when it starts again.
|
||||
"""
|
||||
key, folder = loop.window.name, epic.paths.home / scheduler.STATE / epic.key
|
||||
recorded_as = parked.parked_chunks(folder)
|
||||
if isinstance(recorded_as, Failure):
|
||||
return Stop(1, recorded_as.reason)
|
||||
if starts and key not in recorded_as:
|
||||
return ""
|
||||
# What the chunk is parked on; None while it isn't parked.
|
||||
on = recorded_as[key].on if key in recorded_as else None
|
||||
# What the window's name says the chunk awaits, and the last reason said for staying parked.
|
||||
shown = said = None
|
||||
while True:
|
||||
now = gates.gate(key, loop.root, epic.forge_at)
|
||||
if on is None and not isinstance(now, gates.Gate):
|
||||
if isinstance(now, Failure):
|
||||
say(f"cannot tell whether the chunk's pull request waits on a gate, so the chunk isn't parked: {now.reason}", epic.effects, stdout=True)
|
||||
return ""
|
||||
stays = None
|
||||
if isinstance(now, gates.Gate) and now.on != on:
|
||||
failed = parked.park(folder, parked.Parked(key, now.worktree, now.pull, str(loop.session.id), now.on))
|
||||
if failed is not None:
|
||||
return Stop(1, failed.reason)
|
||||
say(f"parked: pull request #{now.pull} awaits {now.on}", epic.effects, stdout=True)
|
||||
elif isinstance(now, Failure):
|
||||
stays = f"cannot read the gate: {now.reason}"
|
||||
elif isinstance(now, gates.Open):
|
||||
full = resumed(repo, loop, epic)
|
||||
if full is None:
|
||||
failed = parked.unpark(folder, key)
|
||||
return Stop(1, failed.reason) if failed is not None else f"{DRIVER_PARKED} its pull request awaited {on}. It resumes now: {now.news}."
|
||||
stays = f"{now.news}, but the chunk stays parked: {full}"
|
||||
awaits = parked.SLOT if isinstance(now, gates.Open) else now.on if isinstance(now, gates.Gate) else shown or on
|
||||
if awaits != shown:
|
||||
tmux.park(loop.window, str(awaits), epic.effects.tmux)
|
||||
shown = awaits
|
||||
if on is None:
|
||||
# Once the window shows as parked, so the chunks that start count it as running nowhere.
|
||||
started = chunks_started(epic, key)
|
||||
say(f"could not start the next chunk: {started.reason}" if isinstance(started, Failure) else f"started {', '.join(started.started) or 'no other'}", epic.effects, stdout=True)
|
||||
on = now.on if isinstance(now, gates.Gate) else on
|
||||
if stays is not None and stays != said:
|
||||
say(f"{stays}; trying again every {POLL_SECONDS} seconds", epic.effects, stdout=True)
|
||||
said = stays
|
||||
epic.effects.sleep(POLL_SECONDS)
|
||||
|
||||
|
||||
# How the news for a chunk that was parked opens.
|
||||
DRIVER_PARKED = f"{chunk_prompt.DRIVER} parked the chunk while"
|
||||
|
||||
|
||||
def resumed(repo: str, loop: Loop, epic: EpicRun) -> str | None:
|
||||
"""scheduler.resume for the parked chunk in loop's window, whose work is in the repo called repo, on the epic's cap and repos read afresh: None once it holds a running slot, else why it stays parked."""
|
||||
scheduled = scheduled_epic(epic)
|
||||
if isinstance(scheduled, Failure):
|
||||
return scheduled.reason
|
||||
return scheduler.resume(scheduled, repo, loop.window, epic.effects.tmux)
|
||||
|
||||
|
||||
def issue_read(key: str, epic: EpicRun) -> EpicIssue | Stop:
|
||||
"""The epic's issue called key, read afresh."""
|
||||
issues = epic.store.read()
|
||||
@@ -223,19 +322,19 @@ def issue_read(key: str, epic: EpicRun) -> EpicIssue | Stop:
|
||||
return found[0] if found else Stop(1, f"{key} is no issue of the epic {epic.key}")
|
||||
|
||||
|
||||
def chunk_root(issue: EpicIssue, epic: EpicRun) -> tuple[str, bool] | Stop:
|
||||
"""The folder the issue's chunk works in, and whether it is a repo's checkout: the epic's folder holds work without a repo. The chunk keeps both to its end, whatever the issue's labels become. The stop where the labels make the issue no chunk."""
|
||||
def chunk_root(issue: EpicIssue, epic: EpicRun) -> tuple[str, str | None] | Stop:
|
||||
"""The folder the issue's chunk works in, and the name of the repo whose checkout it is: None for the epic's folder, which holds work without a repo. The chunk keeps both to its end, whatever the issue's labels become. The stop where the labels make the issue no chunk."""
|
||||
pick = recorded(issue)
|
||||
if isinstance(pick, Failure):
|
||||
return Stop(1, pick.reason)
|
||||
if isinstance(pick, ClaudeWork):
|
||||
folder = epic.paths.home / scheduler.STATE / epic.key
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
return str(folder), False
|
||||
return str(folder), None
|
||||
if not isinstance(pick, Repo):
|
||||
return Stop(1, f"{issue.key} is no chunk: {'it is a human\'s task' if isinstance(pick, HumanTask) else 'no label names its repo'}")
|
||||
found = locate(pick.name, [], epic.config, epic.paths.home, epic.effects.git)
|
||||
return Stop(1, found.reason) if isinstance(found, Failure) else (str(real(found.checkout)), True)
|
||||
return Stop(1, found.reason) if isinstance(found, Failure) else (str(real(found.checkout)), pick.name)
|
||||
|
||||
|
||||
def handed_on(key: str, epic: EpicRun) -> Stop:
|
||||
@@ -247,8 +346,8 @@ def handed_on(key: str, epic: EpicRun) -> Stop:
|
||||
return Stop(0, "")
|
||||
|
||||
|
||||
def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun) -> Cycle | Stop:
|
||||
"""The cycle loop_run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
|
||||
def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun, news: str = "") -> Cycle | Stop:
|
||||
"""The cycle loop_run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read. news is what `parked_through` tells the worker of a chunk that was parked, and opens its prompt. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
|
||||
issue = issue_read(loop.window.name, epic)
|
||||
if isinstance(issue, Stop):
|
||||
return issue
|
||||
@@ -274,7 +373,7 @@ def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun) -> C
|
||||
project = ladder.project_rung(models.worker)
|
||||
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, loop), lambda: RoutingIssue(issue) if "L" in issue.scores else None)
|
||||
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", epic.effects, stdout=True)
|
||||
return Cycle(config=setup_version(epic.paths.claude_dir, epic.effects.git), effects=epic.effects, facts="", judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=chunk_prompt.task(issue, epic.key, loop.root if in_repo else None, epic.launcher, language or epic.config.language), prompts=chunk_prompt.WORKER, taken_in=lambda entries: TakenIn(), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
|
||||
return Cycle(config=setup_version(epic.paths.claude_dir, epic.effects.git), effects=epic.effects, facts="", judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=chunk_prompt.task(issue, epic.key, loop.root if in_repo else None, epic.launcher, language or epic.config.language), prompts=chunk_prompt.unparked(news) if news else chunk_prompt.WORKER, taken_in=lambda entries: TakenIn(), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
|
||||
|
||||
|
||||
def repoless_models(epic: EpicRun) -> Models:
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
"""The parked chunks of an epic: each keeps its worktree, pull request and session while that pull request waits on a long gate, in a file that survives a reboot (docs/decisions.md → `do-epic: drives one Jira epic …`, "Long gates").
|
||||
|
||||
A chunk is parked from the moment its file is written until the file is removed, whatever its window does: `scheduler` starts a window again for a parked chunk that has none, and that window goes on polling."""
|
||||
import dataclasses
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
|
||||
from failure import Failure
|
||||
|
||||
# Under the epic's folder, the folder that holds one file per parked chunk, named for its issue.
|
||||
FOLDER = "parked"
|
||||
# What a parked chunk awaits, in its window's name, once its worker has something to do but no running slot is free.
|
||||
SLOT = "a slot"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Parked:
|
||||
"""A parked chunk."""
|
||||
# The key of the issue it works.
|
||||
issue: str
|
||||
worktree: str
|
||||
# Its pull request's number.
|
||||
pull: str
|
||||
# The ID of its worker's claude session, which `claude --resume` takes.
|
||||
session: str
|
||||
# What its pull request waits on, as the words after "awaits" (`gates.CHECKS`, `gates.APPROVAL`, `gates.NEW_REVIEW`).
|
||||
on: str
|
||||
|
||||
def waits(self, shown: str) -> str:
|
||||
"""Why the chunk doesn't run, for the human. shown is what the name of its live window says it awaits, which is newer than the file's `on`; empty where no window lives."""
|
||||
if shown == SLOT:
|
||||
return f"it is parked: its worker has work again on pull request #{self.pull} and awaits {SLOT}"
|
||||
return f"it is parked: its pull request #{self.pull} awaits {shown or self.on}"
|
||||
|
||||
|
||||
def file_of(folder: pathlib.Path, issue: str) -> pathlib.Path:
|
||||
return folder / FOLDER / f"{issue}.json"
|
||||
|
||||
|
||||
def park(folder: pathlib.Path, chunk: Parked) -> Failure | None:
|
||||
"""Records chunk as parked under the epic's folder, in place of what was recorded for its issue. The file is whole or absent, whenever the machine stops."""
|
||||
file = file_of(folder, chunk.issue)
|
||||
partial = file.with_suffix(".partial")
|
||||
try:
|
||||
file.parent.mkdir(parents=True, exist_ok=True)
|
||||
partial.write_text(json.dumps(dataclasses.asdict(chunk), indent="\t") + "\n", encoding="utf-8")
|
||||
os.replace(partial, file)
|
||||
except OSError as failure:
|
||||
return Failure(f"cannot record the parked chunk in {file}: {failure.strerror}")
|
||||
return None
|
||||
|
||||
|
||||
def unpark(folder: pathlib.Path, issue: str) -> Failure | None:
|
||||
"""Records that the issue's chunk is parked no longer."""
|
||||
try:
|
||||
file_of(folder, issue).unlink(missing_ok=True)
|
||||
except OSError as failure:
|
||||
return Failure(f"cannot remove {file_of(folder, issue)}: {failure.strerror}")
|
||||
return None
|
||||
|
||||
|
||||
def parked_chunks(folder: pathlib.Path) -> dict[str, Parked] | Failure:
|
||||
"""The epic's parked chunks, by issue. A Failure where a file cannot be read as one, since a chunk taken as not parked would start a second time."""
|
||||
found: dict[str, Parked] = {}
|
||||
for file in sorted((folder / FOLDER).glob("*.json")):
|
||||
try:
|
||||
fields = json.loads(file.read_text(encoding="utf-8"))
|
||||
chunk = Parked(**fields)
|
||||
except (OSError, ValueError, TypeError) as failure:
|
||||
return Failure(f"cannot read the parked chunk in {file}: {failure}; fix or remove the file")
|
||||
if chunk.issue != file.stem or not all(isinstance(value, str) for value in fields.values()):
|
||||
return Failure(f"cannot read the parked chunk in {file}: its fields are not those of {file.stem}'s chunk; fix or remove the file")
|
||||
found[chunk.issue] = chunk
|
||||
return found
|
||||
+59
-10
@@ -1,6 +1,6 @@
|
||||
"""Which of an epic's issues run as chunks now, each in its own window of the epic's tmux session (docs/decisions.md → `do-epic: drives one Jira epic …`).
|
||||
|
||||
Three limits hold a chunk back: the epic's cap of chunks at once, its repo's `parallel-chunks`, counted across every epic, and a loop-claude that runs chunks in its repo. do-epic checks a repo and starts a chunk in it while holding the lock loop-claude holds when it starts one (`chunk_windows.starting`), so the two never start chunks in one checkout together."""
|
||||
Three limits hold a chunk back: the epic's cap of chunks at once, its repo's `parallel-chunks`, counted across every epic, and a loop-claude that runs chunks in its repo. A parked chunk (`parked`) counts toward none of them: its window only polls the forge, and `resume` gives it a running slot under the same limits once its worker has something to do. do-epic checks a repo and starts a chunk in it while holding the lock loop-claude holds when it starts one (`chunk_windows.starting`), so the two never start chunks in one checkout together."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
@@ -11,6 +11,8 @@ from do_epic.backend.protocol import EpicIssue
|
||||
from failure import Failure
|
||||
from do_epic.repo_pick import HumanTask, recorded, Repo
|
||||
from project_config.settings import PARALLEL_CHUNKS
|
||||
from do_epic.parked import parked_chunks
|
||||
from chunk_pr.approval import PARKS
|
||||
from transcript_archive.project_key import project_key
|
||||
from do_epic.split import SPLIT_LABEL
|
||||
from agent_cycle import tmux
|
||||
@@ -107,24 +109,32 @@ def refusal(repo: str, room: RepoRoom) -> str | None:
|
||||
|
||||
|
||||
def start_chunks(epic: Epic, issues: list[EpicIssue], own: str | None, run: tmux.Tmux) -> Started | Failure:
|
||||
"""Starts a do-epic in a new window for each of the epic's chunks that may start now. own is the issue of the chunk that asks, which has finished: it counts as running nowhere, and no stopped chunk starts again. Where own is None, the human's `do-epic <EPIC-KEY>` asks, and each stopped chunk starts again. A Failure says why a window could not start; those before it run."""
|
||||
"""Starts a do-epic in a new window for each of the epic's chunks that may start now. own is the issue of the chunk that asks, which has finished or parked: it counts as running nowhere, and no stopped chunk starts again. Where own is None, the human's `do-epic <EPIC-KEY>` asks, and each stopped chunk starts again. A parked chunk waits in its window; one without a live window, as after a reboot, starts again as a stopped chunk does, whatever the limits, since it goes on parked. A Failure says why a window could not start; those before it run."""
|
||||
session = tmux.session_name(tmux.EPIC, epic.key)
|
||||
found, waits = chunks(issues)
|
||||
started: list[str] = []
|
||||
parked = parked_chunks(epic.folder)
|
||||
if isinstance(parked, Failure):
|
||||
return parked
|
||||
try:
|
||||
with starting(epic.folder):
|
||||
windows = tmux.listed(session, run)
|
||||
running = {window.name for window in windows if not window.dead} - {own}
|
||||
stopped = {window.name for window in windows if window.dead} - running
|
||||
live = {window.name: window for window in windows if not window.dead and window.name != own}
|
||||
running = {window.name for window in windows if not window.dead and not window.parked} - {own}
|
||||
stopped = {window.name for window in windows if window.dead} - set(live)
|
||||
# The windows this call started that take a running slot.
|
||||
taken = 0
|
||||
for chunk in found:
|
||||
if chunk.issue in running or chunk.issue == own:
|
||||
continue
|
||||
if own and chunk.issue in stopped:
|
||||
if chunk.issue in parked and chunk.issue in live:
|
||||
waits.append(Waits(chunk.issue, parked[chunk.issue].waits(live[chunk.issue].awaits)))
|
||||
elif own and (chunk.issue in stopped or chunk.issue in parked):
|
||||
waits.append(Waits(chunk.issue, f"its window stopped; run `do-epic {epic.key}` to start it again"))
|
||||
elif len(running) + len(started) >= epic.cap:
|
||||
elif chunk.issue not in parked and len(running) + taken >= epic.cap:
|
||||
waits.append(Waits(chunk.issue, f"the epic runs as many chunks as its `{CAP}` line allows: {epic.cap}"))
|
||||
else:
|
||||
now = start_chunk(epic, chunk, own, windows, run)
|
||||
now = start_chunk(epic, chunk if chunk.issue not in parked else Chunk(chunk.issue, None), own, windows, run)
|
||||
if isinstance(now, Failure):
|
||||
return now
|
||||
if isinstance(now, str):
|
||||
@@ -132,15 +142,16 @@ def start_chunks(epic: Epic, issues: list[EpicIssue], own: str | None, run: tmux
|
||||
else:
|
||||
windows = now
|
||||
started.append(chunk.issue)
|
||||
taken += chunk.issue not in parked
|
||||
except OSError as failure:
|
||||
return Failure(f"cannot lock {failure.filename or epic.folder / LOCK}: {failure.strerror}")
|
||||
return Started(started, sorted(waits, key=lambda wait: [issue.key for issue in issues].index(wait.issue)))
|
||||
|
||||
|
||||
def start_chunk(epic: Epic, chunk: Chunk, own: str | None, windows: list[tmux.Listed], run: tmux.Tmux) -> list[tmux.Listed] | str | Failure:
|
||||
"""Starts chunk's window beside windows, the epic's session's, and returns the session's windows afterwards; the reason where its repo takes no chunk now, and a Failure where tmux started none. It holds the repo's lock from the check to the start. Raises OSError where that lock cannot be taken."""
|
||||
"""Starts chunk's window beside windows, the epic's session's, and returns the session's windows afterwards; the reason where its repo takes no chunk now, and a Failure where tmux started none. It holds the repo's lock from the check to the start. Raises OSError where that lock cannot be taken. A chunk without a repo starts in the epic's folder. So does a parked chunk, since it takes no room in its repo."""
|
||||
session = tmux.session_name(tmux.EPIC, epic.key)
|
||||
handed = {name: value for name, value in epic.env.items() if name.startswith(HANDED)}
|
||||
handed = {**{name: value for name, value in epic.env.items() if name.startswith(HANDED)}, PARKS: "1"}
|
||||
command = [str(epic.launcher), CHUNK_FLAG, chunk.issue, epic.key]
|
||||
if chunk.repo is None:
|
||||
return tmux.start_chunk(tmux.Window(session, chunk.issue, ""), str(epic.folder), handed, command, windows, False, run)
|
||||
@@ -155,8 +166,46 @@ def start_chunk(epic: Epic, chunk: Chunk, own: str | None, windows: list[tmux.Li
|
||||
|
||||
|
||||
def room_in(checkout: str, limit: int, leaving: tuple[str, str | None], home: pathlib.Path, run: tmux.Tmux) -> RepoRoom:
|
||||
"""The room in a repo's checkout under its limit. leaving is the session and name of a window whose chunk finished, which no longer counts."""
|
||||
"""The room in a repo's checkout under its limit. leaving is the session and name of the window that asks, which doesn't count: its chunk finished or is parked."""
|
||||
theirs = [placed for placed in epic_chunks(checkout, run) if (placed.session, placed.window.name) != leaving]
|
||||
loop_session = tmux.session_name(tmux.LOOP, project_key(checkout, home))
|
||||
loops = any(not window.dead for window in tmux.listed(loop_session, run)) and tmux.owner(loop_session, run) == checkout
|
||||
return RepoRoom(limit, len(theirs), loops)
|
||||
|
||||
|
||||
def resume(epic: Epic, repo: str, window: tmux.Window, run: tmux.Tmux) -> str | None:
|
||||
"""Takes a running slot for the parked chunk in window, whose worker has something to do again in the repo called repo: None once the window counts as a running chunk, else why the chunk still waits. The limits are those a starting chunk meets, checked under the same locks. The window itself never counts against them: one started again for a parked chunk, as after a reboot, isn't named as parked until its first read of the gate."""
|
||||
try:
|
||||
with starting(epic.folder):
|
||||
running = [listed for listed in tmux.listed(window.session, run) if not listed.dead and not listed.parked and listed.name != window.name]
|
||||
if len(running) >= epic.cap:
|
||||
return f"the epic runs as many chunks as its `{CAP}` line allows: {epic.cap}"
|
||||
located = epic.locate(repo)
|
||||
if isinstance(located, Failure):
|
||||
return located.reason
|
||||
checkout, limit = located
|
||||
with starting(project_lock(str(checkout), epic.home)):
|
||||
if reason := refusal(repo, room_in(str(checkout), limit, (window.session, window.name), epic.home, run)):
|
||||
return reason
|
||||
tmux.unpark(window, str(checkout), run)
|
||||
return None
|
||||
except OSError as failure:
|
||||
return f"cannot lock {failure.filename or epic.folder / LOCK}: {failure.strerror}"
|
||||
|
||||
|
||||
def status(key: str, home: pathlib.Path, issues: list[EpicIssue], run: tmux.Tmux) -> list[str] | Failure:
|
||||
"""A line per open issue of the epic called key, in the tracker's rank order: that its chunk runs, or why it waits. It doesn't start anything and checks no limit, so the line of a chunk that has yet to start doesn't say which limit holds it back."""
|
||||
found, waits = chunks(issues)
|
||||
parked = parked_chunks(home / STATE / key)
|
||||
if isinstance(parked, Failure):
|
||||
return parked
|
||||
windows = tmux.listed(tmux.session_name(tmux.EPIC, key), run)
|
||||
live = {window.name: window for window in windows if not window.dead}
|
||||
again = f"run `do-epic {key}` to start it again"
|
||||
reasons = {wait.issue: wait.reason for wait in waits}
|
||||
for chunk in found:
|
||||
if chunk.issue in parked:
|
||||
reasons[chunk.issue] = parked[chunk.issue].waits(live[chunk.issue].awaits) if chunk.issue in live else f"{parked[chunk.issue].waits('')}, and its window stopped; {again}"
|
||||
elif chunk.issue not in live:
|
||||
reasons[chunk.issue] = f"its window stopped; {again}" if chunk.issue in {window.name for window in windows} else f"it has not started; `do-epic {key}` starts it once the epic and its repo have room"
|
||||
return [f"{issue.key} waits: {reasons[issue.key]}" if issue.key in reasons else f"{issue.key} runs" for issue in issues if issue.key in reasons or issue.key in live]
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from do_epic.gates import APPROVAL, CHECKS, gate, Gate, NEW_REVIEW, Open
|
||||
from failure import Failure
|
||||
|
||||
WORKTREES = "worktree /code/billing\nHEAD aaa\nbranch refs/heads/main\n\nworktree /code/billing-ABC-21\nHEAD abc\nbranch refs/heads/ABC-21\n"
|
||||
HEAD = "abc" * 4
|
||||
|
||||
|
||||
def thread(resolved: bool, last: str, path: str = "a.py") -> dict:
|
||||
return {"isResolved": resolved, "path": path, "comments": {"nodes": [{"author": {"login": last}}]}}
|
||||
|
||||
|
||||
def review(state: str, at: str, by: str = "reviewer", on: str = HEAD) -> dict:
|
||||
"""A review as `gh pr view --json reviews` prints it: made by `by` at `at`, on the commit `on`."""
|
||||
return {"author": {"login": by}, "state": state, "submittedAt": at, "commit": {"oid": on}}
|
||||
|
||||
|
||||
def comment(by: str, at: str) -> dict:
|
||||
return {"author": {"login": by}, "createdAt": at}
|
||||
|
||||
|
||||
class Forge:
|
||||
"""Stands in for git and the forge's CLI in the checkout and the chunk's worktree; `ran` holds each folder a command ran in. Only the CLI of origin's forge answers: `gh` fails on a Gitea origin, and `tea` on a GitHub one."""
|
||||
|
||||
def __init__(self, origin: str = "git@github.com:o/billing.git", pulls: str = f"7\t{HEAD}\tfalse\n", draft: bool = False, decision: str = "REVIEW_REQUIRED", buckets: tuple[str, ...] = ("pass",), threads: tuple[dict, ...] = (), status: str = '{"state": "success", "total_count": 1}', worktrees: str = WORKTREES, fails: str = "", reviews: tuple[dict, ...] = (), comments: tuple[dict, ...] = (), title: str = "Add x") -> None:
|
||||
self.cli = "gh" if "github.com" in origin else "tea"
|
||||
self.ran: list[str] = []
|
||||
self.commands: list[list[str]] = []
|
||||
self.fails = fails
|
||||
self.answers = {
|
||||
"git worktree": worktrees,
|
||||
"git remote": origin + "\n",
|
||||
"git rev-parse": HEAD + "\n",
|
||||
"git symbolic-ref": "origin/main\n",
|
||||
"gh pr list": pulls,
|
||||
"gh pr view": json.dumps({"isDraft": draft, "reviewDecision": decision, "author": {"login": "worker"}, "reviews": list(reviews), "comments": list(comments)}),
|
||||
"gh pr checks": json.dumps([{"bucket": bucket} for bucket in buckets]),
|
||||
"gh api graphql": json.dumps({"data": {"repository": {"pullRequest": {"author": {"login": "worker"}, "reviewThreads": {"nodes": list(threads)}}}}}),
|
||||
"tea api repos/{owner}/{repo}/pulls?state=open&limit=50": json.dumps([{"number": 7, "merged": False, "head": {"label": "ABC-21", "sha": HEAD, "repo": {"full_name": "o/billing"}}, "base": {"repo": {"full_name": "o/billing"}}}]),
|
||||
"tea api repos/{owner}/{repo}/pulls/7": json.dumps({"number": 7, "title": title}),
|
||||
"tea api repos/{owner}/{repo}/branches/main": '{"name": "main"}',
|
||||
"tea api repos/{owner}/{repo}/commits/" + HEAD + "/status": status,
|
||||
}
|
||||
|
||||
def at(self, folder: str):
|
||||
def run(command: list[str]) -> str | Failure:
|
||||
self.ran.append(folder)
|
||||
self.commands.append(command)
|
||||
if command[0] in ("gh", "tea") and command[0] != self.cli:
|
||||
return Failure(f"{command[0]} cannot read this origin")
|
||||
key = " ".join(command) if command[0] == "tea" else " ".join(command[:3] if command[0] == "gh" else command[:2])
|
||||
return Failure(f"{key} failed") if self.fails and key.startswith(self.fails) else self.answers[key]
|
||||
return run
|
||||
|
||||
def gate(self) -> Gate | Open | Failure:
|
||||
return gate("ABC-21", "/code/billing", self.at)
|
||||
|
||||
|
||||
class GateOfAChunk(unittest.TestCase):
|
||||
def test_a_pull_request_whose_checks_passed_awaits_the_approval_its_protection_requires(self):
|
||||
forge = Forge()
|
||||
self.assertEqual(forge.gate(), Gate("/code/billing-ABC-21", "7", APPROVAL))
|
||||
self.assertEqual((forge.ran[0], set(forge.ran[1:])), ("/code/billing", {"/code/billing-ABC-21"}))
|
||||
self.assertEqual(forge.commands[:4], [["git", "worktree", "list", "--porcelain"], ["git", "remote", "get-url", "origin"], ["git", "rev-parse", "HEAD"], ["git", "symbolic-ref", "--short", "refs/remotes/origin/HEAD"]])
|
||||
self.assertIn(["gh", "pr", "view", "7", "--json", "isDraft,reviewDecision,author,reviews,comments"], forge.commands)
|
||||
self.assertIn(["gh", "pr", "checks", "7", "--json", "bucket"], forge.commands)
|
||||
self.assertTrue(any(command[:3] == ["gh", "api", "graphql"] and "number=7" in command for command in forge.commands), forge.commands)
|
||||
|
||||
def test_a_review_decision_gh_prints_in_no_known_shape_is_a_failure(self):
|
||||
forge = Forge()
|
||||
forge.answers["gh pr view"] = "[1]"
|
||||
self.assertEqual(forge.gate(), Failure("gh pr view printed no review decision: [1]"))
|
||||
|
||||
def test_a_pull_request_awaits_its_checks_while_one_runs_whatever_its_reviews_say(self):
|
||||
for decision in ("REVIEW_REQUIRED", "APPROVED", "CHANGES_REQUESTED", ""):
|
||||
with self.subTest(decision=decision):
|
||||
self.assertEqual(Forge(decision=decision, buckets=("pass", "pending")).gate(), Gate("/code/billing-ABC-21", "7", CHECKS))
|
||||
|
||||
def test_a_gitea_pull_request_awaits_only_its_checks(self):
|
||||
gitea = "https://gitea.larvit.se/o/billing.git"
|
||||
self.assertEqual(Forge(gitea, status='{"state": "pending", "total_count": 1}').gate(), Gate("/code/billing-ABC-21", "7", CHECKS))
|
||||
self.assertEqual(Forge(gitea).gate(), Open("pull request #7 passed its checks"))
|
||||
|
||||
def test_a_gitea_draft_awaits_nothing_though_its_checks_run(self):
|
||||
gitea = "https://gitea.larvit.se/o/billing.git"
|
||||
for title in ("WIP: Add x", "[wip] Add x"):
|
||||
with self.subTest(title=title):
|
||||
self.assertEqual(Forge(gitea, title=title, status='{"state": "pending", "total_count": 1}').gate(), Open("pull request #7 is a draft"))
|
||||
self.assertEqual(Forge(gitea, title="Add WIP: x", status='{"state": "pending", "total_count": 1}').gate(), Gate("/code/billing-ABC-21", "7", CHECKS))
|
||||
unread = Forge(gitea)
|
||||
unread.answers["tea api repos/{owner}/{repo}/pulls/7"] = "[1]"
|
||||
self.assertEqual(unread.gate(), Failure("tea api printed no title of pull request 7: [1]"))
|
||||
|
||||
def test_requested_changes_are_news_until_the_worker_pushed_or_replied_and_then_await_a_new_review(self):
|
||||
asked = review("CHANGES_REQUESTED", "2026-11-18T10:00:00Z")
|
||||
before, after, old = "2026-11-18T09:00:00Z", "2026-11-18T11:00:00Z", "fed" * 4
|
||||
news, waits = Open("a reviewer requested changes on pull request #7"), Gate("/code/billing-ABC-21", "7", NEW_REVIEW)
|
||||
cases = {
|
||||
"nobody answered": ((asked,), (), news),
|
||||
"the worker pushed": ((review("CHANGES_REQUESTED", "2026-11-18T10:00:00Z", on=old),), (), waits),
|
||||
"the worker replied in a thread": ((asked, review("COMMENTED", after, by="worker")), (), waits),
|
||||
"the worker commented": ((asked,), (comment("worker", after),), waits),
|
||||
"the worker spoke only before": ((review("COMMENTED", before, by="worker"), asked, review("PENDING", None, by="worker")), (comment("worker", before),), news), # type: ignore[arg-type]
|
||||
"only a reviewer spoke after": ((asked, review("COMMENTED", after)), (comment("reviewer", after),), news),
|
||||
"a reviewer asked again on the new head": ((asked, review("CHANGES_REQUESTED", before, on=old), review("COMMENTED", "2026-11-18T09:30:00Z", by="worker")), (), news),
|
||||
"GitHub lists no such review": ((review("APPROVED", before, on=old),), (comment("worker", after),), news),
|
||||
}
|
||||
for case, (reviews, comments, found) in cases.items():
|
||||
with self.subTest(case=case):
|
||||
self.assertEqual(Forge(decision="CHANGES_REQUESTED", reviews=reviews, comments=comments).gate(), found)
|
||||
self.assertEqual(NEW_REVIEW, "a new review")
|
||||
|
||||
def test_a_reviewers_unanswered_comment_is_news_though_the_worker_answered_the_requested_changes(self):
|
||||
forge = Forge(decision="CHANGES_REQUESTED", reviews=(review("CHANGES_REQUESTED", "2026-11-18T10:00:00Z", on="fed" * 4),), threads=(thread(False, "reviewer"),))
|
||||
self.assertEqual(forge.gate(), Open("a reviewer's comment on pull request #7 waits for an answer, in a.py"))
|
||||
|
||||
def test_the_worker_goes_on_where_nothing_but_the_worker_moves_the_pull_request(self):
|
||||
cases = {
|
||||
"pull request #7 passed its checks and is approved": Forge(decision="APPROVED"),
|
||||
"pull request #7 passed its checks": Forge(decision=""),
|
||||
"pull request #7 is a draft": Forge(draft=True, buckets=("pending",)),
|
||||
"a check on pull request 7 failed; fix it and push": Forge(buckets=("fail",)),
|
||||
"a reviewer requested changes on pull request #7": Forge(decision="CHANGES_REQUESTED"),
|
||||
"a reviewer's comment on pull request #7 waits for an answer, in a.py, b.py": Forge(threads=(thread(False, "reviewer", "b.py"), thread(False, "reviewer"), thread(True, "reviewer", "c.py"), thread(False, "worker", "d.py"))),
|
||||
"no pull request from origin's ABC-21 is open at abcabcabcabc; it may be merged or closed already": Forge(pulls=""),
|
||||
"no worktree of /code/billing has the branch ABC-21 checked out": Forge(worktrees="worktree /code/billing\nHEAD aaa\nbranch refs/heads/main\n"),
|
||||
}
|
||||
for news, forge in cases.items():
|
||||
with self.subTest(news=news):
|
||||
self.assertEqual(forge.gate(), Open(news))
|
||||
|
||||
def test_a_read_that_fails_is_a_failure_and_says_nothing_of_the_gate(self):
|
||||
for command in ("git worktree", "git remote", "gh pr list", "gh pr view", "gh pr checks", "gh api graphql"):
|
||||
with self.subTest(command=command):
|
||||
self.assertEqual(Forge(fails=command).gate(), Failure(f"{command} failed"))
|
||||
for endpoint in ("pulls?state=open&limit=50", "pulls/7", "branches/main"):
|
||||
with self.subTest(endpoint=endpoint):
|
||||
self.assertEqual(Forge("https://gitea.larvit.se/o/billing.git", fails=f"tea api repos/{{owner}}/{{repo}}/{endpoint}").gate(), Failure(f"tea api repos/{{owner}}/{{repo}}/{endpoint} failed"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+215
-13
@@ -18,10 +18,12 @@ from do_epic.epic_record import EpicRecord
|
||||
from agent_cycle import escalation_mark, judge, tmux
|
||||
from failure import Failure
|
||||
from loop_claude.test_loop_claude import FakeEffects, judged, Worker
|
||||
from do_epic.test_gates import Forge
|
||||
from do_epic.test_scheduler import issue
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from loop_claude.loop.reload import LoopRun
|
||||
from do_epic.backend.fake import MemoryEpic
|
||||
from do_epic import parked
|
||||
from loop_state.paths import paths_from
|
||||
from model_routing.route import Route
|
||||
from model_routing.ladder import Rung
|
||||
@@ -50,14 +52,35 @@ class DoEpicTest(unittest.TestCase):
|
||||
(folder / "repo-rule.md").write_text("A label that equals a repo's name names that repo.\n", encoding="utf-8")
|
||||
self.issues = [issue("ABC-20"), dataclasses.replace(issue("ABC-21", "do-epic-claude"), detail="Tidy the docs.", scores={"L": 3, "E": 2, "I": 3})]
|
||||
self.description: str | Failure = DESCRIPTION
|
||||
# What the forge answers each read of a chunk's gate, in turn; the last answers every later read. No worktree has the chunk's branch, so no chunk parks.
|
||||
self.forges = [Forge(worktrees="worktree /code/api\nbranch refs/heads/main\n")]
|
||||
self.gate_reads = 0
|
||||
|
||||
def forge_at(self, folder: str):
|
||||
def run(command: list[str]) -> str | Failure:
|
||||
# A read of a gate opens with the checkout's worktrees.
|
||||
self.gate_reads += command[:2] == ["git", "worktree"]
|
||||
return self.forges[min(self.gate_reads, len(self.forges)) - 1].at(folder)(command)
|
||||
return run
|
||||
|
||||
def backend(self, config: object, key: str) -> tuple[MemoryEpic, EpicRecord]:
|
||||
return MemoryEpic(key, list(self.issues)), EpicRecord(Description(self.description)) # type: ignore[arg-type]
|
||||
|
||||
def worker(self) -> Worker:
|
||||
return Worker(paths_from(self.home).claude_code_transcripts.parent)
|
||||
|
||||
def repo_chunk(self) -> pathlib.Path:
|
||||
"""Makes ABC-20 the one issue, a chunk in the repo api whose checkout holds its premises; the folder of the repo's config."""
|
||||
checkout = self.home / "code" / "api"
|
||||
folder = configured(self, checkout, home=self.home)
|
||||
(checkout / "README.md").write_text("## Goals\nServe the API.\n\n## Audience\nIts callers.\n", encoding="utf-8")
|
||||
self.issues = [issue("ABC-20", "repo:api")]
|
||||
return folder
|
||||
|
||||
def run_main(self, argv: list[str], fake: FakeEffects, env: dict[str, str] | None = None) -> tuple[int, str, str]:
|
||||
out, err = io.StringIO(), io.StringIO()
|
||||
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
|
||||
code = main.main(argv, env or {}, paths_from(self.home), pathlib.Path("/b/do-epic"), fake.effects(), self.backend)
|
||||
code = main.main(argv, env or {}, paths_from(self.home), pathlib.Path("/b/do-epic"), fake.effects(), self.backend, forge_at=self.forge_at)
|
||||
return code, out.getvalue(), err.getvalue()
|
||||
|
||||
|
||||
@@ -89,7 +112,7 @@ class TheCommand(DoEpicTest):
|
||||
self.assertIn("started ABC-21 in the tmux session epic-ABC-12\n", out)
|
||||
self.assertIn("ABC-20 waits: no label names its repo\n", out)
|
||||
started = [args for args in fake.tmux_calls if args[0] == "new-session"]
|
||||
self.assertEqual(started, [["new-session", "-d", "-s", "epic-ABC-12", "-n", "ABC-21", "-c", str(self.home / ".local/state/do-epic/ABC-12"), "-e", "LOOP_CLAUDE_WORKER_MODEL=m", "-P", "-F", "#{window_id}", "--", "/b/do-epic", "--chunk", "ABC-21", "ABC-12"]])
|
||||
self.assertEqual(started, [["new-session", "-d", "-s", "epic-ABC-12", "-n", "ABC-21", "-c", str(self.home / ".local/state/do-epic/ABC-12"), "-e", "LOOP_CLAUDE_WORKER_MODEL=m", "-e", "CHUNK_PR_PARKS=1", "-P", "-F", "#{window_id}", "--", "/b/do-epic", "--chunk", "ABC-21", "ABC-12"]])
|
||||
# The session is no checkout's, so nothing marks it as one's.
|
||||
self.assertNotIn(tmux.OWNER, [args[3] for args in fake.tmux_calls if args[0] == "set-option"])
|
||||
|
||||
@@ -122,10 +145,10 @@ class ACycleOfAChunkWithoutARepo(DoEpicTest):
|
||||
epic = main.EpicRun("ABC-12", config, store, record, paths, env or {}, pathlib.Path("/b/do-epic"), fake.effects())
|
||||
place = main.chunk_root(self.issues[1], epic)
|
||||
assert isinstance(place, tuple)
|
||||
root, in_repo = place
|
||||
root, repo = place
|
||||
loop = Loop(root=root, session=main.session_named("A ABC-12 do-epic ABC-21", paths.claude_code_transcripts), window=tmux.Window("epic-ABC-12", "ABC-21", "%1"), paths=paths, env=env or {}, launcher=pathlib.Path("/b/do-epic"), effects=fake.effects())
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
found = main.next_cycle(LoopRun(1, root, "", "", chunk="ABC-21"), loop, in_repo, epic)
|
||||
found = main.next_cycle(LoopRun(1, root, "", "", chunk="ABC-21"), loop, repo is not None, epic)
|
||||
assert isinstance(found, main.Cycle), found
|
||||
return found
|
||||
|
||||
@@ -253,8 +276,6 @@ class TheHumansRun(DoEpicTest):
|
||||
|
||||
|
||||
class AChunksRun(DoEpicTest):
|
||||
def worker(self) -> Worker:
|
||||
return Worker(paths_from(self.home).claude_code_transcripts.parent)
|
||||
|
||||
def test_runs_cycles_until_the_judge_rules_it_finished_then_starts_the_next_chunk(self) -> None:
|
||||
self.issues.append(issue("ABC-22", "do-epic-claude"))
|
||||
@@ -329,13 +350,11 @@ class AChunksRun(DoEpicTest):
|
||||
self.assertIn("The records: the epic's scoring and decisions, then the repo's working rules and decisions, then the human's rules for every project:\nFrom the epic ABC-12's description, ## Decisions:\nKeep the v1 endpoints.\n\nFrom AGENTS.md:\nSquash every merge.", judge_prompt)
|
||||
self.assertIn("writing it into the epic's description under ## Decisions, the repo's goals file's Goals or Audience, AGENTS.md, docs/decisions.md, the repo's plan or a comment", judge_prompt)
|
||||
|
||||
def repo_chunk(self) -> pathlib.Path:
|
||||
"""Makes ABC-20 the one issue, a chunk in the repo api whose checkout holds its premises; the folder of the repo's config."""
|
||||
checkout = self.home / "code" / "api"
|
||||
folder = configured(self, checkout, home=self.home)
|
||||
(checkout / "README.md").write_text("## Goals\nServe the API.\n\n## Audience\nIts callers.\n", encoding="utf-8")
|
||||
self.issues = [issue("ABC-20", "repo:api")]
|
||||
return folder
|
||||
def test_a_chunk_without_a_repo_reads_no_forge(self) -> None:
|
||||
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD)
|
||||
self.assertEqual(self.run_main(["--chunk", "ABC-21", "ABC-12"], fake, MODELS)[0], 0)
|
||||
self.assertEqual(self.gate_reads, 0)
|
||||
self.assertTrue(next(what[what.index("-p") + 1] for seam, what in fake.calls if seam == "stream_claude").startswith("Work the Jira issue ABC-21 "))
|
||||
|
||||
def test_a_chunk_in_a_repo_routes_on_the_repos_model(self) -> None:
|
||||
folder = self.repo_chunk()
|
||||
@@ -440,5 +459,188 @@ class WhatAWorkerJudgeAndHumanRead(DoEpicTest):
|
||||
self.assertEqual(escalation_mark.asks_higher_rung(chunk_prompt.DRIVER), "asking do-epic for a higher rung")
|
||||
|
||||
|
||||
WORKTREE = "worktree /code/api\nbranch refs/heads/main\n\nworktree /code/api-ABC-20\nbranch refs/heads/ABC-20\n"
|
||||
PARKED_WINDOW = "ABC-20 awaits an approval\t0\t@9\n"
|
||||
|
||||
|
||||
def awaiting(**answers: object) -> Forge:
|
||||
"""A forge where ABC-20's pull request, #7, passed its checks and still wants its review, but for answers."""
|
||||
return Forge(worktrees=WORKTREE, **answers) # type: ignore[arg-type]
|
||||
|
||||
|
||||
class AChunkWhosePullRequestWaitsOnAGate(DoEpicTest):
|
||||
def setUp(self) -> None:
|
||||
super().setUp()
|
||||
self.repo_chunk()
|
||||
self.issues.append(issue("ABC-22", "do-epic-claude"))
|
||||
self.file = parked.file_of(self.home / ".local/state/do-epic/ABC-12", "ABC-20")
|
||||
self.session = main.session_named("A ABC-12 do-epic ABC-20", paths_from(self.home).claude_code_transcripts)
|
||||
|
||||
def parked_run(self, fake: FakeEffects) -> tuple[int, str, str, list[str]]:
|
||||
"""The chunk's run, as its exit code, what it printed, and each prompt its workers got."""
|
||||
code, out, err = self.run_main(["--chunk", "ABC-20", "ABC-12"], fake, MODELS)
|
||||
return code, out.replace("<1790000000> ", ""), err, [what[what.index("-p") + 1] for seam, what in fake.calls if seam == "stream_claude"]
|
||||
|
||||
def test_it_parks_frees_its_slot_and_works_again_once_approved(self) -> None:
|
||||
self.forges = [awaiting(), awaiting(), awaiting(decision="APPROVED")]
|
||||
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("finished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
|
||||
states: list[object] = []
|
||||
fake.effects = lambda made=fake.effects(): dataclasses.replace(made, sleep=lambda seconds: states.append((seconds, parked.parked_chunks(self.file.parent.parent)))) # type: ignore[method-assign]
|
||||
checkout = str(self.home / "code/api")
|
||||
code, out, err, prompts = self.parked_run(fake)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertEqual(states, [(60, {"ABC-20": parked.Parked("ABC-20", "/code/api-ABC-20", "7", str(self.session.id), "an approval")})] * 2)
|
||||
self.assertFalse(self.file.exists())
|
||||
self.assertIn("parked: pull request #7 awaits an approval\nstarted ABC-22\n", out)
|
||||
self.assertEqual(len(prompts), 2)
|
||||
self.assertTrue(prompts[0].startswith("Work the Jira issue ABC-20 "), prompts[0])
|
||||
self.assertEqual(set(self.forges[0].ran[:1] + self.forges[1].ran[:1]), {checkout})
|
||||
self.assertTrue(prompts[1].startswith(f"do-epic parked the chunk while its pull request awaited an approval. It resumes now: pull request #7 passed its checks and is approved. {chunk_prompt.FEEDBACK}\n\nWork the Jira issue ABC-20 "), prompts[1])
|
||||
self.assertIn("the chunk finished; ", out)
|
||||
shown = [args for args in fake.tmux_calls if args[0] == "rename-window" or tmux.ROOT in args and "%1" in args]
|
||||
self.assertEqual(shown, [["rename-window", "-t", "%1", "ABC-20 awaits an approval"], ["rename-window", "-t", "%1", "ABC-20"], ["set-option", "-w", "-t", "%1", tmux.ROOT, checkout]])
|
||||
|
||||
def test_an_unfinished_chunk_parks_too_and_a_reviewers_comment_resumes_it(self) -> None:
|
||||
self.forges = [awaiting(buckets=("pending",)), awaiting(), awaiting(threads=({"isResolved": False, "path": "a.py", "comments": {"nodes": [{"author": {"login": "reviewer"}}]}},))]
|
||||
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("unfinished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW + "ABC-22\t1\t@10\n")
|
||||
code, out, err, prompts = self.parked_run(fake)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertIn("parked: pull request #7 awaits its checks\nstarted no other\nparked: pull request #7 awaits an approval\n", out)
|
||||
self.assertIn("its pull request awaited an approval. It resumes now: a reviewer's comment on pull request #7 waits for an answer, in a.py. Where a reviewer left feedback, fix what is reasonable", prompts[1])
|
||||
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits its checks", "ABC-20 awaits an approval", "ABC-20"])
|
||||
|
||||
def test_a_chunk_parked_when_its_window_ended_polls_before_any_cycle_and_resumes_its_session(self) -> None:
|
||||
parked.park(self.file.parent.parent, parked.Parked("ABC-20", "/code/api-ABC-20", "7", str(self.session.id), "an approval"))
|
||||
self.worker()(["--session-id", str(self.session.id)])
|
||||
self.forges = [awaiting(fails="gh pr list"), awaiting(decision="CHANGES_REQUESTED")]
|
||||
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
|
||||
code, out, err, prompts = self.parked_run(fake)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertNotIn("parked: ", out)
|
||||
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits an approval", "ABC-20"])
|
||||
self.assertEqual(len(prompts), 1)
|
||||
self.assertTrue(prompts[0].startswith("do-epic parked the chunk while its pull request awaited an approval. It resumes now: a reviewer requested changes on pull request #7. "), prompts[0])
|
||||
self.assertFalse(self.file.exists())
|
||||
|
||||
def test_a_chunk_with_work_again_stays_parked_until_a_slot_is_free(self) -> None:
|
||||
self.description = DESCRIPTION.replace("2", "1")
|
||||
self.forges = [awaiting(), awaiting(decision="APPROVED")]
|
||||
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("finished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW + "ABC-22\t0\t@10\n")
|
||||
slept: list[bool] = []
|
||||
|
||||
def sleep(seconds: float) -> None:
|
||||
slept.append(self.file.exists())
|
||||
if len(slept) == 3:
|
||||
fake.windows = PARKED_WINDOW
|
||||
fake.effects = lambda made=fake.effects(): dataclasses.replace(made, sleep=sleep) # type: ignore[method-assign]
|
||||
code, out, err, prompts = self.parked_run(fake)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertEqual(slept, [True] * 3)
|
||||
self.assertEqual(out.count("pull request #7 passed its checks and is approved, but the chunk stays parked: the epic runs as many chunks as its `**Max parallel chunks:**` line allows: 1; trying again every 60 seconds\n"), 1)
|
||||
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits an approval", "ABC-20 awaits a slot", "ABC-20"])
|
||||
self.assertEqual(len(prompts), 2)
|
||||
|
||||
def test_a_chunk_parks_and_resumes_though_the_next_chunk_cannot_start(self) -> None:
|
||||
self.forges = [awaiting(), awaiting(decision="APPROVED")]
|
||||
worker = self.worker()
|
||||
|
||||
def unreadable_after(*args: object) -> object:
|
||||
self.description = Failure("Jira answered 503")
|
||||
return worker(*args) # type: ignore[arg-type]
|
||||
fake = FakeEffects(streams=[unreadable_after, self.worker()], claude=[judged("finished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
|
||||
slept: list[bool] = []
|
||||
# The epic's description cannot be read from the first worker's run until the parked chunk first sleeps.
|
||||
description = Description("")
|
||||
object.__setattr__(description, "read", lambda: self.description)
|
||||
self.backend = lambda config, key: (MemoryEpic(key, list(self.issues)), EpicRecord(description)) # type: ignore[method-assign,arg-type]
|
||||
|
||||
def sleep(seconds: float) -> None:
|
||||
slept.append(self.file.exists())
|
||||
self.description = DESCRIPTION
|
||||
fake.effects = lambda made=fake.effects(): dataclasses.replace(made, sleep=sleep) # type: ignore[method-assign]
|
||||
code, out, err, prompts = self.parked_run(fake)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertIn("parked: pull request #7 awaits an approval\ncould not start the next chunk: Jira answered 503\n", out)
|
||||
self.assertEqual(slept, [True])
|
||||
self.assertFalse(self.file.exists())
|
||||
self.assertEqual(len(prompts), 2)
|
||||
|
||||
def test_a_forge_that_cannot_be_read_parks_no_chunk_and_unparks_none(self) -> None:
|
||||
self.forges = [awaiting(fails="gh pr checks")]
|
||||
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD)
|
||||
code, out, err, _ = self.parked_run(fake)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertIn("cannot tell whether the chunk's pull request waits on a gate, so the chunk isn't parked: gh pr checks failed\nthe chunk finished; ", out)
|
||||
self.assertFalse(self.file.exists())
|
||||
self.forges = [awaiting(), awaiting(fails="gh pr list"), awaiting(fails="gh pr list"), awaiting(decision="APPROVED")]
|
||||
self.gate_reads = 0
|
||||
fake = FakeEffects(streams=[self.worker(), self.worker()], claude=[judged("finished"), judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
|
||||
code, out, err, prompts = self.parked_run(fake)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertEqual(out.count("cannot read the gate: gh pr list failed; trying again every 60 seconds\n"), 1)
|
||||
self.assertEqual([args[3] for args in fake.tmux_calls if args[0] == "rename-window"], ["ABC-20 awaits an approval", "ABC-20"])
|
||||
self.assertEqual(len(prompts), 2)
|
||||
|
||||
def test_stops_where_the_chunk_cannot_be_recorded_as_parked_or_the_record_cannot_be_read(self) -> None:
|
||||
self.forges = [awaiting()]
|
||||
self.file.parent.parent.mkdir(parents=True)
|
||||
self.file.parent.write_text("a file")
|
||||
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD)
|
||||
code, _, err, _ = self.parked_run(fake)
|
||||
self.assertEqual(code, 1)
|
||||
self.assertIn(f"cannot record the parked chunk in {self.file}: ", err)
|
||||
self.file.parent.unlink()
|
||||
parked.park(self.file.parent.parent, parked.Parked("ABC-20", "/w", "7", "s", "an approval"))
|
||||
self.file.write_text("{")
|
||||
code, _, err, prompts = self.parked_run(FakeEffects(git=UNREAD))
|
||||
self.assertEqual((code, prompts), (1, []))
|
||||
self.assertIn(f"cannot read the parked chunk in {self.file}: ", err)
|
||||
|
||||
def test_stops_where_the_chunk_cannot_be_recorded_as_parked_no_longer(self) -> None:
|
||||
self.forges = [awaiting(), awaiting(decision="APPROVED")]
|
||||
fake = FakeEffects(streams=[self.worker()], claude=[judged("finished")], git=UNREAD, windows=PARKED_WINDOW)
|
||||
fake.effects = lambda made=fake.effects(): dataclasses.replace(made, sleep=lambda seconds: self.file.parent.chmod(0o555)) # type: ignore[method-assign]
|
||||
self.addCleanup(lambda: self.file.parent.chmod(0o755))
|
||||
code, _, err, prompts = self.parked_run(fake)
|
||||
self.assertEqual((code, len(prompts)), (1, 1))
|
||||
self.assertIn(f"cannot remove {self.file}: Permission denied", err)
|
||||
|
||||
|
||||
class TheStatus(DoEpicTest):
|
||||
def test_prints_what_each_open_issue_does_and_starts_nothing(self) -> None:
|
||||
self.issues = [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api"), issue("ABC-22", "repo:api"), issue("ABC-23", "repo:api", blockers=("ABC-20",)), issue("ABC-24", "repo:api"), issue("ABC-25", "repo:api", done=True), issue("ABC-26", "repo:api"), issue("ABC-27")]
|
||||
folder = self.home / ".local/state/do-epic/ABC-12"
|
||||
parked.park(folder, parked.Parked("ABC-20", "/code/api-ABC-20", "7", "s", "an approval"))
|
||||
parked.park(folder, parked.Parked("ABC-26", "/code/api-ABC-26", "9", "s", "its checks"))
|
||||
parked.park(folder, parked.Parked("ABC-28", "/code/api-ABC-28", "11", "s", "an approval"))
|
||||
self.issues.insert(7, issue("ABC-28", "repo:api"))
|
||||
fake = FakeEffects(windows="ABC-20 awaits an approval\t0\t@1\nABC-21 ?\t0\t@2\nABC-22\t1\t@3\nABC-28 awaits a slot\t0\t@4\n")
|
||||
code, out, err = self.run_main(["ABC-12", "--status"], fake)
|
||||
self.assertEqual((code, err), (0, ""))
|
||||
self.assertEqual(out.replace("<1790000000> ", "").splitlines(), [
|
||||
"ABC-20 waits: it is parked: its pull request #7 awaits an approval",
|
||||
"ABC-21 runs",
|
||||
"ABC-22 waits: its window stopped; run `do-epic ABC-12` to start it again",
|
||||
"ABC-23 waits: ABC-20 blocks it",
|
||||
"ABC-24 waits: it has not started; `do-epic ABC-12` starts it once the epic and its repo have room",
|
||||
"ABC-26 waits: it is parked: its pull request #9 awaits its checks, and its window stopped; run `do-epic ABC-12` to start it again",
|
||||
"ABC-28 waits: it is parked: its worker has work again on pull request #11 and awaits a slot",
|
||||
"ABC-27 waits: no label names its repo",
|
||||
])
|
||||
self.assertEqual({args[0] for args in fake.tmux_calls}, {"-V", "list-windows"})
|
||||
self.assertTrue(all("epic-ABC-12" in " ".join(args) for args in fake.tmux_calls if args[0] == "list-windows"), fake.tmux_calls)
|
||||
|
||||
def test_says_so_where_no_issue_is_open_or_the_epic_cannot_be_read(self) -> None:
|
||||
self.issues = [issue("ABC-20", "repo:api", done=True)]
|
||||
self.assertIn("the epic ABC-12 has no open issue\n", self.run_main(["ABC-12", "--status"], FakeEffects())[1])
|
||||
store = MemoryEpic("ABC-12", [])
|
||||
store.read = lambda: Failure("Jira answered 503") # type: ignore[method-assign]
|
||||
self.backend = lambda config, key: (store, EpicRecord(Description(self.description))) # type: ignore[method-assign,arg-type]
|
||||
code, _, err = self.run_main(["ABC-12", "--status"], FakeEffects())
|
||||
self.assertEqual(code, 1)
|
||||
self.assertIn("Jira answered 503\n", err)
|
||||
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from failure import Failure
|
||||
from do_epic.parked import file_of, park, Parked, parked_chunks, unpark
|
||||
|
||||
CHUNK = Parked("ABC-21", "/code/billing-ABC-21", "7", "0c0f", "an approval")
|
||||
|
||||
|
||||
class ParkedChunks(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.folder = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory()))
|
||||
|
||||
def test_an_epic_without_a_parked_chunk_has_none(self):
|
||||
self.assertEqual(parked_chunks(self.folder), {})
|
||||
|
||||
def test_a_parked_chunk_is_read_back_by_its_issue_until_it_is_unparked(self):
|
||||
self.assertIsNone(park(self.folder, CHUNK))
|
||||
self.assertEqual(parked_chunks(self.folder), {"ABC-21": CHUNK})
|
||||
self.assertIsNone(unpark(self.folder, "ABC-21"))
|
||||
self.assertEqual(parked_chunks(self.folder), {})
|
||||
|
||||
def test_parking_again_replaces_what_the_chunk_waits_on_and_leaves_no_partial_file(self):
|
||||
park(self.folder, CHUNK)
|
||||
park(self.folder, Parked("ABC-21", "/code/billing-ABC-21", "7", "0c0f", "its checks"))
|
||||
self.assertEqual(parked_chunks(self.folder)["ABC-21"].on, "its checks")
|
||||
self.assertEqual([file.name for file in (self.folder / "parked").iterdir()], ["ABC-21.json"])
|
||||
|
||||
def test_unparking_a_chunk_that_is_not_parked_does_nothing(self):
|
||||
self.assertIsNone(unpark(self.folder, "ABC-21"))
|
||||
|
||||
def test_says_why_the_human_sees_no_chunk_run(self):
|
||||
self.assertEqual(CHUNK.waits(""), "it is parked: its pull request #7 awaits an approval")
|
||||
self.assertEqual(CHUNK.waits("its checks"), "it is parked: its pull request #7 awaits its checks")
|
||||
self.assertEqual(CHUNK.waits("a slot"), "it is parked: its worker has work again on pull request #7 and awaits a slot")
|
||||
|
||||
def test_fails_naming_the_file_where_it_holds_no_parked_chunk_or_another_issues(self):
|
||||
park(self.folder, CHUNK)
|
||||
file = file_of(self.folder, "ABC-21")
|
||||
for text in ("{", '{"issue": "ABC-21"}', file.read_text().replace("ABC-21", "ABC-22"), file.read_text().replace('"7"', "7")):
|
||||
with self.subTest(text=text):
|
||||
file.write_text(text)
|
||||
failed = parked_chunks(self.folder)
|
||||
self.assertIsInstance(failed, Failure)
|
||||
self.assertIn(f"cannot read the parked chunk in {file}: ", failed.reason)
|
||||
|
||||
def test_fails_naming_the_file_it_cannot_remove(self):
|
||||
park(self.folder, Parked("ABC-21", "/w", "7", "s", "an approval"))
|
||||
file_of(self.folder, "ABC-21").parent.chmod(0o555)
|
||||
self.addCleanup(file_of(self.folder, "ABC-21").parent.chmod, 0o755)
|
||||
self.assertEqual(unpark(self.folder, "ABC-21"), Failure(f"cannot remove {file_of(self.folder, 'ABC-21')}: Permission denied"))
|
||||
|
||||
def test_fails_where_the_folder_cannot_be_written(self):
|
||||
(self.folder / "parked").write_text("a file")
|
||||
self.assertIn(f"cannot record the parked chunk in {file_of(self.folder, 'ABC-21')}: ", park(self.folder, CHUNK).reason)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -6,10 +6,11 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from do_epic.scheduler import Chunk, CHUNK_FLAG, chunks, Epic, refusal, RepoRoom, start_chunks, Started, Waits
|
||||
from do_epic.scheduler import Chunk, CHUNK_FLAG, chunks, Epic, refusal, RepoRoom, resume, start_chunks, Started, Waits
|
||||
from do_epic import chunk_prompt
|
||||
from do_epic.backend.protocol import EpicIssue
|
||||
from failure import Failure
|
||||
from do_epic.parked import file_of, park, Parked
|
||||
from do_epic.epic_record import parallel_chunks_in
|
||||
from loop_claude.cycle import prompts
|
||||
from model_routing.route import Route
|
||||
@@ -102,6 +103,12 @@ class Tmux:
|
||||
session = args[args.index("-s") + 1] if args[0] == "new-session" else args[args.index("-t") + 1].removeprefix("=").removesuffix(":")
|
||||
self.windows.append((session, args[args.index("-n") + 1], False, args[args.index("-c") + 1]))
|
||||
out = f"@{len(self.windows) - 1}\n"
|
||||
elif args[0] == "rename-window":
|
||||
at = int(args[2].removeprefix("%"))
|
||||
self.windows[at] = (self.windows[at][0], args[3], *self.windows[at][2:])
|
||||
elif args[0] == "set-option" and tmux.ROOT in args:
|
||||
at = int(args[args.index("-t") + 1].removeprefix("%").removeprefix("@"))
|
||||
self.windows[at] = (*self.windows[at][:3], "" if "-u" in args else args[-1])
|
||||
elif args[0] == "kill-window":
|
||||
self.windows[int(args[2].removeprefix("@"))] = ("", "", True, "")
|
||||
return subprocess.CompletedProcess(args, 0, out, "")
|
||||
@@ -137,6 +144,90 @@ class StartingAnEpicsChunks(unittest.TestCase):
|
||||
self.assertIn("LOOP_CLAUDE_X=1", run.calls[[args[0] for args in run.calls].index("new-session")])
|
||||
self.assertNotIn("OTHER=2", run.calls[[args[0] for args in run.calls].index("new-session")])
|
||||
|
||||
def parked(self, issue_key: str, on: str = "an approval") -> None:
|
||||
self.assertIsNone(park(self.home / ".local/state/do-epic/ABC-12", Parked(issue_key, f"/code/api-{issue_key}", "7", "0c0f", on)))
|
||||
|
||||
def test_a_parked_chunk_waits_in_its_window_and_takes_no_slot_of_the_epic_or_its_repo(self) -> None:
|
||||
self.parked("ABC-20")
|
||||
run = Tmux(("epic-ABC-12", "ABC-20 awaits an approval", False, ""))
|
||||
found = self.start(self.epic(1, api=1), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api"), issue("ABC-22", "repo:api", blockers=("ABC-20",))], run)
|
||||
self.assertEqual(found, Started(["ABC-21"], [Waits("ABC-20", "it is parked: its pull request #7 awaits an approval"), Waits("ABC-22", "ABC-20 blocks it")]))
|
||||
|
||||
def test_a_parked_chunk_whose_window_awaits_a_slot_says_so(self) -> None:
|
||||
self.parked("ABC-20")
|
||||
run = Tmux(("epic-ABC-12", "ABC-20 awaits a slot", False, ""), ("epic-ABC-12", "ABC-21", False, str(self.home / "code/api")))
|
||||
found = self.start(self.epic(1, api=1), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api")], run)
|
||||
self.assertEqual(found, Started([], [Waits("ABC-20", "it is parked: its worker has work again on pull request #7 and awaits a slot")]))
|
||||
|
||||
def test_a_parked_chunk_without_a_window_starts_again_whatever_the_limits_in_the_epics_folder(self) -> None:
|
||||
self.parked("ABC-20")
|
||||
for windows in ((), (("epic-ABC-12", "ABC-20 awaits an approval", True, ""),)):
|
||||
with self.subTest(windows=windows):
|
||||
run = Tmux(*windows, ("epic-ABC-12", "ABC-21", False, str(self.home / "code/api")))
|
||||
found = self.start(self.epic(1, api=1), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api")], run)
|
||||
self.assertEqual(found, Started(["ABC-20"], []))
|
||||
self.assertEqual(run.windows[-1], ("epic-ABC-12", "ABC-20", False, str(self.home / ".local/state/do-epic/ABC-12")))
|
||||
self.assertEqual([name for _, name, dead, _ in run.windows if dead], [""] * len(windows))
|
||||
|
||||
def test_a_parked_chunk_that_starts_again_leaves_the_epics_cap_to_the_others(self) -> None:
|
||||
self.parked("ABC-20")
|
||||
found = self.start(self.epic(1, api=2), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api")], Tmux())
|
||||
self.assertEqual(found, Started(["ABC-20", "ABC-21"], []))
|
||||
|
||||
def test_a_chunk_that_hands_on_starts_no_parked_chunk_whose_window_stopped(self) -> None:
|
||||
self.parked("ABC-20")
|
||||
found = self.start(self.epic(2, api=2), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api")], Tmux(), own="ABC-21")
|
||||
self.assertEqual(found, Started([], [Waits("ABC-20", "its window stopped; run `do-epic ABC-12` to start it again")]))
|
||||
|
||||
def test_a_chunk_that_parks_frees_its_slot_for_the_next(self) -> None:
|
||||
self.parked("ABC-20")
|
||||
run = Tmux(("epic-ABC-12", "ABC-20 awaits an approval", False, ""))
|
||||
found = self.start(self.epic(1, api=1), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api")], run, own="ABC-20")
|
||||
self.assertEqual(found, Started(["ABC-21"], []))
|
||||
|
||||
def test_stops_on_a_parked_chunk_it_cannot_read_and_starts_none(self) -> None:
|
||||
self.parked("ABC-20")
|
||||
file_of(self.home / ".local/state/do-epic/ABC-12", "ABC-20").write_text("{")
|
||||
run = Tmux()
|
||||
found = start_chunks(self.epic(2, api=2), [issue("ABC-21", "repo:api")], None, run)
|
||||
assert isinstance(found, Failure)
|
||||
self.assertIn("cannot read the parked chunk in ", found.reason)
|
||||
self.assertEqual(run.started(), [])
|
||||
|
||||
def test_hands_each_chunk_the_variable_that_makes_its_merge_park(self) -> None:
|
||||
run = Tmux()
|
||||
self.start(self.epic(1, api=1), [issue("ABC-20", "repo:api")], run)
|
||||
self.assertIn("CHUNK_PR_PARKS=1", run.calls[[args[0] for args in run.calls].index("new-session")])
|
||||
|
||||
def resumed(self, epic: Epic, run: Tmux) -> str | None:
|
||||
"""resume for the parked chunk in the first window, which changes the repo api."""
|
||||
return resume(epic, "api", tmux.Window("epic-ABC-12", "ABC-20", "%0"), run)
|
||||
|
||||
def test_a_parked_chunk_resumes_into_a_free_slot_and_counts_in_its_repo_again(self) -> None:
|
||||
run = Tmux(("epic-ABC-12", "ABC-20 awaits an approval", False, ""), ("epic-ABC-12", "ABC-21 awaits its checks", False, ""))
|
||||
self.assertIsNone(self.resumed(self.epic(1, api=1), run))
|
||||
self.assertEqual(run.windows[0], ("epic-ABC-12", "ABC-20", False, str(self.home / "code/api")))
|
||||
|
||||
def test_a_parked_chunk_whose_window_started_again_doesnt_count_itself_as_running(self) -> None:
|
||||
for root in (str(self.home / ".local/state/do-epic/ABC-12"), str(self.home / "code/api")):
|
||||
with self.subTest(root=root):
|
||||
run = Tmux(("epic-ABC-12", "ABC-20", False, root))
|
||||
self.assertIsNone(self.resumed(self.epic(1, api=1), run))
|
||||
self.assertEqual(run.windows[0], ("epic-ABC-12", "ABC-20", False, str(self.home / "code/api")))
|
||||
|
||||
def test_a_parked_chunk_stays_parked_while_the_epic_or_its_repo_is_full(self) -> None:
|
||||
parked = ("epic-ABC-12", "ABC-20 awaits an approval", False, "")
|
||||
cases = {
|
||||
"the epic runs as many chunks as its `**Max parallel chunks:**` line allows: 1": (self.epic(1, api=2), Tmux(parked, ("epic-ABC-12", "ABC-21", False, str(self.home / "code/web")))),
|
||||
"api runs as many chunks as its `parallel-chunks` allows: 1": (self.epic(2, api=1), Tmux(parked, ("epic-XYZ-1", "XYZ-5", False, str(self.home / "code/api")))),
|
||||
"loop-claude runs chunks in api": (self.epic(2, api=1), Tmux(parked, ("loop-api", "item-3", False, str(self.home / "code/api")), owners={"loop-api": str(self.home / "code/api")})),
|
||||
"no project config names api": (self.epic(2), Tmux(parked)),
|
||||
}
|
||||
for reason, (epic, run) in cases.items():
|
||||
with self.subTest(reason=reason):
|
||||
self.assertEqual(self.resumed(epic, run), reason)
|
||||
self.assertEqual(run.windows[0], parked)
|
||||
|
||||
def test_a_running_chunk_counts_and_is_not_started_twice(self) -> None:
|
||||
run = Tmux(("epic-ABC-12", "ABC-20 ?", False, str(self.home / "code/api")))
|
||||
found = self.start(self.epic(2, api=2), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:api"), issue("ABC-22", "repo:api")], run)
|
||||
@@ -147,6 +238,11 @@ class StartingAnEpicsChunks(unittest.TestCase):
|
||||
found = self.start(self.epic(3, api=1, web=1), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:web")], run)
|
||||
self.assertEqual(found, Started(["ABC-21"], [Waits("ABC-20", "api runs as many chunks as its `parallel-chunks` allows: 1")]))
|
||||
|
||||
def test_a_chunk_its_repo_refuses_takes_no_slot_of_the_epics_cap(self) -> None:
|
||||
run = Tmux(("loop-api", "item-3", False, str(self.home / "code/api")), owners={"loop-api": str(self.home / "code/api")})
|
||||
found = self.start(self.epic(1, api=1, web=1), [issue("ABC-20", "repo:api"), issue("ABC-21", "repo:web")], run)
|
||||
self.assertEqual(found, Started(["ABC-21"], [Waits("ABC-20", "loop-claude runs chunks in api")]))
|
||||
|
||||
def test_never_starts_in_a_repo_loop_claude_runs_chunks_in(self) -> None:
|
||||
run = Tmux(("loop-api", "item-3", False, str(self.home / "code/api")), owners={"loop-api": str(self.home / "code/api")})
|
||||
found = self.start(self.epic(3, api=2), [issue("ABC-20", "repo:api")], run)
|
||||
|
||||
+56
-21
@@ -1,5 +1,7 @@
|
||||
"""Finding a pull request from origin's own branch, and waiting for the checks on its head commit."""
|
||||
import dataclasses
|
||||
import fnmatch
|
||||
import json
|
||||
import time
|
||||
import typing
|
||||
|
||||
@@ -10,6 +12,8 @@ from forge.commands import Forge, gitea, Run
|
||||
POLL_SECONDS = 10
|
||||
# How long checks_passed waits on Gitea for the checks before it gives up; CI took about 3 minutes in 2026-11. gh's watch waits as long as the checks run.
|
||||
WAIT_SECONDS = 30 * 60
|
||||
# What gh says of a pull request whose head has no check.
|
||||
NO_CHECKS = "no checks reported"
|
||||
|
||||
|
||||
def open_pull(forge: Forge, branch: str, head: str, run: Run) -> str | None | Failure:
|
||||
@@ -57,37 +61,68 @@ def required_checks(forge: Forge, base: str, run: Run) -> set[str] | Failure:
|
||||
return Failure(f"tea api answered branch {base} in a shape that can't be read: {str(branch)[:200]}")
|
||||
|
||||
|
||||
def checks_passed(forge: Forge, number: str, head: str, required: set[str], run: Run, sleep: typing.Callable[[float], None] = time.sleep, settle: int = 0) -> Failure | None:
|
||||
"""None once every check on pull request number, whose head commit is head, has passed, or where it has none for settle seconds; a Failure where one fails, or where they have not all passed within WAIT_SECONDS.
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class CheckFailed(Failure):
|
||||
"""A check on a pull request's head failed, which a new push answers. Any other Failure of a read of the checks says the forge could not be read."""
|
||||
|
||||
On Gitea it also waits for a passed check to match each of required: a check CI has yet to queue is no status at all.
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Pending:
|
||||
"""The checks on a pull request's head have not all passed yet, and none failed."""
|
||||
# The checks, or glob patterns, that the protection requires and no passed check matches yet. Empty where a check that reported is still running.
|
||||
missing: list[str]
|
||||
|
||||
|
||||
def checks_now(forge: Forge, number: str, head: str, required: set[str], run: Run, settled: bool = True) -> Pending | Failure | None:
|
||||
"""The checks on pull request number, whose head commit is head, read once: None where every one passed, Pending while one still runs, a CheckFailed where one failed, and any other Failure where the forge cannot be read.
|
||||
|
||||
On Gitea a passed check must also match each of required: a check CI has yet to queue is no status at all. A head without any check counts as passed once settled, and as pending before, since CI may not have queued its checks yet. On GitHub such a head always counts as passed: gh cannot tell a head whose checks have yet to start from a head that has none.
|
||||
"""
|
||||
if forge == "gh":
|
||||
listed = run(["gh", "pr", "checks", number, "--json", "bucket"])
|
||||
if isinstance(listed, Failure):
|
||||
return None if NO_CHECKS in listed.reason else listed
|
||||
try:
|
||||
buckets = {check["bucket"] for check in json.loads(listed)}
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh pr checks printed no checks: {listed.strip()[:200]}")
|
||||
if buckets & {"fail", "cancel"}:
|
||||
return CheckFailed(f"a check on pull request {number} failed")
|
||||
return Pending([]) if "pending" in buckets else None
|
||||
status = gitea(run, f"repos/{{owner}}/{{repo}}/commits/{head}/status")
|
||||
if isinstance(status, Failure):
|
||||
return status
|
||||
try:
|
||||
count, state = status["total_count"], status["state"]
|
||||
passed = [check["context"] for check in status.get("statuses") or [] if check["status"] == "success"]
|
||||
except (KeyError, TypeError, AttributeError):
|
||||
return Failure(f"tea api printed no commit status for {head}: {str(status)[:200]}")
|
||||
if state not in ("pending", "success"):
|
||||
return CheckFailed(f"a check on pull request {number} ended in {state}")
|
||||
# Gitea matches a required context as a glob, and reads a commit with no status as pending, with total_count 0.
|
||||
missing = sorted(pattern for pattern in required if not fnmatch.filter(passed, pattern))
|
||||
return None if (count == 0 and settled or state == "success") and not missing else Pending(missing)
|
||||
|
||||
|
||||
def checks_passed(forge: Forge, number: str, head: str, required: set[str], run: Run, sleep: typing.Callable[[float], None] = time.sleep, settle: int = 0) -> Failure | None:
|
||||
"""None once every check on pull request number, whose head commit is head, has passed, or where it has none for settle seconds; a Failure where one fails, or where they have not all passed within WAIT_SECONDS. checks_now says what counts as passed on Gitea."""
|
||||
if forge == "gh":
|
||||
for waited in range(0, settle + 1, POLL_SECONDS):
|
||||
watched = run(["gh", "pr", "checks", number, "--watch", "--fail-fast"])
|
||||
if not isinstance(watched, Failure):
|
||||
return None
|
||||
if "no checks reported" not in watched.reason:
|
||||
if NO_CHECKS not in watched.reason:
|
||||
return watched
|
||||
if waited < settle:
|
||||
sleep(POLL_SECONDS)
|
||||
return None
|
||||
for _ in range(WAIT_SECONDS // POLL_SECONDS + 1):
|
||||
status = gitea(run, f"repos/{{owner}}/{{repo}}/commits/{head}/status")
|
||||
if isinstance(status, Failure):
|
||||
return status
|
||||
try:
|
||||
count, state = status["total_count"], status["state"]
|
||||
passed = [check["context"] for check in status.get("statuses") or [] if check["status"] == "success"]
|
||||
except (KeyError, TypeError, AttributeError):
|
||||
return Failure(f"tea api printed no commit status for {head}: {str(status)[:200]}")
|
||||
if state not in ("pending", "success"):
|
||||
return Failure(f"a check on pull request {number} ended in {state}")
|
||||
# Gitea matches a required context as a glob, and reads a commit with no status as pending, with total_count 0.
|
||||
missing = sorted(pattern for pattern in required if not fnmatch.filter(passed, pattern))
|
||||
if (count == 0 and _ * POLL_SECONDS >= settle or state == "success") and not missing:
|
||||
return None
|
||||
if _ < WAIT_SECONDS // POLL_SECONDS:
|
||||
polls = WAIT_SECONDS // POLL_SECONDS
|
||||
now: Pending | Failure | None = None
|
||||
for poll in range(polls + 1):
|
||||
now = checks_now(forge, number, head, required, run, settled=poll * POLL_SECONDS >= settle)
|
||||
if not isinstance(now, Pending):
|
||||
return now
|
||||
if poll < polls:
|
||||
sleep(POLL_SECONDS)
|
||||
waited_on = f"{', '.join(missing)}, which the protection requires, never passed" if missing else "a check is still pending"
|
||||
waited_on = f"{', '.join(now.missing)}, which the protection requires, never passed" if isinstance(now, Pending) and now.missing else "a check is still pending"
|
||||
return Failure(f"gave up on pull request {number} after {WAIT_SECONDS // 60} minutes: {waited_on}")
|
||||
|
||||
+29
-1
@@ -6,7 +6,8 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from forge.pulls import checks_passed, open_pull, POLL_SECONDS, required_checks, WAIT_SECONDS
|
||||
from forge.pulls import CheckFailed, checks_now, checks_passed, open_pull, Pending, POLL_SECONDS, required_checks, WAIT_SECONDS
|
||||
from failure import Failure
|
||||
from forge.commands import forge_of, gitea, url_beside
|
||||
|
||||
|
||||
@@ -69,6 +70,33 @@ class RequiredChecks(unittest.TestCase):
|
||||
self.assertEqual(required_checks("gh", "main", self.fail), set())
|
||||
|
||||
|
||||
class ChecksNow(unittest.TestCase):
|
||||
def test_reads_on_gh_once_and_says_a_check_is_pending(self):
|
||||
read = []
|
||||
self.assertEqual(checks_now("gh", "7", "abc", set(), lambda command: read.append(command) or '[{"bucket": "pass"}, {"bucket": "pending"}]'), Pending([]))
|
||||
self.assertEqual(read, [["gh", "pr", "checks", "7", "--json", "bucket"]])
|
||||
|
||||
def test_takes_on_gh_passed_and_skipped_checks_and_no_checks_as_passed(self):
|
||||
self.assertIsNone(checks_now("gh", "7", "abc", set(), lambda command: '[{"bucket": "pass"}, {"bucket": "skipping"}]'))
|
||||
self.assertIsNone(checks_now("gh", "7", "abc", set(), lambda command: Failure("gh pr checks: no checks reported on the 'x' branch")))
|
||||
|
||||
def test_fails_on_gh_where_a_check_failed_or_was_cancelled_or_gh_fails(self):
|
||||
for bucket in ("fail", "cancel"):
|
||||
self.assertEqual(checks_now("gh", "7", "abc", set(), lambda command: f'[{{"bucket": "pending"}}, {{"bucket": "{bucket}"}}]'), CheckFailed("a check on pull request 7 failed"))
|
||||
self.assertEqual(checks_now("gh", "7", "abc", set(), lambda command: Failure("no network")), Failure("no network"))
|
||||
self.assertEqual(checks_now("tea", "1", "abc", set(), tea(['{"state": "failure", "total_count": 1}'])), CheckFailed("a check on pull request 1 ended in failure"))
|
||||
self.assertEqual(checks_now("gh", "7", "abc", set(), lambda command: "[1]").reason, "gh pr checks printed no checks: [1]")
|
||||
|
||||
def test_reads_on_tea_once_and_names_each_required_check_that_has_not_passed(self):
|
||||
self.assertEqual(checks_now("tea", "1", "abc", {"lint"}, tea(['{"state": "pending", "total_count": 0, "statuses": []}'])), Pending(["lint"]))
|
||||
self.assertEqual(checks_now("tea", "1", "abc", set(), tea(['{"state": "pending", "total_count": 1}'])), Pending([]))
|
||||
self.assertIsNone(checks_now("tea", "1", "abc", set(), tea(['{"state": "success", "total_count": 1}'])))
|
||||
|
||||
def test_takes_a_commit_with_no_status_on_tea_as_pending_until_settled(self):
|
||||
self.assertEqual(checks_now("tea", "1", "abc", set(), tea(['{"state": "pending", "total_count": 0}']), settled=False), Pending([]))
|
||||
self.assertIsNone(checks_now("tea", "1", "abc", set(), tea(['{"state": "pending", "total_count": 0}'])))
|
||||
|
||||
|
||||
class ChecksPassed(unittest.TestCase):
|
||||
def test_waits_on_tea_while_a_check_is_pending(self):
|
||||
slept = []
|
||||
|
||||
@@ -66,8 +66,8 @@ def project_lock(root: str, home: pathlib.Path) -> pathlib.Path:
|
||||
|
||||
|
||||
def epic_chunks(root: str, run: tmux.Tmux) -> list[tmux.Placed]:
|
||||
"""The windows of do-epic's sessions whose chunks run in the checkout root."""
|
||||
return [placed for placed in tmux.everywhere(run) if placed.session.startswith(f"{tmux.EPIC}-") and placed.root == root and not placed.window.dead]
|
||||
"""The windows of do-epic's sessions whose chunks run in the checkout root. A parked chunk doesn't run."""
|
||||
return [placed for placed in tmux.everywhere(run) if placed.session.startswith(f"{tmux.EPIC}-") and placed.root == root and not placed.window.dead and not placed.window.parked]
|
||||
|
||||
|
||||
def held_items(chunks: typing.Iterable[str], key: str, claude_code_transcripts: pathlib.Path) -> set[int]:
|
||||
|
||||
+1
-1
@@ -153,7 +153,7 @@ Valid while `~/.claude` is a checkout of this repo.
|
||||
- **Releases.** An epic item has no release. It ships into its repo's unreleased work: a bullet under `CHANGELOG.md` → `## Unreleased`, or nothing where the repo's plan is in Jira.
|
||||
- **Repos.** A repo's name is its main checkout's folder name, and a project config names that checkout. A missing checkout is cloned from the host of the epic's other repos; where unsure, do-epic asks. Unsure means the other repos don't all sit under one host and owner (the same host spelt as https and as ssh counts as two), an origin can't be read or isn't an https, ssh or scp-style address, the folder to clone into holds something else, or the clone fails. A repo no config names is cloned into the clone folder, and the human is asked to write its config. Each chunk uses its repo's model, routing, and language for commits and PRs. Jira text is in the epic's language, or else the machine's default.
|
||||
- **Running.** The epic caps the chunks running at once, and each repo's `parallel-chunks` still applies. do-epic and loop-claude never run chunks in the same repo at once. The cap is a line of the epic's description, `**Max parallel chunks:** 2`, and 1 without it, as `parallel-chunks` is. A repo's `parallel-chunks` counts the chunks of every epic in its checkout. Each window is named for its issue's key, and a tmux window option, `@chunk-root`, names the folder its chunk works in, which is how loop-claude sees do-epic's chunks in a checkout. do-epic and loop-claude each hold the checkout's `starting` lock from checking the checkout's chunks to starting a window there. A chunk without a repo works in `~/.local/state/do-epic/<EPIC-KEY>/`. do-epic archives its cycles under `~/.claude/do-epic-transcripts/`, not in loop-claude's archive, whose readers parse loop-claude's prompts. An open issue runs once a label records its repo; until do-epic picks repos itself, an issue without one waits, and `do-epic <EPIC-KEY>` says so.
|
||||
- **Long gates.** A chunk whose PR waits on a gate, such as a human approval, parks. It keeps its worktree, PR and session, and its state is in a file that survives a reboot. A parked chunk takes no running slot, and the items it blocks wait. do-epic polls the forge and resumes the chunk's session when the gate passes or a reviewer comments. It fixes reasonable feedback, replies to the rest, and changes the setup so review would have caught it. `do-epic <key> --status` and the tmux window names show what each chunk waits on.
|
||||
- **Long gates.** A chunk whose PR waits on a gate, such as a human approval, parks. It keeps its worktree, PR and session, and its state is in a file that survives a reboot. A parked chunk takes no running slot, and the items it blocks wait. do-epic polls the forge and resumes the chunk's session when the gate passes or a reviewer comments. It fixes reasonable feedback, replies to the rest, and changes the setup so review would have caught it. `do-epic <key> --status` and the tmux window names show what each chunk waits on. A gate is a pull request's pending checks or, on GitHub, the approval its branch protection requires; a draft waits on neither, since its worker has yet to mark it ready. On Gitea a draft is a pull request whose title opens with `WIP:` or `[WIP]`, Gitea's default `WORK_IN_PROGRESS_PREFIXES`. Changes a reviewer requested resume the chunk only while that review was made on the pull request's head commit and the pull request's author has not commented or replied after it; once the worker pushed or replied, the chunk parks on a new review, since GitHub keeps the decision until the reviewer reviews again. The chunk's window reads the forge once a minute, which no forge's rate limit notices. A chunk resumes only into a free slot, under the limits a starting chunk meets, and stays parked until then. `do-epic <key>` starts a parked chunk's window again whatever the limits, since the chunk stays parked. do-epic sets `CHUNK_PR_PARKS=1` for each worker, so `chunk-pr merge` fails at once where the pull request awaits an approval, and the worker's run doesn't hold a slot while the pull request waits.
|
||||
- **Human tasks.** do-epic DMs the human when one blocks the next item, and polls until its issue closes.
|
||||
- **Findings.** A finding about the epic's goals is filed in the epic. A finding about a repo's goals follows that repo's rules. Where unsure which, do-epic asks. A set of new items toward an epic goal needs the human's approval before work starts; a finding filed during a chunk doesn't.
|
||||
- **Finishing.** When the epic holds no open item: release-review passes on each repo the epic changed, then the items they file, then the human confirms each achieve goal and that no maintain goal has a defect. For a goal that isn't met, do-epic proposes items for approval. Then a summary comment on the epic, and closing the epic in Jira, as the very last step.
|
||||
|
||||
@@ -17,8 +17,7 @@
|
||||
| 415 | 0.12.0 | | | **Finish an epic: run release-review passes on each repo it changed, work what they file, have the human confirm each goal, post a summary comment, and close the epic in Jira last.** | 3 | 2 | 3 | 6 | 5 | 1 by 5 | 1.2 |
|
||||
| 418 | 0.12.0 | 415 | | **Describe do-epic in `README.md`'s tools and setup, with its machine settings and `repo-rule.md`.** | 1 | 2 | 1 | 2 | 2 | 1 by 5 | 2.1 |
|
||||
| 420 | 0.12.0 | | | **End a worker that printed its final result and lingers, by killing the background shells it left, after a few minutes, and print each kill with its evidence on loop-claude's terminal.** | 2 | 2 | 2 | 4 | 3 | 1.5 by 6 | 2.0 |
|
||||
| 411 | 0.12.0 | | | **Park a chunk whose PR waits on a long gate, keeping its worktree, PR and session in a state file that survives a reboot, and resume its session when the gate passes or a reviewer comments.** | 3 | 2 | 3 | 6 | 5 | 1 by 5 | 1.2 |
|
||||
| 412 | 0.12.0 | 411 | | **Print each chunk of an epic, running, parked with its gate and since when, or stopped, each human task and each open interrupt with `do-epic <key> --status`, and name a parked chunk's gate in its tmux window's name.** | 2 | 1 | 1 | 2 | 3 | 1 by 5 | 1.9 |
|
||||
| 412 | 0.12.0 | | | **Print each chunk of an epic, running, parked with its gate and since when, or stopped, each human task and each open interrupt with `do-epic <key> --status`, and name a parked chunk's gate in its tmux window's name.** | 2 | 1 | 1 | 2 | 3 | 1 by 5 | 1.9 |
|
||||
| 414 | 0.12.0 | | | **DM the human through the notifier when a human-task issue blocks the epic's next item, and poll until it closes.** | 2 | 2 | 2 | 4 | 3 | 1 by 5 | 1.9 |
|
||||
| 417 | 0.12.0 | | | **Run do-epic end to end against a sandbox epic on a real Jira site with a command of its own, outside `run-tests`.** | 2 | 1 | 1 | 2 | 3 | 1 by 5 | 1.9 |
|
||||
| 440 | 0.12.0 | | | **End the shell that the interrupt test of `mutate-changes` starts within a minute, so a mutation run of this repo never waits on it for ever.** | 1 | 1 | 2 | 2 | 2 | 1 by 4 | 1.7 |
|
||||
@@ -130,18 +129,6 @@ Touches:
|
||||
- `bin/loop_claude/loop/lingering_workers.py`
|
||||
- `README.md`: `## Tools` (small)
|
||||
|
||||
### 411. Park a chunk whose PR waits on a long gate, keeping its worktree, PR and session in a state file that survives a reboot, and resume its session when the gate passes or a reviewer comments.
|
||||
|
||||
A parked chunk takes no running slot, and the items it blocks wait (`docs/decisions.md` → `do-epic: drives one Jira epic …`). On reviewer feedback the resumed chunk fixes what is reasonable, replies to the rest, and changes the setup so review would have caught it. The forge checks need forms that return instead of blocking. `chunk_pr/main.py` `merged` is a place the seats found hard: make it easier to read where it changes.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/chunk_pr/main.py`: `merged`, `merge_turn` (small)
|
||||
- `bin/chunk_pr/approval.py`: `approval`, `unanswered` (small)
|
||||
- `bin/forge/pulls.py`: `checks_passed` (small)
|
||||
- `bin/do_epic/gates.py`
|
||||
- `bin/do_epic/parked.py`
|
||||
|
||||
### 412. Print each chunk of an epic, running, parked with its gate and since when, or stopped, each human task and each open interrupt with `do-epic <key> --status`, and name a parked chunk's gate in its tmux window's name.
|
||||
|
||||
A plain `do-epic <key>` resumes all of them, as after a reboot (`docs/decisions.md` → `do-epic: drives one Jira epic …`). The window suffix generalises the interrupt's " ?".
|
||||
|
||||
Reference in New Issue
Block a user