Merge remote-tracking branch 'origin/main' into item-310
CI / tests (pull_request) Successful in 4m41s
CI / tests (pull_request) Successful in 4m41s
# Conflicts: # todo.md
This commit was merged in pull request #632.
This commit is contained in:
@@ -57,10 +57,13 @@
|
||||
!bin/deploy_release/
|
||||
!bin/deploy_release/*.py
|
||||
!bin/do-epic
|
||||
!bin/do-epic-e2e
|
||||
!bin/do_epic/
|
||||
!bin/do_epic/*.py
|
||||
!bin/do_epic/backend/
|
||||
!bin/do_epic/backend/*.py
|
||||
!bin/do_epic/e2e/
|
||||
!bin/do_epic/e2e/*.py
|
||||
!bin/failure.py
|
||||
!bin/forge/
|
||||
!bin/forge/*.py
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Add `do-epic-e2e`, a command that runs `do-epic` end to end against a sandbox epic on a real Jira site and deletes the epic afterwards.
|
||||
- `do-epic <EPIC-KEY>` plans the epic before it starts chunks: it files a sub-task per repo under an open issue that changes several repos, labels each other unlabelled issue with where it is worked, asking on the terminal where the repo rule leaves that unclear, and puts the scored issues it filed in order of score.
|
||||
- loop-claude compacts a waiting worker's session 10 minutes before its hour-long prompt cache expires, where no answer to an interrupt has come and the context holds at least 60k tokens, so the resume doesn't write the whole context again.
|
||||
- do-epic gives a `do-epic-human` issue that blocks another its own window: it DMs you once through the notifier of one of the epic's repos, takes no running slot, and polls until you close the issue in Jira, when the issues it held up may start.
|
||||
|
||||
@@ -193,6 +193,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `compress-archives` gzips each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them; Install's crontab line runs it daily.
|
||||
- `deploy-release [--cut] [<checkout>]` moves `~/.claude`, or the checkout named, to the newest release, as Update says; with `--cut`, run in the development clone, it first cuts the next release, as Release says.
|
||||
- `do-epic <EPIC-KEY>` runs the chunks of one Jira epic whose items span repos, or have none, each in its own window of the tmux session `epic-<EPIC-KEY>`, on the same cycle, judge and interrupt as loop-claude. The epic's description is its record: goals, audience, scoring and decisions. Before it starts chunks, do-epic plans each open issue that no label places yet, as "Set up a project" says. It files a sub-task per repo where the issue changes several repos. Otherwise it labels the issue with where it is worked, and asks you where that is unclear. Then it puts the scored issues that do-epic itself filed in order of score. An open issue runs once a label names its repo or marks it as Claude's, in Jira's backlog order, as many at once as a line such as `**Max parallel chunks:** 2` in the epic's description allows, 1 without it, and as each repo's `parallel-chunks` allows. A chunk whose pull request waits on a gate parks. An open task of yours that blocks another issue gets a window that DMs you once and reads the epic every minute; once you close the task, the issues it blocked may start. do-epic closes the epic only once you confirm every goal. Install → "Set up a project" says how to set it up; `do-epic --help` states the flags and the contract.
|
||||
- `do-epic-e2e` runs `do-epic` against a sandbox epic on a real Jira site, as Test says.
|
||||
- `judge-replay <model> [--offline]` replays loop-claude's judge on archived verdicts of the projects run on the model. It prints what each rung a review climbs to misses, and the lowest rung below those that returns every verdict, as each candidate above it does. With `--offline` it doesn't call a judge: it counts what a "?" rule, a decision table and a classifier trained on the archive decide of each verdict's kind from the worker's final message, and how many interrupts each misses. `judge-replay --help` states both replays.
|
||||
- `larv-review <command>` holds larv-review's mechanical rules: the chunk's review folder, the paired ruling's base and vote count, each pass's round count and the opening of its round commit's title, the scoring run's trigger and setup; the comprehension history's format, its latest places and its count of how often seats agree; and each reviewer's prompt header. `larv-review --help` states them. `settings.json` → `hooks` runs `larv-review reviewer-context` as each subagent starts. A reviewer under `agents/` whose frontmatter has `omitClaudeMd: true` starts without `CLAUDE.md`, and the command hands it that file's "Shared goal", "Prose" and "Vocabulary" sections.
|
||||
- `loop-claude <task>` runs a project's next chunks, each in its own tmux window, cycle after cycle; `loop-claude --help` states its contract.
|
||||
@@ -226,7 +227,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `branch_chunks/` cuts a default branch into chunks and finds, for each, the items it shipped and the later chunks that fixed its code, for `score_outcomes` and `routing_figures`. It imports `reading_cost`'s record path.
|
||||
- `command_lines/` reads a shell command: its simple commands, and a docker call's subcommand and memory limits, for `command_patterns`, `reading_cost`, `routing_figures`, `rule_hook` and `run_heavy`.
|
||||
- `deploy_release` imports `release_plan`'s `release_standing`.
|
||||
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `planning.py` settles, before the chunks start, where each open issue is worked and the order of the issues do-epic filed, through `split.py`, `repo_pick.py` and `ranking.py`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. `gates.py` reads once whether a chunk's pull request waits on its checks, an approval or a new review, and `parked.py` keeps each chunk that does in a state file, so it takes no running slot while it waits and goes on waiting after a reboot. `chunk_prompt.py` words what a chunk's worker reads. Through `findings.py` that worker files a finding as an issue of the epic, with `--file`, and appends a decision to its description, with `--record-decision`. `finish.py` ends an epic that holds no open issue: its finishing chunk files a release review of each repo the epic changed, has the human confirm each goal, and with `--close-epic` posts the summary on the epic and closes it. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, posts comments, moves a finished chunk's issue to done, and closes the epic.
|
||||
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `planning.py` settles, before the chunks start, where each open issue is worked and the order of the issues do-epic filed, through `split.py`, `repo_pick.py` and `ranking.py`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. `gates.py` reads once whether a chunk's pull request waits on its checks, an approval or a new review, and `parked.py` keeps each chunk that does in a state file, so it takes no running slot while it waits and goes on waiting after a reboot. `chunk_prompt.py` words what a chunk's worker reads. Through `findings.py` that worker files a finding as an issue of the epic, with `--file`, and appends a decision to its description, with `--record-decision`. `finish.py` ends an epic that holds no open issue: its finishing chunk files a release review of each repo the epic changed, has the human confirm each goal, and with `--close-epic` posts the summary on the epic and closes it. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, posts comments, moves a finished chunk's issue to done, and closes the epic. `e2e/` holds `do-epic-e2e`.
|
||||
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and reads its checks, once or until they settle (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
|
||||
- `git_cli/` runs git for every tool (`git_repo.py`), and reads the git calls in a shell command (`git_command.py`).
|
||||
- `judge_replay` imports `agent_cycle`'s judge and verdicts, and `loop_claude`'s premises, records and `ANSWER_DESTINATIONS`.
|
||||
@@ -254,6 +255,8 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
|
||||
Test on the host with `bin/run-tests`, which runs pytest through `uvx`, pinned in `bin/run_tests/main.py`. It prints each failing test with its error and pytest's summary line, and saves the whole output to a file whose path it prints. It passes on any test paths or pytest options it is given, such as `bin/run-tests bin/chunk_pr -x`. It needs `uv` on `PATH`, which fetches pytest, pytest-xdist and mypy, a Docker daemon, Linux, and a user other than root; a test that lacks one fails, naming it. pytest-xdist runs the tests in one worker per CPU of a heavy slot (`docs/decisions.md` → `bin/: the tools support only the latest stable Python, …`). `python3 -m unittest discover -s bin -p 'test_*.py'` runs the same tests one at a time, with the same needs. Test the bench's scripts with `python3 -m unittest discover -s workflow-bench -p 'test_*.py'`.
|
||||
|
||||
`bin/do-epic-e2e` checks `do-epic` against a real Jira site, which `run-tests` never reaches. It makes an epic with issues and sub-tasks in a sandbox project, runs `do-epic --file`, `--record-decision` and `--close-epic` against it in a home folder of its own, and deletes the epic and its issues when it ends. The sandbox project must be team-managed, with a `Done` transition, and its account must be able to create, edit, transition, comment on and delete issues there. Set `DO_EPIC_E2E_JIRA_URL` to the sandbox site's address, `DO_EPIC_E2E_ENV_FILE` to the absolute path of the file holding the account's `ATLASSIAN_EMAIL` and `ATLASSIAN_API_TOKEN`, and `DO_EPIC_E2E_PROJECT` to the project's key. The command fails, naming each setting that is missing. It needs a Docker daemon, as `run-tests` does.
|
||||
|
||||
CI runs `ci.sh` on every pull request, and `main` doesn't merge anything until it passes. The script runs the same tests on the committed HEAD in a pinned Python container. Run it under `run-heavy` to reproduce a CI failure, with Docker on `PATH` and its socket at `/var/run/docker.sock`, which the container shares.
|
||||
|
||||
To mutation-test the lines a branch changes, run `~/.claude/bin/run-heavy ~/.claude/bin/mutate-changes --prefix 'uvx --python "$(command -v python3)" --with pytest==9.2.0 --from "mutmut>=3,<4"'` in its checkout.
|
||||
|
||||
Executable
+11
@@ -0,0 +1,11 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
LAUNCHER = pathlib.Path(__file__).resolve()
|
||||
sys.path.insert(0, str(LAUNCHER.parent))
|
||||
|
||||
from do_epic.e2e import main
|
||||
|
||||
sys.exit(main.main(sys.argv[1:], os.environ, LAUNCHER.parent / "do-epic"))
|
||||
@@ -0,0 +1,44 @@
|
||||
"""The do-epic-e2e command: runs do-epic's commands that need no tmux session and no worker against a sandbox epic on a real Jira site (docs/decisions.md → `do-epic: drives one Jira epic …`)."""
|
||||
import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
from do_epic.e2e import sandbox, scenario, settings
|
||||
|
||||
HELP = f"""usage: do-epic-e2e
|
||||
|
||||
Makes an epic in the Jira sandbox, with issues and sub-tasks, drives do-epic against it and deletes it, whatever the outcome.
|
||||
It reads three environment variables and fails, naming each that is missing:
|
||||
{chr(10).join(f' {name}: {meaning}' for name, meaning in settings.MEANING.items())}
|
||||
It prints a line per check, and exits 0 only where every check held."""
|
||||
|
||||
|
||||
def main(argv: list[str], env: typing.Mapping[str, str], launcher: pathlib.Path) -> int:
|
||||
if argv:
|
||||
print(HELP, file=sys.stdout if argv in (["-h"], ["--help"]) else sys.stderr)
|
||||
return 0 if argv in (["-h"], ["--help"]) else 1
|
||||
found = settings.read(env)
|
||||
if isinstance(found, Failure):
|
||||
print(f"do-epic-e2e cannot start: {found.reason}", file=sys.stderr)
|
||||
return 1
|
||||
report = scenario.Report()
|
||||
left: sandbox.Leftover | None = None
|
||||
try:
|
||||
with sandbox.epic_made(found, "do-epic e2e sandbox epic") as key:
|
||||
if isinstance(key, Failure):
|
||||
print(f"do-epic-e2e: {key.reason}", file=sys.stderr)
|
||||
return 1
|
||||
print(f"Sandbox epic {key} in {found.jira_url}")
|
||||
report = scenario.run(found, key, launcher, env)
|
||||
except sandbox.Leftover as failure:
|
||||
left = failure
|
||||
# The checks are printed even where the epic could not be deleted.
|
||||
for line in report.held:
|
||||
print(f"ok {line}")
|
||||
for line in report.broke:
|
||||
print(f"FAIL {line}")
|
||||
print(f"{len(report.held)} held, {len(report.broke)} broke")
|
||||
if left:
|
||||
print(f"do-epic-e2e: {left}", file=sys.stderr)
|
||||
return 1 if report.broke or left else 0
|
||||
@@ -0,0 +1,75 @@
|
||||
"""The sandbox epic the e2e command makes on the real Jira site and deletes when it ends."""
|
||||
import contextlib
|
||||
import typing
|
||||
|
||||
from atlassian.client import basic_auth, Http, jira_json, ToAdf
|
||||
from failure import Failure
|
||||
from adf_codec.convert import markdown_to_adf
|
||||
from do_epic.e2e.settings import Sandbox
|
||||
from plan_store.jira_epic import searched
|
||||
|
||||
EPIC_TYPE = "Epic"
|
||||
# How a team-managed project's workflow moves an issue to done, as do-epic's `jira-finish-transition` and `jira-finish-status` name it.
|
||||
FINISH_TRANSITION = "Done"
|
||||
FINISH_STATUS = "Done"
|
||||
DESCRIPTION = """**Language:** English
|
||||
|
||||
## Goals
|
||||
|
||||
1. **Achieve:** Prove do-epic's commands against a real Jira site.
|
||||
|
||||
## Audience
|
||||
|
||||
The person who runs do-epic's e2e command.
|
||||
"""
|
||||
|
||||
|
||||
class Leftover(Exception):
|
||||
"""The sandbox epic or one of its issues could not be deleted, so some of them may still be on the site."""
|
||||
|
||||
|
||||
def api(sandbox: Sandbox, path: str) -> str:
|
||||
return f"{sandbox.jira_url}/rest/api/3/{path}"
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def epic_made(sandbox: Sandbox, summary: str, http: Http = jira_json, to_adf: ToAdf = markdown_to_adf) -> typing.Iterator[str | Failure]:
|
||||
"""Makes an epic in the sandbox project and yields its key; a Failure where it cannot be made. Deletes the epic with its issues and sub-tasks on the way out, however the block ends."""
|
||||
authorization = basic_auth(sandbox.env_file)
|
||||
if isinstance(authorization, Failure):
|
||||
yield authorization
|
||||
return
|
||||
description = to_adf(DESCRIPTION)
|
||||
if isinstance(description, Failure):
|
||||
yield Failure(f"cannot convert the sandbox epic's description: {description.reason}")
|
||||
return
|
||||
fields = {"project": {"key": sandbox.project}, "issuetype": {"name": EPIC_TYPE}, "summary": summary, "description": description}
|
||||
made = http("POST", api(sandbox, "issue"), authorization, {"fields": fields})
|
||||
if isinstance(made, Failure):
|
||||
yield Failure(f"cannot make the sandbox epic: {made.reason}")
|
||||
return
|
||||
key = made.get("key") if isinstance(made, dict) else None
|
||||
if not isinstance(key, str):
|
||||
yield Failure(f"Jira answered the epic's creation with {made!r}, which holds no key")
|
||||
return
|
||||
try:
|
||||
yield key
|
||||
finally:
|
||||
deleted = deleted_with_children(sandbox, key, authorization, http)
|
||||
if isinstance(deleted, Failure):
|
||||
raise Leftover(f"the sandbox epic {key} and some of its issues may be left in {sandbox.project}: look for them and delete them by hand ({deleted.reason})")
|
||||
|
||||
|
||||
def deleted_with_children(sandbox: Sandbox, key: str, authorization: str, http: Http) -> None | Failure:
|
||||
"""Deletes the epic's child issues, with their sub-tasks, and then the epic: Jira deletes an epic's sub-tasks along with it, but not its child issues."""
|
||||
found = searched(sandbox.jira_url, f"the issues of {key}", f"parent = {key}", {"fields": ["summary"]}, authorization, http)
|
||||
# The epic is deleted all the same; the first failure is the one to report.
|
||||
problem = found if isinstance(found, Failure) else None
|
||||
entries = [] if isinstance(found, Failure) else found
|
||||
children = [item["key"] for item in entries if isinstance(item, dict) and isinstance(item.get("key"), str)]
|
||||
if len(children) < len(entries):
|
||||
problem = Failure(f"the search for the issues of {key} answered an entry without a key, so that issue is not deleted")
|
||||
for each in [*children, key]:
|
||||
deleted = http("DELETE", api(sandbox, f"issue/{each}?deleteSubtasks=true"), authorization, None)
|
||||
problem = problem or (deleted if isinstance(deleted, Failure) else None)
|
||||
return problem
|
||||
@@ -0,0 +1,133 @@
|
||||
"""What the e2e command does to a sandbox epic: runs `do-epic --file`, `--record-decision` and `--close-epic`, each as a process of its own against the real Jira site, and checks what the site holds after each."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import subprocess
|
||||
import tempfile
|
||||
import typing
|
||||
|
||||
from do_epic.repo_pick import CLAUDE_LABEL, HUMAN_LABEL
|
||||
from project_config.settings import Epic, Finish
|
||||
from do_epic import epic_record, findings, finish
|
||||
from record_store.jira_description import EpicDescription
|
||||
from do_epic.backend.protocol import EpicIssue, EpicStore
|
||||
from failure import Failure
|
||||
from do_epic.e2e.sandbox import FINISH_STATUS, FINISH_TRANSITION
|
||||
from do_epic.backend.jira import JiraEpic
|
||||
from model_routing.ladder_file import read_ladder, SHIPPED
|
||||
from do_epic.e2e.settings import Sandbox
|
||||
|
||||
# Seconds one do-epic command may take.
|
||||
TIMEOUT = 300
|
||||
RULE = "A label that equals a repo's name names that repo.\n"
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class Report:
|
||||
"""What the scenario found: the lines of what held, and of what didn't."""
|
||||
held: list[str] = dataclasses.field(default_factory=list)
|
||||
broke: list[str] = dataclasses.field(default_factory=list)
|
||||
|
||||
def check(self, holds: bool, what: str) -> None:
|
||||
(self.held if holds else self.broke).append(what)
|
||||
|
||||
|
||||
class DoEpic(typing.Protocol):
|
||||
"""What the steps run do-epic through: a Machine, or a stand-in in a test."""
|
||||
|
||||
def do_epic(self, *args: str, stdin: str = "") -> subprocess.CompletedProcess[str]:
|
||||
"""Runs `do-epic args`, feeding it stdin."""
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Machine:
|
||||
"""A home folder of its own, holding the do-epic settings that point at the sandbox, so that do-epic writes its state there. The sandbox's env-file stays where it is. do-epic runs in env with HOME set to the home folder, and is stopped after timeout seconds."""
|
||||
home: pathlib.Path
|
||||
launcher: pathlib.Path
|
||||
env: typing.Mapping[str, str]
|
||||
timeout: float = TIMEOUT
|
||||
|
||||
def do_epic(self, *args: str, stdin: str = "") -> subprocess.CompletedProcess[str]:
|
||||
env = {**self.env, "HOME": str(self.home)}
|
||||
try:
|
||||
return subprocess.run([str(self.launcher), *args], input=stdin, env=env, capture_output=True, text=True, timeout=self.timeout, check=False)
|
||||
except subprocess.TimeoutExpired:
|
||||
return subprocess.CompletedProcess(args, 124, "", f"do-epic {' '.join(args)} ran over {self.timeout} seconds and was stopped")
|
||||
|
||||
|
||||
def machine_made(home: pathlib.Path, sandbox: Sandbox, launcher: pathlib.Path, env: typing.Mapping[str, str]) -> Machine | Failure:
|
||||
folder = home / ".claude-projects" / "do-epic"
|
||||
folder.mkdir(parents=True)
|
||||
ladder = read_ladder(SHIPPED)
|
||||
if isinstance(ladder, Failure):
|
||||
return ladder
|
||||
(folder / "config.toml").write_text(
|
||||
f'jira-url = "{sandbox.jira_url}"\nenv-file = "{sandbox.env_file}"\nrepoless-model = "{ladder.rungs[0].model}"\n'
|
||||
f'jira-finish-transition = "{FINISH_TRANSITION}"\njira-finish-status = "{FINISH_STATUS}"\n', encoding="utf-8")
|
||||
(folder / "repo-rule.md").write_text(RULE, encoding="utf-8")
|
||||
return Machine(home, launcher, env)
|
||||
|
||||
|
||||
def run(sandbox: Sandbox, epic_key: str, launcher: pathlib.Path, env: typing.Mapping[str, str]) -> Report:
|
||||
"""Drives do-epic against the epic epic_key in the sandbox, an epic with no issue yet."""
|
||||
report = Report()
|
||||
epic = Epic(sandbox.jira_url, epic_key)
|
||||
store = JiraEpic(epic, sandbox.env_file, finish=Finish(FINISH_TRANSITION, FINISH_STATUS))
|
||||
record = epic_record.EpicRecord(EpicDescription(epic, sandbox.env_file))
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
machine = machine_made(pathlib.Path(home).resolve(), sandbox, launcher, env)
|
||||
if isinstance(machine, Failure):
|
||||
report.check(False, f"set up do-epic's settings: {machine.reason}")
|
||||
return report
|
||||
items_filed(report, store, epic_key)
|
||||
finding_filed(report, machine, store, epic_key)
|
||||
decision_recorded(report, machine, record.section, epic_key)
|
||||
closed(report, machine, store, epic_key)
|
||||
return report
|
||||
|
||||
|
||||
def read(report: Report, store: EpicStore, what: str) -> list[EpicIssue]:
|
||||
issues = store.read()
|
||||
if isinstance(issues, Failure):
|
||||
report.check(False, f"read the epic's issues {what}: {issues.reason}")
|
||||
return []
|
||||
return issues
|
||||
|
||||
|
||||
def items_filed(report: Report, store: EpicStore, epic_key: str) -> None:
|
||||
"""Files an item, its sub-task and a second item in the epic, and checks that a read answers them in the tracker's order."""
|
||||
parent = store.file(epic_key, "Sandbox item with a sub-task", "The first item.", (HUMAN_LABEL,))
|
||||
second = store.file(epic_key, "Sandbox second item", "", ())
|
||||
child = store.file(parent, "Sandbox sub-task", "", ()) if isinstance(parent, str) else parent
|
||||
for what, key in (("an item", parent), ("a second item", second), ("a sub-task", child)):
|
||||
report.check(isinstance(key, str), f"file {what} in the epic" + (f": {key.reason}" if isinstance(key, Failure) else ""))
|
||||
issues = read(report, store, "after filing")
|
||||
report.check([issue.summary for issue in issues] == ["Sandbox item with a sub-task", "Sandbox sub-task", "Sandbox second item"], f"read the filed issues back in order, the sub-task after its parent (found {[issue.summary for issue in issues]})")
|
||||
|
||||
|
||||
def finding_filed(report: Report, machine: DoEpic, store: EpicStore, epic_key: str) -> None:
|
||||
done = machine.do_epic(epic_key, findings.FILE, "Sandbox finding", findings.LABEL, CLAUDE_LABEL, stdin="Why the finding matters.\n")
|
||||
report.check(done.returncode == 0 and " Filed " in done.stdout, f"`do-epic {findings.FILE}` files the finding and says so (exit {done.returncode}: {done.stdout.strip() or done.stderr.strip()})")
|
||||
found = [issue for issue in read(report, store, "after the finding") if issue.summary == "Sandbox finding"]
|
||||
report.check(len(found) == 1 and found[0].labels == (CLAUDE_LABEL,) and found[0].detail == "Why the finding matters." and found[0].by_loop, f"the finding is an issue of the epic with its label, its detail and the loop's mark (found {found})")
|
||||
|
||||
|
||||
def decision_recorded(report: Report, machine: DoEpic, section: typing.Callable[[str], str | Failure], epic_key: str) -> None:
|
||||
done = machine.do_epic(epic_key, findings.RECORD_DECISION, "Sandbox decision", stdin="The text of the decision.\n")
|
||||
report.check(done.returncode == 0 and "Recorded the decision" in done.stdout, f"`do-epic {findings.RECORD_DECISION}` records the decision and says so (exit {done.returncode}: {done.stdout.strip() or done.stderr.strip()})")
|
||||
decisions = section("Decisions")
|
||||
report.check(isinstance(decisions, str) and "Sandbox decision" in decisions and "The text of the decision." in decisions, f"the epic's description holds the decision under ## Decisions (found {decisions!r})")
|
||||
goals = section("Goals")
|
||||
report.check(isinstance(goals, str) and "Prove do-epic's commands" in goals, f"the decision leaves the description's ## Goals as it was (found {goals!r})")
|
||||
|
||||
|
||||
def closed(report: Report, machine: DoEpic, store: EpicStore, epic_key: str) -> None:
|
||||
early = machine.do_epic(epic_key, finish.CLOSE, stdin="Too early.\n")
|
||||
still = store.closed()
|
||||
report.check(early.returncode == 1 and " open" in early.stderr and still is False, f"`do-epic {finish.CLOSE}` refuses while issues are open and leaves the epic open (exit {early.returncode}: {early.stderr.strip()}; epic closed: {still!r})")
|
||||
for issue in read(report, store, "before finishing them"):
|
||||
done = store.finish_issue(issue.key)
|
||||
report.check(done is None, f"finish {issue.key}" + (f": {done.reason}" if isinstance(done, Failure) else ""))
|
||||
late = machine.do_epic(epic_key, finish.CLOSE, stdin="Everything the sandbox epic held is done.\n")
|
||||
report.check(late.returncode == 0 and "closed it" in late.stdout, f"`do-epic {finish.CLOSE}` posts the summary and closes the epic (exit {late.returncode}: {late.stdout.strip() or late.stderr.strip()})")
|
||||
now = store.closed()
|
||||
report.check(now is True, f"Jira counts the epic as done (found {now!r})")
|
||||
@@ -0,0 +1,48 @@
|
||||
"""What the e2e command needs to know about its sandbox: three environment variables, so that neither the site nor the credentials' place is in the repo."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import re
|
||||
import typing
|
||||
|
||||
from atlassian.client import basic_auth
|
||||
from failure import Failure
|
||||
from project_config.settings import SITE
|
||||
|
||||
JIRA_URL = "DO_EPIC_E2E_JIRA_URL"
|
||||
ENV_FILE = "DO_EPIC_E2E_ENV_FILE"
|
||||
PROJECT = "DO_EPIC_E2E_PROJECT"
|
||||
PROJECT_KEY = r"[A-Z][A-Z0-9]+"
|
||||
MEANING = {
|
||||
JIRA_URL: "the sandbox Jira site's https address without a path, such as https://acme.atlassian.net",
|
||||
ENV_FILE: "the absolute path of the file holding the sandbox account's ATLASSIAN_EMAIL and ATLASSIAN_API_TOKEN",
|
||||
PROJECT: "the key of the sandbox project, a team-managed space where the account may create, edit, transition (to Done), comment on and delete issues, such as SBX",
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Sandbox:
|
||||
jira_url: str
|
||||
env_file: pathlib.Path
|
||||
project: str
|
||||
|
||||
|
||||
def read(env: typing.Mapping[str, str]) -> Sandbox | Failure:
|
||||
"""The sandbox that env names; a Failure naming each setting that is missing or malformed, and why the sandbox's env-file cannot be used."""
|
||||
values = {name: env.get(name, "").strip() for name in MEANING}
|
||||
url, project = values[JIRA_URL], values[PROJECT]
|
||||
env_file = pathlib.Path(values[ENV_FILE]) if values[ENV_FILE] else None
|
||||
usable = env_file if env_file is not None and env_file.is_absolute() else None
|
||||
faults = [
|
||||
*(f"{name} is missing: set it to {meaning}" for name, meaning in MEANING.items() if not values[name]),
|
||||
*([f"{JIRA_URL} is {url!r}, not an https address without a path"] if url and not re.fullmatch(SITE, url) else []),
|
||||
*([f"{PROJECT} is {project!r}, not a project key such as SBX"] if project and not re.fullmatch(PROJECT_KEY, project) else []),
|
||||
*([f"{ENV_FILE} is {values[ENV_FILE]!r}, not an absolute path"] if env_file is not None and usable is None else []),
|
||||
]
|
||||
if usable is not None:
|
||||
login = basic_auth(usable)
|
||||
if isinstance(login, Failure):
|
||||
faults.append(login.reason)
|
||||
if faults:
|
||||
return Failure("; ".join(faults))
|
||||
assert env_file is not None
|
||||
return Sandbox(url, env_file, project)
|
||||
@@ -0,0 +1,90 @@
|
||||
import contextlib
|
||||
import io
|
||||
from unittest import mock
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import typing
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from do_epic.e2e.settings import ENV_FILE, JIRA_URL, PROJECT
|
||||
from failure import Failure
|
||||
from do_epic.e2e import main, sandbox, scenario, settings
|
||||
|
||||
|
||||
class MainTest(unittest.TestCase):
|
||||
def run_main(self, argv: list[str]) -> tuple[int, str, str]:
|
||||
out, err = io.StringIO(), io.StringIO()
|
||||
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
|
||||
code = main.main(argv, {}, pathlib.Path("do-epic"))
|
||||
return code, out.getvalue(), err.getvalue()
|
||||
|
||||
def test_fails_naming_each_missing_setting_before_it_touches_anything(self) -> None:
|
||||
code, out, err = self.run_main([])
|
||||
self.assertEqual((code, out), (1, ""))
|
||||
for name in (JIRA_URL, ENV_FILE, PROJECT):
|
||||
self.assertIn(name, err)
|
||||
|
||||
def test_prints_its_help_for_help_and_refuses_other_words(self) -> None:
|
||||
code, out, err = self.run_main(["--help"])
|
||||
self.assertEqual((code, err), (0, ""))
|
||||
self.assertIn(JIRA_URL, out)
|
||||
self.assertEqual(self.run_main(["-h"])[:2], (0, out))
|
||||
code, out, err = self.run_main(["--status"])
|
||||
self.assertEqual((code, out), (1, ""))
|
||||
self.assertIn(main.HELP, err)
|
||||
|
||||
|
||||
class ReportedTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
made = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(made.cleanup)
|
||||
env_file = pathlib.Path(made.name) / "jira.env"
|
||||
env_file.write_text("ATLASSIAN_EMAIL=a@example.com\nATLASSIAN_API_TOKEN=secret\n", encoding="utf-8")
|
||||
env_file.chmod(0o600)
|
||||
self.env = {JIRA_URL: "https://acme.atlassian.net", ENV_FILE: str(env_file), PROJECT: "SBX"}
|
||||
|
||||
def run_scenario(self, deleted: bool, report: scenario.Report | None = None, made: str | Failure = "SBX-1") -> tuple[int, str, str]:
|
||||
@contextlib.contextmanager
|
||||
def epic_made(*args: object) -> typing.Iterator[str | Failure]:
|
||||
self.made_with = args
|
||||
yield made
|
||||
if not deleted:
|
||||
raise sandbox.Leftover("SBX-1 is still there")
|
||||
|
||||
out, err = io.StringIO(), io.StringIO()
|
||||
report = report or scenario.Report(["a check held"], ["a check broke"])
|
||||
with mock.patch.object(sandbox, "epic_made", epic_made), mock.patch.object(scenario, "run", return_value=report) as ran:
|
||||
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
|
||||
code = main.main([], self.env, pathlib.Path("do-epic"))
|
||||
self.ran_with = ran.call_args.args if ran.called else None
|
||||
return code, out.getvalue(), err.getvalue()
|
||||
|
||||
def test_prints_each_check_and_fails_where_one_broke(self) -> None:
|
||||
code, out, err = self.run_scenario(True)
|
||||
self.assertEqual((code, err), (1, ""))
|
||||
self.assertIn("ok a check held\nFAIL a check broke\n1 held, 1 broke", out)
|
||||
|
||||
def test_prints_the_checks_and_names_the_epic_left_behind_when_it_cannot_be_deleted(self) -> None:
|
||||
code, out, err = self.run_scenario(False)
|
||||
self.assertEqual(code, 1)
|
||||
self.assertIn("ok a check held", out)
|
||||
self.assertIn("SBX-1 is still there", err)
|
||||
|
||||
def test_makes_the_epic_in_the_sandbox_and_runs_the_scenario_on_it(self) -> None:
|
||||
code, out, err = self.run_scenario(True, scenario.Report(["a check held"]))
|
||||
sandbox_set = settings.read(self.env)
|
||||
self.assertEqual((code, err), (0, ""))
|
||||
self.assertEqual(self.made_with, (sandbox_set, "do-epic e2e sandbox epic"))
|
||||
self.assertEqual(self.ran_with, (sandbox_set, "SBX-1", pathlib.Path("do-epic"), self.env))
|
||||
self.assertEqual(out, "Sandbox epic SBX-1 in https://acme.atlassian.net\nok a check held\n1 held, 0 broke\n")
|
||||
|
||||
def test_fails_without_a_scenario_where_the_epic_cannot_be_made(self) -> None:
|
||||
code, out, err = self.run_scenario(True, made=Failure("Jira said no"))
|
||||
self.assertEqual((code, out, self.ran_with), (1, "", None))
|
||||
self.assertEqual(err, "do-epic-e2e: Jira said no\n")
|
||||
|
||||
def test_fails_where_every_check_held_but_the_epic_stays(self) -> None:
|
||||
self.assertEqual(self.run_scenario(False, scenario.Report(["a check held"]))[0], 1)
|
||||
@@ -0,0 +1,113 @@
|
||||
import base64
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from do_epic.e2e.sandbox import epic_made, Leftover
|
||||
from failure import Failure
|
||||
from do_epic.e2e.settings import Sandbox
|
||||
|
||||
|
||||
class JiraStub:
|
||||
"""Records each request, and answers as Jira does: a key for a new issue, the epic's children for a search."""
|
||||
|
||||
def __init__(self, refuses: str | None = None) -> None:
|
||||
self.requests: list[tuple[str, str]] = []
|
||||
self.authorizations: set[str] = set()
|
||||
self.bodies: list[object] = []
|
||||
self.refuses = refuses
|
||||
self.search_answer: object = {"issues": [{"key": "SBX-2"}, {"key": "SBX-3"}]}
|
||||
self.later_pages: list[object] = []
|
||||
|
||||
def __call__(self, method: str, url: str, authorization: str, body: object) -> object:
|
||||
self.requests.append((method, url.split("/rest/api/3/")[1]))
|
||||
self.authorizations.add(authorization)
|
||||
self.bodies.append(body)
|
||||
if self.refuses and url.endswith(self.refuses):
|
||||
return Failure(f"{method} {url} answered 403")
|
||||
if method == "POST" and url.endswith("/issue"):
|
||||
return {"key": "SBX-1"}
|
||||
if method == "POST":
|
||||
return self.later_pages.pop(0) if self.later_pages and isinstance(body, dict) and "nextPageToken" in body else self.search_answer
|
||||
return None
|
||||
|
||||
|
||||
class EpicMadeTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
made = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(made.cleanup)
|
||||
env_file = pathlib.Path(made.name) / "jira.env"
|
||||
env_file.write_text("ATLASSIAN_EMAIL=a@example.com\nATLASSIAN_API_TOKEN=secret\n", encoding="utf-8")
|
||||
env_file.chmod(0o600)
|
||||
self.sandbox = Sandbox("https://acme.atlassian.net", env_file, "SBX")
|
||||
|
||||
def test_deletes_each_child_issue_and_then_the_epic_when_the_block_ends(self) -> None:
|
||||
jira = JiraStub()
|
||||
with epic_made(self.sandbox, "An epic", jira, lambda markdown: {"type": "doc"}) as key:
|
||||
self.assertEqual(key, "SBX-1")
|
||||
self.assertEqual(jira.authorizations, {"Basic " + base64.b64encode(b"a@example.com:secret").decode()})
|
||||
self.assertEqual(jira.bodies[1], {"jql": "parent = SBX-1", "fields": ["summary"], "maxResults": 100})
|
||||
self.assertEqual(jira.requests[1:], [
|
||||
("POST", "search/jql"),
|
||||
("DELETE", "issue/SBX-2?deleteSubtasks=true"),
|
||||
("DELETE", "issue/SBX-3?deleteSubtasks=true"),
|
||||
("DELETE", "issue/SBX-1?deleteSubtasks=true"),
|
||||
])
|
||||
|
||||
def test_deletes_the_children_on_every_page_of_the_search(self) -> None:
|
||||
jira = JiraStub()
|
||||
jira.search_answer = {"issues": [{"key": "SBX-2"}], "nextPageToken": "more"}
|
||||
jira.later_pages = [{"issues": [{"key": "SBX-3"}]}]
|
||||
with epic_made(self.sandbox, "An epic", jira, lambda markdown: {"type": "doc"}):
|
||||
pass
|
||||
self.assertEqual(jira.bodies[2], {"jql": "parent = SBX-1", "fields": ["summary"], "maxResults": 100, "nextPageToken": "more"})
|
||||
self.assertEqual([url for method, url in jira.requests if method == "DELETE"], [f"issue/SBX-{n}?deleteSubtasks=true" for n in (2, 3, 1)])
|
||||
|
||||
def test_deletes_the_rest_and_says_so_when_a_found_issue_has_no_key(self) -> None:
|
||||
jira = JiraStub()
|
||||
jira.search_answer = {"issues": [{"key": "SBX-2"}, {"summary": "no key"}]}
|
||||
with self.assertRaises(Leftover) as left:
|
||||
with epic_made(self.sandbox, "An epic", jira, lambda markdown: {"type": "doc"}):
|
||||
pass
|
||||
self.assertIn("without a key", str(left.exception))
|
||||
self.assertEqual([url for method, url in jira.requests if method == "DELETE"], ["issue/SBX-2?deleteSubtasks=true", "issue/SBX-1?deleteSubtasks=true"])
|
||||
|
||||
def test_deletes_them_when_the_block_raises(self) -> None:
|
||||
jira = JiraStub()
|
||||
with self.assertRaises(ZeroDivisionError):
|
||||
with epic_made(self.sandbox, "An epic", jira, lambda markdown: {"type": "doc"}):
|
||||
1 / 0
|
||||
self.assertEqual(jira.requests[-1], ("DELETE", "issue/SBX-1?deleteSubtasks=true"))
|
||||
|
||||
def test_names_the_epic_left_behind_when_a_delete_is_refused(self) -> None:
|
||||
with self.assertRaises(Leftover) as left:
|
||||
with epic_made(self.sandbox, "An epic", JiraStub(refuses="SBX-3?deleteSubtasks=true"), lambda markdown: {"type": "doc"}):
|
||||
pass
|
||||
self.assertIn("SBX-1", str(left.exception))
|
||||
|
||||
def test_deletes_the_epic_all_the_same_and_says_so_when_the_search_for_its_issues_answers_nothing_usable(self) -> None:
|
||||
jira = JiraStub()
|
||||
jira.search_answer = {"errorMessages": ["no"]}
|
||||
with self.assertRaises(Leftover) as left:
|
||||
with epic_made(self.sandbox, "An epic", jira, lambda markdown: {"type": "doc"}):
|
||||
pass
|
||||
self.assertIn("something other than a page of issues", str(left.exception))
|
||||
self.assertEqual(jira.requests[1:], [("POST", "search/jql"), ("DELETE", "issue/SBX-1?deleteSubtasks=true")])
|
||||
|
||||
def test_deletes_the_epic_all_the_same_and_gives_the_reason_when_the_search_for_its_issues_is_refused(self) -> None:
|
||||
jira = JiraStub(refuses="search/jql")
|
||||
with self.assertRaises(Leftover) as left:
|
||||
with epic_made(self.sandbox, "An epic", jira, lambda markdown: {"type": "doc"}):
|
||||
pass
|
||||
self.assertIn("POST https://acme.atlassian.net/rest/api/3/search/jql answered 403", str(left.exception))
|
||||
self.assertNotIn("something other than a page of issues", str(left.exception))
|
||||
self.assertEqual(jira.requests[1:], [("POST", "search/jql"), ("DELETE", "issue/SBX-1?deleteSubtasks=true")])
|
||||
|
||||
def test_yields_a_failure_and_deletes_nothing_when_the_epic_cannot_be_made(self) -> None:
|
||||
jira = JiraStub(refuses="/issue")
|
||||
with epic_made(self.sandbox, "An epic", jira, lambda markdown: {"type": "doc"}) as key:
|
||||
self.assertIsInstance(key, Failure)
|
||||
self.assertEqual(jira.requests, [("POST", "issue")])
|
||||
@@ -0,0 +1,232 @@
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import typing
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from do_epic.repo_pick import CLAUDE_LABEL
|
||||
from failure import Failure
|
||||
from do_epic import findings, finish
|
||||
from do_epic.config import load
|
||||
from do_epic.backend.fake import MemoryEpic
|
||||
from do_epic.e2e.settings import Sandbox
|
||||
from do_epic.e2e import scenario
|
||||
|
||||
EPIC = "SBX-1"
|
||||
|
||||
|
||||
def done(code: int, out: str = "", err: str = "") -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.CompletedProcess([], code, out, err)
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class Scripted:
|
||||
"""A do-epic that answers each call with the next of its answers, each a function that may also change the epic as do-epic would, and notes the call."""
|
||||
answers: list[typing.Callable[[], subprocess.CompletedProcess[str]]]
|
||||
calls: list[tuple[tuple[str, ...], str]] = dataclasses.field(default_factory=list)
|
||||
|
||||
def do_epic(self, *args: str, stdin: str = "") -> subprocess.CompletedProcess[str]:
|
||||
self.calls.append((args, stdin))
|
||||
return self.answers.pop(0)()
|
||||
|
||||
|
||||
class ReportTest(unittest.TestCase):
|
||||
def test_sorts_each_check_into_what_held_and_what_broke(self) -> None:
|
||||
report = scenario.Report()
|
||||
report.check(True, "one")
|
||||
report.check(False, "two")
|
||||
self.assertEqual((report.held, report.broke), (["one"], ["two"]))
|
||||
|
||||
|
||||
class ItemsFiledTest(unittest.TestCase):
|
||||
def test_holds_where_the_epic_answers_the_issues_in_order(self) -> None:
|
||||
report = scenario.Report()
|
||||
scenario.items_filed(report, MemoryEpic(EPIC), EPIC)
|
||||
self.assertEqual(report.broke, [])
|
||||
self.assertEqual(len(report.held), 4)
|
||||
|
||||
def test_breaks_where_the_issues_come_back_in_another_order(self) -> None:
|
||||
class Reversed(MemoryEpic):
|
||||
def read(self) -> list[typing.Any] | Failure:
|
||||
found = super().read()
|
||||
return found if isinstance(found, Failure) else found[::-1]
|
||||
|
||||
report = scenario.Report()
|
||||
scenario.items_filed(report, Reversed(EPIC), EPIC)
|
||||
self.assertEqual(len(report.held), 3)
|
||||
self.assertEqual(len(report.broke), 1)
|
||||
self.assertIn("read the filed issues back in order", report.broke[0])
|
||||
|
||||
def test_breaks_naming_why_where_the_epic_refuses_every_call(self) -> None:
|
||||
report = scenario.Report()
|
||||
scenario.items_filed(report, MemoryEpic(EPIC, fails="Jira is down"), EPIC)
|
||||
self.assertEqual(report.held, [])
|
||||
self.assertEqual(report.broke[:3], [f"file {what} in the epic: Jira is down" for what in ("an item", "a second item", "a sub-task")])
|
||||
self.assertEqual(report.broke[3], "read the epic's issues after filing: Jira is down")
|
||||
self.assertIn("found []", report.broke[4])
|
||||
|
||||
|
||||
class FindingFiledTest(unittest.TestCase):
|
||||
def filing(self, store: MemoryEpic, labels: tuple[str, ...] = (CLAUDE_LABEL,), code: int = 0, out: str = "10:00 Filed SBX-100") -> typing.Callable[[], subprocess.CompletedProcess[str]]:
|
||||
def answer() -> subprocess.CompletedProcess[str]:
|
||||
store.file(EPIC, "Sandbox finding", "Why the finding matters.", labels)
|
||||
return done(code, out)
|
||||
return answer
|
||||
|
||||
def test_holds_where_do_epic_files_the_finding_with_its_label_and_detail(self) -> None:
|
||||
store = MemoryEpic(EPIC)
|
||||
machine = Scripted([self.filing(store)])
|
||||
report = scenario.Report()
|
||||
scenario.finding_filed(report, machine, store, EPIC)
|
||||
self.assertEqual((report.broke, len(report.held)), ([], 2))
|
||||
self.assertEqual(machine.calls, [((EPIC, findings.FILE, "Sandbox finding", findings.LABEL, CLAUDE_LABEL), "Why the finding matters.\n")])
|
||||
|
||||
def test_breaks_where_do_epic_fails_and_files_nothing(self) -> None:
|
||||
report = scenario.Report()
|
||||
scenario.finding_filed(report, Scripted([lambda: done(1, err="no")]), MemoryEpic(EPIC), EPIC)
|
||||
self.assertEqual(report.held, [])
|
||||
self.assertIn("exit 1: no", report.broke[0])
|
||||
self.assertIn("found []", report.broke[1])
|
||||
|
||||
def test_breaks_where_the_finding_lacks_the_label_asked_for(self) -> None:
|
||||
store = MemoryEpic(EPIC)
|
||||
report = scenario.Report()
|
||||
scenario.finding_filed(report, Scripted([self.filing(store, labels=())]), store, EPIC)
|
||||
self.assertEqual(len(report.held), 1)
|
||||
self.assertEqual(len(report.broke), 1)
|
||||
self.assertIn("with its label", report.broke[0])
|
||||
|
||||
def test_breaks_where_do_epic_exits_cleanly_but_says_nothing_of_filing(self) -> None:
|
||||
store = MemoryEpic(EPIC)
|
||||
report = scenario.Report()
|
||||
scenario.finding_filed(report, Scripted([self.filing(store, out="")]), store, EPIC)
|
||||
self.assertEqual(len(report.held), 1)
|
||||
self.assertIn("files the finding and says so", report.broke[0])
|
||||
|
||||
|
||||
class DecisionRecordedTest(unittest.TestCase):
|
||||
SECTIONS: typing.ClassVar[dict[str, str | Failure]] = {"Decisions": "### Sandbox decision\nThe text of the decision.", "Goals": "1. Prove do-epic's commands work."}
|
||||
|
||||
def recorded(self, answer: subprocess.CompletedProcess[str], **sections: str | Failure) -> tuple[scenario.Report, Scripted]:
|
||||
machine = Scripted([lambda: answer])
|
||||
report = scenario.Report()
|
||||
found = {**self.SECTIONS, **sections}
|
||||
scenario.decision_recorded(report, machine, lambda name: found[name], EPIC)
|
||||
return report, machine
|
||||
|
||||
def test_holds_where_the_decision_is_in_the_description_and_the_goals_are_as_they_were(self) -> None:
|
||||
report, machine = self.recorded(done(0, "10:00 Recorded the decision"))
|
||||
self.assertEqual((report.broke, len(report.held)), ([], 3))
|
||||
self.assertEqual(machine.calls, [((EPIC, findings.RECORD_DECISION, "Sandbox decision"), "The text of the decision.\n")])
|
||||
|
||||
def test_breaks_where_the_decision_is_not_recorded(self) -> None:
|
||||
report, _ = self.recorded(done(1, err="no"), Decisions="")
|
||||
self.assertEqual(len(report.held), 1)
|
||||
self.assertIn("exit 1: no", report.broke[0])
|
||||
self.assertIn("holds the decision under ## Decisions", report.broke[1])
|
||||
|
||||
def test_breaks_where_only_half_of_the_decision_is_recorded(self) -> None:
|
||||
report, _ = self.recorded(done(0, "Recorded the decision"), Decisions="### Sandbox decision")
|
||||
self.assertEqual(len(report.broke), 1)
|
||||
|
||||
def test_breaks_where_the_goals_changed(self) -> None:
|
||||
report, _ = self.recorded(done(0, "Recorded the decision"), Goals="")
|
||||
self.assertEqual(len(report.broke), 1)
|
||||
self.assertIn("leaves the description's ## Goals", report.broke[0])
|
||||
|
||||
def test_breaks_without_raising_where_the_description_cannot_be_read(self) -> None:
|
||||
report, _ = self.recorded(done(0, "Recorded the decision"), Decisions=Failure("Jira is down"), Goals=Failure("Jira is down"))
|
||||
self.assertEqual((len(report.held), len(report.broke)), (1, 2))
|
||||
|
||||
|
||||
class ClosedTest(unittest.TestCase):
|
||||
def closing(self, store: MemoryEpic, early: typing.Callable[[], subprocess.CompletedProcess[str]] | None = None, late: typing.Callable[[], subprocess.CompletedProcess[str]] | None = None) -> scenario.Report:
|
||||
def closes() -> subprocess.CompletedProcess[str]:
|
||||
store.close("Everything the sandbox epic held is done.")
|
||||
return done(0, "10:00 Posted the summary and closed it")
|
||||
|
||||
store.file(EPIC, "An open issue", "", ())
|
||||
report = scenario.Report()
|
||||
self.machine = Scripted([early or (lambda: done(1, err="SBX-100 is still open")), late or closes])
|
||||
scenario.closed(report, self.machine, store, EPIC)
|
||||
return report
|
||||
|
||||
def test_holds_where_do_epic_refuses_while_an_issue_is_open_and_closes_the_epic_after(self) -> None:
|
||||
store = MemoryEpic(EPIC)
|
||||
report = self.closing(store)
|
||||
self.assertEqual((report.broke, len(report.held)), ([], 4))
|
||||
self.assertEqual(store.comments[EPIC], ["Everything the sandbox epic held is done."])
|
||||
self.assertEqual(self.machine.calls, [((EPIC, finish.CLOSE), "Too early.\n"), ((EPIC, finish.CLOSE), "Everything the sandbox epic held is done.\n")])
|
||||
self.assertTrue(all(issue.done for issue in store.issues))
|
||||
|
||||
def test_breaks_where_the_early_close_closes_the_epic(self) -> None:
|
||||
store = MemoryEpic(EPIC)
|
||||
|
||||
def closes_early() -> subprocess.CompletedProcess[str]:
|
||||
store.close("Too early.")
|
||||
return done(1, err="SBX-100 is still open")
|
||||
|
||||
report = self.closing(store, early=closes_early)
|
||||
self.assertEqual(len(report.broke), 1)
|
||||
self.assertIn("refuses while issues are open", report.broke[0])
|
||||
|
||||
def test_breaks_where_the_late_close_exits_cleanly_but_leaves_the_epic_open(self) -> None:
|
||||
report = self.closing(MemoryEpic(EPIC), late=lambda: done(0, "Posted the summary and closed it"))
|
||||
self.assertEqual(len(report.broke), 1)
|
||||
self.assertIn("Jira counts the epic as done (found False)", report.broke[0])
|
||||
|
||||
def test_breaks_where_the_late_close_fails(self) -> None:
|
||||
report = self.closing(MemoryEpic(EPIC), late=lambda: done(1, err="refused"))
|
||||
self.assertIn("exit 1: refused", report.broke[0])
|
||||
|
||||
def test_breaks_without_raising_where_the_epic_cannot_be_read(self) -> None:
|
||||
report = self.closing(MemoryEpic(EPIC, fails="Jira is down"))
|
||||
self.assertEqual(len(report.held), 1)
|
||||
self.assertIn("read the epic's issues before finishing them: Jira is down", report.broke)
|
||||
self.assertIn("Jira counts the epic as done (found Failure(reason='Jira is down'))", report.broke)
|
||||
|
||||
|
||||
class MachineTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
made = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(made.cleanup)
|
||||
self.folder = pathlib.Path(made.name).resolve()
|
||||
self.env_file = self.folder / "jira.env"
|
||||
self.env_file.write_text("ATLASSIAN_EMAIL=a@example.com\nATLASSIAN_API_TOKEN=secret\n", encoding="utf-8")
|
||||
self.env_file.chmod(0o600)
|
||||
self.sandbox = Sandbox("https://acme.atlassian.net", self.env_file, "SBX")
|
||||
|
||||
def script(self, body: str) -> pathlib.Path:
|
||||
launcher = self.folder / "launcher"
|
||||
launcher.write_text(f"#!/bin/sh\n{body}\n", encoding="utf-8")
|
||||
launcher.chmod(0o755)
|
||||
return launcher
|
||||
|
||||
def test_writes_settings_that_do_epic_loads_and_that_point_at_the_sandbox(self) -> None:
|
||||
home = self.folder / "home"
|
||||
machine = scenario.machine_made(home, self.sandbox, pathlib.Path("do-epic"), {"A": "b"})
|
||||
assert not isinstance(machine, Failure), machine
|
||||
self.assertEqual((machine.home, machine.launcher, machine.env), (home, pathlib.Path("do-epic"), {"A": "b"}))
|
||||
config = load(home)
|
||||
assert not isinstance(config, Failure), config
|
||||
self.assertEqual((config.jira_url, config.env_file, config.repo_rule), ("https://acme.atlassian.net", self.env_file, scenario.RULE))
|
||||
self.assertEqual((config.finish.transition, config.finish.status) if config.finish else None, ("Done", "Done"))
|
||||
|
||||
def test_runs_do_epic_in_the_given_environment_with_the_machines_home_and_the_words_and_stdin_given(self) -> None:
|
||||
machine = scenario.Machine(self.folder, self.script('echo "$HOME $KEPT $*"; cat'), {"KEPT": "yes", "HOME": "/elsewhere"})
|
||||
finished = machine.do_epic("ABC-1", "--file", stdin="detail")
|
||||
self.assertEqual((finished.returncode, finished.stdout), (0, f"{self.folder} yes ABC-1 --file\ndetail"))
|
||||
|
||||
def test_answers_a_failing_do_epic_with_its_exit_and_output(self) -> None:
|
||||
finished = scenario.Machine(self.folder, self.script("echo out; echo err >&2; exit 3"), {}).do_epic("ABC-1")
|
||||
self.assertEqual((finished.returncode, finished.stdout, finished.stderr), (3, "out\n", "err\n"))
|
||||
|
||||
def test_stops_a_do_epic_that_runs_over_its_time_and_says_so(self) -> None:
|
||||
finished = scenario.Machine(self.folder, self.script("exec sleep 30"), {}, timeout=0.2).do_epic("ABC-1")
|
||||
self.assertEqual(finished.returncode, 124)
|
||||
self.assertIn("ran over 0.2 seconds", finished.stderr)
|
||||
self.assertIn("ABC-1", finished.stderr)
|
||||
@@ -0,0 +1,54 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from do_epic.e2e.settings import ENV_FILE, JIRA_URL, PROJECT, read, Sandbox
|
||||
from failure import Failure
|
||||
|
||||
|
||||
class SettingsTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
made = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(made.cleanup)
|
||||
self.env_file = pathlib.Path(made.name) / "jira.env"
|
||||
self.env_file.write_text("ATLASSIAN_EMAIL=a@example.com\nATLASSIAN_API_TOKEN=secret\n", encoding="utf-8")
|
||||
self.env_file.chmod(0o600)
|
||||
self.env = {JIRA_URL: "https://acme.atlassian.net", ENV_FILE: str(self.env_file), PROJECT: "SBX"}
|
||||
|
||||
def failure(self, env: dict[str, str]) -> str:
|
||||
found = read(env)
|
||||
assert isinstance(found, Failure), found
|
||||
return found.reason
|
||||
|
||||
def test_reads_the_site_the_env_file_and_the_project(self) -> None:
|
||||
self.assertEqual(read(self.env), Sandbox("https://acme.atlassian.net", self.env_file, "SBX"))
|
||||
|
||||
def test_names_each_setting_that_is_missing(self) -> None:
|
||||
reason = self.failure({})
|
||||
for name in (JIRA_URL, ENV_FILE, PROJECT):
|
||||
self.assertIn(name, reason)
|
||||
self.assertNotIn(PROJECT, self.failure({k: v for k, v in self.env.items() if k != JIRA_URL}))
|
||||
|
||||
def test_treats_a_blank_setting_as_missing(self) -> None:
|
||||
self.assertIn(f"{ENV_FILE} is missing", self.failure({**self.env, ENV_FILE: " "}))
|
||||
|
||||
def test_names_every_fault_at_once_separated_by_semicolons(self) -> None:
|
||||
self.env_file.chmod(0o644)
|
||||
reason = self.failure({**self.env, PROJECT: "sbx-1"})
|
||||
self.assertTrue(reason.startswith(f"{PROJECT} is"))
|
||||
self.assertIn(f"; {self.env_file} ", reason)
|
||||
|
||||
def test_refuses_a_site_that_is_not_an_https_address_without_a_path(self) -> None:
|
||||
self.assertIn(JIRA_URL, self.failure({**self.env, JIRA_URL: "http://acme.atlassian.net/jira"}))
|
||||
|
||||
def test_refuses_an_env_file_that_cannot_be_read_naming_the_file(self) -> None:
|
||||
gone = self.env_file.parent / "gone.env"
|
||||
self.assertIn(str(gone), self.failure({**self.env, ENV_FILE: str(gone)}))
|
||||
self.env_file.chmod(0o644)
|
||||
self.assertIn(str(self.env_file), self.failure(self.env))
|
||||
|
||||
def test_refuses_an_env_file_that_is_not_an_absolute_path(self) -> None:
|
||||
self.assertIn(ENV_FILE, self.failure({**self.env, ENV_FILE: "jira.env"}))
|
||||
@@ -13,7 +13,6 @@
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 420 | 0.12.0 | | | **End a worker that printed its final result and lingers, by killing the background shells it left, after a few minutes, and print each kill with its evidence on loop-claude's terminal.** | 2 | 2 | 2 | 4 | 3 | 1.5 by 6 | 2.0 |
|
||||
| 417 | 0.12.0 | | | **Run do-epic end to end against a sandbox epic on a real Jira site with a command of its own, outside `run-tests`.** | 2 | 1 | 1 | 2 | 3 | 1 by 5 | 1.9 |
|
||||
| 422 | 0.12.0 | 420 | | **Check every 5 minutes whether a tmux window of the loop looks stuck by other signs than a lingering worker, investigate it with a headless claude, and kill only the process the investigation names.** | 3 | 2 | 3 | 6 | 5 | 1.5 by 4 | 0.8 |
|
||||
| 445 | 0.13.0 | | | **Refuse a plan that holds an item a `Finish item <ID>` commit already removed, so a merge resolution cannot bring a finished item back.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3, 1.1 by 2 | 1.8 |
|
||||
| 365 | 0.13.0 | | | **Have `score-outcomes` read the finished and dropped items of a plan outside the checkout, from the project repo's history or from Jira's issue changelogs.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
|
||||
@@ -86,18 +85,6 @@ Touches:
|
||||
- `bin/loop_claude/loop/lingering_workers.py`
|
||||
- `README.md`: `## Tools` (small)
|
||||
|
||||
### 417. Run do-epic end to end against a sandbox epic on a real Jira site with a command of its own, outside `run-tests`.
|
||||
|
||||
The sandbox is the Jira site `larvit.atlassian.net`, project `SCRUM`: a team-managed test-data space with Epic, Task, Story and Subtask issue types. The test creates its own epic there, with children and sub-tasks, and deletes them when it ends, so it leaves the space's other issues alone. Its account may create, edit, transition, comment on and delete issues in `SCRUM`. The command reads the env-file's path, holding `ATLASSIAN_EMAIL` and `ATLASSIAN_API_TOKEN`, from a setting of its own, never from the repo.
|
||||
|
||||
The test never decides at run time whether to check anything: the command fails, naming what is missing, where the site or its env-file isn't set (`docs/decisions.md` → `do-epic: drives one Jira epic …`).
|
||||
|
||||
Touches:
|
||||
|
||||
- `README.md`: `## Test` (small)
|
||||
- `bin/do-epic-e2e`
|
||||
- `bin/do_epic/e2e/`
|
||||
|
||||
### 422. Check every 5 minutes whether a tmux window of the loop looks stuck by other signs than a lingering worker, investigate it with a headless claude, and kill only the process the investigation names.
|
||||
|
||||
Item 420 ends a worker that printed its final result and lingers. A window can also stall earlier: a test run that hangs on a shell ignoring SIGINT (item 402's first full run), a pane with no output while its process tree does no work, or a background shell older than the chunk's usual length.
|
||||
|
||||
Reference in New Issue
Block a user