Answer architecture round 1: 0 🔴, 36 🟡 (0 misread); chunk_driver holds what both drivers share, and Jira and review-state reads have one home each
CI / tests (pull_request) Successful in 5m11s
CI / tests (pull_request) Successful in 5m11s
This commit is contained in:
@@ -39,6 +39,10 @@
|
||||
!bin/branch_chunks/*.py
|
||||
!bin/chunk-pr
|
||||
!bin/chunk-preflight
|
||||
!bin/chunk_driver/
|
||||
!bin/chunk_driver/*.py
|
||||
!bin/chunk_driver/project/
|
||||
!bin/chunk_driver/project/*.py
|
||||
!bin/chunk_pr/
|
||||
!bin/chunk_pr/*.py
|
||||
!bin/chunk_preflight/
|
||||
|
||||
@@ -222,18 +222,19 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests. Beside the packages, `bin/`'s root holds only the launchers, the tools of one file with their tests, `test_env.py`, `failure.py` and `setup.cfg`. `failure.py`'s `Failure`, which modules across `bin/` return for a failed step, belongs to no single package. `agent_cycle/`, `atlassian/`, `branch_chunks/`, `claude_code/`, `command_lines/`, `forge/`, `git_cli/`, `loop_effects/`, `loop_state/`, `machine_load/`, `model_routing/`, `notifier/`, `plan/`, `plan_store/`, `record_store/`, `release_plan/`, `review_rules/`, `rung_replay/`, `text_files/` and `transcript_archive/` serve more than one tool. No package imports one that imports it back, which `repo_checks/test_package_cycles.py` checks.
|
||||
|
||||
- `adf_codec/` converts markdown to ADF, the rich text of Jira Cloud and Confluence, and back, in a throwaway container running a pinned `@larvit/adf-codec`. It builds the container's image inside a heavy slot.
|
||||
- `agent_cycle/` runs one cycle (`run.py`): a headless claude worker, then the judge of its transcript. It also holds the interrupt, the tmux window and the routing of the worker's rung. The tool that drives the cycles supplies the prompts and file names of its own workflow (`worker_prompts.py`), and reads which items a worker took in. `loop_claude` drives it, and `do_epic` will; `agent_cycle` doesn't import either.
|
||||
- `atlassian/` holds what the Jira and Confluence backends share (`client.py`): one Atlassian account's credentials, the request both backends send, and each service's error messages.
|
||||
- `agent_cycle/` runs one cycle (`run.py`): a headless claude worker, then the judge of its transcript. It also holds the interrupt, the tmux window and the routing of the worker's rung. The tool that drives the cycles supplies the prompts and file names of its own workflow (`worker_prompts.py`), and reads which items a worker took in. `loop_claude` and `do_epic` drive it; `agent_cycle` doesn't import either.
|
||||
- `atlassian/` holds what the Jira and Confluence backends share (`client.py`): one Atlassian account's credentials, the request both backends send, and each service's error messages. `jira.py` holds what every reader of Jira shares: a search read page by page, the head of each entry it answers, an issue's links and the transition that finishes it, for `plan_store` and `do_epic`.
|
||||
- `branch_chunks/` cuts a default branch into chunks and finds, for each, the items it shipped and the later chunks that fixed its code, for `score_outcomes` and `routing_figures`. It imports `reading_cost`'s record path.
|
||||
- `chunk_driver/` holds what `loop_claude` and `do_epic` share as they drive chunks through `agent_cycle`: how a cycle starts (`cycle_start.py`) and closes (`chunk_run.py`), the claim on a checkout before a chunk starts in it (`claims.py`), the prompts every worker gets (`prompts.py`), and a checkout's premises, records and saved models (`project/`). It imports neither tool, and `repo_checks` fails any package that imports one of the two.
|
||||
- `command_lines/` reads a shell command: its simple commands, and a docker call's subcommand and memory limits, for `command_patterns`, `reading_cost`, `routing_figures`, `rule_hook` and `run_heavy`.
|
||||
- `deploy_release` imports `release_plan`'s `release_standing`.
|
||||
- `do_epic/` holds `do-epic`, the tool that drives one Jira epic. `main.py` starts the epic's chunks and runs one chunk's cycles in its window, through `agent_cycle`. `planning.py` settles, before the chunks start, where each open issue is worked and the order of the issues do-epic filed, through `split.py`, `repo_pick.py` and `ranking.py`. `scheduler.py` says which issues run now: as many as the epic's cap and each repo's `parallel-chunks` allow, and none in a repo where loop-claude runs chunks. `gates.py` reads once whether a chunk's pull request waits on its checks, an approval or a new review, and `parked.py` keeps each chunk that does in a state file, so it takes no running slot while it waits and goes on waiting after a reboot. `chunk_prompt.py` words what a chunk's worker reads. Through `findings.py` that worker files a finding as an issue of the epic, with `--file`, and appends a decision to its description, with `--record-decision`. `finish.py` ends an epic that holds no open issue: its finishing chunk files a release review of each repo the epic changed, has the human confirm each goal, and with `--close-epic` posts the summary on the epic and closes it. `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section, language line and cap line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`. `repo_pick.py` asks claude, by `repo-rule.md`, where an epic item is worked, and records the answer as a label on the issue. `split.py` has claude predict from the code which repos an item changes, and files a sub-task per repo under it where there are several. `ranking.py` orders an epic's items by score over one chain of goals, the epic's, then code health, then the item's repo's, through `plan`'s ranking, and re-ranks the tracker's backlog to that order through `backend/`. `backend/` reads, files and ranks the epic's issues, adds the labels `repo_pick.py` and `split.py` record, posts comments, moves a finished chunk's issue to done, and closes the epic. `e2e/` holds `do-epic-e2e`.
|
||||
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and reads its checks, once or until they settle (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
|
||||
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and reads its checks, once or until they settle (`pulls.py`), and reads a pull request's review state (`reviews.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
|
||||
- `git_cli/` runs git for every tool (`git_repo.py`), and reads the git calls in a shell command (`git_command.py`).
|
||||
- `judge_replay` imports `agent_cycle`'s judge and verdicts, and `loop_claude`'s premises, records and `ANSWER_DESTINATIONS`.
|
||||
- `judge_replay` imports `agent_cycle`'s judge and verdicts, and `chunk_driver`'s premises, records and `ANSWER_DESTINATIONS`.
|
||||
- `loop_claude` runs `chunk-preflight` as a process of its own.
|
||||
- `loop_effects/` holds `Effects`: the processes, terminal, clock and git a tool reaches, injected so its tests can script them. loop-claude, both replay tools and `reviewer-merge-trials`' driver use it.
|
||||
- `loop_state/` holds where loop-claude keeps its files (`paths.py`), which `rung_replay` reads too.
|
||||
- `loop_effects/` holds `Effects`: the processes, terminal, clock and git a tool reaches, injected so its tests can script them. loop-claude, do-epic, both replay tools and `reviewer-merge-trials`' driver use it. `test_doubles.py` holds the doubles their tests share, such as `FakeEffects`.
|
||||
- `loop_state/` holds where loop-claude and do-epic keep their files (`paths.py`), which `rung_replay` reads too.
|
||||
- `machine_load/` takes the machine's heavy slots, its shared lock files, for `run_heavy` and `adf_codec`, and reads the memory share that loop-claude waits for. It also gives `mutate_changes` a slot's CPU count.
|
||||
- `model_routing/` holds the shipped routing ladder (`routing-ladder.json`) and its steps (`ladder.py`), and reads a ladder from that file or a project's own (`ladder_file.py`), for loop-claude, `rule_hook`, `do_epic`, both replay tools and `subagent-rung`. It finds a project's config folder through `project_config`. It also holds the list prices, which `routing_figures` and `transcript_archive` read.
|
||||
- `notifier/` sends the human a DM through the notifier a project's config names, for `notify` and for loop-claude through `loop_effects`. Its backends are `mattermost.py` and `slack.py`.
|
||||
@@ -246,7 +247,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `rest_client/` sends a JSON request to a REST API and reads its credentials from an env file, for `notifier`, `atlassian` and `record_store`.
|
||||
- `review_rules/` holds the larv-review rules that several tools apply: its passes, its review folder, its round commit titles and the comprehension history's format.
|
||||
- `routing_figures/` measures loop-claude's archive, and each of its projects' default branches, for loop-claude's routing calls.
|
||||
- `rung_replay/` holds what `judge_replay` and `review_replay` share: the parallel replay runner, the argument reader, the candidate rungs and the exits, and `judge_replay`'s rung search. It reads `agent_cycle`'s retry count and `loop_claude`'s models file, and finds that file through `loop_state`'s paths. Its `archive.py` keeps each replay's runs and its report in a `replays/` folder of loop-claude's archive.
|
||||
- `rung_replay/` holds what `judge_replay` and `review_replay` share: the parallel replay runner, the argument reader, the candidate rungs and the exits, and `judge_replay`'s rung search. It reads `agent_cycle`'s retry count and `chunk_driver`'s models file, and finds that file through `loop_state`'s paths. Its `archive.py` keeps each replay's runs and its report in a `replays/` folder of loop-claude's archive.
|
||||
- `test_env.py` serves the tests.
|
||||
- `text_files/` reads a Markdown file's `##` sections, quotes a passage and joins names into a list, reads the text piped into a command, stamps times in UTC, replaces a file in one step, private where it is new, and gzips a file beside its plain name, by which it reads it afterwards.
|
||||
- `transcript_archive/` stores loop-claude's transcripts. Its `project_key.py` names each project's archive folder, loop-claude session and models-file entry.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""One cycle of a headless claude worker and the judge of its transcript, for every entry point that drives such cycles: loop-claude today, do-epic next (docs/decisions.md → `do-epic: drives one Jira epic …`).
|
||||
"""One cycle of a headless claude worker and the judge of its transcript, for every entry point that drives such cycles: loop-claude and do-epic (docs/decisions.md → `do-epic: drives one Jira epic …`).
|
||||
|
||||
run.py runs a cycle through the states in states.py: worker.py runs the worker, judge.py rules on its transcript, interrupt.py puts a question to the human in the tmux window tmux.py names, and routing.py picks the rung the worker runs on next. An entry point hands the cycle what is its own: its prompts and file names (worker_prompts.py) and how it reads the plan items a worker took in (run.py's Cycle.taken_in). No module here imports an entry point's package: loop_claude imports this one, so bin/repo_checks/test_package_cycles.py fails on such an import. A reader finds a module by its name, so the modules stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows that.
|
||||
run.py runs a cycle through the states in states.py: worker.py runs the worker, judge.py rules on its transcript, interrupt.py puts a question to the human in the tmux window tmux.py names, and routing.py picks the rung the worker runs on next. An entry point hands the cycle what is its own: its prompts and file names (worker_prompts.py) and how it reads the plan items a worker took in (run.py's Cycle.taken_in). What two entry points share beyond the cycle lives in bin/chunk_driver/. No module here imports an entry point's package, and bin/repo_checks/test_package_cycles.py fails on such an import. A reader finds a module by its name, so the modules stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows that.
|
||||
"""
|
||||
|
||||
@@ -34,7 +34,7 @@ class Cycle:
|
||||
|
||||
Each step reads the few fields it needs by name, so the fields stay at one level. docs/decisions.md → `principles: about seven things per level, …` allows more than seven where a reader finds them by name.
|
||||
"""
|
||||
# The claude-config release and commit the cycle runs under, such as `v0.8.0 (<sha>)`, or why they are unknown: loop/main.py's setup_version.
|
||||
# The claude-config release and commit the cycle runs under, such as `v0.8.0 (<sha>)`, or why they are unknown: chunk_driver/cycle_start.py's setup_version.
|
||||
config: str
|
||||
effects: Effects
|
||||
# What chunk-preflight found, or why it could not run, for a fresh worker's prompt; empty when the session was open as the cycle began.
|
||||
|
||||
@@ -0,0 +1,231 @@
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import io
|
||||
import json
|
||||
import pathlib
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
import uuid
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from loop_effects.test_doubles import assistant, FakeEffects, human, NOW, STAMP, Streamed
|
||||
from failure import Failure
|
||||
from agent_cycle import interrupt
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from machine_load import memory
|
||||
from loop_state import paths
|
||||
from model_routing.ladder import Rung
|
||||
from agent_cycle.stop import Stop
|
||||
from agent_cycle import tmux
|
||||
from text_files.utc_stamp import utc
|
||||
from agent_cycle.test_tmux import WINDOW
|
||||
|
||||
|
||||
class Interrupts(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.home = pathlib.Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, self.home)
|
||||
self.paths = paths.paths_from(self.home)
|
||||
self.transcript = self.home / "t.jsonl"
|
||||
self.transcript.write_text(json.dumps(human("a")) + "\n")
|
||||
self.session = interrupt.WorkerSession(str(self.home), uuid.uuid7(), Rung("m", "high"), Rung("m", "high"), str(self.transcript), {"LOOP_CLAUDE_WORKER_RUNG": "m at high effort"})
|
||||
|
||||
def put(self, fake, answer_here=True, effects=None, question="Which one?"):
|
||||
"""interrupt's answer and what it said, without the bell it rang, which self.bells counts."""
|
||||
with contextlib.redirect_stderr(io.StringIO()) as said:
|
||||
answer = interrupt.interrupt(question, answer_here, self.session, WINDOW, LOOP_CLAUDE, effects or fake.effects())
|
||||
self.bells = said.getvalue().count(tmux.BELL)
|
||||
return answer, said.getvalue().replace(tmux.BELL, "")
|
||||
|
||||
def typed_there(self, args):
|
||||
"""The claude opened on the terminal: the human types a line in the worker's session, and it ends."""
|
||||
with open(self.transcript, "a") as handle:
|
||||
handle.write(json.dumps(human("cut it")) + "\n")
|
||||
return 0
|
||||
|
||||
def worked(self, minutes_ago, written_1h=5000, read=70000, written_5m=0):
|
||||
"""Appends the worker's last message, sent minutes_ago before NOW, whose usage wrote written_1h tokens to the hour-long cache, written_5m to the five-minute one, and read read tokens from the cache."""
|
||||
said = assistant({"type": "text", "text": "Which one?"})
|
||||
said["timestamp"] = utc(NOW - minutes_ago * 60)
|
||||
said["message"] |= {"id": "msg-1", "model": "m", "usage": {"input_tokens": 2, "cache_creation": {"ephemeral_5m_input_tokens": written_5m, "ephemeral_1h_input_tokens": written_1h}, "cache_read_input_tokens": read, "output_tokens": 100}}
|
||||
with open(self.transcript, "a") as handle:
|
||||
handle.write(json.dumps(said) + "\n")
|
||||
|
||||
def fired(self, fake, answer):
|
||||
"""An input that first fires every timer started, then returns answer as the line the human typed."""
|
||||
def typed(prompt):
|
||||
for _, action in fake.timers:
|
||||
action()
|
||||
return answer
|
||||
return typed
|
||||
|
||||
def test_compacts_the_workers_session_ten_minutes_before_its_hour_long_prompt_cache_expires_and_keeps_waiting(self):
|
||||
self.worked(minutes_ago=10)
|
||||
fake = FakeEffects(streams=[Streamed(0)])
|
||||
fake.scripts["input"] = [self.fired(fake, "Use B.")]
|
||||
answer, said = self.put(fake)
|
||||
self.assertEqual(answer, "Use B.")
|
||||
self.assertEqual([delay for delay, _ in fake.timers], [40 * 60])
|
||||
self.assertEqual([what for seam, what in fake.calls if seam == "stream_claude"], [["--model", "m", "--effort", "high", "--permission-mode", "bypassPermissions", "-p", "/compact", "--resume", str(self.session.id), "--output-format", "stream-json", "--verbose"]])
|
||||
self.assertEqual(fake.worker_environments, [self.session.env])
|
||||
self.assertEqual(fake.worker_folders, [self.session.root])
|
||||
self.assertIn(f"\n{STAMP} compacting the worker's claude session before its prompt cache expires; an answer typed meanwhile is read once that ends\n\n{STAMP} compacted the worker's claude session\n", said)
|
||||
self.assertNotIn("still waiting", said)
|
||||
|
||||
def test_compacts_on_the_rung_the_worker_ran_on_where_a_ruling_stepped_it_up_and_talk_opens_claude_on_the_new_rung(self):
|
||||
self.worked(minutes_ago=10)
|
||||
self.session = dataclasses.replace(self.session, rung=Rung("n", "medium"))
|
||||
fake = FakeEffects(streams=[Streamed(0)], on_terminal_claude=[self.typed_there])
|
||||
fake.scripts["input"] = [self.fired(fake, "/talk")]
|
||||
self.assertEqual(self.put(fake)[0], interrupt.Talked())
|
||||
(compaction,), (opened,) = fake.made("stream_claude"), fake.made("claude_on_terminal")
|
||||
self.assertEqual(compaction[:4], ["--model", "m", "--effort", "high"])
|
||||
self.assertEqual(opened[:4], ["--model", "n", "--effort", "medium"])
|
||||
|
||||
def test_compacts_until_the_prompt_cache_expires_and_not_from_then_on(self):
|
||||
for name, late, compactions in (("a second before", 50 * 60 - 1, 1), ("as it expires", 50 * 60, 0)):
|
||||
with self.subTest(name):
|
||||
self.transcript.write_text(json.dumps(human("a")) + "\n")
|
||||
self.worked(minutes_ago=10)
|
||||
fake = FakeEffects(streams=[Streamed(0)])
|
||||
clock = [NOW]
|
||||
|
||||
def typed(prompt):
|
||||
clock[0] = NOW + late
|
||||
for _, action in fake.timers:
|
||||
action()
|
||||
return "Use B."
|
||||
fake.scripts["input"] = [typed]
|
||||
answer, said = self.put(fake, effects=dataclasses.replace(fake.effects(), now=lambda: clock[0]))
|
||||
self.assertEqual(answer, "Use B.")
|
||||
self.assertEqual(len(fake.made("stream_claude")), compactions)
|
||||
self.assertEqual(said.count("compact"), 2 * compactions, said)
|
||||
|
||||
def test_a_compaction_doesnt_wait_for_memory(self):
|
||||
self.worked(minutes_ago=10)
|
||||
fake = FakeEffects(streams=[Streamed(0)], free_memory=[memory.Memory(available=3, total=16)])
|
||||
fake.scripts["input"] = [self.fired(fake, "Use B.")]
|
||||
said = self.put(fake)[1]
|
||||
self.assertEqual([seam for seam, _ in fake.calls if seam in ("memory", "sleep", "stream_claude")], ["stream_claude"])
|
||||
self.assertNotIn("waiting to start the worker", said)
|
||||
|
||||
def test_an_answer_before_the_timer_fires_stops_it_and_doesnt_compact_anything(self):
|
||||
self.worked(minutes_ago=10)
|
||||
fake = FakeEffects(inputs=["Use B."])
|
||||
self.assertEqual(self.put(fake)[0], "Use B.")
|
||||
self.assertEqual([call for call in fake.calls if call[0] in ("stop_timer", "stream_claude")], [("stop_timer", 40 * 60)])
|
||||
|
||||
def test_stops_the_timer_before_talk_opens_the_workers_session(self):
|
||||
self.worked(minutes_ago=10)
|
||||
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[self.typed_there])
|
||||
self.assertEqual(self.put(fake)[0], interrupt.Talked())
|
||||
self.assertEqual([call[0] for call in fake.calls if call[0] in ("stop_timer", "claude_on_terminal")], ["stop_timer", "claude_on_terminal"])
|
||||
|
||||
def test_starts_no_timer_under_60k_tokens_of_context_past_the_margin_on_a_five_minute_cache_or_where_claude_opens_at_once(self):
|
||||
for name, minutes_ago, written_1h, written_5m, read, answer_here in (("small", 10, 5000, 0, 54997, True), ("margin begun", 50, 5000, 0, 70000, True), ("five minutes", 1, 0, 5000, 70000, True), ("nothing written", 1, 0, 0, 70000, True), ("opens at once", 10, 5000, 0, 70000, False)):
|
||||
with self.subTest(name):
|
||||
self.transcript.write_text(json.dumps(human("a")) + "\n")
|
||||
self.worked(minutes_ago, written_1h, read, written_5m)
|
||||
fake = FakeEffects(inputs=["Use B."], on_terminal_claude=[self.typed_there])
|
||||
self.put(fake, answer_here=answer_here)
|
||||
self.assertEqual(fake.timers, [])
|
||||
|
||||
def test_starts_the_timer_at_60k_tokens_of_context(self):
|
||||
self.worked(minutes_ago=10, read=54998)
|
||||
fake = FakeEffects(inputs=["Use B."])
|
||||
self.put(fake)
|
||||
self.assertEqual([delay for delay, _ in fake.timers], [40 * 60])
|
||||
|
||||
def test_starts_no_timer_where_the_transcript_cannot_be_read_and_still_asks(self):
|
||||
self.transcript.unlink()
|
||||
fake = FakeEffects(inputs=["Use B."])
|
||||
self.assertEqual(self.put(fake)[0], "Use B.")
|
||||
self.assertEqual(fake.timers, [])
|
||||
|
||||
def test_a_compaction_that_hits_a_usage_limit_says_the_limits_message_and_keeps_waiting(self):
|
||||
self.worked(minutes_ago=10)
|
||||
fake = FakeEffects(streams=[Streamed(0, {"type": "result", "api_error_status": 429, "result": "You've hit your session limit"})])
|
||||
fake.scripts["input"] = [self.fired(fake, "Use B.")]
|
||||
answer, said = self.put(fake)
|
||||
self.assertEqual(answer, "Use B.")
|
||||
self.assertIn(f"\n{STAMP} could not compact the worker's claude session: You've hit your session limit\n", said)
|
||||
|
||||
def test_a_compaction_that_fails_says_why_and_keeps_waiting(self):
|
||||
self.worked(minutes_ago=10)
|
||||
fake = FakeEffects(streams=[Streamed(1)])
|
||||
fake.scripts["input"] = [self.fired(fake, "Use B.")]
|
||||
answer, said = self.put(fake)
|
||||
self.assertEqual(answer, "Use B.")
|
||||
self.assertIn(f"\n{STAMP} could not compact the worker's claude session: claude exited 1\n", said)
|
||||
|
||||
def test_stops_off_a_terminal_before_asking(self):
|
||||
fake = FakeEffects(on_terminal=False)
|
||||
self.assertEqual(self.put(fake), (Stop(2, "the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"), f"\n{STAMP} Which one?\n"))
|
||||
self.assertEqual(fake.calls, [])
|
||||
|
||||
def test_dms_the_question_naming_the_project_on_a_terminal_or_off_one(self):
|
||||
for on_terminal in (True, False):
|
||||
with self.subTest(on_terminal=on_terminal):
|
||||
fake = FakeEffects(inputs=["Use B."], on_terminal=on_terminal)
|
||||
self.put(fake)
|
||||
self.assertEqual(fake.dms, [f"loop-claude in {self.home} asks, in the tmux window loop-proj:item-7:\n\nWhich one?"])
|
||||
|
||||
def test_marks_its_window_and_rings_the_bell_while_it_waits_and_clears_the_mark_once_answered(self):
|
||||
fake = FakeEffects(inputs=["Use B."])
|
||||
answer, said = self.put(fake)
|
||||
self.assertEqual(answer, "Use B.")
|
||||
self.assertEqual(self.bells, 1)
|
||||
self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"], ["rename-window", "-t", "%4", "item-7"]])
|
||||
|
||||
def test_a_loop_that_stops_unanswered_leaves_its_window_marked(self):
|
||||
fake = FakeEffects(on_terminal=False)
|
||||
self.put(fake)
|
||||
self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"]])
|
||||
|
||||
def test_drops_the_keys_typed_before_the_question_ahead_of_the_dm_so_an_answer_typed_while_it_sends_counts(self):
|
||||
fake = FakeEffects(inputs=["Use B."])
|
||||
seen = []
|
||||
effects = dataclasses.replace(fake.effects(), dm=lambda text, directory: seen.append(list(fake.calls)))
|
||||
self.assertEqual(self.put(fake, effects=effects)[0], "Use B.")
|
||||
self.assertEqual(seen, [[("discard_typed", None)]])
|
||||
|
||||
def test_says_why_a_dm_failed_and_still_takes_the_answer(self):
|
||||
fake = FakeEffects(inputs=["Use B."], dm_failures=[Failure("cannot reach https://chat.example: timed out")])
|
||||
self.assertEqual(self.put(fake), ("Use B.", f"\n{STAMP} Which one?\n{STAMP} could not DM you: cannot reach https://chat.example: timed out\n"))
|
||||
|
||||
def test_returns_the_line_typed_after_the_question_stripped_dropping_what_was_typed_before(self):
|
||||
fake = FakeEffects(inputs=[" Use B. "])
|
||||
self.assertEqual(self.put(fake), ("Use B.", f"\n{STAMP} Which one?\n"))
|
||||
self.assertEqual(fake.calls, [("discard_typed", None), ("input", "Answer (/talk opens claude, Enter stops the loop): ")])
|
||||
|
||||
def test_talks_on_talk_or_at_once_where_the_answer_is_not_asked_for_here(self):
|
||||
for typed, answer_here in (([" /talk "], True), ([], False)):
|
||||
with self.subTest(typed=typed):
|
||||
fake = FakeEffects(inputs=typed, on_terminal_claude=[self.typed_there])
|
||||
self.assertEqual(self.put(fake, answer_here)[0], interrupt.Talked())
|
||||
self.assertEqual(len(fake.made("input")), len(typed))
|
||||
self.assertEqual(fake.calls[0], ("discard_typed", None))
|
||||
|
||||
def test_stops_on_an_empty_answer_or_the_inputs_end(self):
|
||||
for typed, why, said in ((" ", "nothing was typed", ""), (EOFError(), "the input ended", "\n")):
|
||||
with self.subTest(why=why):
|
||||
self.assertEqual(self.put(FakeEffects(inputs=[typed])), (Stop(2, f"the human is needed to answer the question above, but {why}"), f"\n{STAMP} Which one?\n{said}"))
|
||||
|
||||
def test_stops_when_nothing_was_typed_in_the_session(self):
|
||||
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[0])
|
||||
stop, _ = self.put(fake)
|
||||
self.assertEqual(fake.made("claude_on_terminal")[0][:4], ["--model", "m", "--effort", "high"])
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended without anything typed"))
|
||||
|
||||
def test_hands_the_claude_that_takes_the_terminal_the_workers_environment(self):
|
||||
fake = FakeEffects(inputs=["/talk"], on_terminal_claude=[0])
|
||||
self.put(fake)
|
||||
self.assertEqual(fake.terminal_environments, [{"LOOP_CLAUDE_WORKER_RUNG": "m at high effort", "CLAUDE_CODE_SANDBOXED": "1"}])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,130 @@
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from loop_effects.test_doubles import assistant, FakeEffects, human, judged, kinds
|
||||
from failure import Failure
|
||||
from agent_cycle.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished
|
||||
from agent_cycle import judge
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from chunk_driver import prompts
|
||||
from chunk_driver.project import records
|
||||
from model_routing.ladder import Rung
|
||||
from agent_cycle.judge import verdict_from
|
||||
|
||||
|
||||
class Grounded(unittest.TestCase):
|
||||
def test_every_verdict_must_quote_the_transcript(self):
|
||||
self.assertTrue(judge.grounded("Needs you: x", "so.\n\nNeeds you: x\n"))
|
||||
self.assertFalse(judge.grounded("invented", "the transcript"))
|
||||
self.assertFalse(judge.grounded("", "the transcript"))
|
||||
self.assertFalse(judge.grounded(" ", "the transcript"))
|
||||
transcript = "done.\n\n**Your step:**\n```\n./run.sh\n```\nRun it."
|
||||
self.assertTrue(judge.grounded("Your step:\n```\n./run.sh\n```\nRun it.", transcript))
|
||||
self.assertTrue(judge.grounded("Your step: ``` ./run.sh ``` Run it.", transcript))
|
||||
self.assertFalse(judge.grounded("**", transcript))
|
||||
|
||||
def test_an_elided_quote_holds_when_its_fragments_appear_in_order(self):
|
||||
transcript = "**Decision for you:** security, or not.\n\nI recommend one more round of each.\n\nI recommend skipping it: it costs."
|
||||
self.assertTrue(judge.grounded("**Decision for you:** security, … ... I recommend one more round of each ... I recommend skipping it: …", transcript))
|
||||
self.assertFalse(judge.grounded("I recommend skipping it ... Decision for you", transcript))
|
||||
self.assertFalse(judge.grounded("Decision for you ... invented", transcript))
|
||||
self.assertFalse(judge.grounded("... …", transcript))
|
||||
self.assertTrue(judge.grounded("a ... b ... c", "x a b a c"))
|
||||
|
||||
|
||||
class Verdict(unittest.TestCase):
|
||||
def test_needs_every_field_the_schema_requires(self):
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Finished("e"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": False, "evidence": "e", "finished": False, "question": "ignored", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Unfinished("e"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": True, "evidence": "e", "finished": True, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Interrupt("e", "Which one?"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": True, "evidence": "e", "finished": False, "question": "Cut 0.2.0?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": True}), Interrupt("e", "Cut 0.2.0?", release_ready=True))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answered_before": "", "left_open": "", "reversed": "", "answer": "", "interrupt": True, "evidence": "e", "finished": False, "question": "Cut 0.2.0?"}), Failure("the judge's reply has no release_ready of type boolean"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), QuotedAnswer("e", "Which one?", "A."))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "A.", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Interrupt("e", "Which one?"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "A.", "interrupt": False, "evidence": "e", "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge quoted an answer without a question"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": "no", "evidence": "", "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge's reply has no interrupt of type boolean"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": False, "evidence": None, "finished": True, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge's reply has no evidence of type string"))
|
||||
self.assertEqual(verdict_from({"escalation": "", "answer": "", "interrupt": True, "evidence": "x", "finished": False, "question": " ", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}), Failure("the judge interrupted without a question"))
|
||||
self.assertEqual(verdict_from(None), Failure("the judge returned no reply"))
|
||||
|
||||
def test_carries_the_earlier_answer_an_interrupt_names(self):
|
||||
reply = {"answer": "", "escalation": "", "interrupt": True, "evidence": "e", "finished": False, "release_ready": False, "answered_before": "Grant one more round.", "left_open": "one more", "question": "Which one?", "reversed": ""}
|
||||
self.assertEqual(verdict_from(reply), Interrupt("e", "Which one?", answered_before="Grant one more round.", left_open="one more"))
|
||||
self.assertEqual(verdict_from({**reply, "interrupt": False, "finished": True}), Finished("e"))
|
||||
|
||||
def test_takes_an_escalation_quote_on_an_unfinished_verdict(self):
|
||||
reply = {"answer": "", "escalation": "round 2: 🔴", "interrupt": False, "evidence": "e", "finished": False, "question": "", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}
|
||||
self.assertEqual(verdict_from(reply), Unfinished("e", "round 2: 🔴"))
|
||||
self.assertEqual(verdict_from({**reply, "finished": True}), Finished("e"))
|
||||
self.assertEqual(verdict_from({**reply, "escalation": None}), Failure("the judge's reply has no escalation of type string"))
|
||||
|
||||
def test_takes_an_escalation_quote_on_an_interrupt(self):
|
||||
reply = {"answer": "", "escalation": "round 2: 🔴", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}
|
||||
self.assertEqual(verdict_from(reply), Interrupt("e", "Which one?", "round 2: 🔴"))
|
||||
|
||||
def test_drops_an_escalation_quote_the_transcript_does_not_hold_word_for_word_from_an_interrupt_or_an_unfinished_verdict(self):
|
||||
shown = "The agent: round 2: 🔴. Which one?"
|
||||
self.assertEqual(judge.quotes_checked(Interrupt("e", "q", "round 2: 🔴"), judge.Grounds(shown, shown, "")), Interrupt("e", "q", "round 2: 🔴"))
|
||||
self.assertEqual(judge.quotes_checked(Interrupt("e", "q", "round 2: … 🔴"), judge.Grounds(shown, shown, "")), Interrupt("e", "q"))
|
||||
self.assertEqual(judge.quotes_checked(Unfinished("e", "round 3: 🔴"), judge.Grounds(shown, shown, "")), Unfinished("e"))
|
||||
|
||||
def test_turns_an_interrupt_that_re_asks_a_human_answer_into_that_answer_quoted_back(self):
|
||||
answer = "Grant one more round; ask again if a 🔴 survives it."
|
||||
shown = f"[loop] Asked.\n\n[human] {answer}\n\n[step] Round 2: a 🔴 survived."
|
||||
grounds = judge.Grounds(shown, shown, "")
|
||||
asking = "Grant a third round? Your answer left open: a 🔴 survives it"
|
||||
kept = Interrupt("[step] Round 2", asking, answered_before=answer, left_open="a 🔴 survives it")
|
||||
self.assertEqual(judge.quotes_checked(kept, grounds), kept)
|
||||
re_asked = Interrupt("[step] Round 2", "Grant a third round?", "Round 2: a 🔴", answered_before=answer, left_open="a 🔴 survives it")
|
||||
self.assertEqual(judge.quotes_checked(re_asked, grounds), Unfinished(answer, "Round 2: a 🔴"))
|
||||
self.assertEqual(judge.quotes_checked(dataclasses.replace(kept, left_open=""), grounds), Unfinished(answer))
|
||||
self.assertEqual(judge.quotes_checked(dataclasses.replace(kept, left_open="a 🔴 survived"), grounds), Unfinished(answer))
|
||||
invented = dataclasses.replace(re_asked, answered_before="Grant two more rounds.")
|
||||
self.assertEqual(judge.quotes_checked(invented, grounds), invented)
|
||||
unnamed = Interrupt("[step] Round 2", "Grant a third round?")
|
||||
self.assertEqual(judge.quotes_checked(unnamed, grounds), unnamed)
|
||||
|
||||
def test_takes_an_escalation_quote_on_a_quoted_answer(self):
|
||||
reply = {"answer": "A.", "escalation": "round 2: 🔴", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?", "answered_before": "", "left_open": "", "reversed": "", "release_ready": False}
|
||||
self.assertEqual(verdict_from(reply), QuotedAnswer("e", "Which one?", "A.", "round 2: 🔴"))
|
||||
|
||||
def test_drops_an_escalation_quote_the_transcript_does_not_hold_word_for_word_from_a_quoted_answer(self):
|
||||
shown = "The agent: round 2: 🔴. Which one?"
|
||||
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "A.", "round 2: 🔴"), judge.Grounds(shown, shown, "A.")), QuotedAnswer("e", "q", "A.", "round 2: 🔴"))
|
||||
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "A.", "round 2: … 🔴"), judge.Grounds(shown, shown, "A.")), QuotedAnswer("e", "q", "A."))
|
||||
|
||||
def test_puts_a_quoted_answer_the_records_do_not_hold_to_the_human_with_the_escalation_it_carried(self):
|
||||
self.assertEqual(judge.quotes_checked(QuotedAnswer("e", "q", "not there", "round 2: 🔴"), judge.Grounds("The agent: round 2: 🔴.", "The agent: round 2: 🔴.", "A.")), Interrupt("e", "q", "round 2: 🔴"))
|
||||
|
||||
def test_puts_a_quoted_answer_the_records_do_not_hold_word_for_word_to_the_human(self):
|
||||
self.assertEqual(judge.quotes_checked(QuotedAnswer("s", "q", "not there"), judge.Grounds("shown", "shown", "the records")), Interrupt("s", "q"))
|
||||
self.assertEqual(judge.quotes_checked(QuotedAnswer("s", "q", "the records"), judge.Grounds("shown", "shown", "From the records: the records")), QuotedAnswer("s", "q", "the records"))
|
||||
|
||||
def test_drops_an_escalation_quote_found_only_in_the_transcript_entries_before_the_workers_last_step_up(self):
|
||||
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), assistant({"type": "text", "text": "round 3: 🔴"}), assistant({"type": "text", "text": "working"})]
|
||||
def judged_on(mark, escalation):
|
||||
return judge.judge_once("/r", judge.Brief("Do it.", "## Goals", "", records.RECORDS_NAMED, prompts.ANSWER_DESTINATIONS, kinds(*entries), mark, LOOP_CLAUDE), Rung("judge-model"), FakeEffects(claude=[judged("unfinished", escalation=escalation)]).effects())
|
||||
self.assertEqual(judged_on(0, "round 2: 🔴"), Unfinished("working", "round 2: 🔴", entries_read=3))
|
||||
self.assertEqual(judged_on(1, "round 2: 🔴"), Unfinished("working", entries_read=3))
|
||||
self.assertEqual(judged_on(1, "round 3: 🔴"), Unfinished("working", "round 3: 🔴", entries_read=3))
|
||||
self.assertEqual(judged_on(1, "round 2: 🔴\n\nround 3: 🔴"), Unfinished("working", entries_read=3))
|
||||
self.assertEqual(judged_on(3, "round 3: 🔴"), Unfinished("working", entries_read=3))
|
||||
|
||||
def test_drops_an_escalation_quote_that_only_loop_claudes_prompts_or_the_humans_lines_repeat_after_the_step_up(self):
|
||||
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), {**human("as the work remaining: round 2: 🔴"), "promptSource": "sdk"}, human("round 2: 🔴 again"), assistant({"type": "text", "text": "working"})]
|
||||
verdict = judge.judge_once("/r", judge.Brief("Do it.", "## Goals", "", records.RECORDS_NAMED, prompts.ANSWER_DESTINATIONS, kinds(*entries), 1, LOOP_CLAUDE), Rung("judge-model"), FakeEffects(claude=[judged("unfinished", escalation="round 2: 🔴")]).effects())
|
||||
self.assertEqual(verdict, Unfinished("working", entries_read=4))
|
||||
|
||||
def test_takes_evidence_from_before_the_workers_last_step_up(self):
|
||||
entries = [assistant({"type": "text", "text": "round 2: 🔴"}), assistant({"type": "text", "text": "working"})]
|
||||
verdict = judge.judge_once("/r", judge.Brief("Do it.", "## Goals", "", records.RECORDS_NAMED, prompts.ANSWER_DESTINATIONS, kinds(*entries), 1, LOOP_CLAUDE), Rung("judge-model"), FakeEffects(claude=[judged("unfinished", evidence="round 2: 🔴")]).effects())
|
||||
self.assertEqual(verdict, Unfinished("round 2: 🔴", entries_read=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,51 @@
|
||||
import contextlib
|
||||
import io
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from failure import Failure
|
||||
from loop_effects.test_doubles import FakeEffects, NOW, STAMP
|
||||
from claude_code.reply import Limited
|
||||
from agent_cycle import retry
|
||||
from agent_cycle.stop import Stop
|
||||
|
||||
|
||||
class Retried(unittest.TestCase):
|
||||
def test_retries_a_minute_apart_and_stops_on_the_third_failure(self):
|
||||
fake = FakeEffects()
|
||||
results = [Failure("one"), "done"]
|
||||
self.assertEqual(retry.retried("worker", lambda: results.pop(0), fake.effects()), "done")
|
||||
self.assertEqual(fake.calls, [("sleep", 60)])
|
||||
fake = FakeEffects()
|
||||
self.assertEqual(retry.retried("judge", lambda: Failure("no"), fake.effects()), Stop(1, "the judge failed three times running: no"))
|
||||
self.assertEqual(fake.calls, [("sleep", 60), ("sleep", 60)])
|
||||
|
||||
def test_hands_each_failure_to_the_keeper_and_stops_where_it_stops(self):
|
||||
kept = []
|
||||
def keep(failure):
|
||||
kept.append(failure)
|
||||
return Stop(1, "no room") if len(kept) == 2 else None
|
||||
fake = FakeEffects()
|
||||
self.assertEqual(retry.retried("judge", lambda: Failure("no"), fake.effects(), keep), Stop(1, "no room"))
|
||||
self.assertEqual(kept, [Failure("no")] * 2)
|
||||
self.assertEqual(fake.calls, [("sleep", 60)])
|
||||
|
||||
def test_waits_out_a_usage_limit_which_ends_a_run_of_failures(self):
|
||||
fake = FakeEffects()
|
||||
results = [Failure("one"), Failure("two"), Limited(NOW + 3600, "limit"), Failure("three"), Limited(None, "limit"), Limited(NOW - 600, "limit"), Failure("four"), Failure("five"), Failure("six")]
|
||||
self.assertEqual(retry.retried("worker", lambda: results.pop(0), fake.effects()), Stop(1, "the worker failed three times running: six"))
|
||||
self.assertEqual(fake.calls, [("sleep", 60), ("sleep", 60), ("sleep", 3720), ("sleep", 60), ("sleep", 600), ("sleep", 600), ("sleep", 60), ("sleep", 60)])
|
||||
|
||||
def test_says_what_claude_said_and_when_the_next_try_is_due(self):
|
||||
results = [Limited(NOW + 3600, "You've hit your session limit"), "done"]
|
||||
with contextlib.redirect_stderr(io.StringIO()) as said:
|
||||
retry.retried("judge", lambda: results.pop(0), FakeEffects().effects())
|
||||
self.assertEqual(said.getvalue(), f"{STAMP} the judge hit a usage limit (You've hit your session limit); retrying at <{NOW + 3720}>\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,90 @@
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from claude_code import entries
|
||||
from loop_effects.test_doubles import FakeEffects, judged, ran
|
||||
from agent_cycle import judge, spend
|
||||
from agent_cycle.verdicts import Judged, Unfinished
|
||||
from transcript_archive import outcomes
|
||||
from claude_code.tokens import Tokens, tokens_of
|
||||
|
||||
|
||||
def turn(message_id, model, at, **usage):
|
||||
return entries.AssistantTurn(message_id=message_id, model=model, timestamp=at, usage=usage)
|
||||
|
||||
|
||||
class Spend(unittest.TestCase):
|
||||
def test_reads_the_cache_writes_by_lifetime_and_counts_one_without_the_split_as_five_minutes(self):
|
||||
self.assertEqual(tokens_of({"input_tokens": 1, "cache_creation_input_tokens": 9, "cache_creation": {"ephemeral_5m_input_tokens": 2, "ephemeral_1h_input_tokens": 3}, "cache_read_input_tokens": 4, "output_tokens": 5}), Tokens(1, 2, 3, 4, 5))
|
||||
self.assertEqual(tokens_of({"input_tokens": 1, "cache_creation_input_tokens": 9}), Tokens(1, 9, 0, 0, 0))
|
||||
self.assertEqual(tokens_of(None), Tokens())
|
||||
|
||||
def test_counts_a_messages_context_as_every_prompt_token_and_no_output(self):
|
||||
self.assertEqual(Tokens(1, 2, 3, 4, 5).context(), 10)
|
||||
|
||||
def test_counts_a_message_written_in_the_stamped_second_or_without_a_readable_time_as_before_since(self):
|
||||
thread = [turn("m1", "a", "2026-11-01T10:00:01Z", output_tokens=7), turn("m2", "a", "not a time", output_tokens=3), turn("m3", "a", "2026-11-01T10:00:02Z", output_tokens=1)]
|
||||
self.assertEqual(spend.spent("2026-11-01T10:00:01Z", thread, [], "j", Tokens()), {("worker", "a"): Tokens(output=1)})
|
||||
|
||||
def test_sums_two_subagents_of_one_name_on_one_model(self):
|
||||
helpers = [spend.Subagent("Explore", [turn("s1", "b", "2026-11-01T10:00:00Z", input_tokens=3)]), spend.Subagent("Explore", [turn("s2", "b", "2026-11-01T10:00:00Z", input_tokens=4)])]
|
||||
self.assertEqual(spend.spent("", [], helpers, "j", Tokens()), {("Explore", "b"): Tokens(input=7)})
|
||||
|
||||
def test_counts_each_message_once_by_its_id_and_only_after_since(self):
|
||||
thread = [
|
||||
turn("m1", "a", "2026-11-01T10:00:00Z", output_tokens=7),
|
||||
turn("m2", "a", "2026-11-01T10:00:02Z", input_tokens=1, output_tokens=2),
|
||||
turn("m2", "a", "2026-11-01T10:00:02Z", input_tokens=1, output_tokens=2),
|
||||
turn("m3", "b", "2026-11-01T10:00:03Z", cache_read_input_tokens=5),
|
||||
]
|
||||
self.assertEqual(spend.thread_spend(thread, "2026-11-01T10:00:01Z"), {"a": Tokens(1, 0, 0, 0, 2), "b": Tokens(0, 0, 0, 5, 0)})
|
||||
self.assertEqual(spend.thread_spend(thread, ""), {"a": Tokens(1, 0, 0, 0, 9), "b": Tokens(0, 0, 0, 5, 0)})
|
||||
|
||||
def test_names_each_spender_and_leaves_out_one_that_spent_nothing(self):
|
||||
worker = [turn("m1", "a", "2026-11-01T10:00:00Z", output_tokens=7)]
|
||||
helper = spend.Subagent('Explore "Search"', [turn("s1", "b", "2026-11-01T10:00:00Z", input_tokens=3)])
|
||||
self.assertEqual(spend.spent("", worker, [helper], "j", Tokens()), {("worker", "a"): Tokens(output=7), ('Explore "Search"', "b"): Tokens(input=3)})
|
||||
self.assertEqual(spend.spent("", [], [], "j", Tokens(input=1)), {("judge", "j"): Tokens(input=1)})
|
||||
|
||||
def test_names_a_subagent_by_its_agent_type_and_description(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
folder = pathlib.Path(root, "s", "subagents")
|
||||
folder.mkdir(parents=True)
|
||||
for name, meta in [("agent-a", {"agentType": "Explore", "description": "Search the hooks"}), ("agent-b", None), ("agent-c", ["Explore"]), ("agent-d", {"description": "Läs"}), ("agent-e", {"agentType": "Ex\nConfig: x"})]:
|
||||
(folder / f"{name}.jsonl").write_text(json.dumps({"type": "assistant", "message": {"id": name, "model": "m", "content": [], "usage": {"output_tokens": 1}}, "timestamp": "2026-11-01T10:00:00Z"}) + "\n")
|
||||
if meta:
|
||||
(folder / f"{name}.meta.json").write_text(json.dumps(meta))
|
||||
found = spend.subagents(str(pathlib.Path(root, "s.jsonl")))
|
||||
self.assertEqual([subagent.name for subagent in found], ['Explore "Search the hooks"', "subagent", "subagent", 'subagent "Läs"', "Ex Config: x"])
|
||||
self.assertEqual(spend.spent("", [], found[:2], "j", Tokens()), {('Explore "Search the hooks"', "m"): Tokens(output=1), ("subagent", "m"): Tokens(output=1)})
|
||||
self.assertEqual(spend.subagents(str(pathlib.Path(root, "none.jsonl"))), [])
|
||||
|
||||
def test_renders_the_spend_then_the_list_prices_of_each_model_it_knows(self):
|
||||
found = {("worker", "claude-opus-5-5"): Tokens(1, 2, 3, 4, 5), ("judge", "unpriced"): Tokens(input=6)}
|
||||
self.assertEqual(outcomes.spent_block("2026-11-01T10:00:00Z", found), """Spent since 2026-11-01T10:00:00Z, in tokens of input, cache write 5m, cache write 1h, cache read, output:
|
||||
- worker, claude-opus-5-5: 1, 2, 3, 4, 5
|
||||
- judge, unpriced: 6, 0, 0, 0, 0
|
||||
|
||||
List prices in dollars per million tokens of input, cache write 5m, cache write 1h, cache read, output:
|
||||
- claude-opus-5-5: 4, 5, 8, 0.2, 20""")
|
||||
self.assertEqual(outcomes.spent_block("", {("judge", "unpriced"): Tokens(input=6)}), "Spent since the chunk began, in tokens of input, cache write 5m, cache write 1h, cache read, output:\n- judge, unpriced: 6, 0, 0, 0, 0")
|
||||
self.assertEqual(outcomes.spent_block("", {}), "")
|
||||
|
||||
def test_keeps_the_whole_typed_answer_that_holds_the_judges_quote_and_its_tokens(self):
|
||||
shown = "[human] Use A.\n[human] No, keep the old name. Use X.\nworking: keep the new name"
|
||||
typed = ("Use A.", "No, keep the old name. Use X.")
|
||||
for quoted, kept in [("No, keep the old name. Use X.", "No, keep the old name. Use X."), ("keep the old name", "No, keep the old name. Use X."), ("Use", "No, keep the old name. Use X."), ("keep the new name", ""), ("No, keep the name.", "")]:
|
||||
with self.subTest(quoted=quoted):
|
||||
answer = json.loads(judged("unfinished", reversed=quoted).stdout)
|
||||
fake = FakeEffects(claude=[ran(json.dumps({**answer, "usage": {"input_tokens": 2, "output_tokens": 3}}))])
|
||||
self.assertEqual(judge.run_judge("/r", [], "", judge.Grounds(shown, shown, "", typed), fake.effects()), Unfinished("working", judged=Judged(tokens=Tokens(input=2, output=3), reversed=kept)))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,69 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from agent_cycle.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished
|
||||
from model_routing.route import Route
|
||||
from agent_cycle import routing
|
||||
from model_routing.ladder import Rung
|
||||
from agent_cycle import states
|
||||
|
||||
|
||||
TYPED = states.TypedAnswer("Which one?", "Use B.")
|
||||
|
||||
|
||||
class Transitions(unittest.TestCase):
|
||||
def test_each_state_moves_on_its_events_and_only_on_them(self):
|
||||
answered = states.Ruled(QuotedAnswer("e", "Which one?", "A."))
|
||||
legal = (
|
||||
(states.Working(), states.WorkerRan("/t"), states.Judging("/t")),
|
||||
(states.Working(QuotedAnswer("e", "Which one?", "A."), 3), states.WorkerRan("/t"), states.Judging("/t", 3)),
|
||||
(states.Judging("/t"), states.Ruled(Interrupt("e", "Which one?")), states.Interrupting("/t", Interrupt("e", "Which one?"))),
|
||||
(states.Judging("/t", 3), states.Ruled(Interrupt("e", "Which one?")), states.Interrupting("/t", Interrupt("e", "Which one?"), 3)),
|
||||
(states.Judging("/t"), answered, states.Working(QuotedAnswer("e", "Which one?", "A."), 1)),
|
||||
(states.Judging("/t", 2), answered, states.Working(QuotedAnswer("e", "Which one?", "A."), 3)),
|
||||
(states.Judging("/t", 3), answered, states.Interrupting("/t", QuotedAnswer("e", "Which one?", "A."), 3)),
|
||||
(states.Judging("/t"), states.Ruled(Finished("e")), states.Closing("/t", Finished("e"))),
|
||||
(states.Judging("/t", 3), states.Ruled(Finished("e")), states.Closing("/t", Finished("e"))),
|
||||
(states.Judging("/t"), states.Ruled(Unfinished("e")), states.Closing("/t", Unfinished("e"))),
|
||||
(states.Judging("/t", 3), states.Ruled(Unfinished("e")), states.Closing("/t", Unfinished("e"))),
|
||||
(states.Interrupting("/t", Interrupt("e", "q")), states.HumanTalked(), states.Judging("/t")),
|
||||
(states.Interrupting("/t", Interrupt("e", "q"), 3), states.HumanTalked(), states.Judging("/t", 3)),
|
||||
(states.Interrupting("/t", Interrupt("e", "q")), states.HumanTyped(TYPED), states.Working(TYPED)),
|
||||
(states.Interrupting("/t", Interrupt("e", "q"), 3), states.HumanTyped(TYPED), states.Working(TYPED, 3)),
|
||||
(states.Working(TYPED, 3), states.WorkerRan("/t"), states.Judging("/t", 3)),
|
||||
(states.Closing("/t", Finished("e")), states.Archived(), states.Ended(Finished("e"))),
|
||||
(states.Closing("/t", Unfinished("e")), states.Archived(), states.Ended(Unfinished("e"))),
|
||||
)
|
||||
for state, event, after in legal:
|
||||
with self.subTest(state=state, event=event):
|
||||
self.assertEqual(states.transition(state, event), after)
|
||||
every_state = (states.Working(), states.Working(QuotedAnswer("e", "Which one?", "A."), 3), states.Working(TYPED, 3), states.Judging("/t"), states.Judging("/t", 3), states.Interrupting("/t", Interrupt("e", "q")), states.Interrupting("/t", Interrupt("e", "q"), 3), states.Closing("/t", Finished("e")), states.Ended(Finished("e")))
|
||||
every_event = (states.WorkerRan("/t"), states.Ruled(Interrupt("e", "Which one?")), answered, states.Ruled(Finished("e")), states.Ruled(Unfinished("e")), states.HumanTalked(), states.HumanTyped(TYPED), states.Archived())
|
||||
for state in every_state:
|
||||
for event in every_event:
|
||||
if (state, event) not in [(state, event) for state, event, _ in legal]:
|
||||
with self.subTest(state=state, event=event), self.assertRaises(ValueError):
|
||||
states.transition(state, event)
|
||||
|
||||
def test_a_rulings_step_up_reaches_the_worker_it_resumes_through_the_answer_the_human_types_and_no_further(self):
|
||||
ran_on = Rung("claude-opus-5-5", "xhigh")
|
||||
stepped = routing.StepUp(Route(ran_on), Route(Rung("claude-fable-5-2", "low"), escalated_at=3))
|
||||
interrupt, quoted = Interrupt("e", "q", "round 2: 🔴"), QuotedAnswer("e", "q", "A.", "round 2: 🔴")
|
||||
for state, event, after in (
|
||||
(states.Judging("/t"), states.Ruled(quoted, stepped), states.Working(quoted, 1, True)),
|
||||
(states.Judging("/t", 3), states.Ruled(quoted, stepped), states.Interrupting("/t", quoted, 3, ran_on)),
|
||||
(states.Judging("/t"), states.Ruled(interrupt, stepped), states.Interrupting("/t", interrupt, 0, ran_on)),
|
||||
(states.Interrupting("/t", interrupt, 0, ran_on), states.HumanTyped(TYPED), states.Working(TYPED, 0, True)),
|
||||
(states.Interrupting("/t", interrupt, 0, ran_on), states.HumanTalked(), states.Judging("/t")),
|
||||
(states.Working(TYPED, 0, True), states.WorkerRan("/t"), states.Judging("/t")),
|
||||
):
|
||||
with self.subTest(state=state, event=event):
|
||||
self.assertEqual(states.transition(state, event), after)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -6,7 +6,7 @@ import pathlib
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from loop_claude.test_loop_claude import FakeEffects, NOW, ran, Streamed
|
||||
from loop_effects.test_doubles import FakeEffects, NOW, ran, Streamed
|
||||
from model_routing.ladder import Rung
|
||||
from loop_effects.child_shells import Shell
|
||||
from agent_cycle import stuck_workers, worker
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from failure import Failure
|
||||
from loop_effects.test_doubles import FakeEffects
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from agent_cycle import tmux
|
||||
|
||||
|
||||
WINDOW = tmux.Window("loop-proj", "item-7", "%4")
|
||||
|
||||
|
||||
class TmuxCommands(unittest.TestCase):
|
||||
def test_refuses_a_tmux_older_than_needed_naming_both_versions(self):
|
||||
for found, reason in (("tmux 3.2", None), ("tmux 3.8", None), ("tmux next-3.4", None), ("tmux 10.0", None),
|
||||
("tmux 3.1c", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 3.1c"),
|
||||
("tmux 2.9a", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found tmux 2.9a"),
|
||||
("", "loop-claude runs each chunk in a tmux window, and needs tmux 3.2 or later; it found no tmux")):
|
||||
with self.subTest(found=found):
|
||||
fake = FakeEffects(tmux_version=found)
|
||||
self.assertEqual(tmux.unusable(fake.tmux, LOOP_CLAUDE), reason)
|
||||
self.assertEqual(fake.tmux_calls, [["-V"]])
|
||||
|
||||
def test_names_the_session_for_the_project(self):
|
||||
self.assertEqual(tmux.session_name(tmux.LOOP, "claude-config"), "loop-claude-config")
|
||||
# tmux reads `.` and `:` in a target as separators.
|
||||
self.assertEqual(tmux.session_name(tmux.LOOP, "my.site:2 x"), "loop-my-site-2-x")
|
||||
self.assertEqual(tmux.session_name(tmux.LOOP, "Acme/Billing"), "loop-Acme-Billing")
|
||||
|
||||
def test_lists_the_windows_without_the_waiting_mark(self):
|
||||
fake = FakeEffects(windows="item-372 ?\t0\t@1\nitem-380\t1\t@2\nno-item\t0\t@5\n")
|
||||
self.assertEqual(tmux.listed("loop-proj", fake.tmux), [tmux.Listed("item-372", "@1", False, asks=True), tmux.Listed("item-380", "@2", True), tmux.Listed("no-item", "@5", False)])
|
||||
self.assertEqual(fake.tmux_calls, [["list-windows", "-t", "=loop-proj", "-F", "#{window_name}\t#{pane_dead}\t#{window_id}"]])
|
||||
self.assertEqual(tmux.listed("loop-proj", FakeEffects().tmux), [])
|
||||
failed = lambda args: subprocess.CompletedProcess(args, 1, "item-1\t0\t@1\n", "no server")
|
||||
self.assertEqual(tmux.listed("loop-proj", failed), [])
|
||||
|
||||
def test_the_first_window_makes_the_session_and_each_later_one_joins_it(self):
|
||||
fake = FakeEffects()
|
||||
window = tmux.Window("loop-proj", "item-7", "")
|
||||
self.assertIsNone(tmux.start_window(window, "/r", {"LOOP_CLAUDE_X": "1"}, ["/b/loop-claude", "--chunk", "item-7", "ship it;"], True, True, fake.tmux))
|
||||
self.assertIsNone(tmux.start_window(window, "/r", {}, ["/b/loop-claude"], False, True, fake.tmux))
|
||||
self.assertEqual(fake.tmux_calls, [
|
||||
["new-session", "-d", "-s", "loop-proj", "-n", "item-7", "-c", "/r", "-e", "LOOP_CLAUDE_X=1", "-P", "-F", "#{window_id}", "--", "/b/loop-claude", "--chunk", "item-7", "ship it\;"],
|
||||
["set-option", "-t", "=loop-proj:", "@loop-root", "/r"],
|
||||
["set-option", "-w", "-t", "@9", "@chunk-root", "/r"],
|
||||
["set-option", "-w", "-t", "@9", "remain-on-exit", "failed"],
|
||||
["new-window", "-d", "-t", "=loop-proj:", "-n", "item-7", "-c", "/r", "-P", "-F", "#{window_id}", "--", "/b/loop-claude"],
|
||||
["set-option", "-w", "-t", "@9", "@chunk-root", "/r"],
|
||||
["set-option", "-w", "-t", "@9", "remain-on-exit", "failed"],
|
||||
])
|
||||
|
||||
def test_says_why_tmux_started_no_window(self):
|
||||
failed = lambda args: subprocess.CompletedProcess(args, 1, "", "duplicate session: loop-proj\n")
|
||||
silent = lambda args: subprocess.CompletedProcess(args, 2, "", "")
|
||||
window = tmux.Window("loop-proj", "item-7", "")
|
||||
self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, True, failed), Failure("tmux could not start the window loop-proj:item-7: duplicate session: loop-proj"))
|
||||
self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, True, silent), Failure("tmux could not start the window loop-proj:item-7: it exited 2"))
|
||||
|
||||
def test_fails_where_it_cannot_mark_a_session_it_made_as_the_checkouts_but_not_where_remain_on_exit_fails(self):
|
||||
def answering(failing: str):
|
||||
def run(args):
|
||||
fails = args[:1] == ["set-option"] and failing in args
|
||||
return subprocess.CompletedProcess(args, 1 if fails else 0, "" if fails else "@9\n", "no such session\n" if fails else "")
|
||||
return run
|
||||
window = tmux.Window("loop-proj", "item-7", "")
|
||||
self.assertEqual(tmux.start_window(window, "/r", {}, ["x"], True, True, answering(tmux.OWNER)), Failure("tmux started the session loop-proj but could not mark it as this checkout's: no such session"))
|
||||
self.assertIsNone(tmux.start_window(window, "/r", {}, ["x"], True, True, answering("remain-on-exit")))
|
||||
self.assertIsNone(tmux.start_window(window, "/r", {}, ["x"], False, True, answering(tmux.OWNER)))
|
||||
|
||||
def test_reads_the_checkout_the_session_names(self):
|
||||
fake = FakeEffects(owner="/c/proj")
|
||||
self.assertEqual(tmux.owner("loop-proj", fake.tmux), "/c/proj")
|
||||
self.assertEqual(fake.tmux_calls, [["show-options", "-qv", "-t", "=loop-proj:", "@loop-root"]])
|
||||
|
||||
def test_marks_a_waiting_window_by_its_pane_and_clears_the_mark(self):
|
||||
fake = FakeEffects()
|
||||
tmux.mark(WINDOW, True, fake.tmux)
|
||||
tmux.mark(WINDOW, False, fake.tmux)
|
||||
tmux.close(tmux.Listed("item-7", "@3", True), fake.tmux)
|
||||
self.assertEqual(fake.tmux_calls, [["rename-window", "-t", "%4", "item-7 ?"], ["rename-window", "-t", "%4", "item-7"], ["kill-window", "-t", "@3"]])
|
||||
self.assertEqual(WINDOW.target, "loop-proj:item-7")
|
||||
|
||||
def test_shows_the_session_by_attaching_or_inside_tmux_by_switching(self):
|
||||
self.assertEqual(tmux.show("loop-proj", False), ["attach-session", "-t", "=loop-proj"])
|
||||
self.assertEqual(tmux.show("loop-proj", True), ["switch-client", "-t", "=loop-proj"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,135 @@
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import io
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from claude_code import entries
|
||||
from failure import Failure
|
||||
from loop_effects.test_doubles import FakeEffects, NOW, ran, Streamed
|
||||
from agent_cycle import judge, lingering_workers, worker
|
||||
from claude_code.reply import Limited
|
||||
from machine_load import memory
|
||||
from model_routing.ladder import Rung
|
||||
from loop_effects.child_shells import Shell
|
||||
from text_files.utc_stamp import utc
|
||||
|
||||
|
||||
class Limits(unittest.TestCase):
|
||||
def test_the_worker_reports_the_latest_reset_its_stream_names(self):
|
||||
rejected = [{"type": "rate_limit_event", "rate_limit_info": {"status": "rejected", "resetsAt": resets_at}} for resets_at in (NOW + 60, NOW + 600)]
|
||||
allowed = {"type": "rate_limit_event", "rate_limit_info": {"status": "allowed", "resetsAt": NOW + 9000}}
|
||||
limited = {"type": "result", "api_error_status": 429, "result": "You've hit your session limit"}
|
||||
fake = FakeEffects(streams=[Streamed(1, rejected[1], allowed, rejected[0], limited)])
|
||||
self.assertEqual(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), fake.effects()), Limited(NOW + 600, "You've hit your session limit"))
|
||||
|
||||
def test_a_limit_outranks_the_exit_code_and_its_message_fits_one_line(self):
|
||||
def limited(**said):
|
||||
return {"type": "result", "api_error_status": 429, **said}
|
||||
self.assertEqual(worker.worker_outcome(-9, limited(result="limit\n resets 7pm"), []), Limited(None, "limit resets 7pm"))
|
||||
self.assertEqual(worker.worker_outcome(1, limited(result=" \n"), [NOW]), Limited(NOW, "claude gave no message"))
|
||||
self.assertEqual(worker.worker_outcome(0, limited(), []), Limited(None, "claude gave no message"))
|
||||
self.assertEqual(worker.worker_outcome(0, None, [NOW]), Failure("claude exited 0 without a result event"))
|
||||
self.assertIsNone(worker.worker_outcome(0, {"type": "result"}, [NOW]))
|
||||
|
||||
def test_the_judge_reports_a_limit_whatever_its_exit_code(self):
|
||||
reply = json.dumps({"type": "result", "api_error_status": 429, "is_error": True, "result": "You've hit your session limit · resets 7:40pm (Europe/Stockholm)"})
|
||||
fake = FakeEffects(claude=[ran(reply, code=1, stderr="limited")])
|
||||
self.assertEqual(judge.run_judge("/r", [], "", judge.Grounds("", "", ""), fake.effects()), Limited(None, "You've hit your session limit · resets 7:40pm (Europe/Stockholm)"))
|
||||
|
||||
def test_reads_only_a_rejecting_events_whole_seconds(self):
|
||||
for info, want in [({"status": "rejected", "resetsAt": NOW}, NOW), ({"status": "allowed", "resetsAt": NOW}, None), ({"status": "rejected", "resetsAt": True}, None), ({"status": "rejected"}, None), ("rejected", None)]:
|
||||
with self.subTest(info=info):
|
||||
self.assertEqual(entries.rejected_until({"type": "rate_limit_event", "rate_limit_info": info}), want)
|
||||
|
||||
|
||||
class LingeringWorkers(unittest.TestCase):
|
||||
RESULT = {"type": "result"}
|
||||
|
||||
def worker_with(self, *events, held=()):
|
||||
"""Runs a worker whose claude prints events and holds shells `held`; returns the claude, what loop-claude said and the record's text."""
|
||||
clock = [NOW]
|
||||
streamed = Streamed(0, *events, clock=clock, held=held)
|
||||
fake = FakeEffects(streams=[streamed])
|
||||
effects = dataclasses.replace(fake.effects(), now=lambda: clock[0])
|
||||
with tempfile.TemporaryDirectory() as folder, contextlib.redirect_stdout(io.StringIO()) as said:
|
||||
# The archive folder of a project's first stop doesn't exist yet.
|
||||
archive = pathlib.Path(folder, "transcripts", "project")
|
||||
worker.run_worker("/r", [], {}, archive, Rung("m"), effects)
|
||||
record = archive / lingering_workers.RECORD
|
||||
return streamed, said.getvalue(), record.read_text() if record.exists() else ""
|
||||
|
||||
def shell(self, written=NOW - 1000, runs_busy_tool=False, holds_slot=False):
|
||||
return Shell(pid=7, started=NOW - 1000, command="until false; do sleep 15; done", written=written, runs_busy_tool=runs_busy_tool, holds_slot=holds_slot)
|
||||
|
||||
def test_a_worker_that_lingers_after_its_result_loses_its_quiet_shell_and_the_stop_is_said_and_recorded_in_utc(self):
|
||||
shell = self.shell()
|
||||
streamed, said, record = self.worker_with(self.RESULT, None, held=[shell])
|
||||
self.assertEqual(streamed.stopped, [shell])
|
||||
line = f"stopped shell 7, which held a worker after its final result; it had run 1300 s and written nothing for 1300 s: {shell.command}"
|
||||
self.assertEqual(said, f"<{NOW + 300}> {line}\n")
|
||||
self.assertEqual(record, f"- {utc(NOW + 300)} {line}\n")
|
||||
|
||||
def test_a_worker_keeps_its_shells_where_it_printed_after_its_result_or_printed_none_or_no_check_came_due(self):
|
||||
for events in ((self.RESULT, {"type": "assistant", "message": {"content": []}}, None), (self.RESULT,), (None,), ({"type": "assistant", "message": {"content": []}}, None)):
|
||||
with self.subTest(events=events):
|
||||
streamed, said, record = self.worker_with(*events, held=[self.shell()])
|
||||
self.assertEqual((streamed.stopped, said, record), ([], "", ""))
|
||||
|
||||
def test_a_worker_keeps_every_shell_while_one_of_them_writes_output_runs_a_busy_tool_or_holds_a_heavy_slot(self):
|
||||
for busy in (self.shell(written=NOW + 200), self.shell(runs_busy_tool=True), self.shell(holds_slot=True)):
|
||||
with self.subTest(busy=busy):
|
||||
streamed, _, _ = self.worker_with(self.RESULT, None, held=[self.shell(), busy])
|
||||
self.assertEqual(streamed.stopped, [])
|
||||
|
||||
def test_a_worker_keeps_a_shell_that_wrote_within_the_quiet_time_and_stops_one_that_wrote_before_it(self):
|
||||
# The check comes 300 s after the result, so a shell written at NOW + 300 - QUIET is the latest that is still kept.
|
||||
kept = self.shell(written=NOW + 300 - lingering_workers.QUIET)
|
||||
stopped = self.shell(written=NOW + 300 - lingering_workers.QUIET - 1)
|
||||
self.assertEqual(self.worker_with(self.RESULT, None, held=[kept])[0].stopped, [])
|
||||
self.assertEqual(self.worker_with(self.RESULT, None, held=[stopped])[0].stopped, [stopped])
|
||||
|
||||
def test_a_worker_reads_on_past_a_check_that_found_it_still_working(self):
|
||||
clock = [NOW]
|
||||
effects = dataclasses.replace(FakeEffects(streams=[Streamed(0, None, self.RESULT, clock=clock)]).effects(), now=lambda: clock[0])
|
||||
self.assertIsNone(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), effects))
|
||||
|
||||
def test_a_stop_that_cannot_be_recorded_is_said_on_stderr_and_every_shell_is_still_stopped(self):
|
||||
clock = [NOW]
|
||||
held = [self.shell(), self.shell()]
|
||||
streamed = Streamed(0, self.RESULT, None, clock=clock, held=held)
|
||||
effects = dataclasses.replace(FakeEffects(streams=[streamed]).effects(), now=lambda: clock[0])
|
||||
with tempfile.NamedTemporaryFile() as blocker, contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()) as failed:
|
||||
worker.run_worker("/r", [], {}, pathlib.Path(blocker.name), Rung("m"), effects)
|
||||
self.assertEqual(streamed.stopped, held)
|
||||
self.assertIn("could not record the stop", failed.getvalue())
|
||||
|
||||
def test_every_stop_of_a_check_is_recorded_though_the_archive_folder_exists_after_the_first(self):
|
||||
held = [self.shell(), self.shell()]
|
||||
_, said, record = self.worker_with(self.RESULT, None, held=held)
|
||||
self.assertEqual(record.count("stopped shell 7"), 2)
|
||||
|
||||
|
||||
class MemoryWait(unittest.TestCase):
|
||||
def test_a_worker_starts_only_once_a_quarter_of_memory_is_available(self):
|
||||
fake = FakeEffects(free_memory=[memory.Memory(available=3, total=16), memory.Memory(available=3, total=16), memory.Memory(available=4, total=16)], streams=[Streamed(0)])
|
||||
with contextlib.redirect_stderr(io.StringIO()) as said:
|
||||
self.assertIsNone(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), fake.effects()))
|
||||
self.assertEqual([seam for seam, _ in fake.calls], ["memory", "sleep", "memory", "sleep", "memory", "stream_claude"])
|
||||
self.assertEqual(fake.made("sleep"), [worker.MEMORY_POLL] * 2)
|
||||
self.assertEqual(said.getvalue().count("waiting"), 1, said.getvalue())
|
||||
|
||||
def test_a_worker_starts_at_once_where_memory_cannot_be_read(self):
|
||||
fake = FakeEffects(free_memory=[None], streams=[Streamed(0)])
|
||||
self.assertIsNone(worker.run_worker("/r", [], {}, pathlib.Path("/archive"), Rung("m"), fake.effects()))
|
||||
self.assertEqual([seam for seam, _ in fake.calls], ["memory", "stream_claude"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,4 +1,4 @@
|
||||
"""The verdicts cycle.judge.verdict_from builds from the judge's reply; cycle.run.rule then stamps every verdict but Finished with the items the chunk takes. They sit outside cycle/, since routing reads them and cycle imports routing."""
|
||||
"""The verdicts judge.verdict_from builds from the judge's reply; run.rule then stamps every verdict but Finished with the items the chunk takes. They sit in a module of their own, since routing.py reads them and run.py imports routing.py."""
|
||||
import dataclasses
|
||||
|
||||
from transcript_archive.judge_run import JudgeRun
|
||||
@@ -7,7 +7,7 @@ from claude_code.tokens import Tokens
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class TakenIn:
|
||||
"""The highest L, and the highest E + I, among the plan items the worker names on its latest `Chunk items:` line, as cycle.chunk_items reads them. The item the chunk was routed on counts only where the worker names it."""
|
||||
"""The highest L, and the highest E + I, among the plan items the worker names on its latest `Chunk items:` line, as the entry point reads them (run.py's Cycle.taken_in). The item the chunk was routed on counts only where the worker names it."""
|
||||
# The highest L among them, 0 for none: an escalation steps up from the rung that L picks, and an unfinished verdict also raises the worker to it.
|
||||
highest_likelihood: int = 0
|
||||
# The highest E + I among them, 0 for none: after an unfinished verdict, it routes the next cycle's reviewers.
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
"""What every reader of Jira Cloud's REST API version 3 shares: a search read page by page, the head of each entry it answers, an issue's "is blocked by" links, and the transition that finishes an issue."""
|
||||
import dataclasses
|
||||
import re
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
from project_config.settings import Finish, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION
|
||||
from atlassian.client import Http
|
||||
|
||||
ISSUE_KEY = re.compile(r"([A-Z][A-Z0-9_]*)-([0-9]+)")
|
||||
PAGE = 100
|
||||
# The pages one read asks for at most, so an answer that never names a last page ends the read.
|
||||
MAX_PAGES = 100
|
||||
# The key of the status category Jira Cloud gives every done status.
|
||||
DONE = "done"
|
||||
# What a failed create ends with: a request Jira never answered, or answered brokenly, may still have filed the issue.
|
||||
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under {under} before filing it again"
|
||||
# What a failed move ends with, for the same reason.
|
||||
PERHAPS_MOVED = "unless Jira refused the request, the issue may have moved: look at {key} before finishing the item again"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Link:
|
||||
"""One "is blocked by" link of an issue."""
|
||||
# The key of the issue that blocks.
|
||||
blocker: str
|
||||
# The link's own id, which Jira deletes it by.
|
||||
id: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Entry:
|
||||
"""The head of one entry of a search's answer, which every reader checks alike."""
|
||||
key: str
|
||||
# The entry's `fields`, from which each reader takes the ones it asked for.
|
||||
fields: dict[str, typing.Any]
|
||||
links: tuple[Link, ...]
|
||||
# The issue property the search asked for; empty where the issue has none. A reader checks that it is a JSON object.
|
||||
held: object
|
||||
|
||||
|
||||
def entry_of(raw: object, property_: str) -> Entry | Failure:
|
||||
"""The head of one entry of a search's answer, or which issue is not in the shape Jira documents, and how. property_ names the issue property the search asked for."""
|
||||
key = raw.get("key") if isinstance(raw, dict) else None
|
||||
if not isinstance(raw, dict) or not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
|
||||
return Failure("the search answered an issue without an issue key")
|
||||
# The answer's nested fields are read as Jira documents them; one missing or of another type ends the read here, as a Failure, so no later step meets it.
|
||||
try:
|
||||
fields = raw["fields"]
|
||||
return Entry(key, fields, blocked_by_links(fields), raw.get("properties", {}).get(property_, {}))
|
||||
except (AttributeError, KeyError, TypeError) as failure:
|
||||
return Failure(unshaped(key, failure))
|
||||
|
||||
|
||||
def unshaped(key: str, failure: Exception) -> str:
|
||||
"""Why the issue named key cannot be read, for a field that is missing or of another type than Jira documents."""
|
||||
return f"{key} is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}"
|
||||
|
||||
|
||||
def offer_of(raw: object) -> tuple[str, str, str, str] | None:
|
||||
"""One transition Jira offers, as its ID, its name, the name of the status it leads to and the key of that status's category; None where raw holds no such transition."""
|
||||
to = raw.get("to") if isinstance(raw, dict) else None
|
||||
category = to.get("statusCategory") if isinstance(to, dict) else None
|
||||
if not isinstance(raw, dict) or not isinstance(to, dict) or not isinstance(category, dict):
|
||||
return None
|
||||
id_, name, status, key = raw.get("id"), raw.get("name"), to.get("name"), category.get("key")
|
||||
return (id_, name, status, key) if isinstance(id_, str) and isinstance(name, str) and isinstance(status, str) and isinstance(key, str) else None
|
||||
|
||||
|
||||
def finishing_transition(site: str, key: str, finish: Finish, authorization: str, http: Http) -> str | Failure:
|
||||
"""The ID of the transition finish names, as Jira offers it now for the issue named key, which is a plan item or an epic; or why none of those offered finishes it."""
|
||||
offered = http("GET", f"{site}/rest/api/3/issue/{key}/transitions", authorization, None)
|
||||
if isinstance(offered, Failure):
|
||||
return Failure(f"cannot read {key}'s transitions: {offered.reason}")
|
||||
return transition_of(offered, key, finish)
|
||||
|
||||
|
||||
def move_refusal(site: str, key: str, transition: str, finish: Finish, authorization: str, http: Http) -> str | None:
|
||||
"""Moves the issue named key by the transition with the ID transition, which finishing_transition answered for finish. Why Jira did not take the move; None where it did."""
|
||||
moved = http("POST", f"{site}/rest/api/3/issue/{key}/transitions", authorization, {"transition": {"id": transition}})
|
||||
return f"cannot move {key} by {finish.transition!r}: {moved.reason}" if isinstance(moved, Failure) else None
|
||||
|
||||
|
||||
def transition_of(offered: object, key: str, finish: Finish) -> str | Failure:
|
||||
"""The ID of the transition finish names, among those Jira offered for the issue named key, which are the ones its present status has; or why none of them finishes the item."""
|
||||
raw = offered.get("transitions") if isinstance(offered, dict) else None
|
||||
found = [offer for each in raw if (offer := offer_of(each))] if isinstance(raw, list) else []
|
||||
if not isinstance(raw, list) or len(found) != len(raw):
|
||||
return Failure(f"Jira answered {key}'s transitions in a shape other than the one it documents")
|
||||
named = [each for each in found if each[1] == finish.transition]
|
||||
if not named:
|
||||
others = " and ".join(f"{name!r} to {status!r}" for _, name, status, _ in found)
|
||||
return Failure(f"{key} has no transition {finish.transition!r} from its status, " + (f"only {others}: move the issue in Jira to a status that has it, or name in `{JIRA_FINISH_TRANSITION}` one that leads to a done status" if found else "nor any other the account may make"))
|
||||
id_, _, status, category = next((each for each in named if each[2] == finish.status), named[0])
|
||||
if status != finish.status:
|
||||
return Failure(f"{key}'s transition {finish.transition!r} leads to the status {status!r}, not to {finish.status!r}, which `{JIRA_FINISH_STATUS}` names")
|
||||
# `children` reads an issue as finished by its status's category, so a move to any other category would leave the item in the plan.
|
||||
if category != DONE:
|
||||
return Failure(f"{key}'s transition {finish.transition!r} leads to the status {status!r}, which Jira doesn't count as done, so the item would stay in the plan: name a transition to a done status")
|
||||
return id_
|
||||
|
||||
|
||||
def project_of(key: str) -> str:
|
||||
"""The key of the Jira project an issue key lies in."""
|
||||
return key.rpartition("-")[0]
|
||||
|
||||
|
||||
def rank_refusal(site: str, key: str, before: str, authorization: str, http: Http) -> str | None:
|
||||
"""Ranks the issue named key just before the issue named before; why Jira refused, or None where it didn't. Jira answers a rank it made with no body, and one it refused with an error, or with an entry per issue that holds the errors."""
|
||||
answer = http("PUT", f"{site}/rest/agile/1.0/issue/rank", authorization, {"issues": [key], "rankBeforeIssue": before})
|
||||
if isinstance(answer, Failure):
|
||||
return answer.reason
|
||||
entries = answer.get("entries") if isinstance(answer, dict) else None
|
||||
refused = [entry for entry in entries if isinstance(entry, dict) and entry.get("status") != 200] if isinstance(entries, list) else []
|
||||
return "; ".join(str(error) for entry in refused for error in entry.get("errors") or [f"Jira answered {entry.get('status')}"]) or None
|
||||
|
||||
|
||||
def searched(site: str, what: str, jql: str, asked: dict[str, object], authorization: str, http: Http) -> list[object] | Failure:
|
||||
"""Every issue a JQL search on site finds, page by page. The dict asked holds the request's `fields` and `properties` lists. The text what names the issues sought, in the Failure of a search that doesn't end."""
|
||||
body: dict[str, object] = {"jql": jql, "maxResults": PAGE} | asked
|
||||
found: list[object] = []
|
||||
for _ in range(MAX_PAGES):
|
||||
page = http("POST", f"{site}/rest/api/3/search/jql", authorization, body)
|
||||
if isinstance(page, Failure):
|
||||
return page
|
||||
if not isinstance(page, dict) or not isinstance(page.get("issues"), list):
|
||||
return Failure("the search answered something other than a page of issues")
|
||||
found += page["issues"]
|
||||
if not page.get("nextPageToken"):
|
||||
return found
|
||||
body = body | {"nextPageToken": page["nextPageToken"]}
|
||||
return Failure(f"the search for {what} did not end within {MAX_PAGES} pages")
|
||||
|
||||
|
||||
def blocked_by_links(fields: dict[str, typing.Any]) -> tuple[Link, ...]:
|
||||
"""The "is blocked by" links among an issue's `issuelinks` field. A field not in the shape Jira documents raises a KeyError or TypeError, which the caller turns into a Failure."""
|
||||
return tuple(Link(link["inwardIssue"]["key"], link["id"]) for link in fields["issuelinks"] if "inwardIssue" in link and link["type"]["inward"] == "is blocked by")
|
||||
@@ -0,0 +1,4 @@
|
||||
"""What the entry points that drive chunks share: loop-claude and do-epic.
|
||||
|
||||
Each runs a chunk as cycles of bin/agent_cycle/. cycle_start.py starts a cycle and chunk_run.py closes it, on the Loop this_loop.py holds, once claims.py has claimed the checkout. prompts.py words what every worker is told, project/ reads a checkout's premises, records and saved models, and drivers.py names the entry points. No module here imports an entry point's package, and bin/repo_checks/test_package_cycles.py fails on such an import.
|
||||
"""
|
||||
@@ -0,0 +1,76 @@
|
||||
"""One chunk's run across its cycles, for every entry point that drives chunks: its state (Run, ChunkRun), and how a cycle closes: the run the next cycle starts from, the step-up an unfinished verdict makes archived, and the stop once a chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from agent_cycle.run import archived_step, archived_stop, Concluded
|
||||
from agent_cycle.verdicts import Finished
|
||||
from chunk_driver.this_loop import Loop
|
||||
from agent_cycle.stop import needs_human, Stop
|
||||
from agent_cycle.routing import next_route, step_up
|
||||
from model_routing.ladder import ProjectRung
|
||||
from model_routing.route import Route
|
||||
from claude_code.session import transcript_file
|
||||
|
||||
# How many unfinished verdicts running end the loop, so a worker and a judge that disagree about a chunk cannot cycle unattended without end. A policy number, not derived from the routing steps: well above the two unfinished verdicts running that archived chunks have reached at most, and low enough that a stuck pair is stopped after six cycles. It counts every unfinished verdict, including those of cycles where the human answered an interrupt: a rerun is cheap, and a chunk that needs that many answers is worth a look. The human confirmed both the number and what it counts.
|
||||
UNFINISHED_LIMIT = 6
|
||||
|
||||
|
||||
class Run(typing.Protocol):
|
||||
"""What closing a cycle reads and replaces in an entry point's run of one chunk. ChunkRun below is one; loop-claude's LoopRun is another, since it hands the same fields on at a reload."""
|
||||
# Every run is a dataclass, which is what lets closing a cycle replace its fields.
|
||||
__dataclass_fields__: typing.ClassVar[dict[str, typing.Any]]
|
||||
|
||||
@property
|
||||
def task(self) -> str: ...
|
||||
@property
|
||||
def unfinished_evidence(self) -> str: ...
|
||||
@property
|
||||
def route(self) -> Route | None: ...
|
||||
@property
|
||||
def unfinished_run(self) -> int: ...
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class ChunkRun:
|
||||
"""The run of one chunk, for an entry point that never hands it to another process, such as do-epic."""
|
||||
cycle_number: int = 0
|
||||
task: str = ""
|
||||
# The evidence of the latest unfinished verdict; empty before the chunk's first one, and once it finishes.
|
||||
unfinished_evidence: str = ""
|
||||
# Set by each unfinished verdict; None before the chunk's first one, and once it finishes.
|
||||
route: Route | None = None
|
||||
# How many unfinished verdicts the chunk has had running.
|
||||
unfinished_run: int = 0
|
||||
|
||||
|
||||
def closed_cycle[R: Run](run: R, project: ProjectRung | None, concluded: Concluded, archive_folder: pathlib.Path, driver: str, loop: Loop) -> R | Stop:
|
||||
"""The run after a cycle of driver, the entry point, concluded, with the step-up archived; the stop that needs the human, archived, once the chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
after = after_verdict(run, project, concluded)
|
||||
# A finished chunk drops its route, so it has no step-up, and its run of unfinished verdicts restarts.
|
||||
if after.route is None:
|
||||
return after
|
||||
# The unfinished verdict, archived as the cycle closed, names the route its cycle ended on; the step-up names the one the next cycle runs on.
|
||||
path = transcript_file(loop.session)
|
||||
if path and (failure := archived_step(archive_folder, path, after.task, step_up(concluded.route, after.route), loop.effects)):
|
||||
return Stop(1, failure.reason)
|
||||
if limit := reached_limit(after, driver):
|
||||
# Archived with the route the next cycle would run on, so a rerun resumes there.
|
||||
return archived_stop(archive_folder, loop.session, after.task, after.route, limit, limit.reason, loop.effects)
|
||||
return after
|
||||
|
||||
|
||||
def reached_limit(run: Run, driver: str) -> Stop | None:
|
||||
"""The stop that needs the human once the run holds UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
if run.unfinished_run < UNFINISHED_LIMIT:
|
||||
return None
|
||||
return needs_human("go on with the chunk", f"the judge ruled it unfinished {run.unfinished_run} cycles running; finish the chunk yourself, or rerun {driver} to give it {UNFINISHED_LIMIT} more cycles")
|
||||
|
||||
|
||||
def after_verdict[R: Run](run: R, project: ProjectRung | None, concluded: Concluded) -> R:
|
||||
"""The run the next cycle starts from. A finished chunk drops its route and its run of unfinished verdicts. An unfinished one takes the route routing.next_route makes of the route its cycle ended on, which an interrupt's or a quoted answer's escalation may have raised, and the evidence the judge gave for the verdict; and counts the verdict."""
|
||||
verdict = concluded.verdict
|
||||
if isinstance(verdict, Finished):
|
||||
return dataclasses.replace(run, route=None, unfinished_evidence="", unfinished_run=0)
|
||||
route = next_route(project, concluded.route, verdict)
|
||||
return dataclasses.replace(run, route=route, unfinished_evidence=verdict.evidence, unfinished_run=run.unfinished_run + 1)
|
||||
@@ -0,0 +1,39 @@
|
||||
"""How entry points claim a checkout before they start a chunk in it, so that two never start the same chunk and do-epic never starts one where loop-claude runs.
|
||||
|
||||
An entry point holds `starting` on `project_lock` while it lists the windows of the checkout's chunks and starts new ones. It starts each chunk in a tmux window with CHUNK_FLAG, and hands that window the variables named with HANDED.
|
||||
"""
|
||||
import contextlib
|
||||
import fcntl
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from transcript_archive.project_key import project_key
|
||||
from agent_cycle import tmux
|
||||
|
||||
# Under the home folder, a folder per project key holds LOCK.
|
||||
CLAIMS = ".local/state/loop-claude"
|
||||
# The file an entry point keeps locked with flock while it lists the windows of a project's chunks and starts new ones. Two loops whose chunks finish together then don't start a chunk twice, and do-epic, which holds the same lock, never starts a chunk in a checkout while loop-claude does.
|
||||
LOCK = "starting"
|
||||
# The flag that tells the entry point in a window its chunk, as `--chunk item-372` for loop-claude and `--chunk ABC-34 ABC-12` for do-epic. Nobody types it, so no panel picked it.
|
||||
CHUNK_FLAG = "--chunk"
|
||||
# The prefix of the variables both entry points read. A window gets the tmux server's environment, not the starting terminal's, so they are handed to each window by name.
|
||||
HANDED = "LOOP_CLAUDE_"
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def starting(folder: pathlib.Path) -> typing.Iterator[None]:
|
||||
"""Holds the LOCK in folder: project_lock's for a project, or do-epic's for an epic. Raises OSError where the file cannot be opened or locked."""
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
with open(folder / LOCK, "a") as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
yield
|
||||
|
||||
|
||||
def project_lock(root: str, home: pathlib.Path) -> pathlib.Path:
|
||||
"""The folder of the LOCK that everything starting a chunk in the checkout root holds."""
|
||||
return home / CLAIMS / project_key(root, home)
|
||||
|
||||
|
||||
def epic_chunks(root: str, run: tmux.Tmux) -> list[tmux.Placed]:
|
||||
"""The windows of do-epic's sessions whose chunks run in the checkout root. A parked chunk doesn't run."""
|
||||
return [placed for placed in tmux.everywhere(run) if placed.session.startswith(f"{tmux.EPIC}-") and placed.root == root and not placed.window.dead and not placed.window.parked]
|
||||
@@ -0,0 +1,114 @@
|
||||
"""How a cycle starts, for every entry point that drives chunks: the models and the ladder it routes on, the route its worker runs on, and the Cycle built from them.
|
||||
|
||||
An entry point starts a cycle in this order:
|
||||
|
||||
1. It reads what it alone knows, such as the premises, the records and the task.
|
||||
2. `project_routing` or `routing_on` reads the models and the ladder.
|
||||
3. `announced_route` picks the route and prints the cycle's opening line.
|
||||
4. `cycle_of` builds the Cycle.
|
||||
|
||||
Each read is made afresh every cycle, so an edit between two cycles takes effect in the next.
|
||||
"""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from agent_cycle.run import Cycle
|
||||
from claude_code.entries import Entry
|
||||
from failure import Failure
|
||||
from agent_cycle.routing import first_route, judge_rung, kept, Scored
|
||||
from chunk_driver.project.model_choice import get_or_ask_models, Models
|
||||
from git_cli.git_repo import Git
|
||||
from model_routing.ladder import Ladder, ProjectRung, Rung
|
||||
from transcript_archive.read import latest_worker
|
||||
from chunk_driver.this_loop import Loop
|
||||
from transcript_archive.project_key import project_key
|
||||
from model_routing.ladder_file import project_ladder_file, read_ladder
|
||||
from model_routing.route import Route, route_from
|
||||
from loop_effects.effects import say
|
||||
from agent_cycle.stop import Stop
|
||||
from agent_cycle.verdicts import TakenIn
|
||||
from claude_code.session import transcript_file
|
||||
from agent_cycle.worker_prompts import WorkerPrompts
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Routing:
|
||||
"""What a cycle routes on, as read when it started."""
|
||||
# The project's worker and judge models.
|
||||
models: Models
|
||||
ladder: Ladder
|
||||
# The file the ladder was read from: the shipped one, or the project's own.
|
||||
ladder_file: pathlib.Path
|
||||
|
||||
@property
|
||||
def project(self) -> ProjectRung | None:
|
||||
"""The project's rung on the ladder; None for a worker model off the ladder."""
|
||||
return self.ladder.project_rung(self.models.worker)
|
||||
|
||||
|
||||
def project_routing(loop: Loop, driver: str) -> Routing | Stop:
|
||||
"""The routing of the project checked out at the loop's root: its models, asked of the human where none is saved, and its ladder. The stop where one cannot be read."""
|
||||
models = get_or_ask_models(project_key(loop.root, loop.paths.home), loop.paths, loop.env, driver, loop.effects)
|
||||
if isinstance(models, Stop):
|
||||
return models
|
||||
ladder_file = project_ladder_file(loop.root, loop.paths.home)
|
||||
return Stop(1, ladder_file.reason) if isinstance(ladder_file, Failure) else routing_on(models, ladder_file)
|
||||
|
||||
|
||||
def routing_on(models: Models, ladder_file: pathlib.Path) -> Routing | Stop:
|
||||
"""The routing of models on the ladder ladder_file holds; the stop where it cannot be read."""
|
||||
ladder = read_ladder(ladder_file)
|
||||
return Stop(1, ladder.reason) if isinstance(ladder, Failure) else Routing(models, ladder, ladder_file)
|
||||
|
||||
|
||||
def announced_route(cycle_number: int, kept_route: Route | None, routing: Routing, archive_folder: pathlib.Path, item: typing.Callable[[], Scored | None], loop: Loop) -> Route:
|
||||
"""The route cycle cycle_number runs on, printed with the cycle's opening line. kept_route is the one the chunk's run holds, and item finds what routes a fresh chunk."""
|
||||
route = routed(kept_route, routing.project, routing.models.worker, lambda: archived_route(archive_folder, loop), item)
|
||||
say(f"\n=== cycle {cycle_number}, worker on {route.rung} ===", loop.effects, stdout=True)
|
||||
return route
|
||||
|
||||
|
||||
def routed(route: Route | None, project: ProjectRung | None, model: str, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Scored | None]) -> Route:
|
||||
"""The route the cycle runs on. project is the project's rung, and model the project's model.
|
||||
|
||||
Where project is None, the model is off the ladder, and the worker runs on it. Otherwise the chunk keeps route, or on a rerun the route its archive last names, as routing.kept allows. Else what item finds routes the chunk afresh.
|
||||
"""
|
||||
if project is None:
|
||||
return Route(Rung(model))
|
||||
route = route or archived()
|
||||
return route if route and kept(route, project) else first_route(project, item())
|
||||
|
||||
|
||||
def archived_route(archive_folder: pathlib.Path, loop: Loop) -> Route | None:
|
||||
"""The route named by the latest verdict archived for the loop's open chunk; None for a chunk not yet judged."""
|
||||
transcript = transcript_file(loop.session)
|
||||
worker = latest_worker(archive_folder, transcript) if transcript else None
|
||||
if isinstance(worker, Failure):
|
||||
say(f"routing the chunk afresh: {worker.reason}", loop.effects)
|
||||
return None
|
||||
route = route_from(worker) if worker else None
|
||||
if worker and route is None:
|
||||
say(f"routing the chunk afresh: the archive's latest Worker: line names no route that can be read: {worker}", loop.effects)
|
||||
return route
|
||||
|
||||
|
||||
def cycle_of(loop: Loop, routing: Routing, route: Route, archive_folder: pathlib.Path, *, task: str, facts: str, premises: str, records: str, unfinished_evidence: str, prompts: WorkerPrompts, taken_in: typing.Callable[[list[Entry]], TakenIn]) -> Cycle:
|
||||
"""The cycle that runs in the loop's window on route. The keyword arguments are what the entry point alone knows; Cycle says what each is."""
|
||||
return Cycle(config=setup_version(loop.paths.claude_dir, loop.effects.git), effects=loop.effects, facts=facts, judge=judge_rung(routing.ladder, routing.models.judge), premises=premises, project=routing.project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=task, prompts=prompts, taken_in=taken_in, unfinished_evidence=unfinished_evidence, window=loop.window)
|
||||
|
||||
|
||||
def setup_version(claude_dir: pathlib.Path, git: Git) -> str:
|
||||
"""The release and commit of this setup that the worker's claude reads, as `v0.8.0 (<sha>)`; a commit past the latest release reads as git describe gives it, such as `v0.8.0-3-g1a2b3c4 (<sha>)`. Where git can't tell, `unknown` and why."""
|
||||
described = git("describe", "--tags", "--match", "v*", "--always", cwd=str(claude_dir))
|
||||
commit = git("rev-parse", "HEAD", cwd=str(claude_dir))
|
||||
if isinstance(described, Failure):
|
||||
return unknown_version(described)
|
||||
if isinstance(commit, Failure):
|
||||
return unknown_version(commit)
|
||||
return f"{described.strip()} ({commit.strip()})"
|
||||
|
||||
|
||||
def unknown_version(failure: Failure) -> str:
|
||||
# One line, since the archive reads the line after it as the judge's.
|
||||
return f"unknown: {' '.join(failure.reason.split())}"
|
||||
@@ -0,0 +1,3 @@
|
||||
"""The entry points that drive chunks, each by its name as its workers, its judge and the human read it. A prompt function that takes a driver takes one of these."""
|
||||
LOOP_CLAUDE = "loop-claude"
|
||||
DO_EPIC = "do-epic"
|
||||
@@ -2,7 +2,7 @@ import pathlib
|
||||
|
||||
from failure import Failure
|
||||
from project_config.main import located
|
||||
from loop_claude.project.text_file import read_utf8
|
||||
from chunk_driver.project.text_file import read_utf8
|
||||
from record_store.stores import store
|
||||
|
||||
# The records the judge reads beside the premises, by the name that heads each, with the setting in the project's config that places it; then the human's rules below.
|
||||
@@ -0,0 +1,36 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from failure import Failure
|
||||
from loop_effects.test_doubles import FakeEffects
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from chunk_driver.project import model_choice
|
||||
from agent_cycle.stop import Stop
|
||||
|
||||
|
||||
class ModelPrompt(unittest.TestCase):
|
||||
OFFERED = [("claude-opus-5-5", "Opus 5.5"), ("claude-haiku-4-5", "Haiku 4.5")]
|
||||
|
||||
def test_reads_a_number_as_the_offered_model_and_anything_else_as_a_name(self):
|
||||
self.assertEqual(model_choice.model_picked(" 2 ", self.OFFERED), "claude-haiku-4-5")
|
||||
self.assertEqual(model_choice.model_picked("opus", self.OFFERED), "opus")
|
||||
self.assertEqual(model_choice.model_picked("3", []), "3")
|
||||
self.assertEqual(model_choice.model_picked("3", self.OFFERED), Failure("pick 1 to 2"))
|
||||
self.assertIsNone(model_choice.model_picked(" ", self.OFFERED))
|
||||
|
||||
def test_asks_again_until_an_answer_picks_a_model(self):
|
||||
with tempfile.TemporaryDirectory() as empty:
|
||||
fake = FakeEffects(inputs=["", "my-model"])
|
||||
self.assertEqual(model_choice.ask_for_model("proj", pathlib.Path("/m.json"), pathlib.Path(empty), LOOP_CLAUDE, fake.effects()), "my-model")
|
||||
self.assertEqual(len(fake.made("input")), 2)
|
||||
fake = FakeEffects(inputs=[EOFError()])
|
||||
self.assertEqual(model_choice.ask_for_model("proj", pathlib.Path("/m.json"), pathlib.Path(empty), LOOP_CLAUDE, fake.effects()), Stop(2, "the human is needed to pick proj's model, but the input ended"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,90 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from project_config.test_project_config import configured
|
||||
from failure import Failure
|
||||
from chunk_driver.project import records
|
||||
|
||||
|
||||
class Records(unittest.TestCase):
|
||||
def test_heads_each_record_by_its_name_the_humans_rules_last(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
|
||||
config = configured(self, pathlib.Path(root), home=claude_md.parents[1]) / "config.toml"
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), "")
|
||||
claude_md.parent.mkdir(parents=True)
|
||||
claude_md.write_text("# Global\n")
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), "From ~/.claude/CLAUDE.md:\n# Global")
|
||||
pathlib.Path(root, "docs").mkdir()
|
||||
pathlib.Path(root, "docs", "decisions.md").write_text("# Decisions\n")
|
||||
pathlib.Path(root, "AGENTS.md").write_text("\n# Rules\n")
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), "From AGENTS.md:\n# Rules\n\nFrom docs/decisions.md:\n# Decisions\n\nFrom ~/.claude/CLAUDE.md:\n# Global")
|
||||
pathlib.Path(claude_md.parent, "rules").mkdir()
|
||||
pathlib.Path(claude_md.parent, "rules", "todo-items.md").write_text("# Todo\n")
|
||||
pathlib.Path(claude_md.parent, "rules", "goals-and-audience.md").write_text("# Goals\n")
|
||||
pathlib.Path(claude_md.parent, "rules", "notes.txt").write_text("not a rule\n")
|
||||
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards").mkdir(parents=True)
|
||||
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards", "comments.md").write_text("# Comments\n")
|
||||
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).endswith("From ~/.claude/CLAUDE.md:\n# Global\n\nFrom ~/.claude/rules/goals-and-audience.md:\n# Goals\n\nFrom ~/.claude/rules/todo-items.md:\n# Todo\n\nFrom ~/.claude/skills/larv-review/standards/comments.md:\n# Comments"))
|
||||
config.write_text(config.read_text().replace('"~/.claude/skills/larv-review/standards"', '"~/review-standards"'))
|
||||
pathlib.Path(claude_md.parents[1], "review-standards").mkdir()
|
||||
pathlib.Path(claude_md.parents[1], "review-standards", "quality.md").write_text("# Ours\n")
|
||||
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).endswith("From ~/.claude/rules/todo-items.md:\n# Todo\n\nFrom ~/review-standards/quality.md:\n# Ours"), "the config's standards folder replaces the default")
|
||||
config.write_text(config.read_text().replace('"~/review-standards"', f'"{root}/standards"'))
|
||||
pathlib.Path(root, "standards").mkdir()
|
||||
pathlib.Path(root, "standards", "quality.md").write_text("# Elsewhere\n")
|
||||
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).endswith(f"From {root}/standards/quality.md:\n# Elsewhere"), "a folder outside home is named in full")
|
||||
|
||||
config.write_text(config.read_text().replace('"checkout:AGENTS.md"', '"folder:AGENTS.md"').replace('"checkout:docs/decisions.md"', '"folder:log/decisions.md"'))
|
||||
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).startswith("From ~/.claude/CLAUDE.md:\n# Global"), "the checkout's AGENTS.md and docs/decisions.md are its owners'")
|
||||
pathlib.Path(config.parent, "log").mkdir()
|
||||
pathlib.Path(config.parent, "log", "decisions.md").write_text("# Ours\n")
|
||||
pathlib.Path(config.parent, "AGENTS.md").write_text("# Our rules\n")
|
||||
self.assertTrue(records.read_records(root, claude_md.parents[1], claude_md.parent).startswith("From AGENTS.md:\n# Our rules\n\nFrom docs/decisions.md:\n# Ours\n\nFrom ~/.claude/CLAUDE.md:\n# Global"))
|
||||
config.unlink()
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"no config names the checkout {root}: write ~/.claude-projects/<key>/config.toml, as `project-config --help` shows - the judge cannot read the records."))
|
||||
|
||||
def test_puts_the_records_handed_in_first_and_reads_only_the_humans_rules_without_a_checkout(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
|
||||
configured(self, pathlib.Path(root), home=claude_md.parents[1])
|
||||
claude_md.parent.mkdir(parents=True)
|
||||
claude_md.write_text("# Global\n")
|
||||
pathlib.Path(claude_md.parent, "rules").mkdir()
|
||||
pathlib.Path(claude_md.parent, "rules", "todo-items.md").write_text("# Todo\n")
|
||||
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards").mkdir(parents=True)
|
||||
pathlib.Path(claude_md.parent, "skills", "larv-review", "standards", "comments.md").write_text("# Comments\n")
|
||||
pathlib.Path(root, "AGENTS.md").write_text("# Rules\n")
|
||||
first = (("the epic ABC-1's description, ## Decisions", "Squash.\n"), ("the epic ABC-1's description, ## Scoring", " \n"))
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent, first), "From the epic ABC-1's description, ## Decisions:\nSquash.\n\nFrom AGENTS.md:\n# Rules\n\nFrom ~/.claude/CLAUDE.md:\n# Global\n\nFrom ~/.claude/rules/todo-items.md:\n# Todo\n\nFrom ~/.claude/skills/larv-review/standards/comments.md:\n# Comments")
|
||||
self.assertEqual(records.read_records(None, claude_md.parents[1], claude_md.parent, first), "From the epic ABC-1's description, ## Decisions:\nSquash.\n\nFrom ~/.claude/CLAUDE.md:\n# Global\n\nFrom ~/.claude/rules/todo-items.md:\n# Todo")
|
||||
|
||||
def test_stops_on_a_record_it_cannot_read(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
|
||||
configured(self, pathlib.Path(root), home=claude_md.parents[1])
|
||||
claude_md.mkdir(parents=True)
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"CLAUDE.md in {claude_md.parent}: Is a directory - the judge cannot read the records."))
|
||||
pathlib.Path(root, "docs", "decisions.md").mkdir(parents=True)
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"cannot read {root}/docs/decisions.md: Is a directory - the judge cannot read the records."))
|
||||
pathlib.Path(root, "AGENTS.md").write_bytes(b"# Rules\n\xff\n")
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"AGENTS.md in {root} is not UTF-8 on line 2 - the judge cannot read the records."))
|
||||
|
||||
|
||||
def test_stops_on_a_rule_it_cannot_read(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
claude_md = pathlib.Path(root, "home", ".claude", "CLAUDE.md")
|
||||
configured(self, pathlib.Path(root), home=claude_md.parents[1])
|
||||
rules = pathlib.Path(claude_md.parent, "rules")
|
||||
rules.mkdir(parents=True)
|
||||
pathlib.Path(rules, "todo-items.md").write_bytes(b"\xff\n")
|
||||
self.assertEqual(records.read_records(root, claude_md.parents[1], claude_md.parent), Failure(f"todo-items.md in {rules} is not UTF-8 on line 1 - the judge cannot read the records."))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,20 +1,17 @@
|
||||
"""What loop-claude says to the worker each cycle. model_routing.review_choice chooses where the reviews run. review_launches and reruns put that choice into sentences, and escalation says where escalated subagents run; rung_sentences returns the sentences of all three. In a chunk's first prompt, routed_by places them after its routing sentences where an item routed the chunk, and fresh places them alone otherwise. rungs_held repeats them at the end of a prompt after a step-up or an unfinished verdict."""
|
||||
"""What every entry point says to the worker each cycle. model_routing.review_choice chooses where the reviews run. review_launches and reruns put that choice into sentences, and escalation says where escalated subagents run; rung_sentences returns the sentences of all three. An entry point places them in a chunk's first prompt, as loop-claude does in bin/loop_claude/chunk/fresh_prompt.py. rungs_held repeats them at the end of a prompt after a step-up or an unfinished verdict."""
|
||||
import functools
|
||||
import typing
|
||||
|
||||
from agent_cycle.worker_prompts import answers_recorded, WorkerPrompts
|
||||
from agent_cycle.escalation_mark import asks_higher_rung
|
||||
from model_routing.review_choice import chosen, ReviewChoice
|
||||
from loop_claude.cycle.chunk_items import CHUNK_ITEMS
|
||||
from model_routing.route import ESCALATION_MARKER, Route
|
||||
from agent_cycle.routing import escalation_rungs
|
||||
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES, RECORDS_NAMED
|
||||
from chunk_driver.project.records import HUMAN_RULE_NAMES, RECORD_FILES, RECORDS_NAMED
|
||||
from review_rules.review_passes import INSTRUCTION_FILES, INSTRUCTION_FOLDERS
|
||||
from text_files.markdown_text import listed, quoted
|
||||
from model_routing.ladder import ProjectRung, Rung
|
||||
|
||||
# loop-claude's name as its workers, its judge and the human read it. A function below that takes a driver words a sentence for any entry point, such as do-epic.
|
||||
DRIVER = "loop-claude"
|
||||
# Where the judge may quote an answer from, named as the human and the worker see them; of the goals file the judge sees only the premises.
|
||||
QUOTE_SOURCES = listed(("the goals file's Goals and Audience", *RECORD_FILES, *HUMAN_RULE_NAMES), "or")
|
||||
# Where the worker may write an answer the human typed, so it counts as recorded.
|
||||
@@ -28,45 +25,8 @@ THE_SECURITY_PASS = "the security pass's subagent"
|
||||
|
||||
# Workers only, not CLAUDE.md, which reviewers load too: a reviewer under a like rule missed findings.
|
||||
READING_RULE = "Read in small pieces: never read more than 80 lines of a file at once, nor take a tool output over 3,000 characters into your context."
|
||||
# chunk_items reads the latest such line; three fresh readers picked the label (docs/decisions.md → `routing: loop-claude reads the items a chunk takes`).
|
||||
NAME_THE_ITEMS = f"Name every plan item this chunk takes on one line, as in `{CHUNK_ITEMS} 297 (L 3, E + I 5), 298 (L 3, E + I 7)`, and write the whole line again whenever the set changes."
|
||||
|
||||
|
||||
AFTER_CODE_HEALTH = 'which rank after code health (~/.claude/CLAUDE.md → "Shared goal") unless the goals file voids it'
|
||||
|
||||
|
||||
def fresh(driver: str, task: str, premises: str, route: Route, project: ProjectRung | None, facts: str) -> str:
|
||||
"""A chunk's first prompt; facts is preflight_facts' or preflight_failed's text."""
|
||||
routed = f"{routed_by(driver, route, project)} {recount(route)}" if route.item else " ".join(rung_sentences(driver, route, project))
|
||||
opening = f"{routed}\n\n" if routed else ""
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one. {READING_RULE} {NAME_THE_ITEMS}\n\n{opening}{facts}\n\nThe chunk's plan items, or else the task above, set its scope. The goals below, {AFTER_CODE_HEALTH}, choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file, {premises}"
|
||||
|
||||
|
||||
def routed_by(driver: str, route: Route, project: ProjectRung | None) -> str:
|
||||
return " ".join((
|
||||
f"{driver} runs you on {route.rung}, routed by item {route.item}'s L of {route.likelihood}.",
|
||||
*rung_sentences(driver, route, project),
|
||||
))
|
||||
|
||||
|
||||
def recount(route: Route) -> str:
|
||||
"""Has the worker count the routing item's size again once it knows the code, since the L that routed it was scored before anyone read that code."""
|
||||
return f"After your first read of the code, count item {route.item}'s size again: the lines, files and components you will change. Where that count would change its L, have a subagent that scores plan items re-score it; a re-score whose L picks a rung above yours is escalation trigger 1."
|
||||
|
||||
|
||||
def preflight_facts(printed: str) -> str:
|
||||
"""What chunk-preflight printed, for the worker to take as found."""
|
||||
return f"loop-claude ran `~/.claude/bin/chunk-preflight` at the checkout root just before this run. Take what it printed as found rather than finding it again:\n\n{printed}"
|
||||
|
||||
|
||||
def held_by_others(items: list[int]) -> str:
|
||||
"""Keeps a fresh worker off the plan items the project's other chunks hold, running or stopped."""
|
||||
return f"\n\nOther chunks of this project, running or stopped, hold {'items ' if len(items) > 1 else 'item '}{', '.join(map(str, items))}. Leave those items out of this chunk, even where the facts above list them as items that may join it."
|
||||
|
||||
|
||||
def preflight_failed(reason: str) -> str:
|
||||
"""Has the worker find the facts chunk-preflight would have, its worktree among them."""
|
||||
return f"loop-claude could not run `~/.claude/bin/chunk-preflight`, so find the chunk's facts yourself. Why it could not run: {reason}"
|
||||
|
||||
|
||||
def review_launches(route: Route, project: ProjectRung | None) -> list[str]:
|
||||
@@ -196,6 +156,3 @@ class RecordNames(typing.NamedTuple):
|
||||
def worker_prompts(driver: str, fresh: typing.Callable[[str, str, Route, ProjectRung | None, str], str], naming: str, names: RecordNames = RecordNames()) -> WorkerPrompts:
|
||||
"""The prompts of an entry point's cycles: its own fresh prompt, and the resumed prompt and rung sentences every entry point shares, each naming driver. resumed says what naming is. names is loop-claude's unless the entry point keeps records elsewhere, as do-epic does."""
|
||||
return WorkerPrompts(driver, fresh, functools.partial(resumed, driver, naming), functools.partial(rungs_held, driver), *names)
|
||||
|
||||
|
||||
WORKER = worker_prompts(DRIVER, functools.partial(fresh, DRIVER), NAME_THE_ITEMS)
|
||||
@@ -0,0 +1,40 @@
|
||||
import os
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from chunk_driver import cycle_start
|
||||
from failure import Failure
|
||||
from git_cli.git_repo import git
|
||||
|
||||
|
||||
class SetupVersion(unittest.TestCase):
|
||||
def test_names_the_release_and_the_commit_or_why_git_cannot_tell(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
isolated = {"GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1", "GIT_CEILING_DIRECTORIES": root}
|
||||
with unittest.mock.patch.dict(os.environ, isolated):
|
||||
gone = cycle_start.setup_version(pathlib.Path(root), git)
|
||||
self.assertRegex(gone, r"^unknown: [^\n]*not a git repository")
|
||||
for step in (["init", "-q"], ["-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "--allow-empty", "-m", "one"]):
|
||||
self.assertIsInstance(git(*step, cwd=root), str)
|
||||
head = git("rev-parse", "HEAD", cwd=root).strip()
|
||||
self.assertEqual(cycle_start.setup_version(pathlib.Path(root), git), f"{head[:7]} ({head})")
|
||||
git("tag", "v0.1.0", cwd=root)
|
||||
self.assertEqual(cycle_start.setup_version(pathlib.Path(root), git), f"v0.1.0 ({head})")
|
||||
git("-c", "user.name=t", "-c", "user.email=t@t", "commit", "-q", "--allow-empty", "-m", "two", cwd=root)
|
||||
head = git("rev-parse", "HEAD", cwd=root).strip()
|
||||
self.assertEqual(cycle_start.setup_version(pathlib.Path(root), git), f"v0.1.0-1-g{head[:7]} ({head})")
|
||||
|
||||
def test_names_why_git_cannot_name_the_commit_on_one_line(self):
|
||||
def fake_git(*args, cwd=None):
|
||||
return Failure("fatal: bad HEAD\nhint: repair it") if args[0] == "rev-parse" else "v0.1.0\n"
|
||||
|
||||
self.assertEqual(cycle_start.setup_version(pathlib.Path("/r"), fake_git), "unknown: fatal: bad HEAD hint: repair it")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -10,8 +10,8 @@ from agent_cycle.tmux import Window
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Loop:
|
||||
"""What stays the same through one run of loop-claude; run_loop builds it once per process, so a reload builds it afresh."""
|
||||
# The checkout the loop runs in. LoopRun.root names it too, since a reload hands it on there.
|
||||
"""What stays the same while an entry point drives one chunk in one process. loop-claude builds it once per process, so a reload builds it afresh; do-epic builds one per chunk."""
|
||||
# The checkout the loop runs in. loop-claude's LoopRun.root names it too, since a reload hands it on there.
|
||||
root: str
|
||||
# The worker's session.
|
||||
session: Session
|
||||
@@ -19,6 +19,6 @@ class Loop:
|
||||
window: Window
|
||||
paths: Paths
|
||||
env: typing.Mapping[str, str]
|
||||
# The loop-claude script, whose edits reload the loop and beside which chunk-preflight lies.
|
||||
# The entry point's script, beside which chunk-preflight lies; an edit of loop-claude's reloads its loop.
|
||||
launcher: pathlib.Path
|
||||
effects: Effects
|
||||
@@ -1,11 +1,10 @@
|
||||
"""The human approval that `chunk-pr merge` waits for where the project's config sets `wait-for-approval`."""
|
||||
import json
|
||||
import pathlib
|
||||
import time
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
from forge.commands import Forge, Run
|
||||
from forge.reviews import github_decision, waiting_threads
|
||||
|
||||
# A human answers in hours, so merge polls once a minute and gives up after 90 minutes. That leaves half an hour for the queue and the checks inside the 2 hours a background command may run; a run that needs more is killed before the give-up message prints, and its next run goes on from what it pushed.
|
||||
POLL_SECONDS = 60
|
||||
@@ -21,20 +20,9 @@ def awaited(forge: Forge, wait_for_approval: bool) -> bool | Failure:
|
||||
return wait_for_approval
|
||||
|
||||
|
||||
# Each review thread, with whether it is resolved and who wrote its last comment, beside who opened the pull request. gh fills {owner} and {repo} from the checkout.
|
||||
THREADS = """query($owner: String!, $repo: String!, $number: Int!) { repository(owner: $owner, name: $repo) { pullRequest(number: $number) {
|
||||
author { login }
|
||||
reviewThreads(first: 100) { nodes { isResolved path comments(last: 1) { nodes { author { login } } } } }
|
||||
} } }"""
|
||||
|
||||
|
||||
def approval(number: str, run: Run) -> bool | Failure:
|
||||
"""Whether GitHub reports pull request number approved, as its branch protection asks, read once: False while it still wants a review. A Failure where a reviewer requested changes, or where the protection asks for no review."""
|
||||
viewed = run(["gh", "pr", "view", number, "--json", "reviewDecision"])
|
||||
try:
|
||||
decision = viewed if isinstance(viewed, Failure) else json.loads(viewed)["reviewDecision"]
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh pr view printed no review decision: {str(viewed).strip()[:200]}")
|
||||
decision = github_decision(number, run)
|
||||
if isinstance(decision, Failure):
|
||||
return decision
|
||||
if decision == "CHANGES_REQUESTED":
|
||||
@@ -73,16 +61,3 @@ def unanswered(number: str, run: Run) -> Failure | None:
|
||||
if not waiting:
|
||||
return None
|
||||
return Failure(f"a reviewer's comment on pull request {number} waits for an answer, in {', '.join(waiting)}; answer it in its thread, push any fix, and run chunk-pr merge again")
|
||||
|
||||
|
||||
def waiting_threads(number: str, run: Run) -> list[str] | Failure:
|
||||
"""The files where a reviewer's comment waits for an answer on pull request number, read once: those of each review thread nobody resolved, whose last comment someone other than the pull request's author wrote. So a thread the worker answered, as that author, waits for the reviewer again."""
|
||||
read = run(["gh", "api", "graphql", "-f", f"query={THREADS}", "-F", "owner={owner}", "-F", "repo={repo}", "-F", f"number={number}"])
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
try:
|
||||
pull = json.loads(read)["data"]["repository"]["pullRequest"]
|
||||
author = pull["author"]["login"]
|
||||
return sorted({thread["path"] for thread in pull["reviewThreads"]["nodes"] if not thread["isResolved"] and any(comment["author"]["login"] != author for comment in thread["comments"]["nodes"])})
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh api graphql printed no review threads: {read.strip()[:200]}")
|
||||
|
||||
@@ -16,7 +16,7 @@ from failure import Failure
|
||||
from test_env import isolated_env
|
||||
from project_config.settings import ProjectConfig
|
||||
from forge.commands import Run
|
||||
from chunk_pr.approval import THREADS
|
||||
from forge.reviews import THREADS
|
||||
|
||||
TOOL = pathlib.Path(__file__).resolve().parents[1] / "chunk-pr"
|
||||
GITHUB = "git@github.com:me/project.git"
|
||||
|
||||
+12
-14
@@ -5,16 +5,16 @@ import typing
|
||||
|
||||
from adf_codec.convert import adfs_to_markdown, markdown_to_adf, Refused
|
||||
from atlassian.client import basic_auth, Http, jira_json, ToAdf, ToMarkdown
|
||||
from plan_store.jira_epic import blocked_by_links, DONE, FILED_BY, finishing_transition, ISSUE_KEY, ISSUE_TYPE, move_refusal, PERHAPS_MOVED, project_of, PROPERTY, rank_refusal, scores, searched
|
||||
from atlassian.jira import DONE, entry_of, finishing_transition, ISSUE_KEY, move_refusal, PERHAPS_FILED, PERHAPS_MOVED, project_of, rank_refusal, searched, unshaped
|
||||
from project_config.settings import Epic, Finish, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION
|
||||
from do_epic.backend.protocol import EpicIssue, no_child, no_parent, no_words, unfit, unfit_label
|
||||
from failure import Failure
|
||||
from plan_store.jira_epic import FILED_BY, ISSUE_TYPE, PROPERTY, scores
|
||||
|
||||
FIELDS = ["summary", "description", "labels", "assignee", "status", "parent", "issuelinks"]
|
||||
# How many child issues one search for sub-tasks names, which keeps its JQL well inside the length Jira takes.
|
||||
PARENTS_PER_SEARCH = 50
|
||||
NO_FINISH = "do-epic's settings name no transition to {to} by: set `{transition}` and `{status}` in ~/.claude-projects/do-epic/config.toml, to the names Jira shows, such as \"Mark as done\" and \"Done\""
|
||||
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under {parent} before filing it again"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -74,10 +74,10 @@ class JiraEpic:
|
||||
fields["description"] = description
|
||||
created = self.http("POST", f"{self.epic.site}/rest/api/3/issue", authorization, {"fields": fields | {"issuetype": kind}, "properties": properties})
|
||||
if isinstance(created, Failure):
|
||||
return Failure(f"cannot file the issue under {parent}: {created.reason}; {PERHAPS_FILED.format(parent=parent)}")
|
||||
return Failure(f"cannot file the issue under {parent}: {created.reason}; {PERHAPS_FILED.format(under=parent)}")
|
||||
key = created.get("key") if isinstance(created, dict) else None
|
||||
if not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
|
||||
return Failure(f"Jira answered the new issue without an issue key; {PERHAPS_FILED.format(parent=parent)}")
|
||||
return Failure(f"Jira answered the new issue without an issue key; {PERHAPS_FILED.format(under=parent)}")
|
||||
return key
|
||||
|
||||
def add_label(self, key: str, label: str) -> None | Failure:
|
||||
@@ -129,7 +129,7 @@ class JiraEpic:
|
||||
category = answered["fields"]["status"]["statusCategory"]["key"] # type: ignore[index]
|
||||
except (KeyError, TypeError) as failure:
|
||||
return Failure(f"{self.epic_key}'s status is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}")
|
||||
return bool(category == "done")
|
||||
return bool(category == DONE)
|
||||
|
||||
def close(self, summary: str) -> None | Failure:
|
||||
if self.finish is None:
|
||||
@@ -243,19 +243,17 @@ class JiraEpic:
|
||||
|
||||
def found_of(raw: object) -> Found | Failure:
|
||||
"""The issue in one entry of a search's answer, or which issue is not in the shape Jira documents."""
|
||||
key = raw.get("key") if isinstance(raw, dict) else None
|
||||
if not isinstance(raw, dict) or not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
|
||||
return Failure("the search answered an issue without an issue key")
|
||||
# The answer's nested fields are read as Jira documents them; one missing or of another type ends the read here.
|
||||
entry = entry_of(raw, PROPERTY)
|
||||
if isinstance(entry, Failure):
|
||||
return entry
|
||||
key, fields, held = entry.key, entry.fields, entry.held
|
||||
blockers = tuple(link.blocker for link in entry.links)
|
||||
try:
|
||||
fields = raw["fields"]
|
||||
summary, labels, assignee, status, parent, description = fields["summary"], fields["labels"], fields["assignee"], fields["status"], fields["parent"]["key"], fields["description"]
|
||||
name = assignee["displayName"] if assignee is not None else None
|
||||
state, done = status["name"], status["statusCategory"]["key"] == "done"
|
||||
blockers = tuple(link.blocker for link in blocked_by_links(fields))
|
||||
held = raw.get("properties", {}).get(PROPERTY, {})
|
||||
state, done = status["name"], status["statusCategory"]["key"] == DONE
|
||||
except (AttributeError, KeyError, TypeError) as failure:
|
||||
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}")
|
||||
return Failure(unshaped(key, failure))
|
||||
texts = (summary, *labels, state, parent, *blockers) if isinstance(labels, list) else ()
|
||||
if not texts or not all(isinstance(text, str) for text in texts) or not isinstance(name, str | None) or not all(ISSUE_KEY.fullmatch(each) for each in (parent, *blockers)) or not isinstance(description, dict | None):
|
||||
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: its summary, labels, assignee, status, parent, description or a linked issue's key holds another kind of value")
|
||||
|
||||
@@ -12,7 +12,6 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
from project_config.settings import Epic, Finish
|
||||
from do_epic.backend.contract import EpicStoreContract
|
||||
from failure import Failure
|
||||
from plan_store import jira_epic
|
||||
from do_epic.backend.jira import JiraEpic
|
||||
from adf_codec.convert import Refused
|
||||
|
||||
@@ -328,7 +327,7 @@ class Jira(Case):
|
||||
("ABC-34", "the search answered an issue without an issue key"),
|
||||
({"fields": valid}, "the search answered an issue without an issue key"),
|
||||
({"key": "abc-34", "fields": valid}, "the search answered an issue without an issue key"),
|
||||
({"key": "ABC-34", "fields": {"summary": "Count the runs.", "parent": {"key": "ABC-12"}}}, shape + ": KeyError('labels')"),
|
||||
({"key": "ABC-34", "fields": {"summary": "Count the runs.", "parent": {"key": "ABC-12"}, "issuelinks": []}}, shape + ": KeyError('labels')"),
|
||||
*((({"key": "ABC-34", "fields": valid | changed}, shape + ": its summary, labels, assignee, status, parent, description or a linked issue's key holds another kind of value") for changed in (
|
||||
{"summary": 3}, {"labels": "api"}, {"labels": [3]}, {"assignee": {"displayName": 0}}, {"status": {"name": 0, "statusCategory": {"key": "new"}}}, {"parent": {"key": "ABC"}}, {"parent": {"key": 12}}, {"description": "text"}, {"issuelinks": [linked | {"inwardIssue": {"key": 7}}]}, {"issuelinks": [linked | {"inwardIssue": {"key": "ABC-x"}}]},
|
||||
))),
|
||||
@@ -352,7 +351,7 @@ class Jira(Case):
|
||||
children = [store.file("ABC-12", "Count the runs.", "", ()) for _ in range(2)]
|
||||
for _ in range(3):
|
||||
store.file(str(children[0]), "A task.", "", ())
|
||||
with unittest.mock.patch.object(jira_epic, "MAX_PAGES", 1):
|
||||
with unittest.mock.patch("atlassian.jira.MAX_PAGES", 1):
|
||||
self.assertEqual(store.read(), Failure("the search for the sub-tasks of ABC-12's child issues did not end within 1 pages"))
|
||||
store.file("ABC-12", "Count the runs.", "", ())
|
||||
self.assertEqual(store.read(), Failure("the search for ABC-12's child issues did not end within 1 pages"))
|
||||
|
||||
@@ -1,19 +1,20 @@
|
||||
"""What a do-epic cycle's worker is told: its task, from the issue, and its first prompt. The resumed prompt and the rung sentences are the ones loop-claude's cycles use, naming do-epic; a chunk that was parked is first told why it resumes, and one whose task follows the epic's state is told that task as it resumes."""
|
||||
import dataclasses
|
||||
|
||||
from chunk_driver.drivers import DO_EPIC
|
||||
from do_epic.backend.protocol import EpicIssue
|
||||
from do_epic import findings
|
||||
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES
|
||||
from chunk_driver.project.records import HUMAN_RULE_NAMES, RECORD_FILES
|
||||
from text_files.markdown_text import listed
|
||||
import pathlib
|
||||
from project_config.settings import ProjectConfig
|
||||
from model_routing.ladder import ProjectRung
|
||||
from loop_claude.cycle import prompts
|
||||
from chunk_driver import prompts
|
||||
from model_routing.route import Route
|
||||
from agent_cycle.worker_prompts import WorkerPrompts
|
||||
|
||||
# The entry point's name as the worker, the judge and the human read it.
|
||||
DRIVER = "do-epic"
|
||||
DRIVER = DO_EPIC
|
||||
|
||||
|
||||
def shipping(config: ProjectConfig) -> str:
|
||||
|
||||
@@ -6,7 +6,7 @@ from atlassian.client import basic_auth, Http, jira_json, ToAdf
|
||||
from failure import Failure
|
||||
from adf_codec.convert import markdown_to_adf
|
||||
from do_epic.e2e.settings import Sandbox
|
||||
from plan_store.jira_epic import searched
|
||||
from atlassian.jira import searched
|
||||
|
||||
EPIC_TYPE = "Epic"
|
||||
# How a team-managed project's workflow moves an issue to done, as do-epic's `jira-finish-transition` and `jira-finish-status` name it.
|
||||
|
||||
+4
-49
@@ -2,22 +2,17 @@
|
||||
|
||||
A gate is something only time or a person passes: checks that still run, an approval a reviewer has yet to give, or the new review of a reviewer whose requested changes the worker answered. While a chunk's pull request waits on one, its worker doesn't have anything to do, so do-epic parks the chunk (`parked`). Anything else is news the worker acts on."""
|
||||
import dataclasses
|
||||
import json
|
||||
import typing
|
||||
|
||||
from forge.reviews import ANSWERED, review_decision, waiting_threads
|
||||
from forge.pulls import CheckFailed, checks_now, open_pull, Pending, required_checks
|
||||
from failure import Failure
|
||||
from forge.commands import forge_of, gitea, Run
|
||||
from chunk_pr.approval import waiting_threads
|
||||
from forge.commands import forge_of, Run
|
||||
|
||||
# What a pull request waits on, as the words after "awaits".
|
||||
CHECKS = "its checks"
|
||||
APPROVAL = "an approval"
|
||||
NEW_REVIEW = "a new review"
|
||||
# What `review_decision` returns in place of GitHub's `CHANGES_REQUESTED` once the worker answered those changes: GitHub keeps its decision until the reviewer reviews again.
|
||||
ANSWERED = "CHANGES_ANSWERED"
|
||||
# What the title of a draft opens with on Gitea, in any case: Gitea's default `WORK_IN_PROGRESS_PREFIXES`.
|
||||
GITEA_DRAFT = ("WIP:", "[WIP]")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -41,7 +36,7 @@ class Open:
|
||||
def gate(branch: str, checkout: str, at: typing.Callable[[str], Run]) -> Gate | Open | Failure:
|
||||
"""What the chunk on branch waits on, in a worktree of checkout; at gives the runner of commands in a folder. A Failure where git or the forge cannot be read, which doesn't say anything about the gate.
|
||||
|
||||
A draft never waits on a gate: its worker still works on it. On Gitea only the checks are a gate, since chunk-pr reads approvals on GitHub only (`chunk_pr.approval.awaited`). Changes a reviewer requested are news until the worker answered them (`answered`); from then on the pull request awaits a new review.
|
||||
A draft never waits on a gate: its worker still works on it. On Gitea only the checks are a gate, since chunk-pr reads approvals on GitHub only (`chunk_pr.approval.awaited`). Changes a reviewer requested are news until the worker answered them (`forge.reviews.answered`); from then on the pull request awaits a new review.
|
||||
"""
|
||||
worktree = worktree_of(branch, at(checkout))
|
||||
if not isinstance(worktree, str):
|
||||
@@ -56,7 +51,7 @@ def gate(branch: str, checkout: str, at: typing.Callable[[str], Run]) -> Gate |
|
||||
pull = open_pull(forge, branch, head, run)
|
||||
if not isinstance(pull, str):
|
||||
return pull or Open(f"no pull request from origin's {branch} is open at {head[:12]}; it may be merged or closed already")
|
||||
decision = review_decision(pull, head, run) if forge == "gh" else gitea_decision(pull, run)
|
||||
decision = review_decision(forge, pull, head, run)
|
||||
if not isinstance(decision, str):
|
||||
return decision or Open(f"pull request #{pull} is a draft")
|
||||
required = required_checks(forge, base.removeprefix("origin/"), run)
|
||||
@@ -89,43 +84,3 @@ def worktree_of(branch: str, run: Run) -> str | None | Failure:
|
||||
if f"branch refs/heads/{branch}" in lines and lines[0].startswith("worktree "):
|
||||
return lines[0].removeprefix("worktree ")
|
||||
return None
|
||||
|
||||
|
||||
def review_decision(pull: str, head: str, run: Run) -> str | None | Failure:
|
||||
"""GitHub's review decision on the pull request, whose head commit is head, read once: `APPROVED`, `REVIEW_REQUIRED`, `CHANGES_REQUESTED`, or empty where its branch protection asks for no review. ANSWERED in place of `CHANGES_REQUESTED` once the worker answered those changes. None where it is a draft."""
|
||||
viewed = run(["gh", "pr", "view", pull, "--json", "isDraft,reviewDecision,author,reviews,comments"])
|
||||
if isinstance(viewed, Failure):
|
||||
return viewed
|
||||
try:
|
||||
parsed = json.loads(viewed)
|
||||
if parsed["isDraft"]:
|
||||
return None
|
||||
decision = str(parsed["reviewDecision"] or "")
|
||||
return ANSWERED if decision == "CHANGES_REQUESTED" and answered(parsed, head) else decision
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh pr view printed no review decision: {viewed.strip()[:200]}")
|
||||
|
||||
|
||||
def answered(pull: dict[str, typing.Any], head: str) -> bool:
|
||||
"""Whether the worker answered the newest review that requested changes on pull, as `gh pr view --json author,reviews,comments` prints it, whose head commit is head: the head is no longer the commit that review was made on, or the pull request's author commented or replied in a thread after it. GitHub lists a reply in a thread as a review of its own. False where it lists no review that requested changes. Raises KeyError or TypeError where pull has another shape.
|
||||
|
||||
The review names its commit, so no clock tells whether the worker pushed: a commit's date says when it was made, never when it was pushed.
|
||||
"""
|
||||
asked = max((review for review in pull["reviews"] if review["state"] == "CHANGES_REQUESTED"), key=lambda review: review["submittedAt"], default=None)
|
||||
if asked is None:
|
||||
return False
|
||||
author = pull["author"]["login"]
|
||||
# A review the author has yet to submit has no time, and answers nobody.
|
||||
said = [review["submittedAt"] or "" for review in pull["reviews"] if review["author"]["login"] == author] + [comment["createdAt"] for comment in pull["comments"] if comment["author"]["login"] == author]
|
||||
return asked["commit"]["oid"] != head or any(at > asked["submittedAt"] for at in said)
|
||||
|
||||
|
||||
def gitea_decision(pull: str, run: Run) -> str | None | Failure:
|
||||
"""What stands in for `review_decision` on Gitea, read once: empty, since no review is a gate there, and None where the pull request is a draft, which Gitea tells by its title (GITEA_DRAFT)."""
|
||||
viewed = gitea(run, f"repos/{{owner}}/{{repo}}/pulls/{pull}")
|
||||
if isinstance(viewed, Failure):
|
||||
return viewed
|
||||
try:
|
||||
return None if viewed["title"].upper().startswith(GITEA_DRAFT) else ""
|
||||
except (KeyError, TypeError, AttributeError):
|
||||
return Failure(f"tea api printed no title of pull request {pull}: {str(viewed)[:200]}")
|
||||
|
||||
+27
-37
@@ -8,12 +8,13 @@ import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from loop_claude.loop.chunk_start import archived_route, routed
|
||||
from chunk_driver.cycle_start import announced_route, cycle_of, project_routing, routing_on
|
||||
from chunk_driver.claims import CHUNK_FLAG
|
||||
from do_epic import chunk_prompt, findings, finish, gates, human_tasks, parked, planning, scheduler, split, status
|
||||
from chunk_driver.chunk_run import ChunkRun, closed_cycle
|
||||
from do_epic.repo_pick import ClaudeWork, HumanTask, recorded, Repo
|
||||
from loop_claude.loop.close import closed_cycle
|
||||
from project_config.main import located as config_located
|
||||
from loop_claude.project.records import CONSEQUENCE, read_records
|
||||
from chunk_driver.project.records import CONSEQUENCE, read_records
|
||||
from agent_cycle.run import Cycle, run_cycle
|
||||
from do_epic.config import DoEpicConfig, load
|
||||
from loop_effects.effects import Effects, say
|
||||
@@ -23,21 +24,17 @@ from do_epic.backend.protocol import EpicIssue, EpicStore
|
||||
from do_epic.epic_record import EpicRecord
|
||||
from failure import Failure
|
||||
from agent_cycle.verdicts import Finished, TakenIn
|
||||
from loop_claude.project.model_choice import get_or_ask_models, Models
|
||||
from plan_store.jira_epic import ISSUE_KEY
|
||||
from atlassian.jira import ISSUE_KEY
|
||||
from do_epic.backend.jira import JiraEpic
|
||||
from agent_cycle.routing import judge_rung
|
||||
from claude_code.reply import Limited
|
||||
from do_epic.repos import locate
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from loop_claude.loop.reload import LoopRun
|
||||
from chunk_driver.this_loop import Loop
|
||||
from chunk_driver.project.model_choice import Models
|
||||
from loop_state.paths import Paths
|
||||
from transcript_archive.project_key import project_key
|
||||
from model_routing.ladder_file import project_ladder_file, read_ladder, SHIPPED
|
||||
from loop_claude.project.premises import read_premises
|
||||
from chunk_driver.project.premises import read_premises
|
||||
from forge.commands import Run, runner
|
||||
from claude_code.session import session_named
|
||||
from loop_claude.loop.main import setup_version
|
||||
from model_routing.ladder_file import SHIPPED
|
||||
from agent_cycle.stop import Stop
|
||||
from agent_cycle.direct_message import tell
|
||||
from agent_cycle import tmux
|
||||
@@ -110,13 +107,13 @@ def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher:
|
||||
if stop.reason:
|
||||
say(stop.reason, effects, stdout=not stop.code)
|
||||
# A chunk's stop is DMed, as loop-claude's is, so a chunk left overnight never ends unseen.
|
||||
if stop.reason and argv[:1] == [scheduler.CHUNK_FLAG]:
|
||||
if stop.reason and argv[:1] == [CHUNK_FLAG]:
|
||||
tell(f"do-epic {' '.join(argv)} stopped: {stop.reason}", effects)
|
||||
return stop.code
|
||||
|
||||
|
||||
def run(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, effects: Effects, backend: Backend, stdin: typing.TextIO, forge_at: typing.Callable[[str], Run]) -> Stop:
|
||||
chunk, words = (argv[1], argv[2:]) if len(argv) >= 2 and argv[0] == scheduler.CHUNK_FLAG else (None, argv)
|
||||
chunk, words = (argv[1], argv[2:]) if len(argv) >= 2 and argv[0] == CHUNK_FLAG else (None, argv)
|
||||
status_asked = chunk is None and words[1:] == [STATUS_FLAG]
|
||||
words = words[:1] if status_asked else words
|
||||
# What the words after the epic's key write into the epic, where they are a filing or a decision.
|
||||
@@ -242,7 +239,7 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
|
||||
"""Runs the cycles of the chunk of the issue called key until one stops the loop, or the judge rules the chunk finished. For a human's task it runs no cycle: it waits until the task no longer holds up an issue (`human_tasks.awaited`), then starts the next chunks. A chunk in a repo parks after each cycle that leaves its pull request waiting on a gate, and before its first where it was parked when its window last ended, as at a reboot."""
|
||||
pane = epic.env.get("TMUX_PANE")
|
||||
if not pane:
|
||||
return Stop(1, f"{scheduler.CHUNK_FLAG} is how do-epic starts itself in a tmux window; run do-epic <EPIC-KEY>")
|
||||
return Stop(1, f"{CHUNK_FLAG} is how do-epic starts itself in a tmux window; run do-epic <EPIC-KEY>")
|
||||
# None for the epic's finishing, which is no issue.
|
||||
issue = None if key == epic.key else issue_read(key, epic)
|
||||
if isinstance(issue, Stop):
|
||||
@@ -256,24 +253,24 @@ def chunk_run(key: str, epic: EpicRun) -> Stop:
|
||||
root, repo = place
|
||||
session = session_named(f"A {epic.key} do-epic {key}", epic.paths.claude_code_transcripts)
|
||||
loop = Loop(root=root, session=session, window=tmux.Window(epic.session, key, pane), paths=epic.paths, env=epic.env, launcher=epic.launcher, effects=epic.effects)
|
||||
loop_run = LoopRun(0, root, "", "", chunk=key)
|
||||
run = ChunkRun()
|
||||
# What a chunk that was parked is told as it resumes; empty for a chunk that wasn't.
|
||||
news = parked_through(repo, loop, epic, starts=True) if repo is not None else ""
|
||||
while True:
|
||||
if isinstance(news, Stop):
|
||||
return news
|
||||
loop_run = dataclasses.replace(loop_run, cycle_number=loop_run.cycle_number + 1)
|
||||
cycle = next_cycle(loop_run, loop, repo is not None, epic, news)
|
||||
run = dataclasses.replace(run, cycle_number=run.cycle_number + 1)
|
||||
cycle = next_cycle(run, loop, repo is not None, epic, news)
|
||||
if isinstance(cycle, Stop):
|
||||
return cycle
|
||||
loop_run = dataclasses.replace(loop_run, task=cycle.task)
|
||||
run = dataclasses.replace(run, task=cycle.task)
|
||||
concluded = run_cycle(cycle)
|
||||
if isinstance(concluded, Stop):
|
||||
return concluded
|
||||
closed = closed_cycle(loop_run, cycle.project, concluded, cycle.project_archive, chunk_prompt.DRIVER, loop)
|
||||
closed = closed_cycle(run, cycle.project, concluded, cycle.project_archive, chunk_prompt.DRIVER, loop)
|
||||
if isinstance(closed, Stop):
|
||||
return closed
|
||||
loop_run = closed
|
||||
run = closed
|
||||
news = parked_through(repo, loop, epic, starts=False) if repo is not None else ""
|
||||
# A chunk that was parked has work again, whatever the judge ruled before its pull request passed the gate.
|
||||
if isinstance(concluded.verdict, Finished) and news == "":
|
||||
@@ -399,8 +396,8 @@ def split_items_closed(epic: EpicRun, issues: list[EpicIssue]) -> list[EpicIssue
|
||||
return issues
|
||||
|
||||
|
||||
def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun, news: str = "") -> Cycle | Stop:
|
||||
"""The cycle loop_run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read. news is what `parked_through` tells the worker of a chunk that was parked, and opens its prompt. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
|
||||
def next_cycle(run: ChunkRun, loop: Loop, in_repo: bool, epic: EpicRun, news: str = "") -> Cycle | Stop:
|
||||
"""The cycle run runs next, from the issue, the premises, the records, the models and the routing ladder read afresh; the stop where one of them cannot be read. news is what `parked_through` tells the worker of a chunk that was parked, and opens its prompt. A chunk in a repo routes on that repo's model and ladder, and one without a repo on the machine setting `repoless-model` and the shipped ladder."""
|
||||
premises = premises_read(loop.root if in_repo else None, epic)
|
||||
if isinstance(premises, Failure):
|
||||
return Stop(1, premises.reason)
|
||||
@@ -413,21 +410,14 @@ def next_cycle(loop_run: LoopRun, loop: Loop, in_repo: bool, epic: EpicRun, news
|
||||
work = chunk_work(loop.window.name, loop.root if in_repo else None, language or epic.config.language, epic)
|
||||
if isinstance(work, Stop):
|
||||
return work
|
||||
task, routing = work
|
||||
models = get_or_ask_models(project_key(loop.root, epic.paths.home), epic.paths, epic.env, chunk_prompt.DRIVER, epic.effects) if in_repo else repoless_models(epic)
|
||||
if isinstance(models, Stop):
|
||||
return models
|
||||
ladder_file = project_ladder_file(loop.root, epic.paths.home) if in_repo else SHIPPED
|
||||
if isinstance(ladder_file, Failure):
|
||||
return Stop(1, ladder_file.reason)
|
||||
ladder = read_ladder(ladder_file)
|
||||
if isinstance(ladder, Failure):
|
||||
return Stop(1, ladder.reason)
|
||||
task, issue = work
|
||||
routing = project_routing(loop, chunk_prompt.DRIVER) if in_repo else routing_on(repoless_models(epic), SHIPPED)
|
||||
if isinstance(routing, Stop):
|
||||
return routing
|
||||
archive_folder = epic.paths.home / ARCHIVE / epic.key
|
||||
project = ladder.project_rung(models.worker)
|
||||
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, loop), lambda: routing)
|
||||
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", epic.effects, stdout=True)
|
||||
return Cycle(config=setup_version(epic.paths.claude_dir, epic.effects.git), effects=epic.effects, facts="", judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=task, prompts=chunk_prompt.unparked(news) if news else chunk_prompt.restating(task) if loop_run.chunk == epic.key else chunk_prompt.WORKER, taken_in=lambda entries: TakenIn(), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
|
||||
route = announced_route(run.cycle_number, run.route, routing, archive_folder, lambda: issue, loop)
|
||||
prompts = chunk_prompt.unparked(news) if news else chunk_prompt.restating(task) if loop.window.name == epic.key else chunk_prompt.WORKER
|
||||
return cycle_of(loop, routing, route, archive_folder, task=task, facts="", premises=premises, records=records, unfinished_evidence=run.unfinished_evidence, prompts=prompts, taken_in=lambda entries: TakenIn())
|
||||
|
||||
|
||||
def chunk_work(key: str, checkout: str | None, language: str, epic: EpicRun) -> tuple[str, RoutingIssue | None] | Stop:
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
"""Which of an epic's issues run as chunks now, each in its own window of the epic's tmux session (docs/decisions.md → `do-epic: drives one Jira epic …`).
|
||||
|
||||
Three limits hold a chunk back: the epic's cap of chunks at once, its repo's `parallel-chunks`, counted across every epic, and a loop-claude that runs chunks in its repo. A parked chunk (`parked`) counts toward none of them: its window only polls the forge, and `resume` gives it a running slot under the same limits once its worker has something to do. do-epic checks a repo and starts a chunk in it while holding the lock loop-claude holds when it starts one (`chunk_windows.starting`), so the two never start chunks in one checkout together."""
|
||||
Three limits hold a chunk back: the epic's cap of chunks at once, its repo's `parallel-chunks`, counted across every epic, and a loop-claude that runs chunks in its repo. A parked chunk (`parked`) counts toward none of them: its window only polls the forge, and `resume` gives it a running slot under the same limits once its worker has something to do. do-epic checks a repo and starts a chunk in it while holding the lock loop-claude holds when it starts one (`chunk_driver.claims.starting`), so the two never start chunks in one checkout together."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from do_epic.epic_record import CAP
|
||||
from loop_claude.loop.chunk_windows import epic_chunks, HANDED, LOCK, project_lock, starting
|
||||
from chunk_driver.claims import CHUNK_FLAG, epic_chunks, HANDED, LOCK, project_lock, starting
|
||||
from do_epic.backend.protocol import EpicIssue
|
||||
from failure import Failure
|
||||
from do_epic.repo_pick import HumanTask, recorded, Repo
|
||||
@@ -21,8 +21,6 @@ from agent_cycle import tmux
|
||||
STATE = ".local/state/do-epic"
|
||||
# Why a human's task with a live window waits, as the epic's status says.
|
||||
HUMAN_WAIT = "it is a human's task; do-epic polls Jira until you close it"
|
||||
# The flag that tells the do-epic in a window its chunk, as `--chunk ABC-34 ABC-12`. Nobody types it, so no panel picked it.
|
||||
CHUNK_FLAG = "--chunk"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
|
||||
@@ -10,7 +10,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
from do_epic.test_main import DESCRIPTION, DoEpicTest
|
||||
from do_epic.epic_record import EpicRecord
|
||||
from failure import Failure
|
||||
from loop_claude.test_loop_claude import FakeEffects
|
||||
from loop_effects.test_doubles import FakeEffects
|
||||
from record_store.held import Held
|
||||
from do_epic import main
|
||||
from do_epic.backend.fake import MemoryEpic
|
||||
|
||||
@@ -9,7 +9,7 @@ from do_epic.test_main import Description, DESCRIPTION, DoEpicTest, MODELS, UNRE
|
||||
from do_epic.config import DoEpicConfig, load
|
||||
from do_epic.epic_record import EpicRecord
|
||||
from failure import Failure
|
||||
from loop_claude.test_loop_claude import FakeEffects, judged, Worker
|
||||
from loop_effects.test_doubles import FakeEffects, judged, Worker
|
||||
from do_epic import findings, finish, main, scheduler
|
||||
from project_config.settings import Finish
|
||||
from do_epic.test_scheduler import issue
|
||||
|
||||
@@ -10,7 +10,7 @@ from project_config.test_project_config import configured
|
||||
from do_epic.test_main import DESCRIPTION, Description, DoEpicTest, MODELS, UNREAD
|
||||
from do_epic.epic_record import EpicRecord
|
||||
from failure import Failure
|
||||
from loop_claude.test_loop_claude import FakeEffects
|
||||
from loop_effects.test_doubles import FakeEffects
|
||||
from do_epic.scheduler import HUMAN_WAIT
|
||||
from do_epic.test_scheduler import issue
|
||||
from do_epic.backend.fake import MemoryEpic
|
||||
|
||||
@@ -12,6 +12,7 @@ import unittest
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from do_epic import chunk_prompt, main
|
||||
from chunk_driver.chunk_run import ChunkRun
|
||||
from project_config.test_project_config import configured
|
||||
from do_epic.config import DoEpicConfig
|
||||
from project_config.settings import Epic, Finish
|
||||
@@ -19,11 +20,10 @@ from do_epic.backend.protocol import EpicIssue
|
||||
from do_epic.epic_record import EpicRecord
|
||||
from agent_cycle import escalation_mark, judge, tmux
|
||||
from failure import Failure
|
||||
from loop_claude.test_loop_claude import FakeEffects, judged, Worker
|
||||
from loop_effects.test_doubles import FakeEffects, judged, Worker
|
||||
from do_epic.test_gates import Forge
|
||||
from do_epic.test_scheduler import issue
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from loop_claude.loop.reload import LoopRun
|
||||
from chunk_driver.this_loop import Loop
|
||||
from do_epic.backend.fake import MemoryEpic
|
||||
from do_epic import parked
|
||||
from loop_state.paths import paths_from
|
||||
@@ -189,7 +189,7 @@ class ACycleOfAChunkWithoutARepo(DoEpicTest):
|
||||
root, repo = place
|
||||
loop = Loop(root=root, session=main.session_named("A ABC-12 do-epic ABC-21", paths.claude_code_transcripts), window=tmux.Window("epic-ABC-12", "ABC-21", "%1"), paths=paths, env=env or {}, launcher=pathlib.Path("/b/do-epic"), effects=fake.effects())
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
found = main.next_cycle(LoopRun(1, root, "", "", chunk="ABC-21"), loop, repo is not None, epic)
|
||||
found = main.next_cycle(ChunkRun(1), loop, repo is not None, epic)
|
||||
assert isinstance(found, main.Cycle), found
|
||||
return found
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from do_epic.test_main import Description, DoEpicTest, structured as says
|
||||
from do_epic.backend.protocol import EpicIssue
|
||||
from do_epic.epic_record import EpicRecord
|
||||
from failure import Failure
|
||||
from loop_claude.test_loop_claude import FakeEffects
|
||||
from loop_effects.test_doubles import FakeEffects
|
||||
from do_epic.test_scheduler import issue
|
||||
from claude_code.reply import Limited
|
||||
from do_epic.config import load
|
||||
|
||||
@@ -6,13 +6,14 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from do_epic.scheduler import Chunk, CHUNK_FLAG, chunks, Epic, HUMAN_WAIT, refusal, RepoRoom, resume, start_chunks, Started, Waits
|
||||
from do_epic.scheduler import Chunk, chunks, Epic, HUMAN_WAIT, refusal, RepoRoom, resume, start_chunks, Started, Waits
|
||||
from chunk_driver.claims import CHUNK_FLAG
|
||||
from do_epic import chunk_prompt
|
||||
from do_epic.backend.protocol import EpicIssue
|
||||
from failure import Failure
|
||||
from do_epic.parked import file_of, park, Parked
|
||||
from do_epic.epic_record import parallel_chunks_in
|
||||
from loop_claude.cycle import prompts
|
||||
from chunk_driver import prompts
|
||||
from model_routing.route import Route
|
||||
from model_routing.ladder import Rung
|
||||
from project_config import settings
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
"""A pull request's review state, read once from its forge: the review decision, whether it is a draft, whether the worker answered the changes a reviewer requested, and the review threads that wait for an answer. chunk-pr and do-epic decide on what this reads, and run no review query of their own."""
|
||||
import json
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
from forge.commands import Forge, gitea, Run
|
||||
|
||||
# What `review_decision` returns in place of GitHub's `CHANGES_REQUESTED` once the worker answered those changes: GitHub keeps its decision until the reviewer reviews again.
|
||||
ANSWERED = "CHANGES_ANSWERED"
|
||||
# What the title of a draft opens with on Gitea, in any case: Gitea's default `WORK_IN_PROGRESS_PREFIXES`.
|
||||
GITEA_DRAFT = ("WIP:", "[WIP]")
|
||||
# Each review thread, with whether it is resolved and who wrote its last comment, beside who opened the pull request. gh fills {owner} and {repo} from the checkout.
|
||||
THREADS = """query($owner: String!, $repo: String!, $number: Int!) { repository(owner: $owner, name: $repo) { pullRequest(number: $number) {
|
||||
author { login }
|
||||
reviewThreads(first: 100) { nodes { isResolved path comments(last: 1) { nodes { author { login } } } } }
|
||||
} } }"""
|
||||
|
||||
|
||||
def review_decision(forge: Forge, pull: str, head: str, run: Run) -> str | None | Failure:
|
||||
"""The review decision on the pull request, whose head commit is head: `answered_decision` on GitHub, `gitea_decision` on Gitea. None where it is a draft."""
|
||||
return answered_decision(pull, head, run) if forge == "gh" else gitea_decision(pull, run)
|
||||
|
||||
|
||||
def github_decision(pull: str, run: Run) -> str | Failure:
|
||||
"""GitHub's review decision on the pull request as it stands, read once: `APPROVED`, `REVIEW_REQUIRED`, `CHANGES_REQUESTED`, or empty where its branch protection asks for no review."""
|
||||
viewed = run(["gh", "pr", "view", pull, "--json", "reviewDecision"])
|
||||
if isinstance(viewed, Failure):
|
||||
return viewed
|
||||
try:
|
||||
decision: str = json.loads(viewed)["reviewDecision"] or ""
|
||||
return decision
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh pr view printed no review decision: {viewed.strip()[:200]}")
|
||||
|
||||
|
||||
def answered_decision(pull: str, head: str, run: Run) -> str | None | Failure:
|
||||
"""GitHub's review decision on the pull request, whose head commit is head, read once: `APPROVED`, `REVIEW_REQUIRED`, `CHANGES_REQUESTED`, or empty where its branch protection asks for no review. ANSWERED in place of `CHANGES_REQUESTED` once the worker answered those changes. None where it is a draft."""
|
||||
viewed = run(["gh", "pr", "view", pull, "--json", "isDraft,reviewDecision,author,reviews,comments"])
|
||||
if isinstance(viewed, Failure):
|
||||
return viewed
|
||||
try:
|
||||
parsed = json.loads(viewed)
|
||||
if parsed["isDraft"]:
|
||||
return None
|
||||
decision = str(parsed["reviewDecision"] or "")
|
||||
return ANSWERED if decision == "CHANGES_REQUESTED" and answered(parsed, head) else decision
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh pr view printed no review decision: {viewed.strip()[:200]}")
|
||||
|
||||
|
||||
def answered(pull: dict[str, typing.Any], head: str) -> bool:
|
||||
"""Whether the worker answered the newest review that requested changes on pull, as `gh pr view --json author,reviews,comments` prints it, whose head commit is head: the head is no longer the commit that review was made on, or the pull request's author commented or replied in a thread after it. GitHub lists a reply in a thread as a review of its own. False where it lists no review that requested changes. Raises KeyError or TypeError where pull has another shape.
|
||||
|
||||
The review names its commit, so no clock tells whether the worker pushed: a commit's date says when it was made, never when it was pushed.
|
||||
"""
|
||||
asked = max((review for review in pull["reviews"] if review["state"] == "CHANGES_REQUESTED"), key=lambda review: review["submittedAt"], default=None)
|
||||
if asked is None:
|
||||
return False
|
||||
author = pull["author"]["login"]
|
||||
# A review the author has yet to submit has no time, and answers nobody.
|
||||
said = [review["submittedAt"] or "" for review in pull["reviews"] if review["author"]["login"] == author] + [comment["createdAt"] for comment in pull["comments"] if comment["author"]["login"] == author]
|
||||
return asked["commit"]["oid"] != head or any(at > asked["submittedAt"] for at in said)
|
||||
|
||||
|
||||
def gitea_decision(pull: str, run: Run) -> str | None | Failure:
|
||||
"""What stands in for `answered_decision` on Gitea, read once: empty, since no review is a gate there, and None where the pull request is a draft, which Gitea tells by its title (GITEA_DRAFT)."""
|
||||
viewed = gitea(run, f"repos/{{owner}}/{{repo}}/pulls/{pull}")
|
||||
if isinstance(viewed, Failure):
|
||||
return viewed
|
||||
try:
|
||||
return None if viewed["title"].upper().startswith(GITEA_DRAFT) else ""
|
||||
except (KeyError, TypeError, AttributeError):
|
||||
return Failure(f"tea api printed no title of pull request {pull}: {str(viewed)[:200]}")
|
||||
|
||||
|
||||
def waiting_threads(number: str, run: Run) -> list[str] | Failure:
|
||||
"""The files where a reviewer's comment waits for an answer on pull request number, read once: those of each review thread nobody resolved, whose last comment someone other than the pull request's author wrote. So a thread the worker answered, as that author, waits for the reviewer again."""
|
||||
read = run(["gh", "api", "graphql", "-f", f"query={THREADS}", "-F", "owner={owner}", "-F", "repo={repo}", "-F", f"number={number}"])
|
||||
if isinstance(read, Failure):
|
||||
return read
|
||||
try:
|
||||
pull = json.loads(read)["data"]["repository"]["pullRequest"]
|
||||
author = pull["author"]["login"]
|
||||
return sorted({thread["path"] for thread in pull["reviewThreads"]["nodes"] if not thread["isResolved"] and any(comment["author"]["login"] != author for comment in thread["comments"]["nodes"])})
|
||||
except (ValueError, KeyError, TypeError):
|
||||
return Failure(f"gh api graphql printed no review threads: {read.strip()[:200]}")
|
||||
@@ -1,17 +1,18 @@
|
||||
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, DRIVER
|
||||
from chunk_driver.prompts import ANSWER_DESTINATIONS
|
||||
from claude_code.entries import Entry
|
||||
from agent_cycle import escalation_mark
|
||||
from failure import Failure
|
||||
from agent_cycle.judge import Grounds, judge_rules, TASK_HEADING, transcript_heading
|
||||
from claude_code.transcript import human_answers
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from agent_cycle.worker_prompts import typed_answer
|
||||
|
||||
|
||||
def grounds_of(prompt: str, entries: list[Entry]) -> Grounds | Failure:
|
||||
"""What the quotes of a judge run on prompt are checked against, as judge_prompt would have made it for the transcript's entries: the premises and the records before the task are quotable, the transcript the judge was shown comes before the rules. The prompt is split on the headings judge_prompt writes, at the first of each; a Failure where it lacks one. Rules of an earlier wording stay part of the transcript, a slack that lets an evidence quote them pass."""
|
||||
before_task, task_heading, after_task = prompt.partition(f"\n\n{TASK_HEADING}")
|
||||
heading = transcript_heading(DRIVER)
|
||||
heading = transcript_heading(LOOP_CLAUDE)
|
||||
_, found_heading, shown = after_task.partition(heading)
|
||||
if not (task_heading and found_heading):
|
||||
return Failure(f"the archived prompt lacks the heading {TASK_HEADING.strip()!r} or {heading.strip()!r}, which judge_prompt writes")
|
||||
return Grounds(shown.removesuffix(f"\n\n{judge_rules(ANSWER_DESTINATIONS, DRIVER)}"), escalation_mark.after_mark(entries, 0), before_task, tuple(human_answers(entries, typed_answer)))
|
||||
return Grounds(shown.removesuffix(f"\n\n{judge_rules(ANSWER_DESTINATIONS, LOOP_CLAUDE)}"), escalation_mark.after_mark(entries, 0), before_task, tuple(human_answers(entries, typed_answer)))
|
||||
|
||||
@@ -53,7 +53,7 @@ import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, DRIVER
|
||||
from chunk_driver.prompts import ANSWER_DESTINATIONS
|
||||
from transcript_archive.location import archive_root, transcript_of
|
||||
from transcript_archive.read import archived_outcomes
|
||||
from rung_replay.archive import archiving, print_report, replay_folder
|
||||
@@ -69,9 +69,10 @@ from judge_replay.archived_prompt import grounds_of
|
||||
from transcript_archive.judge_run import JudgeRun
|
||||
from model_routing.ladder import Ladder, ProjectRung, REVIEW_STEPS, Rung
|
||||
from claude_code.reply import Limited
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from model_routing.ladder_file import read_ladder, SHIPPED
|
||||
from loop_claude.project.premises import read_premises
|
||||
from loop_claude.project.records import read_records, RECORDS_NAMED
|
||||
from chunk_driver.project.premises import read_premises
|
||||
from chunk_driver.project.records import read_records, RECORDS_NAMED
|
||||
from claude_code.transcript import read_transcript
|
||||
from agent_cycle.verdicts import Verdict
|
||||
|
||||
@@ -164,7 +165,7 @@ def rule(case: Case, rung: Rung, home: pathlib.Path, effects: Effects) -> Kind |
|
||||
grounds = grounds_of(kept.prompt, entries)
|
||||
if isinstance(grounds, Failure):
|
||||
return verdict_kind(grounds)
|
||||
verdict: Verdict | Failure | Limited = first_reply(lambda: judge_archived(case.root, kept.prompt, grounds, len(entries), rung, DRIVER, effects))
|
||||
verdict: Verdict | Failure | Limited = first_reply(lambda: judge_archived(case.root, kept.prompt, grounds, len(entries), rung, LOOP_CLAUDE, effects))
|
||||
return verdict if isinstance(verdict, Limited) else verdict_kind(verdict)
|
||||
premises, records = read_premises(case.root, home), read_records(case.root, home, home / ".claude")
|
||||
if isinstance(premises, Failure):
|
||||
@@ -172,5 +173,5 @@ def rule(case: Case, rung: Rung, home: pathlib.Path, effects: Effects) -> Kind |
|
||||
if isinstance(records, Failure):
|
||||
return verdict_kind(records)
|
||||
# 0 for the route's mark, as before any step-up: the replay compares verdict kinds, and a mark changes only which escalation quotes count.
|
||||
verdict = first_reply(lambda: judge_once(case.root, Brief(task=case.task, premises=premises, records=records, records_named=RECORDS_NAMED, answer_destinations=ANSWER_DESTINATIONS, entries=entries, escalated_at=0, driver=DRIVER), rung, effects))
|
||||
verdict = first_reply(lambda: judge_once(case.root, Brief(task=case.task, premises=premises, records=records, records_named=RECORDS_NAMED, answer_destinations=ANSWER_DESTINATIONS, entries=entries, escalated_at=0, driver=LOOP_CLAUDE), rung, effects))
|
||||
return verdict if isinstance(verdict, Limited) else verdict_kind(verdict)
|
||||
|
||||
@@ -9,7 +9,7 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from loop_claude.cycle.prompts import ANSWER_DESTINATIONS, DRIVER
|
||||
from chunk_driver.prompts import ANSWER_DESTINATIONS
|
||||
from transcript_archive.outcomes import ANSWERED, FINISHED, INTERRUPTED, UNFINISHED
|
||||
from claude_code.entries import AssistantTurn, HumanTurn, LoopPrompt, Text
|
||||
from judge_replay.replay import Case, case_from, report, reproduced, sample, verdict_kind
|
||||
@@ -20,6 +20,7 @@ from failure import Failure
|
||||
from agent_cycle.verdicts import Finished, Interrupt, QuotedAnswer, Unfinished
|
||||
from judge_replay.archived_prompt import grounds_of
|
||||
from agent_cycle.judge import judge_archived, judge_rules, Rejected, TASK_HEADING, transcript_heading
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from judge_replay import main
|
||||
from rung_replay.search import Miss, Replay, Tried
|
||||
from model_routing.ladder import Rung
|
||||
@@ -330,7 +331,7 @@ class Main(unittest.TestCase):
|
||||
|
||||
class JudgeArchivedRuns(unittest.TestCase):
|
||||
"""judge_archived on a prompt as judge_prompt writes it: the premises, the task, the transcript the judge was shown, and the rules."""
|
||||
PROMPT = f"HEAD: Ship fast.\n\n{TASK_HEADING}Do it.\n\n{transcript_heading(DRIVER)}Worker chose option B.\n[human] Use Postgres.\n\n{judge_rules(ANSWER_DESTINATIONS, DRIVER)}"
|
||||
PROMPT = f"HEAD: Ship fast.\n\n{TASK_HEADING}Do it.\n\n{transcript_heading(LOOP_CLAUDE)}Worker chose option B.\n[human] Use Postgres.\n\n{judge_rules(ANSWER_DESTINATIONS, LOOP_CLAUDE)}"
|
||||
ENTRIES = [AssistantTurn(parts=(Text("Worker chose option B."),)), HumanTurn(text="Use Postgres."), LoopPrompt(text="Keep going.")]
|
||||
|
||||
def judged(self, prompt=PROMPT, **fields):
|
||||
@@ -344,7 +345,7 @@ class JudgeArchivedRuns(unittest.TestCase):
|
||||
|
||||
effects = Effects(claude=claude, claude_on_terminal=unused, discard_typed=unused, dm=unused, execv=unused, git=unused, input=unused, local_time=unused, memory=unused, now=lambda: 1790000000.0, on_terminal=unused, python=unused, sleep=unused, stream_claude=unused, timer=unused, tmux=unused, tmux_on_terminal=unused)
|
||||
grounds = grounds_of(prompt, self.ENTRIES)
|
||||
return (grounds if isinstance(grounds, Failure) else judge_archived("/code/p", prompt, grounds, len(self.ENTRIES), REFERENCE, DRIVER, effects)), cwds
|
||||
return (grounds if isinstance(grounds, Failure) else judge_archived("/code/p", prompt, grounds, len(self.ENTRIES), REFERENCE, LOOP_CLAUDE, effects)), cwds
|
||||
|
||||
def test_runs_in_the_projects_folder_and_stamps_the_verdict_with_the_entries_read(self):
|
||||
verdict, cwds = self.judged()
|
||||
@@ -363,11 +364,11 @@ class JudgeArchivedRuns(unittest.TestCase):
|
||||
self.assertEqual((type(rejected), rejected.cost), (Rejected, 0.25))
|
||||
|
||||
def test_checks_neither_the_task_nor_the_rules_as_the_transcript_or_the_premises(self):
|
||||
self.assertIsInstance(self.judged(evidence=judge_rules(ANSWER_DESTINATIONS, DRIVER))[0], Rejected)
|
||||
self.assertIsInstance(self.judged(evidence=judge_rules(ANSWER_DESTINATIONS, LOOP_CLAUDE))[0], Rejected)
|
||||
self.assertIsInstance(self.judged(answer="Do it.", question="Which?")[0], Interrupt)
|
||||
|
||||
def test_splits_the_prompt_at_the_first_transcript_heading(self):
|
||||
quoting = self.PROMPT.replace("Worker chose", f"Quoted: {transcript_heading(DRIVER)}Worker chose", 1)
|
||||
quoting = self.PROMPT.replace("Worker chose", f"Quoted: {transcript_heading(LOOP_CLAUDE)}Worker chose", 1)
|
||||
self.assertIsInstance(self.judged(quoting, evidence="Quoted:")[0], Unfinished)
|
||||
|
||||
def test_splits_the_prompt_at_the_first_task_heading(self):
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""loop-claude: headless claude cycles, judged, that interrupt the human only for a decision that is theirs (loop/usage.py's help).
|
||||
|
||||
Its packages hold one concern each: cycle/ words the worker's prompts and reads the items it names, loop/ runs the cycles, and project/ reads a checkout's files. loop-claude's paths live in bin/loop_state/, and its effects in bin/loop_effects/. The cycle itself, its judge, interrupts and routing live in bin/agent_cycle/, which doesn't import this package, so that a second entry point can drive it. routed_item.py, at this level, reads the plan for the item a chunk starts on.
|
||||
Its packages hold one concern each: chunk/ words a fresh chunk's first prompt and reads the items a worker names, and loop/ runs the cycles. What it shares with do-epic lives in bin/chunk_driver/. loop-claude's paths live in bin/loop_state/, and its effects in bin/loop_effects/. The cycle itself, its judge, interrupts and routing live in bin/agent_cycle/, which doesn't import this package, so that a second entry point can drive it. routed_item.py, at this level, reads the plan for the item a chunk starts on.
|
||||
"""
|
||||
|
||||
@@ -7,7 +7,7 @@ from agent_cycle.quotes import plain
|
||||
from plan_store.plans import Plan
|
||||
from agent_cycle.verdicts import TakenIn
|
||||
|
||||
# The label opening the line; prompts.NAME_THE_ITEMS asks the worker for it.
|
||||
# The label opening the line; fresh_prompt.NAME_THE_ITEMS asks the worker for it.
|
||||
CHUNK_ITEMS = "Chunk items:"
|
||||
# A line the worker opens with the label, after any `-` or `>` marker or a backtick; what follows names the items.
|
||||
LINE = re.compile(rf"[\s>`-]*{re.escape(CHUNK_ITEMS)}(.*)")
|
||||
@@ -0,0 +1,52 @@
|
||||
"""What loop-claude alone says to its worker: the first prompt of a chunk, with what chunk-preflight found. The resumed prompt and the rung sentences are chunk_driver.prompts', which do-epic's cycles use too."""
|
||||
import functools
|
||||
|
||||
from loop_claude.chunk.chunk_items import CHUNK_ITEMS
|
||||
from chunk_driver.drivers import LOOP_CLAUDE
|
||||
from model_routing.ladder import ProjectRung
|
||||
from chunk_driver.prompts import READING_RULE, rung_sentences, worker_prompts
|
||||
from model_routing.route import Route
|
||||
|
||||
# loop-claude's name as its workers, its judge and the human read it. A function below that takes a driver words a sentence for any entry point, such as do-epic.
|
||||
DRIVER = LOOP_CLAUDE
|
||||
# chunk_items reads the latest such line; three fresh readers picked the label (docs/decisions.md → `routing: loop-claude reads the items a chunk takes`).
|
||||
NAME_THE_ITEMS = f"Name every plan item this chunk takes on one line, as in `{CHUNK_ITEMS} 297 (L 3, E + I 5), 298 (L 3, E + I 7)`, and write the whole line again whenever the set changes."
|
||||
|
||||
AFTER_CODE_HEALTH = 'which rank after code health (~/.claude/CLAUDE.md → "Shared goal") unless the goals file voids it'
|
||||
|
||||
|
||||
def fresh(driver: str, task: str, premises: str, route: Route, project: ProjectRung | None, facts: str) -> str:
|
||||
"""A chunk's first prompt; facts is preflight_facts' or preflight_failed's text."""
|
||||
routed = f"{routed_by(driver, route, project)} {recount(route)}" if route.item else " ".join(rung_sentences(driver, route, project))
|
||||
opening = f"{routed}\n\n" if routed else ""
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one. {READING_RULE} {NAME_THE_ITEMS}\n\n{opening}{facts}\n\nThe chunk's plan items, or else the task above, set its scope. The goals below, {AFTER_CODE_HEALTH}, choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file, {premises}"
|
||||
|
||||
|
||||
def routed_by(driver: str, route: Route, project: ProjectRung | None) -> str:
|
||||
return " ".join((
|
||||
f"{driver} runs you on {route.rung}, routed by item {route.item}'s L of {route.likelihood}.",
|
||||
*rung_sentences(driver, route, project),
|
||||
))
|
||||
|
||||
|
||||
def recount(route: Route) -> str:
|
||||
"""Has the worker count the routing item's size again once it knows the code, since the L that routed it was scored before anyone read that code."""
|
||||
return f"After your first read of the code, count item {route.item}'s size again: the lines, files and components you will change. Where that count would change its L, have a subagent that scores plan items re-score it; a re-score whose L picks a rung above yours is escalation trigger 1."
|
||||
|
||||
|
||||
def preflight_facts(printed: str) -> str:
|
||||
"""What chunk-preflight printed, for the worker to take as found."""
|
||||
return f"loop-claude ran `~/.claude/bin/chunk-preflight` at the checkout root just before this run. Take what it printed as found rather than finding it again:\n\n{printed}"
|
||||
|
||||
|
||||
def held_by_others(items: list[int]) -> str:
|
||||
"""Keeps a fresh worker off the plan items the project's other chunks hold, running or stopped."""
|
||||
return f"\n\nOther chunks of this project, running or stopped, hold {'items ' if len(items) > 1 else 'item '}{', '.join(map(str, items))}. Leave those items out of this chunk, even where the facts above list them as items that may join it."
|
||||
|
||||
|
||||
def preflight_failed(reason: str) -> str:
|
||||
"""Has the worker find the facts chunk-preflight would have, its worktree among them."""
|
||||
return f"loop-claude could not run `~/.claude/bin/chunk-preflight`, so find the chunk's facts yourself. Why it could not run: {reason}"
|
||||
|
||||
|
||||
WORKER = worker_prompts(DRIVER, functools.partial(fresh, DRIVER), NAME_THE_ITEMS)
|
||||
@@ -1,24 +1,18 @@
|
||||
"""How a cycle starts: the route it runs on, from the archive or the plan, and the facts chunk-preflight finds for a fresh chunk."""
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from git_cli.git_repo import default_ref
|
||||
from release_plan.release_standing import DefaultPlan, project_plan, UNSEEN_RELEASES
|
||||
from loop_effects.effects import Effects, say
|
||||
from failure import Failure
|
||||
from agent_cycle.routing import first_route, kept, Scored
|
||||
from loop_claude.chunk import fresh_prompt
|
||||
from loop_claude.loop.chunk_windows import held_beside, item_of
|
||||
from chunk_preflight.main import ITEM
|
||||
from plan.todo_table import Item
|
||||
from transcript_archive.read import latest_worker
|
||||
from project_config.main import located
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from chunk_driver.this_loop import Loop
|
||||
from plan_store.plans import Plan
|
||||
from model_routing.ladder import ProjectRung, Rung
|
||||
from loop_claude.cycle import prompts
|
||||
from model_routing.route import Route, route_from
|
||||
from loop_claude.routed_item import routed_item
|
||||
from claude_code.session import session_open, transcript_file
|
||||
from claude_code.session import session_open
|
||||
|
||||
# Seconds routing waits for origin to name its default branch, asked only where the checkout records none; past them, the chunk runs on the project's rung.
|
||||
ORIGIN_TIMEOUT = 60
|
||||
@@ -32,30 +26,6 @@ def scored_plan(default: DefaultPlan | Failure, effects: Effects) -> Plan | Fail
|
||||
return plan
|
||||
|
||||
|
||||
def routed(route: Route | None, project: ProjectRung | None, model: str, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Scored | None]) -> Route:
|
||||
"""The route the cycle runs on. project is the project's rung, and model the project's model.
|
||||
|
||||
Where project is None, the model is off the ladder, and the worker runs on it. Otherwise the chunk keeps route, or on a rerun the route its archive last names, as routing.kept allows. Else the plan item routes the chunk afresh.
|
||||
"""
|
||||
if project is None:
|
||||
return Route(Rung(model))
|
||||
route = route or archived()
|
||||
return route if route and kept(route, project) else first_route(project, item())
|
||||
|
||||
|
||||
def archived_route(archive_folder: pathlib.Path, loop: Loop) -> Route | None:
|
||||
"""The route named by the latest verdict archived for the loop's open chunk; None for a chunk not yet judged."""
|
||||
transcript = transcript_file(loop.session)
|
||||
worker = latest_worker(archive_folder, transcript) if transcript else None
|
||||
if isinstance(worker, Failure):
|
||||
say(f"routing the chunk afresh: {worker.reason}", loop.effects)
|
||||
return None
|
||||
route = route_from(worker) if worker else None
|
||||
if worker and route is None:
|
||||
say(f"routing the chunk afresh: the archive's latest Worker: line names no route that can be read: {worker}", loop.effects)
|
||||
return route
|
||||
|
||||
|
||||
def chunk_start(loop: Loop) -> tuple[str, DefaultPlan | Failure]:
|
||||
"""The facts for a fresh worker's prompt, empty where the session is open, and what the default branch holds, read once for the cycle: it routes the chunk, through plan_item, and scores the items the worker names, through scored_plan. It reads the default branch as last fetched: for a fresh chunk, by the preflight that runs first; for an open session, which runs none, by whatever fetched last."""
|
||||
fresh = not session_open(loop.session)
|
||||
@@ -66,9 +36,9 @@ def chunk_facts(loop: Loop) -> str:
|
||||
"""What `chunk-preflight --worktree` printed at the checkout root, for a fresh worker's prompt, or why it could not run. The chunk of a window named for an item starts from that item, and either ends with the items the project's other chunks hold."""
|
||||
item = item_of(loop.window.name)
|
||||
done = loop.effects.python([str(loop.launcher.with_name("chunk-preflight")), "--worktree", *([] if item is None else [ITEM, str(item)])], loop.root)
|
||||
facts = prompts.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}") if done.returncode else prompts.preflight_facts(done.stdout.strip())
|
||||
facts = fresh_prompt.preflight_failed(done.stderr.strip() or f"it exited {done.returncode}") if done.returncode else fresh_prompt.preflight_facts(done.stdout.strip())
|
||||
held = held_beside(loop.window.name, loop.root, loop.paths, loop.effects)
|
||||
return facts + (prompts.held_by_others(held) if held else "")
|
||||
return facts + (fresh_prompt.held_by_others(held) if held else "")
|
||||
|
||||
|
||||
def fetched_plan(loop: Loop, announce: bool = True) -> DefaultPlan | Failure:
|
||||
|
||||
@@ -7,7 +7,8 @@ import re
|
||||
import typing
|
||||
|
||||
from touch_predictions.beside import Beside, chunks_beside
|
||||
from loop_claude.cycle import chunk_items
|
||||
from chunk_driver.claims import CHUNK_FLAG, CLAIMS, epic_chunks, HANDED, LOCK, project_lock, starting
|
||||
from loop_claude.chunk import chunk_items
|
||||
from git_cli.git_repo import default_ref
|
||||
from loop_effects.effects import Effects, say
|
||||
from failure import Failure
|
||||
@@ -22,18 +23,10 @@ from claude_code.transcript import read_transcript
|
||||
from claude_code.session import Session, session_named, transcript_file
|
||||
from agent_cycle import tmux
|
||||
|
||||
# Under the home folder, a folder per project key holds LOCK.
|
||||
STATE = ".local/state/loop-claude"
|
||||
# The file an entry point keeps locked with flock while it lists the windows of a project's chunks and starts new ones. Two loops whose chunks finish together then don't start a chunk twice, and do-epic, which holds the same lock, never starts a chunk in a checkout while loop-claude does.
|
||||
LOCK = "starting"
|
||||
# The flag that tells the loop in a window its chunk, as `--chunk item-372`. Nobody types it, so no panel picked it.
|
||||
CHUNK_FLAG = "--chunk"
|
||||
# The chunk of a plan that names no item for the next chunk: a release to plan, a release review, or a task outside the plan.
|
||||
NO_ITEM = "no-item"
|
||||
# Seconds a fetch of origin may take before a chunk starts; past them, the plan reads as last fetched.
|
||||
FETCH_TIMEOUT = 60
|
||||
# The prefix of the variables loop-claude reads. A window gets the tmux server's environment, not the starting terminal's, so they are handed to each window by name.
|
||||
HANDED = "LOOP_CLAUDE_"
|
||||
|
||||
|
||||
def window_name(item_id: int) -> str:
|
||||
@@ -51,25 +44,6 @@ def chunk_session(key: str, chunk: str, claude_code_transcripts: pathlib.Path) -
|
||||
return session_named(f"A {key} loop-claude {chunk}", claude_code_transcripts)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def starting(folder: pathlib.Path) -> typing.Iterator[None]:
|
||||
"""Holds the LOCK in folder: project_lock's for a project, or do-epic's for an epic. Raises OSError where the file cannot be opened or locked."""
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
with open(folder / LOCK, "a") as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
yield
|
||||
|
||||
|
||||
def project_lock(root: str, home: pathlib.Path) -> pathlib.Path:
|
||||
"""The folder of the LOCK that everything starting a chunk in the checkout root holds."""
|
||||
return home / STATE / project_key(root, home)
|
||||
|
||||
|
||||
def epic_chunks(root: str, run: tmux.Tmux) -> list[tmux.Placed]:
|
||||
"""The windows of do-epic's sessions whose chunks run in the checkout root. A parked chunk doesn't run."""
|
||||
return [placed for placed in tmux.everywhere(run) if placed.session.startswith(f"{tmux.EPIC}-") and placed.root == root and not placed.window.dead and not placed.window.parked]
|
||||
|
||||
|
||||
def held_items(chunks: typing.Iterable[str], key: str, claude_code_transcripts: pathlib.Path) -> set[int]:
|
||||
"""The plan items these chunks hold: the one each started from, and those its worker names on its latest `Chunk items:` line so far. A chunk whose transcript cannot be read holds the item it started from."""
|
||||
held: set[int] = set()
|
||||
@@ -193,5 +167,5 @@ def held_beside(chunk: str, root: str, paths: Paths, effects: Effects) -> list[i
|
||||
|
||||
def wait_for_starts(home: pathlib.Path, key: str) -> None:
|
||||
"""Returns once no loop of the project is starting windows: start_chunks holds LOCK while it starts them. Where the lock cannot be taken, it returns at once."""
|
||||
with contextlib.suppress(OSError), starting(home / STATE / key):
|
||||
with contextlib.suppress(OSError), starting(home / CLAIMS / key):
|
||||
pass
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
"""How a cycle closes: the loop run the next cycle starts from, the step-up an unfinished verdict makes archived, and the stop once a chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
|
||||
from agent_cycle.run import archived_step, archived_stop, Concluded
|
||||
from agent_cycle.verdicts import Finished
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from loop_claude.loop.reload import LoopRun
|
||||
from agent_cycle.stop import needs_human, Stop
|
||||
from agent_cycle.routing import next_route, step_up
|
||||
from model_routing.ladder import ProjectRung
|
||||
from claude_code.session import transcript_file
|
||||
|
||||
# How many unfinished verdicts running end the loop, so a worker and a judge that disagree about a chunk cannot cycle unattended without end. A policy number, not derived from the routing steps: well above the two unfinished verdicts running that archived chunks have reached at most, and low enough that a stuck pair is stopped after six cycles. It counts every unfinished verdict, including those of cycles where the human answered an interrupt: a rerun is cheap, and a chunk that needs that many answers is worth a look. The human confirmed both the number and what it counts.
|
||||
UNFINISHED_LIMIT = 6
|
||||
|
||||
|
||||
def closed_cycle(loop_run: LoopRun, project: ProjectRung | None, concluded: Concluded, archive_folder: pathlib.Path, driver: str, loop: Loop) -> LoopRun | Stop:
|
||||
"""The loop run after a cycle of driver, the entry point, concluded, with the step-up archived; the stop that needs the human, archived, once the chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
after = after_verdict(loop_run, project, concluded)
|
||||
# A finished chunk drops its route, so it has no step-up, and its run of unfinished verdicts restarts.
|
||||
if after.route is None:
|
||||
return after
|
||||
# The unfinished verdict, archived as the cycle closed, names the route its cycle ended on; the step-up names the one the next cycle runs on.
|
||||
path = transcript_file(loop.session)
|
||||
if path and (failure := archived_step(archive_folder, path, after.task, step_up(concluded.route, after.route), loop.effects)):
|
||||
return Stop(1, failure.reason)
|
||||
if limit := reached_limit(after, driver):
|
||||
# Archived with the route the next cycle would run on, so a rerun resumes there.
|
||||
return archived_stop(archive_folder, loop.session, after.task, after.route, limit, limit.reason, loop.effects)
|
||||
return after
|
||||
|
||||
|
||||
def reached_limit(loop_run: LoopRun, driver: str) -> Stop | None:
|
||||
"""The stop that needs the human once the loop run holds UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
if loop_run.unfinished_run < UNFINISHED_LIMIT:
|
||||
return None
|
||||
return needs_human("go on with the chunk", f"the judge ruled it unfinished {loop_run.unfinished_run} cycles running; finish the chunk yourself, or rerun {driver} to give it {UNFINISHED_LIMIT} more cycles")
|
||||
|
||||
|
||||
def after_verdict(loop_run: LoopRun, project: ProjectRung | None, concluded: Concluded) -> LoopRun:
|
||||
"""The loop run the next cycle starts from. A finished chunk drops its route and its run of unfinished verdicts. An unfinished one takes the route routing.next_route makes of the route its cycle ended on, which an interrupt's or a quoted answer's escalation may have raised, and the evidence the judge gave for the verdict; and counts the verdict."""
|
||||
verdict = concluded.verdict
|
||||
if isinstance(verdict, Finished):
|
||||
return dataclasses.replace(loop_run, route=None, unfinished_evidence="", unfinished_run=0)
|
||||
route = next_route(project, concluded.route, verdict)
|
||||
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence, unfinished_run=loop_run.unfinished_run + 1)
|
||||
@@ -2,28 +2,27 @@ import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from chunk_driver.cycle_start import announced_route, cycle_of, project_routing
|
||||
from transcript_archive.location import archive_root, project_archive
|
||||
from loop_claude.loop.chunk_start import archived_route, chunk_start, plan_item, routed, scored_plan
|
||||
from loop_claude.loop.routing_check import check_routing
|
||||
from loop_claude.project.checkout import checkout_root
|
||||
from loop_claude.loop.chunk_windows import CHUNK_FLAG, chunk_session, item_of, start_chunks
|
||||
from loop_claude.cycle import chunk_items, prompts
|
||||
from loop_claude.loop.close import closed_cycle
|
||||
from loop_claude.loop.checkout import checkout_root
|
||||
from chunk_driver.claims import CHUNK_FLAG
|
||||
from loop_claude.chunk import chunk_items, fresh_prompt
|
||||
from loop_claude.loop.chunk_windows import chunk_session, item_of, start_chunks
|
||||
from loop_claude.loop.chunk_start import chunk_start, plan_item, scored_plan
|
||||
from chunk_driver.chunk_run import closed_cycle
|
||||
from agent_cycle.run import Cycle, run_cycle
|
||||
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
|
||||
from loop_effects.effects import Effects, say
|
||||
from failure import Failure
|
||||
from agent_cycle.verdicts import Finished
|
||||
from loop_claude.project.model_choice import get_or_ask_models
|
||||
from git_cli.git_repo import Git
|
||||
from loop_claude.loop.usage import HELP
|
||||
from agent_cycle.routing import judge_rung
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from chunk_driver.this_loop import Loop
|
||||
from loop_state.paths import Paths
|
||||
from transcript_archive.project_key import project_key
|
||||
from model_routing.ladder_file import project_ladder_file, read_ladder, SHIPPED
|
||||
from loop_claude.project.premises import read_premises
|
||||
from loop_claude.project.records import read_records
|
||||
from chunk_driver.project.premises import read_premises
|
||||
from chunk_driver.project.records import read_records
|
||||
from model_routing.ladder_file import SHIPPED
|
||||
from agent_cycle.stop import Stop
|
||||
from agent_cycle.direct_message import tell
|
||||
from agent_cycle import tmux
|
||||
@@ -50,7 +49,7 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
if isinstance(loop_run, Stop):
|
||||
return loop_run
|
||||
# Before anything starts: a worktree, a claude session or a tmux window.
|
||||
if reason := tmux.unusable(effects.tmux, prompts.DRIVER):
|
||||
if reason := tmux.unusable(effects.tmux, fresh_prompt.DRIVER):
|
||||
return Stop(1, reason)
|
||||
if loop_run.chunk is None:
|
||||
return windows_opened(loop_run, env, paths, launcher, effects)
|
||||
@@ -71,7 +70,7 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
concluded = run_cycle(cycle)
|
||||
if isinstance(concluded, Stop):
|
||||
return concluded
|
||||
closed = closed_cycle(loop_run, cycle.project, concluded, cycle.project_archive, prompts.DRIVER, loop)
|
||||
closed = closed_cycle(loop_run, cycle.project, concluded, cycle.project_archive, fresh_prompt.DRIVER, loop)
|
||||
if isinstance(closed, Stop):
|
||||
return closed
|
||||
loop_run = closed
|
||||
@@ -113,25 +112,17 @@ def next_cycle(loop_run: LoopRun, loop: Loop) -> Cycle | Stop:
|
||||
records = read_records(loop.root, loop.paths.home, loop.paths.claude_dir)
|
||||
if isinstance(records, Failure):
|
||||
return Stop(1, records.reason)
|
||||
models = get_or_ask_models(project_key(loop.root, loop.paths.home), loop.paths, loop.env, prompts.DRIVER, loop.effects)
|
||||
if isinstance(models, Stop):
|
||||
return models
|
||||
ladder_file = project_ladder_file(loop.root, loop.paths.home)
|
||||
if isinstance(ladder_file, Failure):
|
||||
return Stop(1, ladder_file.reason)
|
||||
ladder = read_ladder(ladder_file)
|
||||
if isinstance(ladder, Failure):
|
||||
return Stop(1, ladder.reason)
|
||||
routing = project_routing(loop, fresh_prompt.DRIVER)
|
||||
if isinstance(routing, Stop):
|
||||
return routing
|
||||
archive_folder = project_archive(loop.root, loop.paths.home, loop.env)
|
||||
# After the reads above, so a loop that stops on one makes no worktree.
|
||||
facts, default = chunk_start(loop)
|
||||
project = ladder.project_rung(models.worker)
|
||||
route = routed(loop_run.route, project, models.worker, lambda: archived_route(archive_folder, loop), lambda: plan_item(default, item_of(loop.window.name), loop.effects))
|
||||
say(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", loop.effects, stdout=True)
|
||||
if ladder_file != SHIPPED:
|
||||
say(own_ladder(ladder_file, models.worker, project is None), loop.effects)
|
||||
route = announced_route(loop_run.cycle_number, loop_run.route, routing, archive_folder, lambda: plan_item(default, item_of(loop.window.name), loop.effects), loop)
|
||||
if routing.ladder_file != SHIPPED:
|
||||
say(own_ladder(routing.ladder_file, routing.models.worker, routing.project is None), loop.effects)
|
||||
plan = scored_plan(default, loop.effects)
|
||||
return Cycle(config=setup_version(loop.paths.claude_dir, loop.effects.git), effects=loop.effects, facts=facts, judge=judge_rung(ladder, models.judge), premises=premises, project=project, project_archive=archive_folder, records=records, root=loop.root, route=route, session=loop.session, task=loop_run.task, prompts=prompts.WORKER, taken_in=lambda entries: chunk_items.taken_in(entries, plan), unfinished_evidence=loop_run.unfinished_evidence, window=loop.window)
|
||||
return cycle_of(loop, routing, route, archive_folder, task=loop_run.task, facts=facts, premises=premises, records=records, unfinished_evidence=loop_run.unfinished_evidence, prompts=fresh_prompt.WORKER, taken_in=lambda entries: chunk_items.taken_in(entries, plan))
|
||||
|
||||
|
||||
def own_ladder(file: pathlib.Path, model: str, unrouted: bool) -> str:
|
||||
@@ -152,19 +143,3 @@ def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopR
|
||||
if not (loop_run and loop_run.root and loop_run.task):
|
||||
return Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}")
|
||||
return loop_run
|
||||
|
||||
|
||||
def setup_version(claude_dir: pathlib.Path, git: Git) -> str:
|
||||
"""The release and commit of this setup that the worker's claude reads, as `v0.8.0 (<sha>)`; a commit past the latest release reads as git describe gives it, such as `v0.8.0-3-g1a2b3c4 (<sha>)`. Where git can't tell, `unknown` and why."""
|
||||
described = git("describe", "--tags", "--match", "v*", "--always", cwd=str(claude_dir))
|
||||
commit = git("rev-parse", "HEAD", cwd=str(claude_dir))
|
||||
if isinstance(described, Failure):
|
||||
return unknown_version(described)
|
||||
if isinstance(commit, Failure):
|
||||
return unknown_version(commit)
|
||||
return f"{described.strip()} ({commit.strip()})"
|
||||
|
||||
|
||||
def unknown_version(failure: Failure) -> str:
|
||||
# One line, since the archive reads the line after it as the judge's.
|
||||
return f"unknown: {' '.join(failure.reason.split())}"
|
||||
|
||||
@@ -5,7 +5,7 @@ import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from chunk_driver.this_loop import Loop
|
||||
from model_routing.route import Route, route_of
|
||||
from loop_effects.effects import say
|
||||
from agent_cycle.stop import Stop
|
||||
|
||||
@@ -6,7 +6,7 @@ from routing_figures.calls import Call, calls, totals
|
||||
from agent_cycle.hand_over import claude_args, ClaudeCommand, hand_over
|
||||
from loop_effects.effects import Effects, say
|
||||
from failure import Failure
|
||||
from loop_claude.loop.this_loop import Loop
|
||||
from chunk_driver.this_loop import Loop
|
||||
from loop_state.paths import Paths
|
||||
from model_routing.ladder_file import read_ladder, SHIPPED
|
||||
from routing_figures.render import render
|
||||
|
||||
@@ -71,7 +71,7 @@ doesn't write the whole context again.
|
||||
|
||||
As a chunk starts, before it is routed, loop-claude runs bin/chunk-preflight --worktree, adding
|
||||
--item <ID> in a window named for an item, and quotes what it prints in the first prompt
|
||||
(prompts.preflight_facts in bin/loop_claude/cycle/prompts.py); chunk-preflight --help says what it
|
||||
(preflight_facts in bin/loop_claude/chunk/fresh_prompt.py); chunk-preflight --help says what it
|
||||
finds and makes. A preflight that fails, or is still running after 600 s and is stopped, leaves the
|
||||
worker to find the facts itself.
|
||||
|
||||
@@ -124,7 +124,8 @@ worker names on its `Chunk items:` line an item that picks a higher rung, or whe
|
||||
in the transcript an escalation trigger that fired and the worker asked for a higher rung. The
|
||||
rules are in bin/agent_cycle/routing.py and bin/model_routing/ladder.py, the choice of
|
||||
each review's rung in bin/model_routing/review_choice.py, and the prompts' wording in
|
||||
bin/loop_claude/cycle/prompts.py and bin/agent_cycle/worker_prompts.py. Each verdict, interrupt,
|
||||
bin/loop_claude/chunk/fresh_prompt.py, bin/chunk_driver/prompts.py and
|
||||
bin/agent_cycle/worker_prompts.py. Each verdict, interrupt,
|
||||
quoted answer, stop and step-up in the archive ends with a Worker: line, the route as JSON
|
||||
(Route.worker_line in bin/model_routing/route.py); a rerun resumes an open chunk on the latest
|
||||
route named there. A plan or an archive that cannot be read or parsed is said on stderr, and the
|
||||
|
||||
+126
-1083
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,190 @@
|
||||
"""Test doubles for the suites that run cycles: FakeEffects stands in for loop_effects.effects.Effects, Worker and Streamed for a worker's claude, and judged for a judge's reply."""
|
||||
import json
|
||||
import pathlib
|
||||
import subprocess
|
||||
import uuid
|
||||
|
||||
from loop_effects.effects import Effects
|
||||
from claude_code import entries
|
||||
from machine_load import memory
|
||||
|
||||
|
||||
CWD = pathlib.Path("/c")
|
||||
NOW = 1790000000
|
||||
def assistant(*parts):
|
||||
return {"type": "assistant", "message": {"content": list(parts)}}
|
||||
|
||||
|
||||
def human(content):
|
||||
return {"type": "user", "message": {"content": content}}
|
||||
|
||||
|
||||
# The window the loops under test run in.
|
||||
class FakeEffects:
|
||||
"""Effects that record each call and answer from a script, one answer per call."""
|
||||
|
||||
def __init__(self, claude=(), on_terminal_claude=(), git=(), inputs=(), python=(), streams=(), on_terminal=True, free_memory=(), dm_failures=(), tmux_version="tmux 3.2", windows=None, owner="/r"):
|
||||
self.calls = []
|
||||
# Each tmux command run. They stay out of self.calls, as the DMs do. tmux prints `windows` for list-windows, or without them finds no session.
|
||||
self.tmux_calls = []
|
||||
self.tmux_version = tmux_version
|
||||
self.windows = windows
|
||||
# The checkout the session's @loop-root option names.
|
||||
self.owner = owner
|
||||
# Each direct message sent. They stay out of self.calls, so a test that lists every call doesn't list them. Without a script, each is sent.
|
||||
self.dms = []
|
||||
# The directory each DM named, in the order of self.dms; None where it named none.
|
||||
self.dm_directories = []
|
||||
self.dm_failures = list(dm_failures)
|
||||
self.scripts = {"claude": list(claude), "claude_on_terminal": list(on_terminal_claude), "git": list(git), "input": list(inputs), "python": list(python), "stream_claude": list(streams)}
|
||||
# Without a script, every read finds all memory available.
|
||||
self.free_memory = list(free_memory)
|
||||
self.terminal = on_terminal
|
||||
self.worker_environments = []
|
||||
self.worker_folders = []
|
||||
self.terminal_environments = []
|
||||
self.terminal_folders = []
|
||||
# Each timer started, as its delay and action, which a test runs where the timer fires.
|
||||
self.timers = []
|
||||
|
||||
def effects(self):
|
||||
return Effects(
|
||||
claude=lambda args, cwd, stdin: self.answer("claude", (args, stdin)),
|
||||
claude_on_terminal=self.on_terminal_claude,
|
||||
discard_typed=lambda: self.calls.append(("discard_typed", None)),
|
||||
dm=self.dm,
|
||||
execv=self.execv,
|
||||
git=lambda *args, cwd=None, timeout=None, guarded=False, errors="strict": self.answer("git", list(args)),
|
||||
input=lambda prompt: self.answer("input", prompt),
|
||||
local_time=lambda seconds: f"<{seconds}>",
|
||||
memory=self.memory,
|
||||
now=lambda: NOW,
|
||||
on_terminal=lambda: self.terminal,
|
||||
python=lambda args, cwd: self.answer("python", (args, cwd)),
|
||||
sleep=lambda seconds: self.calls.append(("sleep", seconds)),
|
||||
stream_claude=self.stream,
|
||||
timer=self.timer,
|
||||
tmux=self.tmux,
|
||||
tmux_on_terminal=lambda args: self.tmux(args).returncode,
|
||||
)
|
||||
|
||||
def timer(self, delay, action):
|
||||
self.timers.append((delay, action))
|
||||
return lambda: self.calls.append(("stop_timer", delay))
|
||||
|
||||
def tmux(self, args):
|
||||
self.tmux_calls.append(args)
|
||||
if args[0] == "list-windows":
|
||||
return subprocess.CompletedProcess(args, 1 if self.windows is None else 0, self.windows or "", "no server")
|
||||
return subprocess.CompletedProcess(args, 0, {"-V": self.tmux_version + "\n", "show-options": self.owner + "\n"}.get(args[0], "@9\n"), "")
|
||||
|
||||
def dm(self, text, directory):
|
||||
self.dms.append(text)
|
||||
self.dm_directories.append(directory)
|
||||
return self.dm_failures.pop(0) if self.dm_failures else None
|
||||
|
||||
def memory(self):
|
||||
self.calls.append(("memory", None))
|
||||
return self.free_memory.pop(0) if self.free_memory else memory.Memory(available=16, total=16)
|
||||
|
||||
def on_terminal_claude(self, args, cwd, env):
|
||||
self.terminal_environments.append(env)
|
||||
self.terminal_folders.append(cwd)
|
||||
return self.answer("claude_on_terminal", args)
|
||||
|
||||
def stream(self, args, cwd, env):
|
||||
self.worker_environments.append(env)
|
||||
self.worker_folders.append(cwd)
|
||||
return self.answer("stream_claude", args)
|
||||
|
||||
def answer(self, seam, what):
|
||||
self.calls.append((seam, what))
|
||||
answer = self.scripts[seam].pop(0)
|
||||
if isinstance(answer, BaseException):
|
||||
raise answer
|
||||
return answer(what) if callable(answer) else answer
|
||||
|
||||
def execv(self, path, args):
|
||||
self.calls.append(("execv", args))
|
||||
raise OSError(2, "No such file or directory")
|
||||
|
||||
def made(self, seam):
|
||||
return [what for called, what in self.calls if called == seam]
|
||||
|
||||
|
||||
def ran(stdout="", code=0, stderr=""):
|
||||
return subprocess.CompletedProcess([], code, stdout, stderr)
|
||||
|
||||
|
||||
def judged(verdict, evidence="working", answer="", escalation="", release_ready=False, cost=None, reversed=""):
|
||||
reply = {"answer": answer, "escalation": escalation, "evidence": evidence, "finished": verdict == "finished", "interrupt": verdict == "interrupt", "question": "Which one?" if verdict in ("interrupt", "quoted") else "", "answered_before": "", "left_open": "", "release_ready": release_ready, "reversed": reversed}
|
||||
return ran(json.dumps({"structured_output": reply, **({} if cost is None else {"total_cost_usd": cost})}))
|
||||
|
||||
|
||||
class Worker:
|
||||
"""A worker claude: writes a line to the session's transcript, then streams a result."""
|
||||
|
||||
def __init__(self, home, prompted=False, text="working", usage=None, sent=None):
|
||||
self.home = home
|
||||
# Whether it records its -p prompt in the transcript first, as claude does.
|
||||
self.prompted = prompted
|
||||
self.text = text
|
||||
# The usage its message records, twice as Claude Code writes a message of two blocks; none where None.
|
||||
self.usage = usage
|
||||
# The time its message records, as utc() writes it; none where None.
|
||||
self.sent = sent
|
||||
|
||||
def __call__(self, args):
|
||||
flag = "--session-id" if "--session-id" in args else "--resume"
|
||||
path = self.home / "projects" / "-fake" / f"{args[args.index(flag) + 1]}.jsonl"
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "a") as handle:
|
||||
if flag == "--session-id":
|
||||
handle.write(json.dumps({"type": "custom-title"}) + "\n" + json.dumps({"type": "queue-operation", "timestamp": str(uuid.uuid7())}) + "\n")
|
||||
if self.prompted:
|
||||
handle.write(json.dumps({**human(args[args.index("-p") + 1]), "promptSource": "sdk"}) + "\n")
|
||||
said = assistant({"type": "text", "text": self.text})
|
||||
if self.usage is not None:
|
||||
said["message"] |= {"id": "msg-1", "model": "claude-opus-5-5", "usage": self.usage}
|
||||
if self.sent is not None:
|
||||
said["timestamp"] = self.sent
|
||||
handle.write((json.dumps(said) + "\n") * (1 if self.usage is None else 2))
|
||||
return Streamed(0)
|
||||
|
||||
|
||||
class Streamed:
|
||||
"""A started claude. An event of None is an idle stretch of CHECK_EVERY seconds, which advances `clock` by that; `held` are the shells it has, and `stopped` collects the ones stopped."""
|
||||
|
||||
def __init__(self, code, *events, clock=None, held=()):
|
||||
self.code = code
|
||||
self.events = events or ({"type": "result"},)
|
||||
self.clock = clock
|
||||
self.held = list(held)
|
||||
self.stopped = []
|
||||
|
||||
def lines(self, idle):
|
||||
for event in self.events:
|
||||
if event is None:
|
||||
self.clock[0] += idle
|
||||
yield None if event is None else json.dumps(event) + "\n"
|
||||
|
||||
def shells(self):
|
||||
return self.held
|
||||
|
||||
def stop(self, shell):
|
||||
self.stopped.append(shell)
|
||||
|
||||
def wait(self):
|
||||
return self.code
|
||||
|
||||
|
||||
# How a line said at NOW opens, with FakeEffects' local_time.
|
||||
STAMP = f"<{NOW}>"
|
||||
|
||||
|
||||
def kinds(*found):
|
||||
return [entries.entry_from(entry) for entry in found]
|
||||
|
||||
|
||||
def said(*texts):
|
||||
return entries.AssistantTurn(parts=tuple(entries.Text(text) for text in texts))
|
||||
+8
-110
@@ -5,7 +5,6 @@ An item is an open child issue. Its ID is the issue's number, so every child mus
|
||||
`add` files an item as a new child issue, marked with FILED_BY, and ranks it among the issues the loop filed, by the score the config's formula gives, as `placing` says. `finish` moves an item's issue to a done status. `edit` changes an issue the loop filed, and its links to the epic's other open items. Of an issue a person filed it sets only the scores, with a comment that tells the team why."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import re
|
||||
import typing
|
||||
|
||||
from adf_codec.convert import adfs_to_markdown, markdown_to_adf, Refused
|
||||
@@ -13,6 +12,7 @@ from atlassian.client import basic_auth, Http, jira_json, ToAdf, ToMarkdown
|
||||
from plan.todo_table import BLOCKED_BY, Item, item_fields, ItemFields, MOVES, SHAPE
|
||||
from plan_store.markdown_edits import bold_cell, plain, unfit_text
|
||||
from plan_store.plans import cells, no_item, Plan, Written
|
||||
from atlassian.jira import entry_of, finishing_transition, ISSUE_KEY, Link, move_refusal, PERHAPS_FILED, PERHAPS_MOVED, project_of, rank_refusal, searched, unshaped
|
||||
from project_config.settings import Epic, Finish, JIRA_FINISH_STATUS, JIRA_FINISH_TRANSITION
|
||||
from failure import Failure
|
||||
from plan.release import parsed, Release, tagged
|
||||
@@ -20,10 +20,6 @@ from plan import ranking
|
||||
|
||||
# The issue property that holds an item's scores, as a JSON object such as {"L": 2, "E": 2, "I": 3, "S": 3, "Moves": "1.6 by 4", "Exempt": ""}.
|
||||
PROPERTY = "plan-item"
|
||||
ISSUE_KEY = re.compile(r"([A-Z][A-Z0-9_]*)-([0-9]+)")
|
||||
PAGE = 100
|
||||
# The pages one read asks for at most, so an answer that never names a last page ends the read.
|
||||
MAX_PAGES = 100
|
||||
# The key in PROPERTY's value that marks an issue the loop filed, and its value. An edit of an issue without it, which a person filed, writes only PROPERTY and a comment.
|
||||
FILED_BY = ("Filed by", "loop")
|
||||
# The issue type of an issue the loop files, which Jira Software's project templates all have; a project without it refuses the issue, saying so.
|
||||
@@ -31,12 +27,6 @@ ISSUE_TYPE = "Task"
|
||||
# The name of the link type whose inward side reads "is blocked by", as Jira Cloud ships it.
|
||||
BLOCKS = "Blocks"
|
||||
SECURITY = "security"
|
||||
# The key of the status category Jira Cloud gives every done status.
|
||||
DONE = "done"
|
||||
# What a failed create ends with: a request Jira never answered, or answered brokenly, may still have filed the issue.
|
||||
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under {epic} before filing the item again"
|
||||
# What a failed move ends with, for the same reason.
|
||||
PERHAPS_MOVED = "unless Jira refused the request, the issue may have moved: look at {key} before finishing the item again"
|
||||
# What a failed edit ends with, for the same reason.
|
||||
PERHAPS_EDITED = "unless Jira refused the request, the issue may have changed: look at {key} before editing the item again"
|
||||
# The scores a caller sets, L, E, I and S; R is worked out from them.
|
||||
@@ -50,15 +40,6 @@ class ReadsScoring(typing.Protocol):
|
||||
"""The scoring `add` scores an item by: the config's formula and bar, with the goal weights that the goals file, the decision log and the comprehension history give."""
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Link:
|
||||
"""One "is blocked by" link of an issue."""
|
||||
# The key of the issue that blocks.
|
||||
blocker: str
|
||||
# The link's own id, which Jira deletes it by.
|
||||
id: str
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Issue:
|
||||
"""A child issue as Jira answered it, checked to hold what an item is read from."""
|
||||
@@ -142,10 +123,10 @@ class JiraPlan:
|
||||
before = placing(answered, filing.item, value, scoring)
|
||||
created = self.http("POST", f"{self.epic.site}/rest/api/3/issue", authorization, {"fields": issue_fields | {"fixVersions": [{"name": version}]}, "properties": [{"key": PROPERTY, "value": filing.scores}]})
|
||||
if isinstance(created, Failure):
|
||||
return Failure(f"cannot file the item under {self.epic.key}: {created.reason}; {PERHAPS_FILED.format(epic=self.epic.key)}")
|
||||
return Failure(f"cannot file the item under {self.epic.key}: {created.reason}; {PERHAPS_FILED.format(under=self.epic.key)}")
|
||||
key = created.get("key") if isinstance(created, dict) else None
|
||||
if not isinstance(key, str) or not ISSUE_KEY.fullmatch(key) or project_of(key) != project_of(self.epic.key):
|
||||
return Failure(f"Jira answered the new issue without an issue key in {project_of(self.epic.key)}; {PERHAPS_FILED.format(epic=self.epic.key)}")
|
||||
return Failure(f"Jira answered the new issue without an issue key in {project_of(self.epic.key)}; {PERHAPS_FILED.format(under=self.epic.key)}")
|
||||
return Written(number(key), self.linked_and_ranked(key, filing.blockers, before, authorization))
|
||||
|
||||
def linked_and_ranked(self, key: str, blockers: tuple[str, ...], before: str | None | Failure, authorization: str) -> tuple[str, ...]:
|
||||
@@ -395,54 +376,6 @@ def revision_of(issue: Issue, named: dict[str, str], detail: str | None, comment
|
||||
return Revision(body, release, tuple(dict.fromkeys(f"{project_of(issue.key)}-{blocker}" for blocker in checked.blocked_by)) if BLOCKED_BY in named else None)
|
||||
|
||||
|
||||
def offer_of(raw: object) -> tuple[str, str, str, str] | None:
|
||||
"""One transition Jira offers, as its ID, its name, the name of the status it leads to and the key of that status's category; None where raw holds no such transition."""
|
||||
to = raw.get("to") if isinstance(raw, dict) else None
|
||||
category = to.get("statusCategory") if isinstance(to, dict) else None
|
||||
if not isinstance(raw, dict) or not isinstance(to, dict) or not isinstance(category, dict):
|
||||
return None
|
||||
id_, name, status, key = raw.get("id"), raw.get("name"), to.get("name"), category.get("key")
|
||||
return (id_, name, status, key) if isinstance(id_, str) and isinstance(name, str) and isinstance(status, str) and isinstance(key, str) else None
|
||||
|
||||
|
||||
def finishing_transition(site: str, key: str, finish: Finish, authorization: str, http: Http) -> str | Failure:
|
||||
"""The ID of the transition finish names, as Jira offers it now for the issue named key, which is a plan item or an epic; or why none of those offered finishes it."""
|
||||
offered = http("GET", f"{site}/rest/api/3/issue/{key}/transitions", authorization, None)
|
||||
if isinstance(offered, Failure):
|
||||
return Failure(f"cannot read {key}'s transitions: {offered.reason}")
|
||||
return transition_of(offered, key, finish)
|
||||
|
||||
|
||||
def move_refusal(site: str, key: str, transition: str, finish: Finish, authorization: str, http: Http) -> str | None:
|
||||
"""Moves the issue named key by the transition with the ID transition, which finishing_transition answered for finish. Why Jira did not take the move; None where it did."""
|
||||
moved = http("POST", f"{site}/rest/api/3/issue/{key}/transitions", authorization, {"transition": {"id": transition}})
|
||||
return f"cannot move {key} by {finish.transition!r}: {moved.reason}" if isinstance(moved, Failure) else None
|
||||
|
||||
|
||||
def transition_of(offered: object, key: str, finish: Finish) -> str | Failure:
|
||||
"""The ID of the transition finish names, among those Jira offered for the issue named key, which are the ones its present status has; or why none of them finishes the item."""
|
||||
raw = offered.get("transitions") if isinstance(offered, dict) else None
|
||||
found = [offer for each in raw if (offer := offer_of(each))] if isinstance(raw, list) else []
|
||||
if not isinstance(raw, list) or len(found) != len(raw):
|
||||
return Failure(f"Jira answered {key}'s transitions in a shape other than the one it documents")
|
||||
named = [each for each in found if each[1] == finish.transition]
|
||||
if not named:
|
||||
others = " and ".join(f"{name!r} to {status!r}" for _, name, status, _ in found)
|
||||
return Failure(f"{key} has no transition {finish.transition!r} from its status, " + (f"only {others}: move the issue in Jira to a status that has it, or name in `{JIRA_FINISH_TRANSITION}` one that leads to a done status" if found else "nor any other the account may make"))
|
||||
id_, _, status, category = next((each for each in named if each[2] == finish.status), named[0])
|
||||
if status != finish.status:
|
||||
return Failure(f"{key}'s transition {finish.transition!r} leads to the status {status!r}, not to {finish.status!r}, which `{JIRA_FINISH_STATUS}` names")
|
||||
# `children` reads an issue as finished by its status's category, so a move to any other category would leave the item in the plan.
|
||||
if category != DONE:
|
||||
return Failure(f"{key}'s transition {finish.transition!r} leads to the status {status!r}, which Jira doesn't count as done, so the item would stay in the plan: name a transition to a done status")
|
||||
return id_
|
||||
|
||||
|
||||
def project_of(key: str) -> str:
|
||||
"""The key of the Jira project an issue key lies in."""
|
||||
return key.rpartition("-")[0]
|
||||
|
||||
|
||||
def children(epic: Epic, authorization: str, http: Http) -> list[object] | Failure:
|
||||
"""The epic's open child issues in backlog order, which the team ranks, apart from where add ranks an issue it files. An issue in a done status is a finished item, so it is no longer in the plan."""
|
||||
# The search answers no issues for an epic the account cannot see, as for one whose children are all done; asking for the epic itself tells them apart.
|
||||
@@ -452,46 +385,16 @@ def children(epic: Epic, authorization: str, http: Http) -> list[object] | Failu
|
||||
return searched(epic.site, f"{epic.key}'s children", f'parent = "{epic.key}" AND statusCategory != Done ORDER BY Rank ASC', {"fields": ["summary", "description", "fixVersions", "issuelinks"], "properties": [PROPERTY]}, authorization, http)
|
||||
|
||||
|
||||
def rank_refusal(site: str, key: str, before: str, authorization: str, http: Http) -> str | None:
|
||||
"""Ranks the issue named key just before the issue named before; why Jira refused, or None where it didn't. Jira answers a rank it made with no body, and one it refused with an error, or with an entry per issue that holds the errors."""
|
||||
answer = http("PUT", f"{site}/rest/agile/1.0/issue/rank", authorization, {"issues": [key], "rankBeforeIssue": before})
|
||||
if isinstance(answer, Failure):
|
||||
return answer.reason
|
||||
entries = answer.get("entries") if isinstance(answer, dict) else None
|
||||
refused = [entry for entry in entries if isinstance(entry, dict) and entry.get("status") != 200] if isinstance(entries, list) else []
|
||||
return "; ".join(str(error) for entry in refused for error in entry.get("errors") or [f"Jira answered {entry.get('status')}"]) or None
|
||||
|
||||
|
||||
def searched(site: str, what: str, jql: str, asked: dict[str, object], authorization: str, http: Http) -> list[object] | Failure:
|
||||
"""Every issue a JQL search on site finds, page by page. The dict asked holds the request's `fields` and `properties` lists. The text what names the issues sought, in the Failure of a search that doesn't end."""
|
||||
body: dict[str, object] = {"jql": jql, "maxResults": PAGE} | asked
|
||||
found: list[object] = []
|
||||
for _ in range(MAX_PAGES):
|
||||
page = http("POST", f"{site}/rest/api/3/search/jql", authorization, body)
|
||||
if isinstance(page, Failure):
|
||||
return page
|
||||
if not isinstance(page, dict) or not isinstance(page.get("issues"), list):
|
||||
return Failure("the search answered something other than a page of issues")
|
||||
found += page["issues"]
|
||||
if not page.get("nextPageToken"):
|
||||
return found
|
||||
body = body | {"nextPageToken": page["nextPageToken"]}
|
||||
return Failure(f"the search for {what} did not end within {MAX_PAGES} pages")
|
||||
|
||||
|
||||
def issue_of(raw: object) -> Issue | Failure:
|
||||
"""The issue in one entry of the search's answer, or which issue is not in the shape Jira documents, and how."""
|
||||
key = raw.get("key") if isinstance(raw, dict) else None
|
||||
if not isinstance(raw, dict) or not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
|
||||
return Failure("the search answered an issue without an issue key")
|
||||
# The answer's nested fields are read as Jira documents them; one missing or of another type ends the read here, as a Failure, so no later step meets it.
|
||||
entry = entry_of(raw, PROPERTY)
|
||||
if isinstance(entry, Failure):
|
||||
return entry
|
||||
key, fields, links, held = entry.key, entry.fields, entry.links, entry.held
|
||||
try:
|
||||
fields = raw["fields"]
|
||||
links = blocked_by_links(fields)
|
||||
held = raw.get("properties", {}).get(PROPERTY, {})
|
||||
summary, description, versions = fields["summary"].strip(), fields["description"], tuple(version["name"] for version in fields["fixVersions"])
|
||||
except (AttributeError, KeyError, TypeError) as failure:
|
||||
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}")
|
||||
return Failure(unshaped(key, failure))
|
||||
if not isinstance(held, dict):
|
||||
return Failure(f"{key}'s issue property {PROPERTY} is not a JSON object")
|
||||
issue = Issue(key, summary, description, versions, links, held)
|
||||
@@ -500,11 +403,6 @@ def issue_of(raw: object) -> Issue | Failure:
|
||||
return issue
|
||||
|
||||
|
||||
def blocked_by_links(fields: dict[str, typing.Any]) -> tuple[Link, ...]:
|
||||
"""The "is blocked by" links among an issue's `issuelinks` field. A field not in the shape Jira documents raises a KeyError or TypeError, which the caller turns into a Failure."""
|
||||
return tuple(Link(link["inwardIssue"]["key"], link["id"]) for link in fields["issuelinks"] if "inwardIssue" in link and link["type"]["inward"] == "is blocked by")
|
||||
|
||||
|
||||
def keys_of(answered: list[object]) -> set[object]:
|
||||
"""The key of each issue in the search's answer."""
|
||||
return {raw.get("key") for raw in answered if isinstance(raw, dict)}
|
||||
|
||||
@@ -171,7 +171,7 @@ class JiraPlan(unittest.TestCase):
|
||||
|
||||
def test_a_search_that_never_names_a_last_page_fails_the_read(self):
|
||||
self.pages = [{"issues": [], "nextPageToken": "more"}] * 2
|
||||
with unittest.mock.patch.object(jira_epic, "MAX_PAGES", 2):
|
||||
with unittest.mock.patch("atlassian.jira.MAX_PAGES", 2):
|
||||
self.assertEqual(self.store().read(), Failure("the search for ABC-12's children did not end within 2 pages"))
|
||||
|
||||
def test_an_issue_in_another_shape_fails_the_read_naming_it(self):
|
||||
@@ -224,7 +224,7 @@ class JiraPlan(unittest.TestCase):
|
||||
ISSUES = "https://acme.atlassian.net/rest/api/3/issue"
|
||||
LINKS = "https://acme.atlassian.net/rest/api/3/issueLink"
|
||||
VERSIONS = "https://acme.atlassian.net/rest/api/3/project/ABC/versions"
|
||||
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under ABC-12 before filing the item again"
|
||||
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under ABC-12 before filing it again"
|
||||
FIELDS = {"Release": "0.1.0", "Tagline": "Count the runs.", "L": "1", "E": "2", "I": "3", "S": "1", "Moves": "1 by 9"}
|
||||
PROPERTY = {"L": 1, "E": 2, "I": 3, "S": 1, "Moves": "1 by 9", "Exempt": "", "Filed by": "loop"}
|
||||
RANK = "https://acme.atlassian.net/rest/agile/1.0/issue/rank"
|
||||
|
||||
@@ -6,7 +6,7 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from loop_claude.project.records import HUMAN_RULE_DIR
|
||||
from chunk_driver.project.records import HUMAN_RULE_DIR
|
||||
from repo_checks import ROOT
|
||||
|
||||
# The review standards this repo ships. A project's config may name another folder as `standards`.
|
||||
|
||||
@@ -24,6 +24,21 @@ def package_imports(bin_folder: pathlib.Path) -> dict[str, set[str]]:
|
||||
return imported
|
||||
|
||||
|
||||
# The entry points that drive chunks. What two of them need lives in bin/chunk_driver/, so no other package imports one, its tests included.
|
||||
DRIVERS = ("do_epic", "loop_claude")
|
||||
|
||||
|
||||
def driver_imports(bin_folder: pathlib.Path, drivers: tuple[str, ...]) -> list[str]:
|
||||
"""Each module under bin_folder that imports a driver's package from outside it, tests included, as `<module> imports <driver>`."""
|
||||
found = []
|
||||
for module in sorted(bin_folder.rglob("*.py")):
|
||||
package = module.relative_to(bin_folder).parts[0]
|
||||
for node in ast.walk(ast.parse(module.read_text(encoding="utf-8"))):
|
||||
names = [alias.name for alias in node.names] if isinstance(node, ast.Import) else [node.module or ""] if isinstance(node, ast.ImportFrom) and node.level == 0 else []
|
||||
found += [f"{module.relative_to(bin_folder)} imports {top}" for name in names if (top := name.split(".")[0]) in drivers and top != package]
|
||||
return sorted(set(found))
|
||||
|
||||
|
||||
def cycle(imported: dict[str, set[str]]) -> list[str]:
|
||||
"""One cycle among the packages, as the packages along it; empty where there is none."""
|
||||
try:
|
||||
@@ -37,6 +52,17 @@ class PackageCycles(unittest.TestCase):
|
||||
def test_no_package_under_bin_imports_one_that_imports_it_back(self):
|
||||
self.assertEqual(cycle(package_imports(ROOT / "bin")), [], "move what both packages need into a package that imports neither")
|
||||
|
||||
def test_no_package_imports_a_driver_of_chunks(self):
|
||||
self.assertEqual(driver_imports(ROOT / "bin", DRIVERS), [], "move what the importer needs into bin/chunk_driver/, or a test double into the package whose code it stands in for")
|
||||
|
||||
def test_a_driver_imported_from_another_package_is_found_in_tests_too(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
folder = pathlib.Path(temp)
|
||||
for package, name, source in (("a", "mod.py", "import a.other\n"), ("b", "test_mod.py", "from a.mod import x\n"), ("b", "mod.py", "def f():\n\timport a\n"), ("c", "mod.py", "import b\n")):
|
||||
(folder / package).mkdir(exist_ok=True)
|
||||
(folder / package / name).write_text(source)
|
||||
self.assertEqual(driver_imports(folder, ("a",)), ["b/mod.py imports a", "b/test_mod.py imports a"])
|
||||
|
||||
def test_reads_an_import_deferred_into_a_function_and_leaves_out_tests(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
folder = pathlib.Path(temp)
|
||||
|
||||
@@ -7,7 +7,7 @@ from failure import Failure
|
||||
from model_routing.ladder import Ladder, Rung
|
||||
from claude_code.reply import Limited
|
||||
from loop_state.paths import paths_from
|
||||
from loop_claude.project.model_choice import read_models_file
|
||||
from chunk_driver.project.model_choice import read_models_file
|
||||
|
||||
# What a tool reads from its arguments, such as the project's rung of the model they name.
|
||||
Parsed = typing.TypeVar("Parsed")
|
||||
|
||||
+4
-4
@@ -249,7 +249,7 @@ loop-claude hands each worker's claude `CLAUDE_CODE_AUTO_COMPACT_WINDOW=200000`
|
||||
|
||||
Why 200k. Item 231 asked for the best means to lower what a chunk spends on context, in dollars, then hours, with no loss of quality. Measured from loop-claude's archive, the 90 chunks with messages after the routing figures' reset of 2026-10-21 cost $1,215 at the list prices read on 2026-10-26 (Opus 5.5: $4 per million input tokens, $5 and $8 for 5-minute and 1-hour cache writes, $0.20 for cache reads, $20 for output; no model but Haiku 5.5 charges more above 200k). That is 0.86 of what the chunks' `cost-state` entries sum to, since some chunks' subagent transcripts are not in the archive. Worker main threads cost $375 (31%): 1-hour cache writes $149, cache reads $153, output $74. Subagents cost $840: 5-minute cache writes $379, output $274, cache reads $187, with the principles auditors at $274 over 342 runs, the stability reviewers $179 and general-purpose subagents $192. A worker starts at 41k tokens of context and peaks at a median 167k, one in ten past 358k, the largest at 848k. The worker's messages above 200k were 32% of its messages and 55% of its cost.
|
||||
|
||||
What reads cost, and whether a tool cuts them, measured at lilleman's ask after those figures: no tool cut a reviewer's reading without losing a finding, except a scout, which kept the stability reviewers' findings but lost the auditors' 🔴 and slowed every round; so none of them ships here. lilleman chose on 2026-10-27: workers trial a reading rule, reviewers get no scout, and the principles audit keeps reading the project whole. Every fresh worker prompt now holds that rule (`READING_RULE` in `bin/loop_claude/cycle/prompts.py`), and `todo.md` item 260 keeps or drops it. Two measures of reading: the characters a thread read, 93M in all, about 27M tokens; and the new input tokens of the messages that took results in, charged to those results, which also count what arrives beside a result, such as the agent's own previous reply. By the second measure tool results caused $620 of the $1,215, and reads (Read, Bash reads, Grep, Glob and git's diff, show and log) $537 of that: the worker's reads $140 (Bash reads $115, Read $12) of its $190 of results, the principles auditors' $133 (Read $109, Grep $22) of $139, the stability reviewers' $97 (Bash reads $79) of $106, general-purpose subagents' $84 of $97, the prose reviewers' $32, the architects' $30 and the product-owner reviewers' $21. By the first measure the worker's reads are about $45. Precision is low everywhere: of the characters a principles auditor read, 13% were in a file its report names; a stability reviewer 8%; a prose reviewer 18%; an architect 11%; the worker 4% in a file it later edited. Reading is mostly to find things and to learn what a project holds. The worker's `sed` ranges are small, median 25 lines, one in ten over 80. Of the worker's read characters, 49% were of a file it had read earlier in the thread: 36% within three messages, paging through it, and 13% later, with the file unchanged, about $6 of the $45; a reviewer's later reads of an unchanged file it had read are 1 to 2%. rule-hook's `repeat-read` now blocks a read of lines already in context.
|
||||
What reads cost, and whether a tool cuts them, measured at lilleman's ask after those figures: no tool cut a reviewer's reading without losing a finding, except a scout, which kept the stability reviewers' findings but lost the auditors' 🔴 and slowed every round; so none of them ships here. lilleman chose on 2026-10-27: workers trial a reading rule, reviewers get no scout, and the principles audit keeps reading the project whole. Every fresh worker prompt now holds that rule (`READING_RULE` in `bin/chunk_driver/prompts.py`), and `todo.md` item 260 keeps or drops it. Two measures of reading: the characters a thread read, 93M in all, about 27M tokens; and the new input tokens of the messages that took results in, charged to those results, which also count what arrives beside a result, such as the agent's own previous reply. By the second measure tool results caused $620 of the $1,215, and reads (Read, Bash reads, Grep, Glob and git's diff, show and log) $537 of that: the worker's reads $140 (Bash reads $115, Read $12) of its $190 of results, the principles auditors' $133 (Read $109, Grep $22) of $139, the stability reviewers' $97 (Bash reads $79) of $106, general-purpose subagents' $84 of $97, the prose reviewers' $32, the architects' $30 and the product-owner reviewers' $21. By the first measure the worker's reads are about $45. Precision is low everywhere: of the characters a principles auditor read, 13% were in a file its report names; a stability reviewer 8%; a prose reviewer 18%; an architect 11%; the worker 4% in a file it later edited. Reading is mostly to find things and to learn what a project holds. The worker's `sed` ranges are small, median 25 lines, one in ten over 80. Of the worker's read characters, 49% were of a file it had read earlier in the thread: 36% within three messages, paging through it, and 13% later, with the file unchanged, about $6 of the $45; a reviewer's later reads of an unchanged file it had read are 1 to 2%. rule-hook's `repeat-read` now blocks a read of lines already in context.
|
||||
|
||||
The trials: four stability rounds and five principles rounds archived on 2026-10-23, each holding 🔴 or 🟡 findings, replayed on the project's rung as `review-replay` replays them, with the reviewer's definition changed through `--agents`; and three worker-like tasks, a search, a rename and an explanation, run on fresh clones at the project's rung. The baseline, the reviewer as it is, found 4 of its own 6 archived findings in each pass, so a variant that finds 4 or 5 of 6 does what the same reviewer does twice. The options todo item 231 named, each simulated on the 90 chunks and, where lilleman asked, tried:
|
||||
|
||||
@@ -435,7 +435,7 @@ Messages keep each file's usual name, such as `todo.md`, as its label, since the
|
||||
|
||||
2026-10-15, lilleman asked for it through todo item 144, implemented by claude. Serves goal 1.1: a cheap reviewer that finds the work below standard can have a more capable one check that ruling, and work below standard gets a more capable rung to fix it, unless the fix changes only prose.
|
||||
|
||||
The work is below standard where systems-architect returns BLOCK, a reviewer reports a finding against the review standards' `quality.md`, or `record-ruling` counts a comprehension seat's vote in a paired ruling as worse. That reviewer or seat may ask, and the worker then relaunches it on the rung loop-claude's prompt names, one above the rung it ran on. `bin/loop_claude/cycle/prompts.py`'s `reruns` names it: the seats run on the project's rung, and the reviewers on the highest of the worker's rung and the one their item's E + I picks. Off the ladder, or for a reviewer or seat already on the top rung, the prompt names none, and its report stands.
|
||||
The work is below standard where systems-architect returns BLOCK, a reviewer reports a finding against the review standards' `quality.md`, or `record-ruling` counts a comprehension seat's vote in a paired ruling as worse. That reviewer or seat may ask, and the worker then relaunches it on the rung loop-claude's prompt names, one above the rung it ran on. `bin/chunk_driver/prompts.py`'s `reruns` names it: the seats run on the project's rung, and the reviewers on the highest of the worker's rung and the one their item's E + I picks. Off the ladder, or for a reviewer or seat already on the top rung, the prompt names none, and its report stands.
|
||||
|
||||
A reviewer's relaunch gets the report that asked and confirms or dismisses each of its findings with a reason, so no finding goes unanswered; a seat's relaunch reads afresh, since a seat reports no findings. The relaunch's report stands in place of the one that asked, in the same round, so it adds no round, and a relaunch cannot ask again. So the round cap still bounds a pass, and one round launches a reviewer at most twice. The ruling that stands is the fourth escalation trigger in `CLAUDE.md`, except where the fix changes only prose: the worker fixes that on its own rung. lilleman ruled that on 2026-11-08 through todo item 276, after PR #432's two quality findings were false claims, one in a docstring and one in a sentence of this file. lilleman also ruled that no ruling raises the judge's rung, which `routing: on a model judge-replay covered` sets.
|
||||
|
||||
@@ -638,7 +638,7 @@ Since 2026-10-16 the reviewers may run above the worker's rung, on the one the h
|
||||
|
||||
## routing: loop-claude reads the items a chunk takes from the worker's latest `Chunk items:` line and scores them by `todo.md` as the default branch held it when the cycle started; the judge reports the escalation and whether a release is ready to cut
|
||||
|
||||
2026-11-11, claude. Serves goal 1.4 and code health. The judge's reply held three facts beside its verdict: the items the worker took into the chunk, with their highest L and E + I; whether a release was ready to cut; and any escalation the worker asked for. Routing lifts the worker's and the reviewers' rungs by the items' levels. Code now reads the items: each chunk's first prompt, and each prompt resuming it, asks the worker for a line opening `Chunk items:`, `bin/loop_claude/cycle/chunk_items.py` reads the latest such line in the worker's own text, and `plan.todo_table` scores the items. So the judge's reply is smaller, and every L and E + I that routing reads comes from `todo.md`, through one module.
|
||||
2026-11-11, claude. Serves goal 1.4 and code health. The judge's reply held three facts beside its verdict: the items the worker took into the chunk, with their highest L and E + I; whether a release was ready to cut; and any escalation the worker asked for. Routing lifts the worker's and the reviewers' rungs by the items' levels. Code now reads the items: each chunk's first prompt, and each prompt resuming it, asks the worker for a line opening `Chunk items:`, `bin/loop_claude/chunk/chunk_items.py` reads the latest such line in the worker's own text, and `plan.todo_table` scores the items. So the judge's reply is smaller, and every L and E + I that routing reads comes from `todo.md`, through one module.
|
||||
|
||||
The judge field trials (`judge-field-trials/`) read each fact by code over the 709 archived cases. Code was wrong in each of the 39 cases where it and the judge disagreed on an escalation, and read a release ready to cut in 3 of the 31 interrupts that asked about a cut, so the judge keeps both. Workers stated an item with its L and E + I in 413 cases, but also stated items they filed, finished or only mentioned, so the worker writes the items on a line that holds only them, which code reads. Three fresh readers picked its label `Chunk items:`, 3 to 0, because it says the items are the chunk's.
|
||||
|
||||
@@ -864,7 +864,7 @@ Valid while loop-claude starts a project's chunks side by side by this pick (`bi
|
||||
|
||||
2026-10-15, lilleman. Serves goal 1.1: a cheaper rung writes worse scores, and a wrong score routes a chunk onto the wrong rung, ranks the plan wrong and drops an item under the bar that should stay. Below the project's rung, a session or subagent may not score an item, move it between releases, split a milestone, delete an item under the bar, or rewrite a filed item. It drafts each new item, and each change to a filed item, such as a refresh a rule asks for. Before the next commit, one subagent on the project's rung or above applies and scores the drafts. One launch covers all of a chunk's drafts (goal 1.4). A security finding gets that subagent at once, since it is filed the moment it is found. Removing a finished item closes work and plans none, so any rung does it. Scoring a delegated task's L runs on any rung too: a launch on the project's rung to score each delegation would cost more than the cheaper task saves (goal 1.4).
|
||||
|
||||
Three places carry the rule. `rules/todo-items.md` states it; it loads with `todo.md`, and `CLAUDE.md` → "Topic rules" asks for it before an item is filed or changed. `CLAUDE.md`'s delegation rule launches the scoring subagent on the rung `bin/subagent-rung --plan` prints: the one the task's L picks, never below the project's, so a hard scoring task still climbs. `bin/loop_claude/cycle/prompts.py`'s `review_launches` names the same command to a worker below the project's rung, never a fixed rung, which would disagree with `--plan` on a scoring task of L 4. Valid while a worker or subagent may run below the project's rung.
|
||||
Three places carry the rule. `rules/todo-items.md` states it; it loads with `todo.md`, and `CLAUDE.md` → "Topic rules" asks for it before an item is filed or changed. `CLAUDE.md`'s delegation rule launches the scoring subagent on the rung `bin/subagent-rung --plan` prints: the one the task's L picks, never below the project's, so a hard scoring task still climbs. `bin/chunk_driver/prompts.py`'s `review_launches` names the same command to a worker below the project's rung, never a fixed rung, which would disagree with `--plan` on a scoring task of L 4. Valid while a worker or subagent may run below the project's rung.
|
||||
|
||||
## todo.md: tools and agents read and change the plan as it stands only through `bin/plan_store` and `todo-item`; a tool that reads what a past commit held, ranks the file or checks its rank still parses `todo.md`
|
||||
|
||||
|
||||
@@ -179,10 +179,11 @@ Touches:
|
||||
|
||||
Touches:
|
||||
- `bin/agent_cycle/judge.py`: `Brief`, `judge_prompt`, `JUDGE_PROMPT_CHARS`
|
||||
- `bin/loop_claude/project/records.py`: `read_records`
|
||||
- `bin/chunk_driver/project/records.py`: `read_records`
|
||||
- `bin/judge_replay/main.py`: `rule` (small)
|
||||
- `bin/judge_replay/replay.py`: `case_from` (small)
|
||||
- `bin/loop_claude/test_loop_claude.py`: `JudgePrompt`, `Records`
|
||||
- `bin/loop_claude/test_loop_claude.py`: `JudgePrompt`
|
||||
- `bin/chunk_driver/project/test_records.py`: `Records`
|
||||
- `bin/judge_replay/test_judge_replay.py`
|
||||
|
||||
### 395. Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.
|
||||
@@ -365,7 +366,7 @@ The maintainability senior and the inherited architect of PR #257's scoring run
|
||||
|
||||
### 443. Measure whether compacting a waiting worker's session cut the cost of 20 chunks with an interrupt, with quality holding.
|
||||
|
||||
The check lilleman set for item 310 on 2026-11-11, which project principle 1 asks for before a cost cut stands: compare the cost entries of 20 archived chunks with an interrupt from before the compaction shipped against 20 from after it. Measure the tokens written to the cache by each resume after an interrupt, and the tokens per finished chunk, and check that review rounds and unfinished verdicts per chunk hold. Where the saving doesn't show, or quality drops, file an item to remove the compaction or retune `COMPACT_FLOOR` and `CACHE_MARGIN` in `bin/loop_claude/cycle/interrupt.py`. Record the outcome as a decision. Start it once 20 chunks with an interrupt have run on 0.12.0 or later.
|
||||
The check lilleman set for item 310 on 2026-11-11, which project principle 1 asks for before a cost cut stands: compare the cost entries of 20 archived chunks with an interrupt from before the compaction shipped against 20 from after it. Measure the tokens written to the cache by each resume after an interrupt, and the tokens per finished chunk, and check that review rounds and unfinished verdicts per chunk hold. Where the saving doesn't show, or quality drops, file an item to remove the compaction or retune `COMPACT_FLOOR` and `CACHE_MARGIN` in `bin/agent_cycle/interrupt.py`. Record the outcome as a decision. Start it once 20 chunks with an interrupt have run on 0.12.0 or later.
|
||||
|
||||
### 5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
|
||||
|
||||
@@ -391,11 +392,11 @@ Project principle 4 asks that a rule a program can check is enforced by a progra
|
||||
|
||||
### 260. Compare the first 20 chunks whose worker got the reading rule with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and drop the rule where any rose.
|
||||
|
||||
Start once 20 chunks have run under the rule. The rule is `READING_RULE` in `bin/loop_claude/cycle/prompts.py`, which every fresh worker prompt holds; `git log --first-parent -S READING_RULE` on the default branch finds the merge that added it. A chunk got it when its first worker started after that merge. Measure as `docs/decisions.md` → `loop-claude: the worker's claude compacts its context before it fills a 200k window…` measured the window: rounds past each pass's first and the 🔴, 🟡 and misread counts from the round commit titles, unfinished verdicts from the outcomes files, hours from the worker's first message to its last, and dollars from the chunk's `cost-state` entry. Compare means, since the decision weighs the rule by all spend.
|
||||
Start once 20 chunks have run under the rule. The rule is `READING_RULE` in `bin/chunk_driver/prompts.py`, which every fresh worker prompt holds; `git log --first-parent -S READING_RULE` on the default branch finds the merge that added it. A chunk got it when its first worker started after that merge. Measure as `docs/decisions.md` → `loop-claude: the worker's claude compacts its context before it fills a 200k window…` measured the window: rounds past each pass's first and the 🔴, 🟡 and misread counts from the round commit titles, unfinished verdicts from the outcomes files, hours from the worker's first message to its last, and dollars from the chunk's `cost-state` entry. Compare means, since the decision weighs the rule by all spend.
|
||||
|
||||
### 51. Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints.
|
||||
|
||||
"Functional core, imperative shell", in `principles/technical-principles.md`: `say` in `bin/loop_effects/effects.py` takes the effects but prints to stderr itself, and `retried` and the rest print through `say`. Two bare `print(file=sys.stderr)` calls remain outside them, in `bin/agent_cycle/interrupt.py` and `bin/loop_claude/project/model_choice.py`. The tests swap the global stream.
|
||||
"Functional core, imperative shell", in `principles/technical-principles.md`: `say` in `bin/loop_effects/effects.py` takes the effects but prints to stderr itself, and `retried` and the rest print through `say`. Two bare `print(file=sys.stderr)` calls remain outside them, in `bin/agent_cycle/interrupt.py` and `bin/chunk_driver/project/model_choice.py`. The tests swap the global stream.
|
||||
|
||||
### 201. Save the human's tmux setups and restore them after a reboot, so work is up and running again quickly.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user