Merge pull request 'Stop loop-claude and ask the human after six unfinished verdicts running' (#371) from item-59 into main
This commit was merged in pull request #371.
This commit is contained in:
@@ -2,6 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- `loop-claude` now stops with exit 2 when the judge rules one chunk unfinished 6 cycles running, so a worker and a judge that disagree can no longer cycle unattended without end. Rerunning resumes the chunk and gives it 6 more cycles.
|
||||
- larv-review now launches a round's reviewers in up to three waves, each wave's reviewers in one message, in place of one reviewer at a time: first the architect, then the principles auditors with the security and overrides passes, then the stability, product-owner and prose passes. Each wave now lasts as long as its slowest reviewer, where a round used to last as long as all its reviewers one after another.
|
||||
- loop-claude's worker, and every subagent it launches, now run with a 200k auto-compact window, handed as `CLAUDE_CODE_AUTO_COMPACT_WINDOW`: Claude Code compacts their context at about 170k tokens, where it used to let it grow to about 967k. loop-claude's judge no longer reads a compaction summary as words you typed. `CLAUDE.md` has a new section, "Compact instructions", saying what a summary keeps.
|
||||
- On Linux, `run-heavy` now caps the memory of a container its command makes with `docker run` or `docker create`. A container that sets no limit gets a memory limit and a swap limit, both the slot's share: three quarters of the machine's memory, split across the heavy slots. A container that sets one of the two limits keeps it, and gets the other as large. This covers containers a project's own scripts start, which rule-hook can't see. A `docker compose` service, and a docker named by its full path, get no cap. rule-hook's `docker-run-memory` now lets a `docker run` under run-heavy pass.
|
||||
|
||||
@@ -76,9 +76,13 @@ def ruling_step_up(cycle: Cycle, verdict: Verdict) -> StepUp | None:
|
||||
|
||||
|
||||
def stopped(cycle: Cycle, stop: Stop, why: str) -> Stop:
|
||||
path = transcript_file(cycle.session)
|
||||
failure = archive(cycle.project_archive, path, cycle.task, outcomes.stopped(cycle.route.worker_line(), why), cycle.effects.now()) if path else None
|
||||
# A failed archive exits 1 whatever the stop; when the stop was this archive failing, its reason is said once.
|
||||
return archived_stop(cycle.project_archive, cycle.session, cycle.task, cycle.route, stop, why, cycle.effects)
|
||||
|
||||
|
||||
def archived_stop(project_archive: pathlib.Path, session: Session, task: str, route: Route, stop: Stop, why: str, effects: Effects) -> Stop:
|
||||
"""stop, once archived with the route the worker holds. A failed archive exits 1 whatever the stop; when the stop was this archive failing, its reason is said once."""
|
||||
path = transcript_file(session)
|
||||
failure = archive(project_archive, path, task, outcomes.stopped(route.worker_line(), why), effects.now()) if path else None
|
||||
return Stop(1, f"{failure.reason}\n{stop.reason}".rstrip()) if failure and failure.reason != stop.reason else stop
|
||||
|
||||
|
||||
|
||||
@@ -106,11 +106,12 @@ Exit 1, saying why, when:
|
||||
more than 2 minutes past. The judge's reply gives its reset time only as text, so the judge is
|
||||
always retried every 10 minutes. stderr says when the next try is due.
|
||||
|
||||
Exit 2, when the human is needed and cannot answer:
|
||||
Exit 2, when the human is needed to go on:
|
||||
- stdin or stdout is not a terminal, when a model must be picked or an interrupt answered
|
||||
- the input ends while a model is picked
|
||||
- an interrupt's answer is empty, or the input ends before one is typed
|
||||
- the claude opened for /talk, or for a release ready to cut, ends with nothing typed
|
||||
- the judge rules a chunk unfinished 6 cycles running; rerun to give it 6 more
|
||||
Rerunning resumes any open session.
|
||||
|
||||
"""
|
||||
@@ -122,9 +123,9 @@ import typing
|
||||
from loop_claude.routing import after_unfinished, first_route, kept, routed_item
|
||||
from transcript_archive.write import archive
|
||||
from transcript_archive.location import archive_root, project_archive
|
||||
from loop_claude.cycle.run import archived_stop, Concluded, Cycle, run_cycle
|
||||
from loop_claude.loop.routing_check import check_routing
|
||||
from loop_claude.project.checkout import checkout_root
|
||||
from loop_claude.cycle.run import Concluded, Cycle, run_cycle
|
||||
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
|
||||
from chunk_pick import default_plan, DefaultPlan
|
||||
from git_repo import default_ref
|
||||
@@ -134,6 +135,7 @@ from loop_claude.verdicts import Finished, Unfinished
|
||||
from loop_claude.project.model_choice import get_or_ask_models
|
||||
from todo_table import Item
|
||||
from transcript_archive.read import latest_worker
|
||||
from loop_claude.stop import needs_human, Stop
|
||||
from transcript_archive import outcomes
|
||||
from loop_claude.paths import Paths
|
||||
from project_key import project_key
|
||||
@@ -143,10 +145,11 @@ from loop_claude.project.premises import read_premises
|
||||
from loop_claude.project.records import read_records
|
||||
from model_routing.route import Route, route_from
|
||||
from claude_code.session import Session, session_named, session_open, transcript_file
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
# Seconds routing waits for origin to name its default branch, asked only where the checkout records none; past them, the chunk runs on the project's rung.
|
||||
ORIGIN_TIMEOUT = 60
|
||||
# How many unfinished verdicts running end the loop, so a worker and a judge that disagree about a chunk cannot cycle unattended without end. A policy number, not derived from the routing steps: well above the two unfinished verdicts running that archived chunks have reached at most, and low enough that a stuck pair is stopped after six cycles. It counts every unfinished verdict, including those of cycles where the human answered an interrupt: a rerun is cheap, and a chunk that needs that many answers is worth a look.
|
||||
UNFINISHED_LIMIT = 6
|
||||
|
||||
|
||||
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
|
||||
@@ -189,9 +192,10 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
concluded = run_cycle(Cycle(effects=effects, facts=facts, judge_model=models.judge, paths=paths, premises=premises, project=project, project_archive=archive_folder, records=records, root=loop_run.root, route=route, session=session, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence))
|
||||
if isinstance(concluded, Stop):
|
||||
return concluded
|
||||
loop_run = after_verdict(loop_run, project, concluded)
|
||||
if stop := archive_step(archive_folder, session, loop_run.task, concluded.route, loop_run.route, effects):
|
||||
return stop
|
||||
closed = closed_cycle(loop_run, project, concluded, archive_folder, session, effects)
|
||||
if isinstance(closed, Stop):
|
||||
return closed
|
||||
loop_run = closed
|
||||
if stop := check_routing(archive_root(paths.home, env), project or Rung(models.worker), paths, effects):
|
||||
return stop
|
||||
|
||||
@@ -207,12 +211,31 @@ def routed(route: Route | None, project: Rung | None, model: str, archived: typi
|
||||
return route if route and kept(route, project) else first_route(project, item())
|
||||
|
||||
|
||||
def closed_cycle(loop_run: LoopRun, project: Rung | None, concluded: Concluded, archive_folder: pathlib.Path, session: Session, effects: Effects) -> LoopRun | Stop:
|
||||
"""The loop run after a cycle concluded, with the step-up archived; the stop that needs the human, archived, once the chunk has had UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
after = after_verdict(loop_run, project, concluded)
|
||||
if stop := archive_step(archive_folder, session, after.task, concluded.route, after.route, effects):
|
||||
return stop
|
||||
if limit := reached_limit(after):
|
||||
# An unfinished verdict always leaves a route; the `or` only satisfies LoopRun.route's optional type. The stop is archived with that route, the one the next cycle would run on, so a rerun resumes there.
|
||||
return archived_stop(archive_folder, session, after.task, after.route or concluded.route, limit, limit.reason, effects)
|
||||
return after
|
||||
|
||||
|
||||
def reached_limit(loop_run: LoopRun) -> Stop | None:
|
||||
"""The stop that needs the human once the loop run holds UNFINISHED_LIMIT unfinished verdicts running."""
|
||||
if loop_run.unfinished_run < UNFINISHED_LIMIT:
|
||||
return None
|
||||
return needs_human("go on with the chunk", f"the judge ruled it unfinished {loop_run.unfinished_run} cycles running; finish the chunk yourself, or rerun loop-claude to give it {UNFINISHED_LIMIT} more cycles")
|
||||
|
||||
|
||||
def after_verdict(loop_run: LoopRun, project: Rung | None, concluded: Concluded) -> LoopRun:
|
||||
"""The loop run the next cycle starts from. A finished chunk drops its route. An unfinished one takes the route stepped_route makes, and the evidence the judge gave for the verdict."""
|
||||
"""The loop run the next cycle starts from. A finished chunk drops its route and its run of unfinished verdicts. An unfinished one takes the route stepped_route makes, the evidence the judge gave for the verdict, and counts the verdict."""
|
||||
verdict = concluded.verdict
|
||||
if isinstance(verdict, Finished):
|
||||
return dataclasses.replace(loop_run, route=None, unfinished_evidence="")
|
||||
return dataclasses.replace(loop_run, route=stepped_route(project, concluded.route, verdict), unfinished_evidence=verdict.evidence)
|
||||
return dataclasses.replace(loop_run, route=None, unfinished_evidence="", unfinished_run=0)
|
||||
route = stepped_route(project, concluded.route, verdict)
|
||||
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence, unfinished_run=loop_run.unfinished_run + 1)
|
||||
|
||||
|
||||
def stepped_route(project: Rung | None, ended_on: Route, verdict: Unfinished) -> Route:
|
||||
|
||||
@@ -24,6 +24,8 @@ class LoopRun:
|
||||
unfinished_evidence: str
|
||||
# Set by each unfinished verdict; None before the chunk's first one, and once it finishes.
|
||||
route: Route | None = None
|
||||
# How many unfinished verdicts the chunk has had running; a finished chunk and a rerun each set it to 0. It stays apart from route and unfinished_evidence, though all three change together at each verdict: merging or renaming a field stops a running loop at its next reload.
|
||||
unfinished_run: int = 0
|
||||
|
||||
|
||||
def decode_handed(handed: str, kind: type[T]) -> T | None:
|
||||
|
||||
@@ -647,6 +647,36 @@ class Cycles(unittest.TestCase):
|
||||
self.assertEqual((run.code, run.roles()), (1, ["worker"]))
|
||||
self.assertIn("claude left no transcript for session", run.stderr)
|
||||
|
||||
def test_exits_2_after_six_unfinished_verdicts_running_and_a_rerun_gets_six_more(self):
|
||||
checkout = Checkout(self)
|
||||
six = [step for _ in range(6) for step in (worker(), judge())]
|
||||
run = checkout.run(six + [worker()])
|
||||
self.assertEqual((run.code, run.roles()), (2, ["worker", "judge"] * 6))
|
||||
self.assertIn("the judge ruled it unfinished 6 cycles running", run.stderr.strip().splitlines()[-1])
|
||||
self.assertIn("rerun loop-claude to give it 6 more cycles", run.stderr.strip().splitlines()[-1])
|
||||
stops = [outcome for outcome in checkout.archived().values() if "Stopped: " in outcome]
|
||||
self.assertEqual(len(stops), 1)
|
||||
self.assertIn(f"\n\nWorker: {OFF_LADDER}\n", stops[0])
|
||||
run = checkout.run(six[:-2] + [worker()])
|
||||
self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"] * 6))
|
||||
|
||||
def test_archives_the_stop_on_the_route_the_sixth_verdict_stepped_to_and_a_rerun_resumes_there(self):
|
||||
checkout = Checkout(self)
|
||||
checkout.env["LOOP_CLAUDE_WORKER_MODEL"] = "claude-opus-5-5"
|
||||
checkout.commit_plan(CHEAP)
|
||||
run = checkout.run([step for _ in range(6) for step in (worker(), judge())] + [worker()])
|
||||
self.assertEqual(run.code, 2)
|
||||
stepped = '<UTC> Stepped up\n\nWorker: {"model": "claude-opus-5-5", "effort": "xhigh", "item": 4, "likelihood": 2, "escape_and_impact": 3}\n\n<UTC> Stopped: '
|
||||
self.assertEqual(len([outcome for outcome in checkout.archived().values() if stepped in outcome]), 1)
|
||||
run = checkout.run([worker()])
|
||||
self.assertEqual([(argument(call, "--model"), argument(call, "--effort")) for call in run.calls if call["role"] == "worker"], [("claude-opus-5-5", "xhigh")])
|
||||
|
||||
def test_five_unfinished_verdicts_and_a_finished_one_do_not_add_up_to_six(self):
|
||||
checkout = Checkout(self)
|
||||
five = [step for _ in range(5) for step in (worker(), judge())]
|
||||
run = checkout.run(five + [worker(), judge(finished=True)] + five + [worker()])
|
||||
self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"] * 12))
|
||||
|
||||
|
||||
class Interrupt(unittest.TestCase):
|
||||
def test_exits_2_off_a_terminal_and_resumes_on_the_next_run(self):
|
||||
@@ -868,7 +898,7 @@ class Reload(unittest.TestCase):
|
||||
self.assertEqual(run.roles(), ["worker", "judge"] * 5)
|
||||
self.assertIn("keeping the running script", run.stderr)
|
||||
reloads = [json.loads(argv[1]) for argv in run.starts if argv[:1] == ["--reloaded"]]
|
||||
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working", "route": OFF_LADDER_ROUTE} for cycle in (2, 3, 4)])
|
||||
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working", "route": OFF_LADDER_ROUTE, "unfinished_run": cycle} for cycle in (2, 3, 4)])
|
||||
self.assertIn("=== cycle 5,", run.stdout)
|
||||
|
||||
outside = checkout.temp / "outside"
|
||||
|
||||
@@ -1010,7 +1010,8 @@ class Steps(unittest.TestCase):
|
||||
def test_a_finished_verdict_clears_the_route_and_the_evidence(self):
|
||||
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="old", route=Route(self.OPUS))
|
||||
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Finished("x"), run_with.route)), dataclasses.replace(run_with, route=None, unfinished_evidence=""))
|
||||
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", 4), run_with.route)), dataclasses.replace(run_with, route=Route(Rung("claude-opus-5-5", "high"), unfinished=1), unfinished_evidence="new"))
|
||||
self.assertEqual(main.after_verdict(dataclasses.replace(run_with, unfinished_run=3), self.OPUS, run.Concluded(Finished("x"), run_with.route)).unfinished_run, 0)
|
||||
self.assertEqual(main.after_verdict(run_with, self.OPUS, run.Concluded(Unfinished("new", 4), run_with.route)), dataclasses.replace(run_with, route=Route(Rung("claude-opus-5-5", "high"), unfinished=1), unfinished_evidence="new", unfinished_run=1))
|
||||
|
||||
def test_steps_from_the_route_its_cycle_ended_on_which_a_ruling_may_have_raised(self):
|
||||
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=Route(self.OPUS, 3, 3, 4))
|
||||
@@ -1517,6 +1518,8 @@ class HelpLimits(unittest.TestCase):
|
||||
self.assertIn(f"still running after {PYTHON_TIMEOUT} s", said)
|
||||
self.assertIn(f"Once {measure.CYCLES_PER_CHECK} cycles have closed across all projects", said)
|
||||
self.assertIn(f"run with a {worker.COMPACT_WINDOW // 1000}k auto-compact window", said)
|
||||
self.assertIn(f"unfinished {main.UNFINISHED_LIMIT} cycles running", said)
|
||||
self.assertIn(f"rerun to give it {main.UNFINISHED_LIMIT} more", said)
|
||||
|
||||
|
||||
class ExitContract(unittest.TestCase):
|
||||
@@ -1531,7 +1534,7 @@ class ExitContract(unittest.TestCase):
|
||||
("cycle/interrupt.py", "talk", "1", "f'could not start claude for the interrupt:", "claude cannot start for an interrupt"),
|
||||
("cycle/interrupt.py", "talk", "1", "f'could not tell whether you typed anything:", read),
|
||||
("cycle/retry.py", "retried", "1", "f'the {who} failed three times running:", "the worker or the judge fails three times running"),
|
||||
("cycle/run.py", "stopped", "1", "f'{failure.reason}", read),
|
||||
("cycle/run.py", "archived_stop", "1", "f'{failure.reason}", read),
|
||||
("cycle/run.py", "work", "1", "f'claude left no transcript for session", "claude leaves no transcript"),
|
||||
("cycle/run.py", "keep_answer", "1", "failure.reason", read),
|
||||
("cycle/run.py", "rule", "1", "entries.reason", read),
|
||||
|
||||
@@ -2767,3 +2767,28 @@ Junior A, decided by:
|
||||
|
||||
- Locality: `bin/rule_hook/rules.py:154` `unlimited_container`: rule-hook copies run-heavy's PATH trick, and nothing in run_heavy points back to it
|
||||
- Locality: `bin/run_heavy/main.py:65` `capping`: one process sets RUN_HEAVY_MEMORY and PATH, and another reads them, so a docker run spans two processes
|
||||
|
||||
## 2026-10-28T01:47:11Z, PR #371 at 51faf71, against 3bde9cd
|
||||
|
||||
Ruling: same
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | same | same | same | same | same |
|
||||
| Inherited architect | same | same | same | same | same |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Locality, Overall: `bin/loop_claude/loop/reload.py:28` `LoopRun.unfinished_run`: its name is one word from `Route.unfinished`, and the two reset at different times
|
||||
- Locality: `bin/loop_claude/loop/main.py:214` `closed_cycle`: the order of archiving the step-up before the limit is checked has to be known, and the `or` fallback contradicts the comment above it
|
||||
- Shape: `bin/loop_claude/cycle/worker.py:15` `AUTO_COMPACT_WINDOW`: the constant named for the window holds the environment variable's name
|
||||
- Self-sufficiency: `bin/loop_claude/loop/main.py:151` `UNFINISHED_LIMIT`: the number rests on archive evidence the reader can't see
|
||||
- Navigation: `bin/loop_claude/cycle/run.py:82` `archived_stop`: a loop-level stop goes through a cycle-level helper
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Locality, Overall: `bin/loop_claude/loop/main.py:214` `closed_cycle`: the order of its steps is easy to break, and the `or` fallback contradicts the comment above it
|
||||
- Locality: `bin/loop_claude/loop/reload.py:28` `LoopRun.unfinished_run`: a rerun sets it to 0 only through the dataclass default, which the call site at `loop_run_from` doesn't say
|
||||
- Navigation: `bin/loop_claude/cycle/judge.py:150` `run_judge`: the path from a rejected verdict to exit 1 crosses three files, and only some misquotes count
|
||||
- Navigation, Shape: `bin/round_titles.py:8` `CAP`: the fourth-round cap lives in a module named for commit titles
|
||||
- Self-sufficiency: `bin/loop_claude/loop/main.py:1` module docstring: the exit list maps a code to a cause, but no bullet links to the unit that raises it
|
||||
|
||||
@@ -18,7 +18,6 @@
|
||||
| 239 | 0.4.0 | | owner | **Block a Read, `cat`, `sed` or `head` of a file range already in the session's context, with a note saying so, and let it through once a compaction dropped the range or the file changed.** | 3 | 2 | 3 | 6 | 4 | 1.4 by 3 | 2.0 |
|
||||
| 226 | 0.5.0 | | question | **Answer whether rule-hook's `run-heavy` rule should also catch a mutation run or docker build behind `uv run`, `poetry run`, `python3 -m`, `sudo`, `docker image build` and `docker compose up --build`, or narrow the `machine load:` decision's wording.** | 2 | 2 | 3 | 5 | 3 | 1.2 by 2, 1.5 by 2 | 4.0 |
|
||||
| 235 | 0.5.0 | | decision | **Compare the first 20 chunks whose worker ran under the 200k compaction window with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and file an item where any rose.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 2, 1.4 by 3 | 6.8 |
|
||||
| 59 | 0.5.0 | | | **Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.** | 2 | 1 | 3 | 4 | 3 | 1.4 by 6 | 6.0 |
|
||||
| 167 | 0.5.0 | | | **Refuse a `/security-review` or `/claude-security` Skill call below the project's rung, as rule-hook refuses such an Agent call.** | 3 | 2 | 3 | 6 | 2 | 1.1 by 4 | 5.6 |
|
||||
| 6 | 0.5.0 | | decision | **Move what `loop-claude` writes out of `~/.claude.json`, and ask the human about each write that must stay.** | 3 | 3 | 3 | 7 | 3 | 1.1 by 5 | 5.3 |
|
||||
| 240 | 0.5.0 | | decision | **Give loop-claude's worker prompt the reading rule "never read more than 80 lines of a file at once, nor take a tool output over 3,000 characters into your context", and keep it only where the 20 chunks after it hold the 20 before's review rounds, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk.** | 1 | 2 | 3 | 4 | 2 | 1.4 by 3 | 4.0 |
|
||||
@@ -103,10 +102,6 @@ Found by review of PR #370. Today `uv run mutmut run`, `poetry run mutmut run`,
|
||||
|
||||
`docs/decisions.md` → `loop-claude: the worker's claude compacts its context before it fills a 200k window…` expects a saving of about 6% of all spend with no rise in the other figures. Measure from loop-claude's archive as that decision did: rounds past each pass's first and the 🔴, 🟡 and misread counts from the round commit titles, unfinished verdicts from the outcomes files, hours from the worker's first message to its last, and dollars at list prices, cache reads included, from the chunk's `cost-state` entry, which counts the compaction calls that no message's usage records; item 234 puts such dollars into the routing figures. A chunk ran under the window when its first worker started after that decision merged.
|
||||
|
||||
### 59. Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.
|
||||
|
||||
Raised by the product-owner review of PR #211. The resumed worker gets the judge's quote of the work remaining, but a worker that still thinks the chunk done can answer each cycle the same way, and each cycle costs a worker and a judge run (goal 1.4).
|
||||
|
||||
### 167. Refuse a `/security-review` or `/claude-security` Skill call below the project's rung, as rule-hook refuses such an Agent call.
|
||||
|
||||
rule-hook checks Agent calls (item 163) but no Skill calls. A worker that invokes `/security-review` in its own context, instead of in a subagent, runs the pass on its own rung, and nothing blocks it. Check Skill calls too: `breaches_of` in bin/rule_hook/main.py takes a Skill call whose skill is `security-review` or `claude-security` as a review launch, and settings.json's PreToolUse matcher gains Skill. Found by the systems-architect's review of PR #295.
|
||||
|
||||
Reference in New Issue
Block a user