Merge pull request 'Resume an unfinished chunk with the judge's quote of the work remaining' (#214) from judge-evidence into main

This commit was merged in pull request #214.
This commit is contained in:
2026-10-04 17:17:28 +02:00
10 changed files with 54 additions and 41 deletions
+1
View File
@@ -2,6 +2,7 @@
## Unreleased
- When `loop-claude`'s judge rules a chunk unfinished, the resumed worker is shown the passage the judge quoted as the work remaining. A loop started before this change exits 1 at its next reload; rerun it.
- larv-review's architecture pass and full security scan get short names, `Architecture` and `Full scan`. Each round's commit is titled with its pass's name in lower case: `Answer full scan round 1: …`.
- larv-review's comprehension floor now says, without another file open, what the scoring run measures. A project's `docs/decisions.md` keeps the last scoring run's scores, and a later run scoring below them blocks the merge until fixed. After three rounds larv-review asks you; an unattended run leaves the PR unmerged with the question. A PR under the floor that scores no lower than the last run merges once you decide to ship. A decision in `docs/decisions.md` that lets every chunk ship under the floor counts as yours. A project without those scores gets them at its next scoring run.
- `loop-claude`'s judge reads your `~/.claude/CLAUDE.md` beside the project's `AGENTS.md` and `docs/decisions.md`. It no longer interrupts for a choice that file settles, and it answers the worker's questions from it.
+1 -1
View File
@@ -65,7 +65,7 @@ def human_text(entry: dict) -> str | None:
return text if isinstance(text, str) else "".join(part_text(part) or "" for part in parts(entry))
# {"type": "user", "promptSource": "sdk", "message": {"content": "Continue the task; the previous run stopped before it was finished."}}
# {"type": "user", "promptSource": "sdk", "message": {"content": "Continue the task; the previous run ended before the chunk was finished."}}
def loop_text(entry: dict) -> str | None:
"""What a -p prompt of loop-claude's told the agent."""
text = content(entry)
+6 -4
View File
@@ -26,12 +26,14 @@ class Cycle:
premises: str
project_key: str
records: str
# The evidence of the previous cycle's unfinished verdict, empty after any other.
remaining: str
root: str
session: Session
task: str
def run_cycle(cycle: Cycle) -> Stop | None:
def run_cycle(cycle: Cycle) -> Stop | Finished | Unfinished:
state: State = Working()
try:
while not isinstance(state, Ended):
@@ -41,7 +43,7 @@ def run_cycle(cycle: Cycle) -> Stop | None:
state = transition(state, event)
except KeyboardInterrupt:
return stopped(cycle, Stop(130, ""), "Ctrl-C")
return None
return state.verdict
def stopped(cycle: Cycle, stop: Stop, why: str) -> Stop:
@@ -71,7 +73,7 @@ def work(cycle: Cycle, answered: QuotedAnswer | None) -> WorkerRan | Stop:
if stop := keep_answer(cycle, answered):
return stop
# session_open runs inside the lambda: a failed attempt may have opened the session.
if stop := retried("worker", lambda: run_worker(cycle.root, worker_args(cycle.task, cycle.session, cycle.models.worker, session_open(cycle.session), answered), cycle.effects), cycle.effects):
if stop := retried("worker", lambda: run_worker(cycle.root, worker_args(cycle.task, cycle.session, cycle.models.worker, session_open(cycle.session), answered, quoted(cycle.remaining)), cycle.effects), cycle.effects):
return stop
path = transcript_file(cycle.session)
return WorkerRan(path) if path is not None else Stop(1, f"claude left no transcript for session {cycle.session.id}")
@@ -115,6 +117,6 @@ def close(cycle: Cycle, transcript_path: str, verdict: Finished | Unfinished) ->
return Archived()
# Blockquoted, so evidence holding blank lines still ends visibly where the next outcome starts.
# Blockquoted, so evidence holding blank lines still ends visibly where the text after it starts.
def quoted(passage: str) -> str:
return "\n".join(f"> {line}" for line in passage.splitlines())
+3 -3
View File
@@ -36,7 +36,7 @@ class Closing:
@dataclasses.dataclass(frozen=True)
class Ended:
"""The cycle is over."""
verdict: Finished | Unfinished
Active = Working | Judging | Interrupting | Closing
@@ -79,6 +79,6 @@ def transition(state: State, event: Event) -> State:
# The human works in the interrupt's claude session, so the judge rules again on the transcript it leaves.
case Interrupting(transcript_path, _, quoted_answers), HumanAnswered():
return Judging(transcript_path, quoted_answers)
case Closing(), Archived():
return Ended()
case Closing(_, verdict), Archived():
return Ended(verdict)
raise ValueError(f"no transition from {state} on {event}")
+3 -2
View File
@@ -39,11 +39,12 @@ def worker_outcome(code: int, result: dict | None, resets: list[int]) -> Failure
return None if result is not None else Failure("claude exited 0 without a result event")
def worker_args(task: str, session: Session, model: str, resuming: bool, answered: QuotedAnswer | None = None) -> list[str]:
def worker_args(task: str, session: Session, model: str, resuming: bool, answered: QuotedAnswer | None = None, remaining: str = "") -> list[str]:
if answered is not None:
start = ["-p", f"Your question: {answered.question}\n\n{QUOTE_SOURCES} answers it:\n\n{answered.answer}\n\nContinue the task.", "--resume", str(session.id)]
elif resuming:
start = ["-p", f"Continue the task; the previous run stopped before it was finished. The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}.", "--resume", str(session.id)]
ruled = f" A judge ruled the chunk unfinished, quoting this passage of your transcript as the work remaining:\n\n{remaining}\n\n" if remaining else " "
start = ["-p", f"Continue the task; the previous run ended before the chunk was finished.{ruled}The chunk is finished once it is merged, or nothing remains that you could do on your own. Even then, it is finished only once {ANSWERS_RECORDED}.", "--resume", str(session.id)]
else:
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
start = ["-p", f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.", "--name", session.name, "--session-id", str(session.id)]
+11 -7
View File
@@ -16,7 +16,8 @@ does a question asked again after its answer. The fourth question they answer in
the human too, with the quote.
A question only the human can answer, or a release ready to cut, is an interrupt: `claude --resume`
on the terminal, every worktree first marked trusted in ~/.claude.json. An unfinished chunk resumes
its session next cycle; the next chunk starts a fresh session.
its session next cycle, and the worker is shown the passage the judge quoted as the work remaining.
The next chunk starts a fresh session.
Cycles continue until an exit below or Ctrl-C. Words: ~/.claude/CLAUDE.md → "Vocabulary".
Each project has one session, named "A <key> loop-claude", where <key> is the checkout's path
@@ -52,8 +53,8 @@ Exit 1, saying why, when:
- a finished chunk's session cannot be deleted
- claude cannot start for an interrupt
- python cannot start for a reload
- a reload cannot read the cycle number, checkout root and task the running script handed it,
likely because the edit changed that handover
- a reload cannot read the state the running script handed it, likely because the edit changed that
handover
- the worker or the judge fails three times running, retried a minute apart within the cycle. A
worker fails when claude cannot start, exits non-zero, is killed, or ends without a result event;
a judge when claude cannot start or exits non-zero, or its reply misses or mistypes a field,
@@ -86,6 +87,7 @@ from loop_claude.project.premises import read_premises
from loop_claude.project.records import read_records
from loop_claude.claude_code.session import session_for
from loop_claude.stop import Stop
from loop_claude.cycle.judge_reply import Unfinished
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
@@ -119,8 +121,10 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
if isinstance(models, Stop):
return models
print(f"\n=== cycle {loop_run.cycle_number} ===", flush=True)
if stop := run_cycle(Cycle(effects=effects, models=models, paths=paths, premises=premises, project_key=key, records=records, root=loop_run.root, session=session, task=loop_run.task)):
return stop
verdict = run_cycle(Cycle(effects=effects, models=models, paths=paths, premises=premises, project_key=key, records=records, remaining=loop_run.remaining, root=loop_run.root, session=session, task=loop_run.task))
if isinstance(verdict, Stop):
return verdict
loop_run = dataclasses.replace(loop_run, remaining=verdict.evidence if isinstance(verdict, Unfinished) else "")
def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopRun | Stop:
@@ -129,8 +133,8 @@ def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopR
if task in ("", "-h", "--help"):
return Stop(0 if task else 1, __doc__.strip())
root = checkout_root(cwd, effects)
return root if isinstance(root, Stop) else LoopRun(0, root, task)
return root if isinstance(root, Stop) else LoopRun(0, "", root, task)
loop_run = decode_handed(argv[1], LoopRun) if len(argv) == 2 else None
if not (loop_run and loop_run.root and loop_run.task):
return Stop(1, f"the edited loop-claude cannot read the cycle number, checkout root and task the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}")
return Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(argv[1:])}")
return loop_run
+1
View File
@@ -17,6 +17,7 @@ T = typing.TypeVar("T")
class LoopRun:
"""What a reload hands the edited script, as JSON on argv."""
cycle_number: int
remaining: str
root: str
task: str
+19 -16
View File
@@ -142,12 +142,13 @@ class Transitions(unittest.TestCase):
(states.Judging("/t", 3), states.Ruled(Unfinished("e")), states.Closing("/t", Unfinished("e"))),
(states.Interrupting("/t", Interrupt("e", "q")), states.HumanAnswered(), states.Judging("/t")),
(states.Interrupting("/t", Interrupt("e", "q"), 3), states.HumanAnswered(), states.Judging("/t", 3)),
(states.Closing("/t", Finished("e")), states.Archived(), states.Ended()),
(states.Closing("/t", Finished("e")), states.Archived(), states.Ended(Finished("e"))),
(states.Closing("/t", Unfinished("e")), states.Archived(), states.Ended(Unfinished("e"))),
)
for state, event, after in legal:
with self.subTest(state=state, event=event):
self.assertEqual(states.transition(state, event), after)
every_state = (states.Working(), states.Working(QuotedAnswer("e", "Which one?", "A."), 3), states.Judging("/t"), states.Judging("/t", 3), states.Interrupting("/t", Interrupt("e", "q")), states.Interrupting("/t", Interrupt("e", "q"), 3), states.Closing("/t", Finished("e")), states.Ended())
every_state = (states.Working(), states.Working(QuotedAnswer("e", "Which one?", "A."), 3), states.Judging("/t"), states.Judging("/t", 3), states.Interrupting("/t", Interrupt("e", "q")), states.Interrupting("/t", Interrupt("e", "q"), 3), states.Closing("/t", Finished("e")), states.Ended(Finished("e")))
every_event = (states.WorkerRan("/t"), states.Ruled(Interrupt("e", "Which one?")), answered, states.Ruled(Finished("e")), states.Ruled(Unfinished("e")), states.HumanAnswered(), states.Archived())
for state in every_state:
for event in every_event:
@@ -175,26 +176,27 @@ class Cycles(unittest.TestCase):
self.paths = paths.paths_from(self.home, {})
self.session = session.session_for("proj", self.paths)
def cycle(self, fake, records=""):
return run.Cycle(effects=fake.effects(), models=model_choice.Models("judge-model", "worker-model"), paths=self.paths, premises="## Goals", project_key="proj", records=records, root=str(self.home), session=self.session, task="Do it.")
def cycle(self, fake, records="", remaining=""):
return run.Cycle(effects=fake.effects(), models=model_choice.Models("judge-model", "worker-model"), paths=self.paths, premises="## Goals", project_key="proj", records=records, remaining=remaining, root=str(self.home), session=self.session, task="Do it.")
def archived_outcomes(self):
return sorted(path.read_text() for path in (self.paths.transcript_archive / "proj").glob("*/*.outcomes.md"))
def test_a_finished_chunk_is_archived_and_its_session_deleted(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
self.assertIsNone(run.run_cycle(self.cycle(fake)))
self.assertEqual(run.run_cycle(self.cycle(fake)), Finished("working"))
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n"])
self.assertIsNone(session.transcript_file(self.session))
def test_an_unfinished_chunks_transcript_is_archived_and_the_next_cycle_resumes_its_session(self):
def test_an_unfinished_chunks_transcript_is_archived_and_the_next_cycle_resumes_its_session_with_the_judges_evidence(self):
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 2, claude=[judged("unfinished")] * 2)
self.assertIsNone(run.run_cycle(self.cycle(fake)))
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
self.assertTrue(session.transcript_file(self.session))
self.assertIsNone(run.run_cycle(self.cycle(fake)))
self.assertEqual(run.run_cycle(self.cycle(fake, remaining="working")), Unfinished("working"))
workers = fake.made("stream_claude")
self.assertIn("--session-id", workers[0])
self.assertIn("--resume", workers[1])
self.assertIn("as the work remaining:\n\n> working\n\nThe chunk is finished once", workers[1][workers[1].index("-p") + 1])
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n"] * 2)
def test_a_cycle_stopped_by_a_failure_or_ctrl_c_archives_what_its_session_holds(self):
@@ -212,7 +214,7 @@ class Cycles(unittest.TestCase):
(folder / "custom-title.json").write_text("first")
(folder.parent / f"{self.session.id}.orphaned-1.jsonl").write_text("o\n")
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)] * 3, claude=[judged("unfinished"), judged("finished"), judged("unfinished")])
self.assertIsNone(run.run_cycle(self.cycle(fake)))
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
[chunk] = (self.paths.transcript_archive / "proj").iterdir()
kept = chunk / "session" / str(self.session.id)
self.assertEqual((chunk / "session" / f"{self.session.id}.orphaned-1.jsonl").read_text(), "o\n")
@@ -220,7 +222,7 @@ class Cycles(unittest.TestCase):
with open(folder / "subagents" / "agent-a.jsonl", "a") as handle:
handle.write("more\n")
(folder / "custom-title.json").write_text("second")
self.assertIsNone(run.run_cycle(self.cycle(fake)))
self.assertEqual(run.run_cycle(self.cycle(fake)), Finished("working"))
self.assertFalse(folder.exists())
self.assertEqual((kept / "subagents" / "agent-a.jsonl").read_text(), "a\nmore\n")
self.assertEqual((kept / "subagents" / "workflows" / "journal.jsonl").stat().st_mtime, 0)
@@ -228,7 +230,7 @@ class Cycles(unittest.TestCase):
self.assertEqual(stat.S_IMODE((kept / "subagents" / "agent-a.jsonl").stat().st_mode), 0o600)
folder.mkdir()
(folder / "custom-title.json").write_text("third")
self.assertIsNone(run.run_cycle(self.cycle(fake)))
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
[next_chunk] = set((self.paths.transcript_archive / "proj").iterdir()) - {chunk}
self.assertEqual(sorted(path.read_text() for path in (next_chunk / "session" / str(self.session.id)).glob("custom-title.json*")), ["third"])
@@ -247,7 +249,7 @@ class Cycles(unittest.TestCase):
handle.write(json.dumps(human("the second")) + "\n")
return 0
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt"), judged("unfinished")], on_terminal_claude=[answer], git=[ran()])
self.assertIsNone(run.run_cycle(self.cycle(fake)))
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nEvidence:\n> working\n", "Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n"])
self.assertIn(str(self.session.id), fake.made("claude_on_terminal")[0])
prompts = [stdin for args, stdin in fake.made("claude")]
@@ -395,6 +397,7 @@ class WorkerArgs(unittest.TestCase):
resumed = worker.worker_args("Do it.", opened, "m", True)
self.assertEqual(resumed[resumed.index("--resume") + 1], str(opened.id))
self.assertNotIn("Do it.", " ".join(resumed))
self.assertTrue(resumed[resumed.index("-p") + 1].startswith("Continue the task; the previous run ended before the chunk was finished. The chunk is finished once"))
class Records(unittest.TestCase):
@@ -565,9 +568,9 @@ class Verdict(unittest.TestCase):
class LoopRuns(unittest.TestCase):
def test_stops_naming_the_reload_when_it_hands_over_anything_but_the_state_with_a_task(self):
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "root": "/r"})], [json.dumps({"cycle_number": 3, "root": "/r", "task": ""})], [json.dumps({"cycle_number": "3", "root": "/r", "task": "do it"})], [json.dumps({"cycle_number": 3, "root": "", "task": "do it"})], [json.dumps({"cycle_number": 3, "root": "/r", "task": "do it", "turns": 1})], [json.dumps({"cycle_number": 3, "failures": 0, "root": "/r", "task": "do it"})]):
for handed in (["3", "2", "/r", "do it"], ["3"], ["[1]"], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": ""})], [json.dumps({"cycle_number": "3", "remaining": "", "root": "/r", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "", "task": "do it"})], [json.dumps({"cycle_number": 3, "remaining": "", "root": "/r", "task": "do it", "turns": 1})], [json.dumps({"cycle_number": 3, "failures": 0, "remaining": "", "root": "/r", "task": "do it"})]):
with self.subTest(handed=handed):
self.assertEqual(main.loop_run_from([main.RELOAD_FLAG, *handed], CWD, FakeEffects().effects()), Stop(1, f"the edited loop-claude cannot read the cycle number, checkout root and task the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(handed)}"))
self.assertEqual(main.loop_run_from([main.RELOAD_FLAG, *handed], CWD, FakeEffects().effects()), Stop(1, f"the edited loop-claude cannot read the state the running one handed it, likely because the edit changed that handover; rerun loop-claude <task>. Handed: {' '.join(handed)}"))
def test_decodes_a_state_whose_annotations_are_strings(self):
@dataclasses.dataclass(frozen=True)
@@ -582,7 +585,7 @@ class LoopRuns(unittest.TestCase):
launcher = pathlib.Path(root, "loop-claude")
launcher.write_text("")
fake = FakeEffects(python=[ran()])
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(1, "/r", "do it"), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
self.assertEqual(reload.exec_if_edited(launcher, "stale", main.LoopRun(1, "", "/r", "do it"), fake.effects()), Stop(1, "could not reload loop-claude: No such file or directory; rerun loop-claude <task>"))
class HelpLimits(unittest.TestCase):
@@ -614,7 +617,7 @@ class ExitContract(unittest.TestCase):
("cycle/run.py", "close", "1", "failure.reason", read),
("cycle/run.py", "close", "1", "failure.reason", "a finished chunk's session cannot be deleted"),
("loop/main.py", "loop_run_from", "0 if task else 1", "__doc__.strip()", "no task is given"),
("loop/main.py", "loop_run_from", "1", 'f"the edited loop-claude cannot read the cycle number,', "a reload cannot read the cycle number, checkout root and task"),
("loop/main.py", "loop_run_from", "1", 'f"the edited loop-claude cannot read the state', "a reload cannot read the state the running script handed it"),
("loop/reload.py", "exec_if_edited", "1", "f'could not reload loop-claude:", "python cannot start for a reload"),
("project/checkout.py", "checkout_root", "1", "f'could not start git:", git),
("project/checkout.py", "checkout_root", "1", "found.stderr.strip()", git),
+3 -2
View File
@@ -345,6 +345,7 @@ class Cycles(unittest.TestCase):
self.assertTrue(argument(workers[0], "-p").startswith(f"{TASK}\n\nContinue any open draft PR"))
self.assertNotIn(TASK, argument(workers[1], "-p"))
self.assertIn("Even then, it is finished only once every answer the human typed in it", argument(workers[1], "-p"))
self.assertIn("as the work remaining:\n\n> started\n\n", argument(workers[1], "-p"))
self.assertEqual({call["cwd"] for call in run.calls}, {str(checkout.root)})
self.assertIn("1. Work.", run.calls[1]["stdin"])
self.assertIn("From docs/decisions.md:\n# Decisions\n\nFrom ~/.claude/CLAUDE.md:\n# Global instructions", run.calls[1]["stdin"])
@@ -681,12 +682,12 @@ class Reload(unittest.TestCase):
self.assertEqual(run.roles(), ["worker", "judge"] * 5)
self.assertIn("keeping the running script", run.stderr)
reloads = [json.loads(argv[1]) for argv in run.starts if argv[:1] == ["--reloaded"]]
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK} for cycle in (2, 3, 4)])
self.assertEqual(reloads, [{"cycle_number": cycle, "remaining": "working", "root": str(checkout.root), "task": TASK} for cycle in (2, 3, 4)])
self.assertIn("=== cycle 5 ===", run.stdout)
outside = checkout.temp / "outside"
outside.mkdir()
state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK}
state = {"cycle_number": 3, "remaining": "", "root": str(checkout.root), "task": TASK}
run = checkout.run([worker()], argv=["--reloaded", json.dumps(state)], cwd=outside)
self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"]))
self.assertEqual(run.calls[0]["cwd"], str(checkout.root))
+6 -6
View File
@@ -6,7 +6,7 @@
`Bar = 9`
`Next ID = 59`
`Next ID = 60`
| Goal | W |
|---|---|
@@ -25,11 +25,11 @@
| ID | Release | Exempt | Item | R | S | A | G | Goals | Score |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 0.1.0 | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 3 | 5 | 8 | 9 | 2, 3 | 27.8 |
| 58 | 0.1.0 | | **Give the worker of a chunk the judge held open the judge's evidence of what remains.** | 3 | 3 | 6 | 6 | 4, 4.4 | 16.3 |
| 23 | 0.1.0 | | **Launch a principles auditor only for a principles file the project shows applies, and rerun only the reviewer whose finding a fix answers.** | 3 | 4 | 7 | 8 | 4.4 | 13.5 |
| 21 | 0.1.0 | | **Give `settings.json`'s model one writer, and sort its keys.** | 3 | 3 | 6 | 7 | 4.3 | 12.9 |
| 49 | 0.1.0 | principle | **Check that `AGENTS.md` lists exactly the titles of `docs/decisions.md`.** | 1 | 2 | 5 | 6 | 4.3 | 12.5 |
| 12 | 0.1.0 | | **Run the test suite in CI on every PR, or record in `docs/decisions.md` why the local run is enough.** | 2 | 4 | 4 | 6 | 4.1 | 11.7 |
| 59 | 0.1.0 | | **Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.** | 3 | 3 | 6 | 6 | 4.4 | 10.9 |
| 5 | 0.1.0 | decision | **Key a project on its checkout's full path, and refuse to start while another session uses that key.** | 5 | 5 | 5 | 6 | 4.1 | 10.5 |
| 4 | 0.1.0 | | **Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.** | 3 | 2 | 4 | 3 | 2 | 9.8 |
| 50 | 0.1.0 | principle | **Grant the bench's runs only the tools a review needs, in place of skipping every permission.** | 2 | 2 | 3 | 4 | 3 | 9.7 |
@@ -48,10 +48,6 @@ It lives under `~/.claude`, which `.gitignore` keeps from the remote, so a lost
The README line covers the bench's transcripts in `workflow-bench/logs/` too (goal 2).
### 58. Give the worker of a chunk the judge held open the judge's evidence of what remains.
Raised by the product-owner review of PR #211: a verdict that the chunk is unfinished resumes the worker with a generic prompt, so a worker that thinks the chunk done repeats itself, and nothing bounds a run of such verdicts (goal 4.4).
### 23. Launch a principles auditor only for a principles file the project shows applies, and rerun only the reviewer whose finding a fix answers.
On this repo the frontend, REST and logging auditors each read the whole project to report "does not apply", and one 🟢 fix reruns ten whole-project reads.
@@ -70,6 +66,10 @@ DRY, in `principles/technical-principles.md`: the list is copied by hand, and se
The repo has no CI, so larv-review's gate is the local run, and larv-review's Gate skips even that when the last stability round left the suite green: an unattended chunk can merge on the worker's own word.
### 59. Bound a run of unfinished verdicts on one chunk, so a worker and judge that disagree cannot cycle unattended without end.
Raised by the product-owner review of PR #211. The resumed worker gets the judge's quote of the work remaining, but a worker that still thinks the chunk done can answer each cycle the same way, and each cycle costs a worker and a judge run (goal 4.4).
### 5. Key a project on its checkout's full path, and refuse to start while another session uses that key.
The key names the project's session, archive folder and models entry. `project_key` drops the leading `/`, so `~/code/a` and `/a` share all three, and one loop can resume or delete the other's session.