Merge remote-tracking branch 'origin/main' into item-309
CI / tests (pull_request) Successful in 4m6s

# Conflicts:
#	docs/reading-cost.md
#	todo.md
This commit was merged in pull request #579.
This commit is contained in:
2026-11-17 07:31:42 +01:00
40 changed files with 1476 additions and 194 deletions
+7
View File
@@ -47,6 +47,9 @@
!bin/command_lines/*.py
!bin/command_patterns/
!bin/command_patterns/*.py
!bin/compress-archives
!bin/compress_archives/
!bin/compress_archives/*.py
!bin/deploy-release
!bin/deploy_release/
!bin/deploy_release/*.py
@@ -184,9 +187,13 @@
!workflow-bench/bench.py
!workflow-bench/compare-prompt.md
!workflow-bench/compare.py
!workflow-bench/handoff-brief-prompt.md
!workflow-bench/handoff-resume-prompt.md
!workflow-bench/handoff_bench.py
!workflow-bench/review_git.py
!workflow-bench/run-prompt.md
!workflow-bench/setup.cfg
!workflow-bench/test_advisor_bench.py
!workflow-bench/test_bench.py
!workflow-bench/test_handoff_bench.py
!workflow-bench/test_review_git.py
+5
View File
@@ -2,6 +2,11 @@
## Unreleased
- Add `workflow-bench/handoff_bench.py`, a bench of how a worker's step to a stronger model hands over the chunk, on six archived L 3 chunks. Sonnet 5.5 left five unfinished at its budget. On those, Opus 5.5 resuming Sonnet 5.5's session passed two and was partial on three, for a mean $0.68; a fresh session with a brief passed one and was partial on four, for $0.78. So a stepped worker still resumes its session.
- todo-item edit --blocked-by sets the "is blocked by" links of a Jira issue the loop filed.
- A `Touches:` list in the description of a Jira issue is read as one in `todo.md` is, a path or name that holds a backtick included: the Jira description spells it in backticks with no padding spaces, and the prediction reader takes that spelling too.
- `todo-item edit --detail` and `project-record save` leave a Jira description or a Confluence page exactly as it is, with its block ids and spacing paragraphs, where the text saved is the text a read showed. An unedited save over a page a teammate saved since still refuses.
- Add `compress-archives`, which gzips each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them. The tools that read those folders read a gzipped file by its plain name. A new crontab line in the README's Install section runs it daily.
- review-replay and judge-replay keep each run, with its prompt, claude's exit code and stream, and the report they print, in a `replays/<tool>-<arguments>-<UTC stamp>/` folder of loop-claude's archive, so a replay that decided a rung can be checked and re-priced later; claude's json replies are now stream-json, whose last line is the reply.
- loop-claude's archive keeps each judge run's prompt and json reply in a judge/ folder beside its chunk's transcripts, and names the run on a `Judge run:` line of the outcome it produced; judge-replay replays a verdict on its archived prompt where the archive holds one.
- A plan kept as a Jira epic takes no changelog bullet: `todo-item finish` there needs neither `--changelog` nor `--no-changelog`, and refuses `--changelog`, since the issues' history is the changelog.
+12 -3
View File
@@ -73,6 +73,14 @@ Then pick a model with `/model`.
Link `loop-claude` onto `PATH`: `ln -s ~/.claude/bin/loop-claude ~/.local/bin/loop-claude`. Every claude it starts, the interactive ones included, skips every permission prompt. loop-claude runs each chunk in its own window of a tmux session per project, so it needs tmux 3.2 or later.
Back up two folders: `~/.claude/loop-claude-transcripts`, where loop-claude keeps every transcript, and `~/.local/state/larv-review`, where larv-review keeps each chunk's reports. To keep them small, add a crontab line that gzips each transcript and review file 30 days after it last changed, and logs each run with its date:
```sh
17 4 * * * { date; $HOME/.claude/bin/compress-archives; } >> $HOME/.local/state/compress-archives.log 2>&1
```
Cron runs it with its own `PATH` and environment. Where the `python3` on cron's `PATH` is older than 3.14, or you set `LOOP_CLAUDE_TRANSCRIPTS`, set `PATH` or that variable in the crontab too.
To get loop-claude's questions and stops as DMs from a bot, write the bot's token, your username or member ID and, for Mattermost, the server into `~/.claude-projects/<key>.env`, in the keys `~/.claude/bin/notify --help` names, run `chmod 600` on it, and add `env-file = "~/.claude-projects/<key>.env"` and `notifier = "mattermost"` (or `"slack"`) to the project's config, as Install → "Set up a project" describes.
### Set up a project
@@ -124,7 +132,7 @@ A read fails, naming the issue, where a child is in another Jira project or name
The loop reads such a plan and files items into it, each as a Task under the epic. `todo-item add` needs the item's release among the Jira project's Fix Versions, named as `1.2.0` or `v1.2.0`. The config states the formula and the bar that a todo.md states under `## Scoring`, such as `jira-score-formula = "100 * V / (2**S + 8)"` and `jira-score-bar = 1`, and fails to load without them. `todo-item add` refuses an item that scores under the bar, unless it is exempt. It ranks the new issue among the issues the loop filed: questions first, then security findings, then by score. A person's issues keep the rank the team gave them, and `rank_todo.py` doesn't rank the plan.
`todo-item edit` and `move` change only an issue the loop filed. Of an issue a person filed, `edit` sets only the scores, and posts its `--comment` on the issue to tell the team why; the summary, description and Fix Version stay the team's to change in Jira, as do every issue's links. For `todo-item finish`, the config names the workflow transition that finishes an item and the status it leads to, both as Jira shows them: `jira-finish-transition = "Mark as done"` and `jira-finish-status = "Done"`. Finish then moves the item's issue by that transition. It refuses a transition that leads to another status, or to one Jira doesn't count as done, and fails where the config sets neither. The four things not supported for a plan outside the checkout, above, are not supported for a plan in Jira either.
`todo-item edit` and `move` change only an issue the loop filed. Of an issue a person filed, `edit` sets only the scores, and posts its `--comment` on the issue to tell the team why; the summary, description and Fix Version stay the team's to change in Jira, as do its links. On an issue the loop filed, `edit --blocked-by` makes and deletes the links to the epic's open items. For `todo-item finish`, the config names the workflow transition that finishes an item and the status it leads to, both as Jira shows them: `jira-finish-transition = "Mark as done"` and `jira-finish-status = "Done"`. Finish then moves the item's issue by that transition. It refuses a transition that leads to another status, or to one Jira doesn't count as done, and fails where the config sets neither. The four things not supported for a plan outside the checkout, above, are not supported for a plan in Jira either.
A client that keeps its goals and audience in an epic's description names that epic as the goals file: `goals-and-audience = "jira:https://acme.atlassian.net/browse/ABC-12"`, which may be the plan's own epic. Such a config keeps its plan in Jira too, and reads both with the same `env-file`. The description needs `## Goals` and `## Audience` as headings, level-2 headings in Jira's editor. The loop only reads the description, so edit it in Jira; `project-record open goals-and-audience` prints a file holding its text. Whoever may edit that description sets the premises every worker and judge follows, so keep its edit rights to the people who own the project's goals.
@@ -171,6 +179,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `chunk-pr open|sync|ready|merge|clean` opens a chunk's pull request as a draft, brings its branch level with origin's default branch, marks it ready and merges it once its checks pass, with `gh` on a GitHub origin and `tea` elsewhere. The chunks of one project merge one at a time: in its turn, `merge` brings the branch level again, finishes the plan items its `--finish` flags name, and waits for the checks; once the pull request is merged, `clean` removes the chunk's worktree and branch and fast-forwards the main checkout. `chunk-pr --help` states the steps.
- `chunk-preflight` finds the mechanical facts a chunk starts from, and with `--worktree` makes its worktree; `loop-claude` runs it with `--worktree` before it routes each chunk, and `chunk-preflight --help` lists the facts.
- `command-patterns [<version>] [--calls <file>]` prints a table of the shell command patterns a release's archived chunks ran, such as `tea pr create`, for an audit of the release, and with `--calls <file>` saves each call as a JSON line; `command-patterns --help` states the columns and the line's keys.
- `compress-archives` gzips each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them; Install's crontab line runs it daily.
- `deploy-release [--cut] [<checkout>]` moves `~/.claude`, or the checkout named, to the newest release, as Update says; with `--cut`, run in the development clone, it first cuts the next release, as Release says.
- `judge-replay <model>` replays loop-claude's judge on archived verdicts of the projects run on the model, and prints what each rung a review climbs to misses, and the lowest rung below them that, like each candidate above it, returns them; `judge-replay --help` states the replay.
- `larv-review <command>` holds larv-review's mechanical rules: the chunk's review folder, the paired ruling's base and vote count, each pass's round count, the scoring run's trigger and setup; the comprehension history's format, its latest places and its count of how often seats agree; and each reviewer's prompt header. `larv-review --help` states them. `settings.json` → `hooks` runs `larv-review reviewer-context` as each subagent starts. A reviewer under `agents/` whose frontmatter has `omitClaudeMd: true` starts without `CLAUDE.md`, and the command hands it that file's "Shared goal", "Prose" and "Vocabulary" sections.
@@ -211,7 +220,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `model_routing/` holds the shipped routing ladder (`routing-ladder.json`) and its steps (`ladder.py`), and reads a ladder from that file or a project's own (`ladder_file.py`), for loop-claude, `rule_hook`, both replay tools and `subagent-rung`. It finds a project's config folder through `project_config`. It also holds the list prices, which `routing_figures` and `transcript_archive` read.
- `notifier/` sends the human a DM through the notifier a project's config names, for `notify` and for loop-claude through `loop_effects`. Its backends are `mattermost.py` and `slack.py`.
- `plan/` is the plan's core, and imports neither `plan_store` nor `release_plan`. It holds an item and the items table's format (`todo_table.py`), releases (`release.py`), the ranking (`ranking.py`, with its score formula in `formula.py` and its blocker order in `blockers.py`), the paths an item's text names (`named_paths.py`) and `citations.py`, which holds `todo.md` and `docs/decisions.md` to each other. `goals_file.py` reads the goals file's sections, goals and goal weights, for the ranking and for `loop_claude`'s premises.
- `plan_store/` reads and writes a project's plan through the store the project's config names (`stores.py`). Every backend meets `plans.py`'s `PlanStore`: a markdown file (`todo_md.py`, which also ranks and checks the file in place, with its text edits in `markdown_edits.py`) or a Jira epic (`jira_epic.py`). `scoring.py` reads the goal weights that both backends rank by. `contract.py` holds the tests every backend that writes must pass, which `jira_epic.py` doesn't yet, since it changes no link, and `fakes.py` a plan in memory.
- `plan_store/` reads and writes a project's plan through the store the project's config names (`stores.py`). Every backend meets `plans.py`'s `PlanStore`: a markdown file (`todo_md.py`, which also ranks and checks the file in place, with its text edits in `markdown_edits.py`) or a Jira epic (`jira_epic.py`). `scoring.py` reads the goal weights that both backends rank by. `contract.py` holds the tests every backend that writes must pass; `jira_epic.py` doesn't yet, and its own tests stand in for Jira. `fakes.py` holds a plan in memory.
- `rank_todo.py`, `settings_filter.py` and `statusline.py` are tools of one file each, tested beside them.
- `record_store/` reads and writes a project's records, each through the store the project's config names (`stores.py`): a markdown file, a Confluence Cloud page (`confluence_page.py`), or, for the goals, a Jira epic's description (`jira_description.py`). `contract.py` holds the tests every backend must pass, and `fakes.py` a store in memory that the tests of other tools use.
- `release_plan/` says which release is next and whether it is ready to cut (`release_standing.py`), from the plan, the changelog and the tags, and picks the item the next chunk takes first (`next_chunk.py`), for `chunk_preflight` and `loop_claude`.
@@ -221,7 +230,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `routing_figures/` measures loop-claude's archive, and each of its projects' default branches, for loop-claude's routing calls.
- `rung_replay/` holds what `judge_replay` and `review_replay` share: the parallel replay runner, the argument reader, the candidate rungs and the exits, and `judge_replay`'s rung search. It reads `loop_claude`'s retry count and models file, and finds that file through `loop_state`'s paths. Its `archive.py` keeps each replay's runs and its report in a `replays/` folder of loop-claude's archive.
- `test_env.py` serves the tests.
- `text_files/` reads a Markdown file's `##` sections, quotes a passage and joins names into a list, stamps times in UTC, and replaces a file in one step, private where it is new.
- `text_files/` reads a Markdown file's `##` sections, quotes a passage and joins names into a list, stamps times in UTC, replaces a file in one step, private where it is new, and gzips a file beside its plain name, by which it reads it afterwards.
- `transcript_archive/` stores loop-claude's transcripts. Its `project_key.py` names each project's archive folder, loop-claude session and models-file entry.
## Test
+2 -2
View File
@@ -3,8 +3,8 @@
It runs @larvit/adf-codec, a Node package, in a throwaway container that holds node_shim/shim.mjs; the shim's header says what its exit status means.
adf-codec has two pairs of calls, and each function here names the one it runs:
- The lossless pair, `*_lossless`, for a document that already exists in Jira or Confluence and is edited and saved back. It keeps each block's localId and the empty paragraphs Jira uses for spacing, so a document read and saved unedited comes back identical.
- The portable pair, with no suffix, for new text written from scratch, and for markdown shown to a person. It drops the localId and the empty paragraphs, so saving a document that exists changes it.
- The portable pair, with no suffix, which the loop's tools use. It drops each block's localId and the empty paragraphs Jira uses for spacing, so saving a document read by it changes the document; a tool that saves one unedited sends nothing instead (`docs/decisions.md` → `project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, …`).
- The lossless pair, `*_lossless`, which keeps both, so a document read and saved unedited comes back identical. Its markdown wraps every block in `!adf:` marker lines, so no tool reads by it yet.
"""
import contextlib
import dataclasses
+5 -2
View File
@@ -1,8 +1,11 @@
import io
import json
import pathlib
import typing
from claude_code.entries import AssistantTurn, Entry, HumanTurn, LoopPrompt, parse_entry, Text, ToolCall
from failure import Failure
from text_files.compressed import read_stored
JUDGED_TOOL_INPUT = 400
@@ -14,9 +17,9 @@ class Typed(typing.NamedTuple):
def read_transcript(path: str) -> list[Entry] | Failure:
"""The entries of the transcript at path, or of its gzip where compress-archives gzipped it."""
try:
with open(path, "rb") as handle:
return [entry for raw in handle if (entry := parse_entry(raw))]
return [entry for raw in io.BytesIO(read_stored(pathlib.Path(path))) if (entry := parse_entry(raw))]
except OSError as failure:
return Failure(f"could not read the transcript {path}: {failure.strerror}")
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env python3
import pathlib
import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from compress_archives import main
sys.exit(main.main(sys.argv[1:]))
View File
+65
View File
@@ -0,0 +1,65 @@
"""Gzip each file of loop-claude's archive and the review folders that no tool changes any more, keeping all of them.
usage: compress-archives
It gzips each file that last changed over 30 days ago: in loop-claude's archive,
$LOOP_CLAUDE_TRANSCRIPTS (default ~/.claude/loop-claude-transcripts), each transcript, *.jsonl;
in the review folders, ~/.local/state/larv-review, every file. A file's last change is the later
of its mtime and ctime, so a copied or unpacked file counts from when it landed.
The gzip lands beside the file as <name>.gz, with the same times. The file is removed only once
its gzip reads back the same bytes. The archive's readers and review-replay read a gzipped file by
its plain name. README.md → Install has the crontab line that runs it daily.
It prints how many files it compressed. It exits 1, saying why, on any argument but --help or -h,
and at the first file it cannot compress, leaving that file and every one after it plain.
"""
import os
import pathlib
import stat
import sys
import time
import typing
from transcript_archive.location import archive_root
from text_files.compressed import compress, SUFFIX
from failure import Failure
from review_rules.review_folder import REPORTS
# Why 30 days: docs/decisions.md → `archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them`.
AGE_DAYS = 30
def main(argv: list[str], env: typing.Mapping[str, str] = os.environ, home: pathlib.Path | None = None, now: float | None = None) -> int:
if argv in (["-h"], ["--help"]):
print(__doc__.strip())
return 0
if argv:
print(f"unknown arguments {' '.join(argv)}; usage: compress-archives", file=sys.stderr)
return 1
home = home or pathlib.Path.home()
cutoff = (time.time() if now is None else now) - AGE_DAYS * 24 * 60 * 60
count = 0
for path in [*old_files(archive_root(home, env), "*.jsonl", cutoff), *old_files(home / REPORTS, "*", cutoff)]:
if failed := compress(path):
print(failed.reason, file=sys.stderr)
return 1
count += 1
print(f"Compressed {count} files.")
return 0
def old_files(root: pathlib.Path, pattern: str, cutoff: float) -> list[pathlib.Path]:
"""Each plain file under root that pattern matches and that last changed before the cutoff, sorted; a link is left alone, since its target lies elsewhere."""
found = []
for path in root.rglob(pattern):
if path.name.endswith(SUFFIX) or path.is_symlink():
continue
try:
status = path.stat()
# Removed while the walk ran, such as a file of review-replay's copy of a review folder.
except FileNotFoundError:
continue
if stat.S_ISREG(status.st_mode) and max(status.st_mtime, status.st_ctime) < cutoff:
found.append(path)
return sorted(found)
@@ -0,0 +1,126 @@
import contextlib
import gzip
import io
from unittest import mock
import os
import pathlib
import sys
import tempfile
import time
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from failure import Failure
from compress_archives import main
DAY = 24 * 60 * 60
# A month ahead, so a file's ctime, which is when the test wrote it, lies over 30 days before NOW.
NOW = time.time() + 31 * DAY
OLD = 30 * DAY + 60
NEW = 30 * DAY - 60
class Main(unittest.TestCase):
"""A home whose archive and review folders each hold files that last changed a minute over 30 days ago, and one that changed a minute under."""
def setUp(self):
self.home = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory()))
self.archive = self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1"
self.reviews = self.home / ".local" / "state" / "larv-review" / "code" / "proj-item-1"
for folder in (self.archive / "session" / "s" / "subagents", self.reviews / "probe"):
folder.mkdir(parents=True)
self.write(self.archive / "a.jsonl", OLD)
self.write(self.archive / "a.outcomes.md", OLD)
self.write(self.archive / "session" / "s" / "subagents" / "agent-a.jsonl", OLD)
self.write(self.archive / "session" / "s" / "subagents" / "agent-a.meta.json", OLD)
self.write(self.archive / "b.jsonl", NEW)
self.write(self.reviews / "stability-round-1.md", OLD)
self.write(self.reviews / "probe" / "out.js", OLD)
self.write(self.reviews / "branch-1.diff", NEW)
def write(self, path, seconds_ago):
path.write_bytes(f"{path.name}\n".encode())
os.utime(path, (NOW - seconds_ago, NOW - seconds_ago))
def run_main(self, argv=(), env=None, now=NOW):
out, err = io.StringIO(), io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main(list(argv), env or {}, self.home, now)
return code, out.getvalue(), err.getvalue()
def kept(self):
return sorted(str(path.relative_to(self.home)) for path in self.home.rglob("*") if path.is_file())
def test_gzips_the_archives_transcripts_and_every_review_file_last_written_over_30_days_ago(self):
code, out, err = self.run_main()
self.assertEqual((code, err), (0, ""))
self.assertEqual(self.kept(), [
".claude/loop-claude-transcripts/proj/c1/a.jsonl.gz",
".claude/loop-claude-transcripts/proj/c1/a.outcomes.md",
".claude/loop-claude-transcripts/proj/c1/b.jsonl",
".claude/loop-claude-transcripts/proj/c1/session/s/subagents/agent-a.jsonl.gz",
".claude/loop-claude-transcripts/proj/c1/session/s/subagents/agent-a.meta.json",
".local/state/larv-review/code/proj-item-1/branch-1.diff",
".local/state/larv-review/code/proj-item-1/probe/out.js.gz",
".local/state/larv-review/code/proj-item-1/stability-round-1.md.gz",
])
self.assertEqual(gzip.decompress((self.archive / "a.jsonl.gz").read_bytes()), b"a.jsonl\n")
self.assertEqual(out, "Compressed 4 files.\n")
self.assertEqual(self.run_main(), (0, "Compressed 0 files.\n", ""))
def test_keeps_a_file_last_written_exactly_30_days_ago_plain(self):
self.write(self.archive / "b.jsonl", 30 * DAY)
self.run_main()
self.assertTrue((self.archive / "b.jsonl").is_file())
def test_keeps_a_file_copied_today_with_its_old_times_plain(self):
now = time.time()
os.utime(self.archive / "a.jsonl", (now - 31 * DAY, now - 31 * DAY))
self.assertEqual(self.run_main(now=now), (0, "Compressed 0 files.\n", ""))
self.assertTrue((self.archive / "a.jsonl").is_file())
def test_measures_the_30_days_from_the_clock_by_default(self):
with mock.patch.object(main.time, "time", return_value=NOW):
self.assertEqual(self.run_main(now=None), (0, "Compressed 4 files.\n", ""))
def test_walks_on_past_a_gzip_a_link_and_a_file_removed_while_it_walks(self):
gone = mock.Mock(**{"is_symlink.return_value": False, "stat.side_effect": FileNotFoundError})
# Mock(name=...) names the mock itself, so the path's name is set after.
gone.name = "gone.jsonl"
packed = self.archive / "c.jsonl.gz"
packed.write_bytes(b"")
link = self.archive / "d.jsonl"
link.symlink_to(self.archive / "a.jsonl")
kept = self.archive / "a.jsonl"
root = mock.Mock(**{"rglob.return_value": [packed, link, gone, kept]})
self.assertEqual(main.old_files(root, "*.jsonl", NOW), [kept])
def test_reads_the_archive_where_loop_claude_transcripts_puts_it(self):
moved = self.home / "elsewhere"
(self.home / ".claude" / "loop-claude-transcripts").rename(moved)
code, _, err = self.run_main(env={"LOOP_CLAUDE_TRANSCRIPTS": str(moved)})
self.assertEqual((code, err), (0, ""))
self.assertTrue((moved / "proj" / "c1" / "a.jsonl.gz").is_file())
def test_runs_where_neither_folder_exists_yet(self):
with tempfile.TemporaryDirectory() as empty:
self.home = pathlib.Path(empty)
self.assertEqual(self.run_main(), (0, "Compressed 0 files.\n", ""))
def test_stops_at_the_first_failure_keeping_that_file_and_every_later_one(self):
with mock.patch.object(main, "compress", return_value=Failure("could not compress x: full")):
code, out, err = self.run_main()
self.assertEqual((code, out, err), (1, "", "could not compress x: full\n"))
self.assertTrue((self.archive / "a.jsonl").is_file())
def test_prints_its_help_and_refuses_other_arguments(self):
for flag in ("--help", "-h"):
code, out, _ = self.run_main([flag])
self.assertEqual(code, 0)
self.assertTrue(out.startswith("Gzip each file"), out)
self.assertEqual(self.run_main(["x", "y"]), (1, "", "unknown arguments x y; usage: compress-archives\n"))
if __name__ == "__main__":
unittest.main()
+8
View File
@@ -12,6 +12,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from transcript_archive.outcomes import ANSWERED, FINISHED, INTERRUPTED, UNFINISHED
from claude_code.entries import AssistantTurn, HumanTurn, LoopPrompt, Text
from judge_replay.replay import Case, case_from, report, reproduced, sample, verdict_kind
from text_files import compressed
from project_config.test_project_config import configured
from loop_effects.effects import Effects
from failure import Failure
@@ -233,6 +234,13 @@ class Main(unittest.TestCase):
self.assertEqual(code, 0)
self.assertIn("replayed failed: could not read the transcript", out)
def test_replays_a_verdict_whose_transcript_is_compressed(self):
self.assertIsNone(compressed.compress(self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1" / "t0.jsonl"))
code, out, err = self.run_main(["claude-opus-5-5"], lambda rung, interrupted: self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}"))
self.assertEqual(code, 0, err)
self.assertNotIn("replayed failed", out)
self.assertIn("The reference judge, claude-opus-5-5 at xhigh effort, reproduced 2.", out)
def test_tries_the_projects_rung_first_and_retries_a_failed_reply_as_the_loop_does(self):
def reply(rung, interrupted):
return self.verdict(interrupted, "not in the transcript" if rung.model != "claude-opus-5-5" else f"said {INTERRUPTED if interrupted else FINISHED}")
+99 -31
View File
@@ -2,7 +2,7 @@
An item is an open child issue. Its ID is the issue's number, so every child must be in the epic's own Jira project: ABC-34 is item 34 of the plan under ABC-12. The summary is the tagline, the description the detail, the earliest Fix Version the release, and an "is blocked by" link a blocker. L, E, I, S, Moves and Exempt sit in one issue property, PROPERTY, which needs no Jira admin to set up; an issue a person filed holds none, and reads as an item without scores.
`add` files an item as a new child issue, marked with FILED_BY, and ranks it among the issues the loop filed, by the score the config's formula gives, as `placing` says. `finish` moves an item's issue to a done status. `edit` changes an issue the loop filed, but never its links. Of an issue a person filed it sets only the scores, with a comment that tells the team why."""
`add` files an item as a new child issue, marked with FILED_BY, and ranks it among the issues the loop filed, by the score the config's formula gives, as `placing` says. `finish` moves an item's issue to a done status. `edit` changes an issue the loop filed, and its links to the epic's other open items. Of an issue a person filed it sets only the scores, with a comment that tells the team why."""
import dataclasses
import pathlib
import re
@@ -50,6 +50,15 @@ class ReadsScoring(typing.Protocol):
"""The scoring `add` scores an item by: the config's formula and bar, with the goal weights that the goals file, the decision log and the comprehension history give."""
@dataclasses.dataclass(frozen=True)
class Link:
"""One "is blocked by" link of an issue."""
# The key of the issue that blocks.
blocker: str
# The link's own id, which Jira deletes it by.
id: str
@dataclasses.dataclass(frozen=True)
class Issue:
"""A child issue as Jira answered it, checked to hold what an item is read from."""
@@ -57,8 +66,8 @@ class Issue:
summary: str
description: dict[str, typing.Any] | None
fix_versions: tuple[str, ...]
# The key of each issue this one "is blocked by".
blockers: tuple[str, ...]
# The issue's "is blocked by" links, one per blocker unless Jira holds a link twice.
links: tuple[Link, ...]
# PROPERTY's value; an empty object where the issue holds none.
scores: dict[str, typing.Any]
@@ -66,6 +75,10 @@ class Issue:
def number(self) -> int:
return number(self.key)
@property
def blockers(self) -> tuple[str, ...]:
return tuple(link.blocker for link in self.links)
@dataclasses.dataclass(frozen=True)
class JiraPlan:
@@ -124,10 +137,8 @@ class JiraPlan:
answered = children(self.epic, authorization, self.http)
if isinstance(answered, Failure):
return answered
held = {raw.get("key") for raw in answered if isinstance(raw, dict)}
# `item` reads only an open child as a blocker, so a link to any other issue would be lost on the next read.
if missing := next((blocker for blocker in filing.blockers if blocker not in held), None):
return Failure(f"{self.label} holds no item {number(missing)} to be blocked by")
if missing := missing_blocker(filing.blockers, keys_of(answered), self.label):
return missing
before = placing(answered, filing.item, value, scoring)
created = self.http("POST", f"{self.epic.site}/rest/api/3/issue", authorization, {"fields": issue_fields | {"fixVersions": [{"name": version}]}, "properties": [{"key": PROPERTY, "value": filing.scores}]})
if isinstance(created, Failure):
@@ -141,9 +152,7 @@ class JiraPlan:
"""Links the filed issue named key as blocked by each of blockers, and ranks it just before the issue named before; a note for each Jira refuses."""
notes = []
for blocker in blockers:
# Jira names the two ends by the link type's sides: the inward issue blocks the outward one, which then reads "is blocked by" it.
linked = self.http("POST", f"{self.epic.site}/rest/api/3/issueLink", authorization, {"type": {"name": BLOCKS}, "inwardIssue": {"key": blocker}, "outwardIssue": {"key": key}})
if isinstance(linked, Failure):
if isinstance(linked := self.link(key, blocker, authorization), Failure):
notes.append(f"filed {key}, but cannot link it as blocked by {blocker}: {linked.reason}; link it in Jira")
if isinstance(before, Failure):
notes.append(f"filed {key}, but cannot rank it: {before.reason}; rank it in Jira")
@@ -151,6 +160,24 @@ class JiraPlan:
notes.append(f"filed {key}, but cannot rank it before {before}: {refused}; rank it in Jira")
return tuple(notes)
def link(self, key: str, blocker: str, authorization: str) -> object:
"""Links the issue named key as blocked by the issue named blocker; a Failure where Jira refuses."""
# Jira names the two ends by the link type's sides: the inward issue blocks the outward one, which then reads "is blocked by" it.
return self.http("POST", f"{self.epic.site}/rest/api/3/issueLink", authorization, {"type": {"name": BLOCKS}, "inwardIssue": {"key": blocker}, "outwardIssue": {"key": key}})
def relinked(self, issue: Issue, wanted: tuple[str, ...] | None, open_keys: set[object], authorization: str) -> tuple[str, ...]:
"""Makes issue blocked by exactly the open children named in wanted: a link for each that it lacks, and a deletion of each link to an open child that wanted drops. A link to any other issue stays, since the team may have made it. None for wanted leaves every link as it is. A note for each request Jira refuses."""
if wanted is None:
return ()
notes = []
for blocker in (blocker for blocker in wanted if blocker not in issue.blockers):
if isinstance(linked := self.link(issue.key, blocker, authorization), Failure):
notes.append(f"edited {issue.key}, but cannot link it as blocked by {blocker}: {linked.reason}; link it in Jira")
for link in (link for link in issue.links if link.blocker in open_keys and link.blocker not in wanted):
if isinstance(deleted := self.http("DELETE", f"{self.epic.site}/rest/api/3/issueLink/{link.id}", authorization, None), Failure):
notes.append(f"edited {issue.key}, but cannot delete the link that makes it blocked by {link.blocker}: {deleted.reason}; delete it in Jira")
return tuple(notes)
def rank_refusal(self, key: str, before: str, authorization: str) -> str | None:
"""Ranks the issue named key just before the issue named before; why Jira refused, or None where it didn't. Jira answers a rank it made with no body, and one it refused with an error, or with an entry per issue that holds the errors."""
answer = self.http("PUT", f"{self.epic.site}/rest/agile/1.0/issue/rank", authorization, {"issues": [key], "rankBeforeIssue": before})
@@ -172,8 +199,10 @@ class JiraPlan:
def child(self, id_: int, authorization: str) -> dict[str, typing.Any] | Failure:
"""Item id_'s issue, as the epic's open children answer it; a Failure where none of them is item id_."""
answered = children(self.epic, authorization, self.http)
if isinstance(answered, Failure):
return answered
return answered if isinstance(answered, Failure) else self.child_in(answered, id_)
def child_in(self, answered: list[object], id_: int) -> dict[str, typing.Any] | Failure:
"""Item id_'s issue among the epic's open children; a Failure where none of them is item id_."""
key = f"{project_of(self.epic.key)}-{id_}"
found = next((raw for raw in answered if isinstance(raw, dict) and raw.get("key") == key), None)
return Failure(no_item(self.label, id_)) if found is None else found
@@ -185,14 +214,24 @@ class JiraPlan:
description = self.to_adf(detail)
return Failure(f"cannot convert the item's detail to a Jira description: {description.reason}") if isinstance(description, Failure) else description
def edit(self, id_: int, fields: dict[str, str], detail: str | None, comment: str | None = None) -> Written | Failure:
"""Sets the fields and the detail in one request, so Jira takes all of the edit or none, and then posts comment on the issue. `revision_of` says what an edit may set.
def shows(self, issue: Issue, detail: str) -> bool:
"""Whether detail is the issue's description as a read shows it. An edit leaves such a description as Jira holds it, since converting the markdown back would drop each block's localId and Jira's spacing paragraphs."""
if issue.description is None:
return not detail.strip()
texts = self.to_markdown([issue.description])
return isinstance(texts, list) and texts[0].strip() == detail.strip()
A comment Jira refuses leaves the edit made, so the write counts as landed, and a note says to post the comment by hand."""
def edit(self, id_: int, fields: dict[str, str], detail: str | None, comment: str | None = None) -> Written | Failure:
"""Sets the fields and the detail in one request, so Jira takes all of those or none, then changes the issue's links one request at a time, and then posts comment on the issue. `revision_of` says what an edit may set.
A link change or a comment Jira refuses leaves the edit made, so the write counts as landed, and a note says what to do by hand."""
authorization = basic_auth(self.env_file)
if isinstance(authorization, Failure):
return authorization
raw = self.child(id_, authorization)
answered = children(self.epic, authorization, self.http)
if isinstance(answered, Failure):
return answered
raw = self.child_in(answered, id_)
if isinstance(raw, Failure):
return raw
issue = issue_of(raw)
@@ -202,8 +241,13 @@ class JiraPlan:
revision = revision_of(issue, cells(fields), detail, comment)
if isinstance(revision, Failure):
return revision
open_keys = keys_of(answered)
if revision.blockers is not None and (missing := missing_blocker(revision.blockers, open_keys, self.label)):
return missing
if revision.blockers is not None and (looped := blocking_loop(answered, issue, revision.blockers, self.label)):
return looped
body = revision.body
if detail is not None:
if detail is not None and not self.shows(issue, detail):
description = self.description_of(detail)
if isinstance(description, Failure):
return description
@@ -217,13 +261,14 @@ class JiraPlan:
return version
body = body | {"fields": body.get("fields", {}) | {"fixVersions": [{"name": version}]}}
url = f"{self.epic.site}/rest/api/3/issue/{key}"
edited = self.http("PUT", url, authorization, body)
edited = self.http("PUT", url, authorization, body) if body else None
if isinstance(edited, Failure):
return Failure(f"cannot edit {key}: {edited.reason}; {PERHAPS_EDITED.format(key=key)}")
notes = self.relinked(issue, revision.blockers, open_keys, authorization)
if said is None:
return Written(id_)
return Written(id_, notes)
posted = self.http("POST", f"{url}/comment", authorization, {"body": said})
return Written(id_, (f"edited {key}, but cannot post the comment: {posted.reason}; post it in Jira",) if isinstance(posted, Failure) else ())
return Written(id_, notes + ((f"edited {key}, but cannot post the comment: {posted.reason}; post it in Jira",) if isinstance(posted, Failure) else ()))
def finish(self, id_: int) -> Written | Failure:
"""Moves item id_'s issue by the transition that the config names, to the status it names. Any open child of the epic may move, whoever filed it."""
@@ -324,18 +369,18 @@ class Revision:
body: dict[str, typing.Any]
# The release to move the issue to; None where the edit names none.
release: str | None
# The key of each issue the edit makes the issue blocked by; None where the edit names no Blocked by.
blockers: tuple[str, ...] | None
def revision_of(issue: Issue, named: dict[str, str], detail: str | None, comment: str | None) -> Revision | Failure:
"""The edit of issue that sets the cells in named, and the detail unless None, or why it cannot be made.
The summary, description and Fix Version of an issue a person filed are the team's, so there an edit may set only SCORED. Jira's screens don't show PROPERTY, so such an edit needs a comment, which tells the team. L, E, I and S are set together, so no issue is left partly scored. An issue the loop filed also keeps the label SECURITY in step with its Exempt. No edit changes a link."""
The summary, description, Fix Version and links of an issue a person filed are the team's, so of such an issue an edit may set only SCORED. Jira's screens don't show PROPERTY, so such an edit needs a comment, which tells the team. L, E, I and S are set together, so no issue is left partly scored. An issue the loop filed also keeps the label SECURITY in step with its Exempt. Only an edit of an issue the loop filed may name the blockers, whose links `JiraPlan.relinked` changes; each must be an open child, which the caller checks."""
held = issue.scores
if BLOCKED_BY in named:
return Failure(f"the loop doesn't change an issue's links: link {issue.key} in Jira")
own = held.get(FILED_BY[0]) == FILED_BY[1]
if not own and ("Item" in named or "Release" in named or detail is not None):
return Failure(f"a person filed {issue.key}, so the loop sets only its scores: change its summary, description and Fix Version in Jira")
if not own and ("Item" in named or "Release" in named or BLOCKED_BY in named or detail is not None):
return Failure(f"a person filed {issue.key}, so the loop sets only its scores: change its summary, description, links and Fix Version in Jira")
if not own and not (comment and comment.strip()):
return Failure(f"a person filed {issue.key}, and Jira doesn't show its team the scores: pass a comment saying that it is scored, and why")
body: dict[str, typing.Any] = {}
@@ -347,7 +392,7 @@ def revision_of(issue: Issue, named: dict[str, str], detail: str | None, comment
if unfit := unfit_text(named.get("Item", ""), detail or ""):
return unfit
release = named.get("Release")
if isinstance(checked := item_fields({name: named[name] for name in ("Release", "Exempt") if name in named}, f"item {issue.key}"), Failure):
if isinstance(checked := item_fields({name: named[name] for name in ("Release", "Exempt", BLOCKED_BY) if name in named}, f"item {issue.key}", id_=issue.number), Failure):
return checked
if any(name in named for name in SCORED):
texts = {name: str(value) for name, value in held.items()} | {name: named[name] for name in SCORED if name in named}
@@ -360,7 +405,7 @@ def revision_of(issue: Issue, named: dict[str, str], detail: str | None, comment
body["properties"] = [{"key": PROPERTY, "value": value}]
if own and (scored.exempt == SECURITY) != (held.get("Exempt") == SECURITY):
body["update"] = {"labels": [{"add" if scored.exempt == SECURITY else "remove": SECURITY}]}
return Revision(body, release)
return Revision(body, release, tuple(dict.fromkeys(f"{project_of(issue.key)}-{blocker}" for blocker in checked.blocked_by)) if BLOCKED_BY in named else None)
def offer_of(raw: object) -> tuple[str, str, str, str] | None:
@@ -431,19 +476,42 @@ def issue_of(raw: object) -> Issue | Failure:
# The answer's nested fields are read as Jira documents them; one missing or of another type ends the read here, as a Failure, so no later step meets it.
try:
fields = raw["fields"]
blockers = tuple(link["inwardIssue"]["key"] for link in fields["issuelinks"] if "inwardIssue" in link and link["type"]["inward"] == "is blocked by")
links = tuple(Link(link["inwardIssue"]["key"], link["id"]) for link in fields["issuelinks"] if "inwardIssue" in link and link["type"]["inward"] == "is blocked by")
held = raw.get("properties", {}).get(PROPERTY, {})
summary, description, versions = fields["summary"].strip(), fields["description"], tuple(version["name"] for version in fields["fixVersions"])
except (AttributeError, KeyError, TypeError) as failure:
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}")
if not isinstance(held, dict):
return Failure(f"{key}'s issue property {PROPERTY} is not a JSON object")
issue = Issue(key, summary, description, versions, blockers, held)
if not isinstance(issue.description, dict | None) or not all(isinstance(name, str) for name in issue.fix_versions + issue.blockers) or not all(ISSUE_KEY.fullmatch(blocker) for blocker in issue.blockers):
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: its description, a Fix Version's name or a linked issue's key holds another kind of value")
issue = Issue(key, summary, description, versions, links, held)
if not isinstance(issue.description, dict | None) or not all(isinstance(name, str) for name in issue.fix_versions + issue.blockers) or not all(isinstance(link.id, str) for link in links) or not all(ISSUE_KEY.fullmatch(blocker) for blocker in issue.blockers):
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: its description, a Fix Version's name or a link's id or linked issue's key holds another kind of value")
return issue
def keys_of(answered: list[object]) -> set[object]:
"""The key of each issue in the search's answer."""
return {raw.get("key") for raw in answered if isinstance(raw, dict)}
def missing_blocker(blockers: tuple[str, ...], held: set[object], label: str) -> Failure | None:
"""None where each of blockers is among the keys held, the plan's open children, else a Failure naming the first that isn't. `item` reads only an open child as a blocker, so a link to any other issue would be lost on the next read."""
missing = next((blocker for blocker in blockers if blocker not in held), None)
return None if missing is None else Failure(f"{label} holds no item {number(missing)} to be blocked by")
def blocking_loop(answered: list[object], issue: Issue, blockers: tuple[str, ...], label: str) -> Failure | None:
"""A Failure where issue, once blocked by blockers, which the caller has checked are open children, would block itself through the items its blockers wait for, and none of them could be taken; None otherwise. A loop the plan already holds without issue isn't this edit's."""
issues = {found.key: found for raw in answered if isinstance(found := issue_of(raw), Issue)}
waits_for = {found.key: found.blockers for found in issues.values()} | {issue.key: blockers}
reached: set[str] = set()
frontier = set(blockers)
while frontier:
reached |= frontier
frontier = {blocker for key in frontier for blocker in waits_for[key] if blocker in issues} - reached
return Failure(f"{label}: item {issue.number} would be blocked by itself through {', '.join(str(number(key)) for key in sorted(reached - {issue.key}, key=number))}; drop one of the blockers") if issue.key in reached else None
def plan_of(issues: list[Issue], epic: Epic, to_markdown: ToMarkdown) -> Plan | Failure:
"""The plan the issues hold, in their order."""
project = project_of(epic.key)
+1 -1
View File
@@ -29,7 +29,7 @@ class Written:
class PlanStore(typing.Protocol):
"""A project's plan. plan_store.contract holds every backend that writes to the same behaviour; `jira_epic` doesn't hold it yet: it changes no link, and its own tests stand in for Jira. A write that fails leaves the plan as it was, except where its Failure says the tracker may have taken the write all the same, as when Jira's answer to a new issue, an edit or a finish is lost."""
"""A project's plan. plan_store.contract holds every backend that writes to the same behaviour; `jira_epic` doesn't hold it yet: its own tests stand in for Jira. A write that fails leaves the plan as it was, except where its Failure says the tracker may have taken the write all the same, as when Jira's answer to a new issue, an edit or a finish is lost."""
@property
def label(self) -> str:
+131 -10
View File
@@ -22,9 +22,14 @@ SEARCH = "https://acme.atlassian.net/rest/api/3/search/jql"
AUTHORIZATION = "Basic " + base64.b64encode(b"me@example.com:s3cret").decode()
def link_id(blocked: str, blocker: str) -> str:
"""The id Jira gives the link that says blocked is blocked by blocker."""
return f"{blocked}<{blocker}"
def issue(key: str, summary: str = "Count the runs.", versions: tuple[str, ...] = ("0.1.0",), scores: dict[str, object] | None = None, blocked_by: tuple[str, ...] = (), description: object = None, blocks: tuple[str, ...] = ()) -> dict[str, object]:
links = [{"type": {"inward": "is blocked by", "outward": "blocks"}, "inwardIssue": {"key": blocker}} for blocker in blocked_by]
links += [{"type": {"inward": "is blocked by", "outward": "blocks"}, "outwardIssue": {"key": blocked}} for blocked in blocks]
links = [{"id": link_id(key, blocker), "type": {"inward": "is blocked by", "outward": "blocks"}, "inwardIssue": {"key": blocker}} for blocker in blocked_by]
links += [{"id": link_id(blocked, key), "type": {"inward": "is blocked by", "outward": "blocks"}, "outwardIssue": {"key": blocked}} for blocked in blocks]
return {
"key": key,
"fields": {"summary": summary, "description": description, "fixVersions": [{"name": name} for name in versions], "issuelinks": links},
@@ -171,7 +176,7 @@ class JiraPlan(unittest.TestCase):
def test_an_issue_in_another_shape_fails_the_read_naming_it(self):
good = issue("ABC-1")
for wrong in ({"key": "ABC-1"}, issue("ABC-1", summary=None), issue("ABC-1", description="text"), good | {"fields": good["fields"] | {"fixVersions": [{"name": 7}]}}, good | {"fields": good["fields"] | {"issuelinks": [{"type": {"inward": "is blocked by"}, "inwardIssue": {"key": 7}}]}}, good | {"fields": good["fields"] | {"issuelinks": [{"type": {"inward": "is blocked by"}, "inwardIssue": {"key": "ABC-x"}}]}}, good | {"properties": None}): # type: ignore[arg-type, operator]
for wrong in ({"key": "ABC-1"}, issue("ABC-1", summary=None), issue("ABC-1", description="text"), good | {"fields": good["fields"] | {"fixVersions": [{"name": 7}]}}, good | {"fields": good["fields"] | {"issuelinks": [{"id": "1", "type": {"inward": "is blocked by"}, "inwardIssue": {"key": 7}}]}}, good | {"fields": good["fields"] | {"issuelinks": [{"id": "1", "type": {"inward": "is blocked by"}, "inwardIssue": {"key": "ABC-x"}}]}}, good | {"fields": good["fields"] | {"issuelinks": [{"type": {"inward": "is blocked by"}, "inwardIssue": {"key": "ABC-2"}}]}}, good | {"fields": good["fields"] | {"issuelinks": [{"id": 7, "type": {"inward": "is blocked by"}, "inwardIssue": {"key": "ABC-2"}}]}}, good | {"properties": None}): # type: ignore[arg-type, operator]
with self.subTest(wrong=wrong):
failure = self.read(wrong) # type: ignore[arg-type]
self.assertIsInstance(failure, Failure)
@@ -487,24 +492,30 @@ class Editing(unittest.TestCase):
self.edited: object = None
self.commented: object = {"id": "10100"}
self.versions: object = [{"name": "0.1.0"}, {"name": "v0.2.0"}]
self.linked: object = None
self.unlinked: object = None
def http(self, method: str, url: str, authorization: str, body: object) -> object:
"""Stands in for Jira: answers the epic, self.versions for the project's Fix Versions, self.open to the search, self.edited to an edit and self.commented to a new comment."""
"""Stands in for Jira: answers the epic, self.versions for the project's Fix Versions, self.open to the search, self.edited to an edit, self.commented to a new comment, self.linked to a new link and self.unlinked to a deleted one."""
self.calls.append((method, url, body))
assert authorization == AUTHORIZATION
if method == "PUT":
return self.edited
if method == "DELETE":
return self.unlinked
if url == LINKS:
return self.linked
return self.commented if url.endswith("/comment") else {SEARCH: {"issues": self.open}, VERSIONS: self.versions}.get(url, {"key": "ABC-12"})
def to_adf(self, markdown: str) -> dict[str, object] | Failure:
return adf(markdown)
def edit(self, id_: int, fields: dict[str, str], detail: str | None = None, comment: str | None = None):
return jira_epic.JiraPlan(EPIC, self.env, self.http, text_of, self.to_adf, scoring=Stated()).edit(id_, fields, detail, comment)
return jira_epic.JiraPlan(EPIC, self.env, self.http, lambda documents: [text_of(document) for document in documents], self.to_adf, scoring=Stated()).edit(id_, fields, detail, comment)
def writes(self) -> list[tuple[str, str, object]]:
"""Each request that changes Jira, in order."""
return [call for call in self.calls if call[0] == "PUT" or call[1].endswith("/comment")]
return [call for call in self.calls if call[0] in ("PUT", "DELETE") or call[1].endswith(("/comment", "/issueLink"))]
def test_sets_the_summary_description_and_scores_of_an_issue_the_loop_filed_in_one_request(self):
self.assertEqual(self.edit(34, {"Tagline": "**Count every run.**", "L": "2", "Moves": ""}, "Why it counts."), Written(34))
@@ -518,7 +529,21 @@ class Editing(unittest.TestCase):
self.edit(34, {"E": "3"})
self.assertEqual([body for _, _, body in self.writes()], [{"fields": {"summary": "Count every run."}}, {"properties": [{"key": "plan-item", "value": PROPERTY | {"E": 3}}]}])
def test_a_detail_unchanged_since_the_read_leaves_the_description_as_jira_holds_it(self):
# Jira's own spacing paragraph and localIds, which a description converted back from markdown would lose.
held = {"type": "doc", "version": 1, "content": [{"type": "paragraph", "attrs": {"localId": "p-1"}, "content": [{"type": "text", "text": "Why it counts."}]}, {"type": "paragraph", "attrs": {"localId": "p-2"}}]}
self.open = [issue("ABC-34", scores=PROPERTY, description=held)]
self.assertEqual(self.edit(34, {"L": "2"}, "Why it counts.\n"), Written(34))
self.assertEqual(self.edit(34, {}, "Why it counts."), Written(34))
self.assertEqual(self.writes(), [("PUT", f"{ISSUES}/ABC-34", {"properties": [{"key": "plan-item", "value": PROPERTY | {"L": 2}}]})])
def test_an_edited_detail_replaces_the_description(self):
self.open = [issue("ABC-34", scores=PROPERTY, description=adf("Why it counts."))]
self.edit(34, {}, "Why it counts twice.")
self.assertEqual(self.writes(), [("PUT", f"{ISSUES}/ABC-34", {"fields": {"description": adf("Why it counts twice.")}})])
def test_an_empty_detail_removes_the_description(self):
self.open = [issue("ABC-34", scores=PROPERTY, description=adf("Why it counts."))]
self.edit(34, {}, " \n")
self.assertEqual(self.writes(), [("PUT", f"{ISSUES}/ABC-34", {"fields": {"description": None}})])
@@ -569,7 +594,7 @@ class Editing(unittest.TestCase):
def test_what_a_person_wrote_or_planned_is_never_edited(self):
for fields, detail in (({"Tagline": "Count every run."}, None), ({}, "Why."), ({}, ""), ({"Release": "0.2.0"}, None), ({"Tagline": "Count.", "Release": "0.2.0", "L": "1"}, "")):
with self.subTest(fields=fields, detail=detail):
self.assertEqual(self.edit(36, fields, detail, COMMENT), Failure("a person filed ABC-36, so the loop sets only its scores: change its summary, description and Fix Version in Jira"))
self.assertEqual(self.edit(36, fields, detail, COMMENT), Failure("a person filed ABC-36, so the loop sets only its scores: change its summary, description, links and Fix Version in Jira"))
self.assertEqual(self.writes(), [])
def test_scoring_an_unscored_issue_in_part_fails_before_any_write(self):
@@ -585,8 +610,96 @@ class Editing(unittest.TestCase):
self.commented = Failure("HTTP 403")
self.assertEqual(self.edit(36, {"L": "2"}, None, COMMENT), Written(36, ("edited ABC-36, but cannot post the comment: HTTP 403; post it in Jira",)))
def test_an_edit_of_the_blockers_fails_since_the_store_changes_no_link(self):
self.assertEqual(self.edit(34, {"Blocked by": "35"}), Failure("the loop doesn't change an issue's links: link ABC-34 in Jira"))
def test_links_each_new_blocker_to_an_issue_the_loop_filed(self):
self.assertEqual(self.edit(34, {"Blocked by": "35, 36"}), Written(34))
self.assertEqual(self.writes(), [("POST", LINKS, {"type": {"name": "Blocks"}, "inwardIssue": {"key": f"ABC-{blocker}"}, "outwardIssue": {"key": "ABC-34"}}) for blocker in (35, 36)])
def test_links_a_blocker_named_twice_once(self):
self.edit(34, {"Blocked by": "35, 35"})
self.assertEqual([body["inwardIssue"]["key"] for _, _, body in self.writes()], ["ABC-35"]) # type: ignore[index]
def test_deletes_the_link_of_each_blocker_the_edit_drops_and_makes_none_for_one_it_keeps(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-35", "ABC-36"))
self.assertEqual(self.edit(34, {"Blocked by": "35"}), Written(34))
self.assertEqual(self.writes(), [("DELETE", f"{LINKS}/{link_id('ABC-34', 'ABC-36')}", None)])
def test_an_empty_blocked_by_deletes_every_link_the_issue_holds(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-35", "ABC-36"), blocks=("ABC-35",))
self.assertEqual(self.edit(34, {"Blocked by": ""}), Written(34))
self.assertEqual(self.writes(), [("DELETE", f"{LINKS}/{link_id('ABC-34', blocker)}", None) for blocker in ("ABC-35", "ABC-36")])
def test_a_link_to_an_issue_that_is_no_open_child_is_never_deleted(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-35", "OPS-9", "ABC-99"))
self.assertEqual(self.edit(34, {"Blocked by": "36"}), Written(34))
self.assertEqual([(method, url) for method, url, _ in self.writes()], [("POST", LINKS), ("DELETE", f"{LINKS}/{link_id('ABC-34', 'ABC-35')}")])
def test_an_edit_that_names_no_blocked_by_leaves_the_links_as_they_are(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-35",))
self.edit(34, {"L": "2"})
self.assertEqual([method for method, _, _ in self.writes()], ["PUT"])
def test_a_blocked_by_the_issue_already_holds_writes_nothing(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-35", "ABC-36"))
self.assertEqual(self.edit(34, {"Blocked by": "36, 35"}), Written(34))
self.assertEqual(self.writes(), [])
def test_a_blocker_swapped_for_another_deletes_one_link_and_makes_one(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-35",))
self.edit(34, {"Blocked by": "36"})
self.assertEqual([(method, url) for method, url, _ in self.writes()], [("POST", LINKS), ("DELETE", f"{LINKS}/{link_id('ABC-34', 'ABC-35')}")])
def test_changes_the_links_after_the_edit_and_before_the_comment(self):
self.assertEqual(self.edit(34, {"L": "2", "Blocked by": "35"}, None, COMMENT), Written(34))
self.assertEqual([(method, url) for method, url, _ in self.writes()], [("PUT", f"{ISSUES}/ABC-34"), ("POST", LINKS), ("POST", f"{ISSUES}/ABC-34/comment")])
def test_a_failed_edit_makes_no_link(self):
self.edited = Failure("HTTP 400")
self.assertIsInstance(self.edit(34, {"L": "2", "Blocked by": "35"}), Failure)
self.assertEqual([method for method, _, _ in self.writes()], ["PUT"])
def test_a_link_jira_refuses_leaves_the_edit_made_with_a_note_and_the_other_links_changed(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-36",))
self.linked = Failure("HTTP 403")
self.assertEqual(self.edit(34, {"Blocked by": "35"}), Written(34, ("edited ABC-34, but cannot link it as blocked by ABC-35: HTTP 403; link it in Jira",)))
self.assertEqual([method for method, _, _ in self.writes()], ["POST", "DELETE"])
def test_a_link_jira_refuses_to_delete_leaves_a_note_and_the_comment_posted(self):
self.open[0] = issue("ABC-34", scores=PROPERTY, blocked_by=("ABC-35", "ABC-36"))
self.unlinked = Failure("HTTP 404")
self.assertEqual(self.edit(34, {"Blocked by": ""}, None, COMMENT), Written(34, ("edited ABC-34, but cannot delete the link that makes it blocked by ABC-35: HTTP 404; delete it in Jira", "edited ABC-34, but cannot delete the link that makes it blocked by ABC-36: HTTP 404; delete it in Jira")))
self.assertEqual(self.writes()[-1][1], f"{ISSUES}/ABC-34/comment")
def test_a_blocker_that_is_no_open_child_fails_before_any_write(self):
self.assertEqual(self.edit(34, {"L": "2", "Blocked by": "35, 40"}), Failure("jira:https://acme.atlassian.net/browse/ABC-12 holds no item 40 to be blocked by"))
self.assertEqual(self.writes(), [])
def test_a_blocker_that_makes_the_issue_wait_for_itself_fails_before_any_write(self):
self.open[1] = issue("ABC-35", scores=PROPERTY, blocked_by=("ABC-34",))
self.open[2] = issue("ABC-36", scores=PROPERTY, blocked_by=("ABC-35",))
self.assertEqual(self.edit(34, {"L": "2", "Blocked by": "36"}), Failure("jira:https://acme.atlassian.net/browse/ABC-12: item 34 would be blocked by itself through 35, 36; drop one of the blockers"))
self.assertEqual(self.writes(), [])
def test_the_loop_names_its_items_by_number_not_by_key(self):
self.open += [issue("ABC-9", scores=PROPERTY, blocked_by=("ABC-10",)), issue("ABC-10", scores=PROPERTY, blocked_by=("ABC-34",))]
failure = self.edit(34, {"Blocked by": "9"})
self.assertTrue(failure.reason.endswith("through 9, 10; drop one of the blockers")) # type: ignore[union-attr]
def test_a_loop_the_plan_holds_without_the_issue_does_not_stop_its_edit(self):
self.open[1] = issue("ABC-35", scores=PROPERTY, blocked_by=("ABC-36",))
self.open[2] = issue("ABC-36", scores=PROPERTY, blocked_by=("ABC-35",))
self.assertEqual(self.edit(34, {"Blocked by": "35"}), Written(34))
self.assertEqual([method for method, _, _ in self.writes()], ["POST"])
def test_a_blocked_by_out_of_format_or_naming_the_issue_itself_fails_before_any_write(self):
for cell in ("ABC-35", "35,36", "34"):
with self.subTest(cell=cell):
self.assertIsInstance(self.edit(34, {"Blocked by": cell}), Failure)
self.assertEqual(self.writes(), [])
def test_the_links_of_an_issue_a_person_filed_are_never_changed(self):
for id_ in (35, 36):
with self.subTest(id_=id_):
self.assertEqual(self.edit(id_, {"Blocked by": "34"}, None, COMMENT), Failure(f"a person filed ABC-{id_}, so the loop sets only its scores: change its summary, description, links and Fix Version in Jira"))
self.assertEqual(self.writes(), [])
def test_an_issue_that_is_no_open_child_of_the_epic_is_not_edited(self):
@@ -655,11 +768,14 @@ class Finishing(unittest.TestCase):
self.open = [issue("ABC-34"), issue("ABC-35")]
self.transitions: object = {"transitions": [transition("11", "Start", "In Progress", "indeterminate"), transition("31", "Mark as done", "Done")]}
self.moved: object = None
self.search_refused: Failure | None = None
def http(self, method: str, url: str, authorization: str, body: object) -> object:
"""Stands in for Jira: answers the epic, self.open to the search, self.transitions for ABC-34's transitions and self.moved to a transition of it."""
"""Stands in for Jira: answers the epic, self.open to the search, or self.search_refused where set, self.transitions for ABC-34's transitions and self.moved to a transition of it."""
self.calls.append((method, url, body))
assert authorization == AUTHORIZATION
if url == SEARCH and self.search_refused:
return self.search_refused
if url == TRANSITIONS:
return self.transitions if method == "GET" else self.moved
return {SEARCH: {"issues": self.open}}.get(url, {"key": "ABC-12"})
@@ -674,6 +790,11 @@ class Finishing(unittest.TestCase):
self.assertEqual(self.finish(), Written(34))
self.assertEqual(self.posts(), [("POST", TRANSITIONS, {"transition": {"id": "31"}})])
def test_a_search_jira_refuses_fails_the_finish_before_any_move(self):
self.search_refused = Failure("HTTP 500")
self.assertEqual(self.finish(), Failure("HTTP 500"))
self.assertEqual(self.posts(), [])
def test_a_config_that_names_no_transition_fails_before_any_request(self):
self.assertEqual(self.finish(by=None), Failure("the config names no transition to finish an item by: set `jira-finish-transition` and `jira-finish-status`, as `project-config --help` shows"))
self.assertEqual(self.calls, [])
+9
View File
@@ -16,6 +16,7 @@ from transcript_archive.write import archive
from claude_code.entries import AssistantTurn
from project_config.templates import BRANCH
from reading_cost.measure import Checkout, Reading
from text_files import compressed
from failure import Failure
from reading_cost import figures, main, measure
from transcript_archive.read import finished_chunks, FinishedChunk
@@ -173,6 +174,14 @@ class ReadingCostTool(unittest.TestCase):
self.assertEqual((ran.returncode, ran.stdout), (1, ""))
self.assertIn("no config names the checkout", ran.stderr)
def test_measures_a_chunk_whose_archived_transcripts_are_compressed(self):
self.archive_chunk(1790000000)
for copy in self.archive.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
ran = self.run_tool(self.checkout)
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertIn("33 new input tokens before the first commit.", (self.checkout / "docs" / "reading-cost.md").read_text())
def test_appends_each_finished_chunk_once_in_the_order_they_finished_from_a_worktree(self):
self.archive_chunk(1790000200)
self.archive_chunk(1790000100)
+23 -5
View File
@@ -2,6 +2,7 @@
A write never overwrites a teammate's edit: it sends the version after the one held() read, and Confluence refuses it where someone saved the page since. The human then decides (lilleman, 2026-11-12)."""
import dataclasses
import hashlib
import json
import pathlib
import typing
@@ -20,9 +21,10 @@ SHAPE = "not in the shape Confluence Cloud's REST API version 2 documents"
@dataclasses.dataclass(frozen=True)
class Seen:
"""The page's version number and title as a read found them or a write left them. A write needs both, so a Held's version holds both, as JSON."""
"""The page's version number and title as a read found them or a write left them, and the digest of the markdown it found or sent. A write needs the version and title, and sends nothing where its text has that digest, so a Held's version holds all three, as JSON. The digest is None where the version came from a request that held no text."""
version: int
title: str
digest: str | None = None
@property
def token(self) -> str:
@@ -64,7 +66,7 @@ class ConfluencePage:
if not isinstance(body, str):
return Failure(f"{self.label} is {SHAPE}: it holds no body in atlas_doc_format")
text = self.converted(body)
return text if isinstance(text, Failure) else Held(text, seen.token)
return text if isinstance(text, Failure) else Held(text, Seen(seen.version, seen.title, digest_of(text)).token)
def converted(self, held: str) -> str | Failure:
"""The markdown of the body held, the JSON of an ADF document; empty where held is."""
@@ -86,6 +88,14 @@ class ConfluencePage:
seen = self.seen_of(over)
if isinstance(seen, Failure):
return seen
# A read's markdown leaves out the localIds and spacing paragraphs Confluence holds, so a write of the same text back would drop them: send nothing. Without a write Confluence answers no 409, so read the version to refuse where a teammate saved since.
if over is not None and seen.digest == digest_of(text):
current = self.fetched("")
if isinstance(current, Failure):
return current
if current[0].version != seen.version:
return self.saved_since(seen.version, f"Confluence holds version {current[0].version}")
return Held(text, over)
document = self.to_adf(text)
if isinstance(document, Failure):
return Failure(f"cannot convert the text for {self.label} to ADF: {document.reason}")
@@ -95,10 +105,14 @@ class ConfluencePage:
body = {"id": self.page.page_id, "status": "current", "title": seen.title, "body": {"representation": "atlas_doc_format", "value": json.dumps(document)}, "version": {"number": seen.version + 1}}
answer = self.http("PUT", self.url, authorization, body)
if isinstance(answer, ErrorAnswer) and answer.status == CONFLICT:
return Failure(f"someone saved {self.label} since the loop read version {seen.version} of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do ({answer.reason})")
return self.saved_since(seen.version, answer.reason)
if isinstance(answer, Failure):
return Failure(f"cannot write {self.label}: {answer.reason}")
return Held(text, Seen(seen.version + 1, seen.title).token)
return Held(text, Seen(seen.version + 1, seen.title, digest_of(text)).token)
def saved_since(self, version: int, evidence: str) -> Failure:
"""The refusal of a write over version, which a teammate's save replaced, as evidence shows."""
return Failure(f"someone saved {self.label} since the loop read version {version} of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do ({evidence})")
def fetched(self, query: str) -> tuple[Seen, object] | Failure:
"""The page's version and title, and the value of its body in atlas_doc_format where query asks for it; None where it doesn't."""
@@ -124,6 +138,10 @@ class ConfluencePage:
return fetched if isinstance(fetched, Failure) else fetched[0]
try:
held: typing.Any = json.loads(over)
return Seen(int(held["version"]), str(held["title"]))
return Seen(int(held["version"]), str(held["title"]), held.get("digest"))
except (ValueError, KeyError, TypeError) as failure:
return Failure(f"cannot read the version of {self.label} to write over, {over!r}: {failure}")
def digest_of(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
+37 -5
View File
@@ -80,18 +80,50 @@ class ConfluencePageTest(ConfluenceCase):
store = self.new_store()
held = store.held()
assert isinstance(held, Held), held
self.assertEqual(store.write("- a decision\n", held.version), Held("- a decision\n", '{"version": 8, "title": "Decisions"}'))
written = store.write("- a decision\n", held.version)
assert isinstance(written, Held), written
self.assertEqual(written.text, "- a decision\n")
self.assertEqual(self.confluence.calls[-1], ("PUT", URL, AUTHORIZATION, {"id": PAGE.page_id, "status": "current", "title": "Decisions", "body": {"representation": "atlas_doc_format", "value": json.dumps(to_adf("- a decision\n"))}, "version": {"number": 8}}))
def test_saving_the_text_a_read_found_leaves_the_page_as_confluence_holds_it(self) -> None:
self.confluence.teammate_saves("- a decision\n")
value = self.confluence.value
held = self.new_store().held()
assert isinstance(held, Held), held
self.assertEqual(self.new_store().write(held.text, held.version), held)
self.assertNotIn("PUT", [call[0] for call in self.confluence.calls])
self.assertEqual(self.confluence.calls[-1], ("GET", URL, AUTHORIZATION, None))
self.assertEqual((self.confluence.version, self.confluence.value), (2, value))
def test_refuses_to_save_the_text_a_read_found_where_a_teammate_saved_since(self) -> None:
self.confluence.teammate_saves("- a decision\n")
held = self.new_store().held()
assert isinstance(held, Held), held
self.confluence.teammate_saves("- their decision\n")
written = self.new_store().write(held.text, held.version)
self.assertEqual(written, Failure(f"someone saved {PAGE.spelling} since the loop read version 2 of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do (Confluence holds version 3)"))
self.assertNotIn("PUT", [call[0] for call in self.confluence.calls])
self.assertEqual(self.new_store().read(), "- their decision\n")
def test_an_unedited_save_fails_where_the_pages_version_cannot_be_read(self) -> None:
held = self.new_store().held()
assert isinstance(held, Held), held
store = ConfluencePage(PAGE, self.env, lambda *_: ErrorAnswer("GET /wiki/api/v2/pages/2293761 answered 404: Not Found", 404), to_markdown, to_adf)
self.assertEqual(store.write(held.text, held.version), Failure(f"cannot read {PAGE.spelling}: GET /wiki/api/v2/pages/2293761 answered 404: Not Found"))
def test_saving_an_edited_text_writes_the_page(self) -> None:
self.confluence.teammate_saves("- a decision\n")
held = self.new_store().held()
assert isinstance(held, Held), held
self.assertIsInstance(self.new_store().write("- another decision\n", held.version), Held)
self.assertEqual(self.new_store().read(), "- another decision\n")
def test_refuses_to_overwrite_a_teammates_save_since_the_version_it_is_handed_and_says_to_ask_the_human(self) -> None:
held = self.new_store().held()
assert isinstance(held, Held), held
self.confluence.teammate_saves("their edit\n")
written = self.new_store().write("my edit\n", held.version)
assert isinstance(written, Failure), written
self.assertIn("someone saved", written.reason)
self.assertIn("ask the human", written.reason)
self.assertIn(PAGE.spelling, written.reason)
self.assertEqual(written, Failure(f"someone saved {PAGE.spelling} since the loop read version 1 of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do (PUT /wiki/api/v2/pages/{PAGE.page_id} answered 409: Version must be incremented on update. Current version is: 2)"))
self.assertEqual(self.new_store().read(), "their edit\n")
def test_keeps_no_version_itself_so_a_read_never_moves_the_version_a_write_names(self) -> None:
+8 -1
View File
@@ -16,6 +16,7 @@ from rung_replay.search import first_reply
from record_store.stores import local_file, store
from project_config.main import located
from review_rules.reviewer_context import principles_line, reviewer_context
from text_files.compressed import read_stored, SUFFIX
from review_rules.review_folder import REPORTS, review_folder
from model_routing.ladder import Rung
from claude_code.entries import structured_output
@@ -146,8 +147,14 @@ def copy_reports(found: Round, folder: pathlib.Path, reports: str) -> None:
"""Copies into reports each report written to folder after found's chunk began and before its round did."""
# The round's own report, any later one, and one from an earlier attempt at the item in the same worktree would each hand the reviewer the findings it is to find.
for written in folder.iterdir() if folder.is_dir() else ():
if written.is_file() and found.began <= written.stat().st_mtime < found.seconds:
if not written.is_file() or not found.began <= written.stat().st_mtime < found.seconds:
continue
name = written.name.removesuffix(SUFFIX)
if name == written.name:
shutil.copy2(written, reports)
# A gzipped report. Where a plain report of the same name stands beside it, that one is newer and the gzip is skipped; the plain report is copied only if it was written before the round.
elif not (folder / name).is_file():
(pathlib.Path(reports) / name).write_bytes(read_stored(folder / name))
def redirected(prompt: str, home: pathlib.Path, folder: pathlib.Path, worktree: str, copies: Copies) -> str | Failure:
+3 -2
View File
@@ -11,6 +11,7 @@ from transcript_archive.project_key import checkout_of
from failure import Failure
from claude_code.transcript import first_timestamped_line, read_transcript
from rung_replay.command import projects_on
from text_files.compressed import read_stored, stored_glob
# A line that opens with a 🔴 or 🟡 mark, after any list, table, heading or bold markup, as review-findings.md has reviewers report them.
FINDING = re.compile(r"^[\s#>*+|-]*(?:\d+[.)]\s*)?[*_]*\s*(?:🔴|🟡)")
@@ -126,8 +127,8 @@ def archived_rounds(reviewer: str, model: str, root: pathlib.Path, home: pathlib
def chunk_rounds(reviewer: str, checkout: str, chunk: pathlib.Path) -> list[Round]:
copies = sorted(chunk.glob("*.jsonl"))
began = parse_entry(first_timestamped_line(copies[0].read_bytes())) if copies else None
copies = stored_glob(chunk, "*.jsonl")
began = parse_entry(first_timestamped_line(read_stored(copies[0]))) if copies else None
if not began or not began.timestamp:
return []
found = []
+28 -3
View File
@@ -15,6 +15,7 @@ import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from review_replay.rounds import authored_by, lost_inputs, recorded_findings, reviewed_path, Round, round_from, sample, stamp_seconds
from text_files import compressed
from project_config.test_project_config import configured, write_config
from loop_effects.effects import Effects, system_effects
from claude_code.entries import entry_from
@@ -375,19 +376,43 @@ class Main(unittest.TestCase):
self.assertIn("claude-opus-5-5 at low effort found 0, 0%, and missed 1 such 🔴: it doesn't qualify.", out)
def review_folder_with_round_1(self):
"""The chunk's review folder, holding round 1's report from before the replayed round began at 10:00, that round's own and a later pass's reports, written after, and an earlier attempt's report, from before the chunk began at 09:15."""
"""The chunk's review folder. The chunk began at 09:15 and the replayed round 2 at 10:00. It holds round 1's report, written at 09:15:00; round 2's report, written after 10:00; the prose pass's report, written at 10:00:00; and an earlier attempt's report, from before 09:15."""
folder = self.home / ".local" / "state" / "larv-review" / "code" / "proj-x"
folder.mkdir(parents=True)
(folder / "stability-round-1.md").write_text("round 1\n")
before = stamp_seconds("2026-10-14T09:50:00Z")
os.utime(folder / "stability-round-1.md", (before, before))
began = stamp_seconds("2026-10-14T09:15:00Z")
os.utime(folder / "stability-round-1.md", (began, began))
(folder / "stability-round-2.md").write_text("🔴 a.txt:1 wrong\n")
(folder / "prose-round-1.md").write_text("later\n")
round_began = stamp_seconds("2026-10-14T10:00:00Z")
os.utime(folder / "prose-round-1.md", (round_began, round_began))
(folder / "stability-round-3.md").write_text("🔴 a.txt:1 wrong\n")
attempt = stamp_seconds("2026-10-14T09:00:00Z")
os.utime(folder / "stability-round-3.md", (attempt, attempt))
return folder
def test_replays_rounds_whose_archived_transcripts_are_compressed(self):
for copy in self.archive.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
code, out, err = self.run_main(["stability", "claude-opus-5-5"], lambda rung: {"found": [1] if rung == LOW else [0, 1]})
self.assertEqual(code, 0, err)
self.assertTrue(out.startswith("Replayed 1 stability rounds of claude-opus-5-5 projects,"), out)
def test_copies_a_compressed_report_from_before_the_round_under_its_plain_name(self):
folder = self.review_folder_with_round_1()
self.assertIsNone(compressed.compress(folder / "stability-round-1.md"))
self.set_prompt(f"Review branch x in {self.home}/code/proj-x. Write your report to ~/.local/state/larv-review/code/proj-x/stability-round-2.md; the last round's is ~/.local/state/larv-review/code/proj-x/stability-round-1.md.")
seen = []
def review(rung):
named = pathlib.Path(re.search(r"report to (\S+)/stability-round-2\.md", self.reviews[-1][4]).group(1))
seen.append((sorted(path.name for path in named.iterdir()), (named / "stability-round-1.md").read_text()))
return {"type": "result", "result": f"review on {rung.model} {rung.effort}"}
code, _, err = self.run_main(["stability", "claude-opus-5-5"], lambda rung: {"found": [0, 1]}, review)
self.assertEqual(code, 0, err)
self.assertEqual(seen[0], (["stability-round-1.md"], "round 1\n"))
def test_points_the_prompts_review_folder_at_a_copy_of_the_reports_from_before_the_round_that_it_removes(self):
folder = self.review_folder_with_round_1()
self.set_prompt(f"Review branch x in {self.home}/code/proj-x. Write your report to ~/.local/state/larv-review/code/proj-x/stability-round-2.md; the last round's is ~/.local/state/larv-review/code/proj-x/stability-round-1.md.")
+3 -2
View File
@@ -3,6 +3,7 @@ import contextlib
import dataclasses
import fcntl
import functools
import io
import json
import pathlib
import typing
@@ -15,6 +16,7 @@ from routing_figures.chunk import Chunk, chunk_figures, ChunkFigures, Subagent
from git_cli.git_repo import default_ref, Git
from claude_code.entries import Entry, parse_entry
from failure import Failure
from text_files.compressed import read_stored
from text_files.file_writes import replace_file
from project_config.main import title_templates
from project_config.templates import Titles
@@ -115,8 +117,7 @@ def branch_history(checkout: str, home: pathlib.Path, titles: Titles, git: Git)
def entries(path: pathlib.Path) -> list[Entry]:
with open(path, "rb") as handle:
return [entry for line in handle if (entry := parse_entry(line))]
return [entry for line in io.BytesIO(read_stored(path)) if (entry := parse_entry(line))]
def subagent(transcript: pathlib.Path) -> Subagent:
@@ -19,6 +19,7 @@ from project_config.templates import BARE, Titles
from branch_chunks.cut import BranchChunk
from routing_figures.calls import Call, calls, totals
from routing_figures.chunk import Chunk, chunk_figures, ChunkFigures, Reports, Subagent
from text_files import compressed
from project_config.test_project_config import configured, write_config
from routing_figures.state import due
from claude_code.entries import entry_from
@@ -352,6 +353,13 @@ class Archive(unittest.TestCase):
self.assertEqual(found.chunks[0].dollars["subagent type"], {"worker": 2.0, "Explore": 2.0})
self.assertEqual(archive.measure_archive(self.root, "/h", "2026-10-14T10:00:00Z", no_checkout), archive.ArchiveChunks([], ()))
def test_measures_a_chunk_whose_transcripts_are_compressed(self):
self.chunk("home/x", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [message("m1", "2026-10-14T09:00:00Z", call("Agent", "a1", subagent_type="Explore"))], ({"agentType": "Explore", "toolUseId": "a1"}, [message("s1", "2026-10-14T09:00:01Z")]))
for copy in self.root.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
found = archive.measure_archive(self.root, "/h", "", no_checkout)
self.assertEqual(found.chunks[0].dollars["subagent type"], {"worker": 2.0, "Explore": 2.0})
def test_counts_each_chunks_later_fixes_from_its_projects_checkout(self):
home = self.root / "home"
env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(home)}
+78
View File
@@ -0,0 +1,78 @@
"""A file kept plain, or gzipped beside its plain name as <name>.gz: reading it by its plain name, and compressing it.
Why gzip and when a file is compressed: docs/decisions.md → `archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them`.
"""
import errno
import gzip
import os
import pathlib
import zlib
from failure import Failure
from text_files.file_writes import replace_file
SUFFIX = ".gz"
def stored(path: pathlib.Path) -> pathlib.Path | None:
"""Where the file named path is kept: path itself, else its gzip; None where neither is."""
# A plain file beside its gzip was written after it was compressed, so it holds the newer bytes. Anything else at path, such as a folder, counts as path, so reading it fails naming path.
return next((kept for kept in (path, gzipped(path)) if kept.exists()), None)
def read_stored(path: pathlib.Path) -> bytes:
"""The bytes of the file named path, read from its gzip where it is kept that way. Raises OSError, FileNotFoundError where neither is kept."""
# Tried in turn rather than checked first: compress-archives may remove the plain file once its gzip is whole, between a check and the read.
try:
return path.read_bytes()
except FileNotFoundError:
pass
packed = gzipped(path)
try:
data = packed.read_bytes()
except FileNotFoundError:
raise FileNotFoundError(errno.ENOENT, "No such file or directory", str(path)) from None
try:
return gzip.decompress(data)
except (EOFError, zlib.error, gzip.BadGzipFile) as failure:
raise OSError(errno.EIO, f"not a whole gzip: {failure}", str(packed)) from failure
def stored_glob(folder: pathlib.Path, pattern: str) -> list[pathlib.Path]:
"""The plain name of each file in folder that pattern matches, whether kept plain or gzipped, sorted."""
return sorted({*folder.glob(pattern), *(path.with_name(path.name.removesuffix(SUFFIX)) for path in folder.glob(pattern + SUFFIX))})
def compress(path: pathlib.Path) -> Failure | None:
"""Replaces path with its gzip, which keeps path's times: writes the gzip, reads it back, and removes path only once it reads back path's bytes. A failure keeps path."""
packed = gzipped(path)
try:
before = path.stat()
data = path.read_bytes()
if unwritten := replace_file(packed, gzip.compress(data, mtime=0)):
return Failure(f"could not compress {path}: {unwritten.reason}")
if gzip.decompress(packed.read_bytes()) != data:
return Failure(f"could not compress {path}: {packed} reads back other bytes")
os.utime(packed, ns=(before.st_atime_ns, before.st_mtime_ns))
# On disk before the plain file goes, so a crash cannot leave an empty gzip in its place.
synced(packed)
synced(packed.parent)
after = path.stat()
# Written while it compressed: the plain file keeps the newer bytes, which readers prefer, and a later run compresses them.
if (after.st_size, after.st_mtime_ns) == (before.st_size, before.st_mtime_ns):
path.unlink()
except (OSError, EOFError, zlib.error) as failure:
return Failure(f"could not compress {path}: {failure}")
return None
def synced(path: pathlib.Path) -> None:
handle = os.open(path, os.O_RDONLY)
try:
os.fsync(handle)
finally:
os.close(handle)
def gzipped(path: pathlib.Path) -> pathlib.Path:
return path.with_name(path.name + SUFFIX)
+129
View File
@@ -0,0 +1,129 @@
import errno
import gzip
from unittest import mock
import os
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from text_files import compressed
from failure import Failure
class Compress(unittest.TestCase):
def test_replaces_the_file_with_a_gzip_of_the_same_bytes_and_times(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b'{"a": 1}\n')
os.utime(plain, (1000, 2000))
self.assertIsNone(compressed.compress(plain))
self.assertFalse(plain.exists())
packed = pathlib.Path(root, "t.jsonl.gz")
# A gzip header's own time is 0, so compressing the same bytes twice writes the same file.
self.assertEqual(packed.read_bytes(), gzip.compress(b'{"a": 1}\n', mtime=0))
self.assertEqual(packed.stat().st_mtime, 2000)
def test_syncs_the_gzip_and_its_folder_before_removing_the_file(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"a\n")
synced = []
fsync = os.fsync
def record(handle):
synced.append((os.readlink(f"/proc/self/fd/{handle}"), plain.exists()))
fsync(handle)
with mock.patch.object(compressed.os, "fsync", record):
self.assertIsNone(compressed.compress(plain))
self.assertEqual(synced, [(os.path.realpath(f"{plain}.gz"), True), (os.path.realpath(root), True)])
def test_a_gzip_it_cannot_write_keeps_the_original_and_says_why(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"kept\n")
with mock.patch.object(compressed, "replace_file", return_value=Failure("disk full")):
self.assertEqual(compressed.compress(plain), Failure(f"could not compress {plain}: disk full"))
self.assertEqual(plain.read_bytes(), b"kept\n")
def test_a_file_it_cannot_read_fails_naming_it(self):
with tempfile.TemporaryDirectory() as root:
folder = pathlib.Path(root, "t.jsonl")
folder.mkdir()
failed = compressed.compress(folder)
self.assertTrue(failed.reason.startswith(f"could not compress {folder}: [Errno {errno.EISDIR}]"), failed.reason)
def test_a_copy_that_reads_back_other_bytes_keeps_the_original_and_fails(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"kept\n")
with mock.patch.object(compressed.gzip, "compress", return_value=gzip.compress(b"other\n")):
failed = compressed.compress(plain)
self.assertIsInstance(failed, Failure)
self.assertIn(str(plain), failed.reason)
self.assertEqual(plain.read_bytes(), b"kept\n")
def test_a_file_written_while_it_compressed_stays_plain(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"one\n")
packing = compressed.gzip.compress
def grow_meanwhile(data, **kwargs):
with open(plain, "ab") as handle:
handle.write(b"two\n")
return packing(data, **kwargs)
with mock.patch.object(compressed.gzip, "compress", grow_meanwhile):
self.assertIsNone(compressed.compress(plain))
self.assertEqual(plain.read_bytes(), b"one\ntwo\n")
self.assertEqual(compressed.read_stored(plain), b"one\ntwo\n")
class Read(unittest.TestCase):
def test_reads_a_file_kept_plain_or_gzipped_by_its_plain_name(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "a.jsonl")
plain.write_bytes(b"a\n")
pathlib.Path(root, "b.jsonl.gz").write_bytes(gzip.compress(b"b\n"))
self.assertEqual(compressed.read_stored(plain), b"a\n")
self.assertEqual(compressed.read_stored(pathlib.Path(root, "b.jsonl")), b"b\n")
self.assertEqual(compressed.stored(pathlib.Path(root, "b.jsonl")), pathlib.Path(root, "b.jsonl.gz"))
self.assertIsNone(compressed.stored(pathlib.Path(root, "c.jsonl")))
with self.assertRaises(FileNotFoundError) as raised:
compressed.read_stored(pathlib.Path(root, "c.jsonl"))
self.assertEqual((raised.exception.errno, raised.exception.strerror, raised.exception.filename), (errno.ENOENT, "No such file or directory", str(pathlib.Path(root, "c.jsonl"))))
def test_the_plain_file_wins_over_an_older_gzip_beside_it(self):
with tempfile.TemporaryDirectory() as root:
pathlib.Path(root, "a.jsonl").write_bytes(b"a\nmore\n")
pathlib.Path(root, "a.jsonl.gz").write_bytes(gzip.compress(b"a\n"))
self.assertEqual(compressed.read_stored(pathlib.Path(root, "a.jsonl")), b"a\nmore\n")
def test_a_truncated_gzip_fails_as_an_os_error(self):
with tempfile.TemporaryDirectory() as root:
pathlib.Path(root, "a.jsonl.gz").write_bytes(gzip.compress(b"a\n" * 100)[:20])
with self.assertRaises(OSError) as raised:
compressed.read_stored(pathlib.Path(root, "a.jsonl"))
self.assertEqual((raised.exception.errno, raised.exception.filename), (errno.EIO, str(pathlib.Path(root, "a.jsonl.gz"))))
self.assertTrue(raised.exception.strerror.startswith("not a whole gzip: "), raised.exception.strerror)
def test_a_file_that_is_no_gzip_fails_as_an_os_error(self):
with tempfile.TemporaryDirectory() as root:
pathlib.Path(root, "a.jsonl.gz").write_bytes(b"plain text\n")
with self.assertRaises(OSError) as raised:
compressed.read_stored(pathlib.Path(root, "a.jsonl"))
self.assertEqual((raised.exception.errno, raised.exception.filename), (errno.EIO, str(pathlib.Path(root, "a.jsonl.gz"))))
def test_globs_each_file_once_by_its_plain_name(self):
with tempfile.TemporaryDirectory() as root:
for name in ("a.jsonl", "b.jsonl.gz", "c.jsonl", "c.jsonl.gz", "d.md"):
pathlib.Path(root, name).write_bytes(b"")
self.assertEqual(compressed.stored_glob(pathlib.Path(root), "*.jsonl"), [pathlib.Path(root, name) for name in ("a.jsonl", "b.jsonl", "c.jsonl")])
if __name__ == "__main__":
unittest.main()
+8 -5
View File
@@ -42,15 +42,18 @@ empties the cell. --detail replaces the detail with the text on stdin, and --cle
Text on stdin without --detail, and --detail without text on stdin, are refused, so no detail is
dropped or wiped by mistake. It prints `Edited item <ID> in <plan>.` It doesn't write anything where
ranking fails or scores the item under the bar.
In a Jira epic, edit changes an issue the loop filed: its summary, description and scores, and
the label `security` with --exempt. It doesn't change a link, so --blocked-by fails.
Of an issue a person filed it sets only the scores, --moves and --exempt, and only with --comment.
In a Jira epic, edit changes an issue the loop filed: its summary, description and scores, the label
`security` with --exempt, and its "is blocked by" links with --blocked-by: it makes a link for each
blocker added and deletes the link of each blocker dropped. Each blocker must be an open child of the
epic that doesn't wait for the issue itself, directly or through others. A link to any other issue stays.
Of an issue a person filed it sets only the scores, --moves and --exempt, and only with --comment;
--blocked-by fails there, since its links are the team's.
The four scores are set together where the issue holds none yet.
--comment is posted as a comment on the Jira issue, for the client's team, in the project's
language. Jira doesn't show the team the scores, so say that the issue is scored, and why.
A todo.md keeps no comment, so there the flag doesn't do anything.
Where Jira takes the edit and then refuses the comment, edit still prints `Edited item <ID> …`,
and says on stderr to post the comment in Jira.
Where Jira takes the edit and then refuses a link, a deletion or the comment, edit still prints `Edited item <ID> …`,
and says on stderr what to do in Jira.
move sets the item's release, as edit sets a cell. It prints `Moved item <ID> to release <version> in <plan>.`
In a Jira epic, move sets the Fix Version of an issue the loop filed; a person moves their own.
+3 -5
View File
@@ -10,7 +10,7 @@ The spelling, which the decision `todo.md: an item of the next release predicts
A path alone is the whole file, and (small) marks a part the item changes only a little. file_parts says how a part is named.
A path or a name that holds a backtick is spelt as a Markdown code span that holds one: in a longer run of backticks than any it holds, with a space inside each end, as in `` ## The `foo` tool ``. So every path and name on one line has a spelling."""
A path or a name that holds a backtick is spelt as a Markdown code span that holds one: in a longer run of backticks than any it holds, with a space inside each end, as in `` ## The `foo` tool ``. So every path and name on one line has a spelling. A description kept in Jira reaches here through adf_codec, which pads a span only where its text opens or ends in a backtick, or opens and ends in a space. So it spells a`b.md as ``a`b.md``; code_span reads that as the name too."""
import dataclasses
import re
@@ -81,10 +81,8 @@ def code_span(line: str, at: int) -> tuple[str, int] | None:
if opened is None or closed is None:
return None
text = line[opened.end():opened.end() + closed.start()]
if len(run) > 1:
# A longer run holds a space inside each end, which keeps a backtick at either end of the text apart from the run.
if len(text) < 3 or not text.startswith(" ") or not text.endswith(" "):
return None
# In a longer run, a space at both ends is padding, which keeps a backtick at either end of the text apart from the run. Without one at both ends, the span is its text. A one-backtick span is always its text, so a name that opens and ends in a space isn't read back from Jira, whose codec pads it; no path or heading is spelt so.
if len(run) > 1 and text.startswith(" ") and text.endswith(" "):
text = text[1:-1]
return (text, opened.end() + closed.end()) if text else None
@@ -1,6 +1,7 @@
import os
import pathlib
import re
import shutil
import subprocess
import sys
import tempfile
@@ -9,6 +10,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from touch_predictions.beside import Beside, chunks_beside
from adf_codec import convert
from touch_predictions import diff_parts, file_parts, main, record
from failure import Failure
from test_env import isolated_env
@@ -31,6 +33,30 @@ Touches:
More prose, with `bin/b.py` named.
"""
needs_docker = unittest.skipIf(shutil.which("docker") is None or subprocess.run(["docker", "info"], capture_output=True, timeout=20).returncode != 0, "adf_codec runs in a container, which needs a docker daemon")
def text(words: str, code: bool = False) -> dict:
return {"type": "text", "text": words, **({"marks": [{"type": "code"}]} if code else {})}
def bullet_of(*spans: dict) -> dict:
return {"type": "listItem", "attrs": {"localId": "i"}, "content": [{"type": "paragraph", "attrs": {"localId": "p"}, "content": list(spans)}]}
# A Touches list as someone types it in Jira's editor: a paragraph, then a bulleted list whose paths and names carry the code mark.
JIRA_TOUCHES = {"type": "doc", "version": 1, "content": [
{"type": "paragraph", "attrs": {"localId": "a"}, "content": [text("A planning call.")]},
{"type": "paragraph", "attrs": {"localId": "b"}, "content": [text("Touches:")]},
{"type": "bulletList", "attrs": {"localId": "c"}, "content": [
bullet_of(text("bin/a.py", True), text(": "), text("main", True), text(", "), text("Plan.branch", True), text(" (small)")),
bullet_of(text("a`b.md", True), text(": "), text("## The `foo` tool", True), text(" (small)"), text(", "), text("`x`", True)),
bullet_of(text("bin/run.sh", True), text(": lines 10-20")),
bullet_of(text("docs/new.md", True)),
]},
{"type": "paragraph", "attrs": {"localId": "d"}, "content": [text("More prose.")]},
]}
PYTHON = """\"\"\"Docstring.\"\"\"
import os
@@ -109,6 +135,28 @@ class PredictedTest(unittest.TestCase):
self.assertEqual(predicted("Touches:\n- `a.md`: `` ## The `foo` tool `` (small), ``` `` ```, `` `x` ``\n"), {"a.md": (Part("## The `foo` tool", small=True), Part("``"), Part("`x`"))})
self.assertEqual(rendered({"a.md": (Part("## The `foo` tool", small=True), Part("``"))}), "- `a.md`: `` ## The `foo` tool `` (small), ``` `` ```")
def test_a_longer_run_without_padding_spaces_is_the_text_between_its_runs(self) -> None:
self.assertEqual(predicted("Touches:\n- ``a`b.md``: ``## The `foo` tool`` (small), ``` `` ```, `` `x` ``\n"), {"a`b.md": (Part("## The `foo` tool", small=True), Part("``"), Part("`x`"))})
def test_a_longer_run_with_a_space_at_one_end_only_keeps_it(self) -> None:
self.assertEqual(predicted("Touches:\n- `a.md`: `` x``, ``y ``, `` a ``\n"), {"a.md": (Part(" x"), Part("y "), Part("a"))})
@needs_docker
def test_a_list_written_in_markdown_and_read_back_from_a_jira_description_reads_as_written(self) -> None:
description = convert.markdown_to_adf(DETAIL)
assert isinstance(description, dict)
self.assertEqual(predicted(convert.adf_to_markdown(description)), predicted(DETAIL))
@needs_docker
def test_a_list_typed_in_jiras_editor_reads_as_its_files_and_parts(self) -> None:
read = convert.adf_to_markdown(JIRA_TOUCHES)
self.assertEqual(predicted(read), {
"bin/a.py": (Part("main"), Part("Plan.branch", small=True)),
"a`b.md": (Part("## The `foo` tool", small=True), Part("`x`")),
"bin/run.sh": (Part("lines 10-20"),),
"docs/new.md": (),
})
def test_a_name_ends_only_at_a_run_as_long_as_the_one_that_opens_it(self) -> None:
self.assertEqual(predicted("Touches:\n- `a.md`: `` a ``` b ``, `c``d`\n"), {"a.md": (Part("a ``` b"), Part("c``d"))})
@@ -119,8 +167,8 @@ class PredictedTest(unittest.TestCase):
touches = {"my file.md": (Part("## todo.md: under `Touches:` in its detail"), Part("## ``a`` and `b`", small=True), Part("`"), Part("## ends in `x`"), Part(" x "), Part("lines 2-3")), "a`b.md": (), "c, d: e.md": (Part("# T"),)}
self.assertEqual(predicted(f"Touches:\n\n{rendered(touches)}\n"), touches)
def test_a_longer_run_without_its_spaces_or_its_end_or_a_name_is_a_failure(self) -> None:
for bullet in ("- `a.md`: ``## A``", "- `a.md`: `` ## A", "- `a.md`: `` ## A `", "- `a.md`: `` ``", "- `a.md`: ", "- `a.md`:", "- `a.md`: `f`, ", "- `a.md`: `f`,`g`", "- ``", "- `a.md` (small)", "- `a.md`: `f` (small) (small)"):
def test_a_bullet_with_an_unclosed_run_a_blank_name_or_a_stray_separator_is_a_failure(self) -> None:
for bullet in ("- `a.md`: `` ## A", "- `a.md`: `` ## A `", "- `a.md`: `` ``", "- `a.md`: ", "- `a.md`:", "- `a.md`: `f`, ", "- `a.md`: `f`,`g`", "- ``", "- `a.md` (small)", "- `a.md`: `f` (small) (small)"):
with self.subTest(bullet):
self.assertIsInstance(predicted(f"Touches:\n\n{bullet}\n"), Failure)
+3
View File
@@ -22,6 +22,9 @@
- judge/ holds each judge run an outcome names on its `Judge run:` line: the prompt the judge read and claude's
json reply, named by the hash of that content (judge_run.JudgeRun). judge-replay reads the prompt back.
compress-archives gzips each .jsonl 30 days after it last changed, as <name>.jsonl.gz beside its plain name,
and every reader reads it by that name (text_files.compressed); a plain file beside its gzip is the newer.
write.archive writes the layout, at every interrupt and as every cycle ends. read.py and the tools that
measure finished chunks read it.
"""
+1 -1
View File
@@ -29,5 +29,5 @@ def outcomes_of(copy: pathlib.Path) -> pathlib.Path:
def transcript_of(outcomes: pathlib.Path) -> pathlib.Path:
"""The transcript copy an outcomes file sits beside; outcomes_of is its inverse."""
"""The plain name of the transcript copy an outcomes file sits beside, which text_files.compressed reads whether gzipped or not; outcomes_of is its inverse."""
return outcomes.with_name(outcomes.name.removesuffix(FILE_SUFFIX) + ".jsonl")
+8 -2
View File
@@ -4,6 +4,7 @@ import pathlib
from transcript_archive.location import chunk_folder, FILE_SUFFIX, transcript_of
from failure import Failure
from transcript_archive.outcomes import FINISHED_AT, spent_stamps, workers
from text_files.compressed import read_stored, stored_glob
@dataclasses.dataclass(frozen=True)
@@ -32,7 +33,7 @@ def finished_chunks(project_archive: pathlib.Path) -> list[FinishedChunk] | Fail
copies.append((stamps[-1], outcomes))
if copies:
finished, outcomes = max(copies)
found.append(FinishedChunk(folder.name, finished, transcript_of(outcomes), sorted(folder.glob("session/*/subagents/*.jsonl"))))
found.append(FinishedChunk(folder.name, finished, transcript_of(outcomes), subagent_transcripts(folder)))
return sorted(found, key=lambda chunk: (chunk.finished, chunk.folder))
@@ -44,6 +45,11 @@ def archived_outcomes(root: pathlib.Path) -> list[tuple[str, pathlib.Path]] | Fa
return Failure(f"could not read the archive {root}: {failure}")
def subagent_transcripts(folder: pathlib.Path) -> list[pathlib.Path]:
"""The plain name of each subagent's transcript archived in a chunk's folder, kept plain or gzipped, sorted."""
return [path for session in sorted(folder.glob("session/*/subagents")) for path in stored_glob(session, "*.jsonl")]
def chunk_outcomes(folder: pathlib.Path) -> list[pathlib.Path]:
"""The outcomes file of each transcript copy in a chunk's folder, sorted."""
return sorted(folder.glob(f"*{FILE_SUFFIX}"))
@@ -52,7 +58,7 @@ def chunk_outcomes(folder: pathlib.Path) -> list[pathlib.Path]:
def read_threads(chunk: FinishedChunk) -> list[bytes] | Failure:
"""The worker's transcript, then each subagent's."""
try:
return [path.read_bytes() for path in (chunk.transcript, *chunk.subagents)]
return [read_stored(path) for path in (chunk.transcript, *chunk.subagents)]
except OSError as failure:
return Failure(f"could not read {failure.filename}: {failure.strerror}")
@@ -10,6 +10,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from transcript_archive.project_key import checkout_of, project_key
from text_files import compressed
from failure import Failure
from transcript_archive.judge_run import JudgeRun
from transcript_archive import outcomes, read, write
@@ -119,6 +120,47 @@ class CopyFor(unittest.TestCase):
self.assertIsNone(write.copy_for(copy, b"second"))
self.assertEqual(write.copy_for(copy, b"first, grown"), copy)
def test_reads_a_compressed_copy_as_the_kept_version(self):
with tempfile.TemporaryDirectory() as temp:
copy = pathlib.Path(temp, "agent-a.jsonl")
copy.write_bytes(b"first\n")
self.assertIsNone(compressed.compress(copy))
self.assertIsNone(write.copy_for(copy, b"first\n"))
self.assertEqual(write.copy_for(copy, b"first\nsecond\n"), copy)
self.assertEqual(write.copy_for(copy, b"other\n"), copy.with_name(f"agent-a.jsonl.{hashlib.sha256(b'other' + bytes([10])).hexdigest()}"))
class Compressed(unittest.TestCase):
"""The archive, its transcript copies and subagents' transcripts gzipped as compress-archives leaves them."""
def archived(self, root: str) -> pathlib.Path:
kept = pathlib.Path(root, "archive")
transcript = pathlib.Path(root, "t.jsonl")
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
subagent = pathlib.Path(root, "t", "subagents", "agent-a.jsonl")
subagent.parent.mkdir(parents=True)
subagent.write_text('{"sub": 1}\n')
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.finished("w", "merged", "j"), NOW))
for copy in kept.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
return transcript
def test_a_compressed_copy_takes_the_next_outcome_and_is_not_copied_again(self):
with tempfile.TemporaryDirectory() as root:
transcript = self.archived(root)
kept = pathlib.Path(root, "archive")
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.stopped("w", "Ctrl-C"), NOW + 1))
self.assertEqual([path.name.endswith(".gz") for path in kept.rglob("*.jsonl*")], [True, True])
self.assertIn("Stopped: Ctrl-C", next(kept.glob("*/*.outcomes.md")).read_text())
def test_reads_a_finished_chunk_whose_copies_are_compressed(self):
with tempfile.TemporaryDirectory() as root:
self.archived(root)
(chunk,) = read.finished_chunks(pathlib.Path(root, "archive"))
self.assertEqual(read.read_threads(chunk), [b'{"timestamp": "2026-10-03T00:00:00Z"}\n', b'{"sub": 1}\n'])
class Outcomes(unittest.TestCase):
def test_the_archive_stamps_what_it_reads_back(self):
+7 -5
View File
@@ -7,6 +7,7 @@ from transcript_archive.location import chunk_folder, outcomes_of
from failure import Failure
from transcript_archive.judge_run import JudgeRun
from text_files.file_writes import make_private, replace_file
from text_files.compressed import read_stored, stored
from claude_code.session import session_files
from transcript_archive.outcomes import TASK
from text_files.utc_stamp import utc
@@ -29,7 +30,7 @@ def archive(project_archive: pathlib.Path, path: str, task: str, outcome: str, a
copy = folder / f"{hashlib.sha256(raw).hexdigest()}.jsonl"
outcomes = outcomes_of(copy)
stamped = f"{utc(at)} {outcome}"
if copy.exists() and outcomes.exists():
if stored(copy) and outcomes.exists():
return add_outcome(outcomes, stamped)
# The copy lands last, so a copy on disk always has its outcomes and its session's files beside it.
for dest, data in ((outcomes, f"{TASK}{task}\n\n{stamped}\n".encode()), (copy, raw)):
@@ -41,7 +42,7 @@ def archive(project_archive: pathlib.Path, path: str, task: str, outcome: str, a
def keep_judge_run(folder: pathlib.Path, run: JudgeRun) -> Failure | None:
kept = folder / run.file
if kept.exists():
if stored(kept):
return None
try:
make_private(kept.parent)
@@ -93,11 +94,12 @@ def files_under(folder: pathlib.Path) -> list[pathlib.Path]:
# Claude Code rewrites some files whole, such as custom-title.json, so a copy is replaced only by a file that grew from it.
# A grown copy lands plain beside its gzip, which compress-archives then replaces without losing anything, since the gzip holds a prefix of it.
def copy_for(copy: pathlib.Path, data: bytes) -> pathlib.Path | None:
if not copy.is_file():
if not stored(copy):
return copy
kept = copy.read_bytes()
kept = read_stored(copy)
if data.startswith(kept):
return copy if data != kept else None
beside = copy.with_name(f"{copy.name}.{hashlib.sha256(data).hexdigest()}")
return None if beside.exists() else beside
return None if stored(beside) else beside
+39 -8
View File
@@ -12,6 +12,7 @@ Each project principle states one rule that the decisions below apply; their ent
# Decisions no project principle covers
- archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them
- bin/: a test parametrizes its cases with `subTest`, since pytest's `parametrize` doesn't reach a unittest `TestCase`
- bin/: the tools run on the host's Python, with no container, and so do their tests outside CI
- bin/: the tools send no OTLP traces, logs or metrics, and loop-claude's archive holds the trace of each cycle
@@ -26,6 +27,7 @@ Each project principle states one rule that the decisions below apply; their ent
- project config: a tool that reads a file's commits says where it does not support that file, and the work goes on
- project config: `language` names the prose's language by its English name, and `commit-title`, `pull-request-title` and `branch` are brace templates over `{id}` and `{summary}`
- project config: `plan` may name a Jira Cloud epic as `jira:` and its address, the plan is its open child issues in backlog order, and an issue's number is the item's ID
- project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, and a save whose text is the text a read showed leaves the document as Jira or Confluence holds it
- project config: `tooling-on-host = true` lets a project with no container setup run its language runtime, package manager, tests and task runner on the host
- review: a release review fixes its findings in the release it reviews, and files under that release only a finding that needs a human answer, adds a feature or outlasts its five rounds
- review: a reviewer or comprehension seat that rules the work below standard may ask to rerun once, one rung up, in the same round; the ruling that stands is an escalation trigger for the worker unless the fix changes only prose, and never for the judge
@@ -58,6 +60,21 @@ Settling a call is the human's. The chunk that settles the calls cites, in `docs
# Decisions
## archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them
2026-11-17, the worker of todo item 306. Serves goal 3, backing up the transcripts, and goal 1, whose measures read them: the routing figures, `reading-cost`, `score-outcomes` and both replays rest on the whole archive, so nothing is pruned. On 2026-11-17 the archive held 7.9 GB and `~/.local/state/larv-review` 1.1 GB, both growing with every chunk.
- **What.** `compress-archives`, run daily from the crontab line in `README.md` → "Install", gzips each file that last changed over 30 days ago. The gzip is `<name>.gz`, beside the plain name, with the same times. In the archive it takes only the transcripts, `*.jsonl`, which held 7.8 GB of the 7.9 GB; the outcomes files stay plain, since loop-claude appends to them every cycle. In the review folders it takes every file, since the probes' build output fills most of them.
- **Last changed.** A file's age is the later of its mtime and its ctime. A copy or an unpacked archive keeps old mtimes, but its ctime is when it landed, so a tree a chunk or a replay unpacks today stays plain while they use it.
- **Safely.** It writes the gzip, reads it back, syncs it to disk, and only then removes the plain file. It removes it only where the gzip read back the same bytes and the plain file didn't change while it was compressed. At the first failure it stops and exits 1, keeping that file plain, as a failed archive stops loop-claude.
- **Readers.** The tools that read old files in either folder, the archive's readers and `review-replay`, read a file by its plain name through `bin/text_files/compressed.py`. A plain file beside its gzip wins, since it was written later. A session file that grows after its copy was compressed lands plain beside the gzip, which holds a prefix of it, so the next run's gzip doesn't lose anything. A report rewritten whole after it was gzipped also lands plain beside its gzip, and the next run's gzip replaces the older one, as rewriting the plain file replaced the older bytes before.
- **Why gzip.** Python 3.14's `compression.zstd` needs a Python built with libzstd, and this machine's lacks `_zstd`; gzip is in every build (goal 2). On 400 archived transcripts older than 30 days, gzip shrank their 609 MB to 23%, at about 29 MB a second.
- **Why 30 days.** A chunk's files stop changing once it merges, and chunks run for hours or days. Since every reader reads both forms, the threshold sets only how long a file stays plain.
- **Backup.** Both folders sit under `$HOME`, on the filesystem that holds `~/code` and `~/.claude` (ai-sandbox's `/dev/vda3` on 2026-11-17), so backing that volume up, as item 1 waits for, covers both. `README.md` → "Install" names them, since the review folders lie outside `~/.claude`.
- **One writer.** Project principle 3 holds: the tool that writes a plain file stays its only writer, and `compress-archives` is the only writer of its gzip.
Valid while every tool that reads old files in the two folders reads through `text_files/compressed.py`.
## bin/: a test parametrizes its cases with `subTest`, since pytest's `parametrize` doesn't reach a unittest `TestCase`
2026-11-12, lilleman, from the release review of 0.9.0. Applies `principles/technical-principles.md` → "Most code should have automated tests", whose "parametrize the cases" asks for one reported result per case. The tests are plain unittest, run by pytest, and pytest's `parametrize` doesn't work on a `TestCase` method; `subTest` is the unittest mechanism that does: each case reports and fails on its own, with its parameters named. A test that loops over its cases itself, outside `subTest`, or holds a branch, is still a breach. Valid while the tests stay in plain unittest.
@@ -184,7 +201,7 @@ Valid while claude takes `--session-id` and resumes by it, and the human runs lo
## loop-claude: the archive keeps the raw transcript and each judge run, a failed archive stops the loop, and a replay keeps its runs and report in a replays/ folder of the archive
2026-09-28, lilleman. Serves goals 1 and 3: saving each transcript locally is the first step of backing it up, and it shows afterwards what an unattended run did. A copy of the `.jsonl`, deduplicated by content hash, with its outcomes beside it; a rendering drops tool results, and a second-resolution stamp names two states within one second as one file. A failed archive exits 1, since a loop that loses its record silently fails the backup goal. The judge's runs are archived too, as `judge/<sha256>.json` in the chunk's folder: the prompt it read and its json reply. Together they are its whole transcript, since it runs with no tools and one turn. A run is kept whole and deduplicated by hash, and the outcome of the verdict, interrupt or rejected reply it produced names it on a `Judge run:` line. Keeping it whole costs little: a prompt is at most `JUDGE_PROMPT_CHARS`, 400,000 characters. Measured 2026-11-17, the archive's 911 judge lines come to at most 0.36 GB of prompts, against 3.5 GB in its 1,211 transcript copies, the largest 23.4 MB. The judge is still priced from its `Judge cost:` line, which is claude's own total for the run, and the Spent block counts the live reply's tokens; the archived reply keeps its `usage` for a later reader, and nothing reads it back. `judge-replay` replays a verdict on its archived prompt where there is one. Valid while transcripts, and the judge's prompts, stay small enough to copy whole.
2026-09-28, lilleman. Serves goals 1 and 3: saving each transcript locally is the first step of backing it up, and it shows afterwards what an unattended run did. A copy of the `.jsonl`, deduplicated by content hash, with its outcomes beside it; a rendering drops tool results, and a second-resolution stamp names two states within one second as one file. A failed archive exits 1, since a loop that loses its record silently fails the backup goal. The judge's runs are archived too, as `judge/<sha256>.json` in the chunk's folder: the prompt it read and its json reply. Together they are its whole transcript, since it runs with no tools and one turn. A run is kept whole and deduplicated by hash, and the outcome of the verdict, interrupt or rejected reply it produced names it on a `Judge run:` line. Keeping it whole costs little: a prompt is at most `JUDGE_PROMPT_CHARS`, 400,000 characters. Measured 2026-11-17, the archive's 911 judge lines come to at most 0.36 GB of prompts, against 3.5 GB in its 1,211 transcript copies, the largest 23.4 MB. The judge is still priced from its `Judge cost:` line, which is claude's own total for the run, and the Spent block counts the live reply's tokens; the archived reply keeps its `usage` for a later reader, and nothing reads it back. `judge-replay` replays a verdict on its archived prompt where there is one. Valid while transcripts, and the judge's prompts, stay small enough to copy whole; `archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them` keeps the copies small on disk.
2026-11-17, lilleman, todo item 304. Serves goals 1.4 and 3. `judge-replay` and `review-replay` each decide which rung a review pass or the judge may run on, so their runs stay checkable and re-priceable once a decision is questioned. They keep their runs in `replays/<tool>-<arguments>-<UTC stamp>/` in the archive's root:
@@ -265,11 +282,11 @@ Valid while no published standard sets either number, the reading-cost window st
## reading cost: a rise is the last 5 chunks' median above the 5 before them, and files or refreshes one plan item
2026-10-08, lilleman. Serves goal 1.1: comprehension never falls, so any rise counts. Per-chunk figures are noisy: a flat trend rises about every other chunk. So a rise refreshes the item and never refiles it, and a chunk that sees no rise leaves the item as it is. The chunk that ships the item resets the trend, so later figures do not refile it for work that chunk already did (goal 1.4). A median, because one chunk that ran a panel of readers before its first commit measured 2.99M tokens against a typical 80k–370k, and a mean let it decide the trend alone. No published standard sets this threshold. Valid while figures arrive once per chunk.
2026-10-08, lilleman. Serves code health and goal 1.4, whose counter-signal is a rising reading cost: any rise counts. Per-chunk figures are noisy: a flat trend rises about every other chunk. So a rise refreshes the item and never refiles it, and a chunk that sees no rise leaves the item as it is. The chunk that ships the item resets the trend, so later figures do not refile it for work that chunk already did (goal 1.4). A median, because one chunk that ran a panel of readers before its first commit measured 2.99M tokens against a typical 80k–370k, and a mean let it decide the trend alone. No published standard sets this threshold. Valid while figures arrive once per chunk.
## reading cost: larv-review measures each finished chunk from loop-claude's archive and appends the figure to `docs/reading-cost.md` in the next chunk's PR
2026-10-08, lilleman. Serves goal 1.1: comprehension measured from real work, at no model cost (goal 1.4). loop-claude archives from the main checkout, outside any PR, so a tracked file it wrote there would dirty that checkout. So `bin/reading-cost` measures each finished chunk `docs/reading-cost.md` lacks from the archive, in the next chunk's worktree. A PR that never merges then loses no figure, and loop-claude needs no change. The figures keep a file of their own, so `docs/comprehension-history.md` keeps one writer, the comprehension check. Serves goal 4 by naming only files git tracks, and token counts. Rejected: loop-claude opening a PR of its own, one more PR per chunk and a tie to the forge; the worker measuring its own transcript before merging, which misses the chunk's tail. The figure counts new input tokens, uncached and cache-written, because workers read mostly through Bash and subagents, where parsed paths alone miss what they read. Valid while loop-claude archives every finished chunk with its subagents' transcripts.
2026-10-08, lilleman. Serves code health and goal 1.4, whose counter-signal is reading cost: measured from real work, at no model cost. loop-claude archives from the main checkout, outside any PR, so a tracked file it wrote there would dirty that checkout. So `bin/reading-cost` measures each finished chunk `docs/reading-cost.md` lacks from the archive, in the next chunk's worktree. A PR that never merges then loses no figure, and loop-claude needs no change. The figures keep a file of their own, so `docs/comprehension-history.md` keeps one writer, the comprehension check. Serves goal 4 by naming only files git tracks, and token counts. Rejected: loop-claude opening a PR of its own, one more PR per chunk and a tie to the forge; the worker measuring its own transcript before merging, which misses the chunk's tail. The figure counts new input tokens, uncached and cache-written, because workers read mostly through Bash and subagents, where parsed paths alone miss what they read. Valid while loop-claude archives every finished chunk with its subagents' transcripts.
## review: `/security-review` is the security pass of record, except where the full scan runs on the same change and replaces it
@@ -359,11 +376,15 @@ Messages keep each file's usual name, such as `todo.md`, as its label, since the
- **The release is the earliest Fix Version named as a semver version**, and an issue with none fails the read, naming it.
- **The loop files an item as a Task under the epic, and marks it as its own** (lilleman's calls of 2026-11-12). `"Filed by": "loop"` in the `plan-item` property tells such an issue from one a person filed, whose summary and description the loop never edits. Task is an issue type every Jira Software project template has; a project without it refuses the issue, saying so. The issue takes the project's Fix Version named as the release, with a leading v or without, and the store refuses an item whose release the project has no Fix Version for, before it files anything. A security finding also gets the label `security`, and its text never holds a secret's value. Jira takes a blocker's link only after the issue exists, so the store first checks that each blocker is an open child. A link Jira still refuses leaves the issue filed: the write counts as landed, with a note naming the link to make by hand. A failed create says the issue may exist all the same, since a lost answer doesn't tell.
- **The config states the formula and the bar, and the store ranks a new issue among the loop's own** (lilleman, 2026-11-12 and 2026-11-15). A plan in Jira has no `## Scoring`, so a config whose `plan` names an epic must set `jira-score-formula` and `jira-score-bar`. It fails to load without them, or with a formula that doesn't parse, since no setting a config needs has a default. A bench of readers picked the spelling: three fresh readers, seated as the person who writes a client's `config.toml`, each picked those keys over `jira-score` and `jira-bar`, `score-formula` and `score-bar`, and bare `score` and `bar`. The prefix matches the other Jira-only settings, "formula" says the value is no score, and "score-bar" says what the bar applies to. The store scores the item as `rank_todo.py` scores a row, with the goal weights from the goals file, the decision log and the comprehension history. It refuses an item under the bar that isn't exempt, or one whose Moves names a goal the goals file lacks, before it files anything. Then it ranks the issue just before the first issue the loop filed that ranks after it. A person's issues don't count, so the team's rank of each stays as it is. The order is todo.md's within a release: a question first, then a security finding, then the higher score, with a tie after the older issue. Questions go first because `rank_todo.py` ranks them ahead of security findings, since an answer may change other items (the loop's choice, 2026-11-16). Release doesn't count, since lilleman's call ranks by score among the loop's issues. A rank Jira refuses, or an issue of the loop's that no longer scores, leaves the new issue filed and last, with a note to rank it by hand.
- **The store edits and moves only an issue the loop filed, and of an issue a person filed it sets only the scores** (lilleman's calls of 2026-11-12). On its own issue it sets the summary, description, scores and Fix Version in one request, so Jira takes all of an edit or none, and it keeps the label `security` in step with Exempt. Of a person's issue it writes only the `plan-item` property, and a comment: the team's screens don't show the property, so the comment tells the team why the scores changed, and the store refuses the edit without one. The comment's text comes from whoever scores, in the project's `language`. Where Jira takes the edit and then refuses the comment, the edit counts as made, and a note says to post the comment by hand. No edit changes a link, until `todo.md` item 389.
- **The store edits and moves only an issue the loop filed, and of an issue a person filed it sets only the scores** (lilleman's calls of 2026-11-12). On its own issue it sets the summary, description, scores and Fix Version in one request, so Jira takes all of an edit or none, and it keeps the label `security` in step with Exempt. It also sets the "is blocked by" links, in requests of their own after that one: a link per blocker added and a deletion per blocker dropped, each blocker an open child, as `add` checks, and none that waits for the item itself, directly or through others. A link to any other issue stays as it is, since a read doesn't show it and the team may have made it. Of a person's issue it writes only the `plan-item` property, and a comment: the team's screens don't show the property, so the comment tells the team why the scores changed, and the store refuses the edit without one. Its links stay the team's, which Jira shows on its board where it doesn't show a score. The comment's text comes from whoever scores, in the project's `language`. Where Jira takes the edit and then refuses a link or the comment, the edit counts as made, and a note says what to do by hand.
- **The loop may re-score a person's issue that already holds scores, and never moves it.** lilleman's calls of 2026-11-12 cover an issue nobody scored, and say nothing of one that holds scores; larv-review chose on 2026-11-16, through todo item 382. A re-score changes only the loop's own reading of the issue, and not what the person wrote, so it is allowed, with a comment as the first scores need. L, E, I and S are set together, so no issue is left partly scored. The Fix Version says when the team ships the issue, and a release is its team's, so the store refuses the move and `CLAUDE.md` has the worker ask the human to move the issue.
- **`todo-item edit --comment` carries the comment, and a plan in a file takes the flag and keeps no comment**, so one command works in every project, as goal 1.3 asks. A bench of readers picked the spelling: three fresh readers, seated as the consultant who runs the loop in a client's project, each picked `--comment` over `--reason`, `--why` and `--because`. Only it says the text is posted where the team reads it, and `--reason` read as `--exempt`'s reason.
- **The store finishes an item with the transition that the config's `jira-finish-transition` names**, which must lead to the status `jira-finish-status` names, since each Jira workflow differs (lilleman's call, 2026-11-12); any open child of the epic may move. It refuses a status outside Jira's done status category: the read counts an issue as finished only in that category, so the item would stay in the plan. Where the issue's present status has no direct transition to the status `jira-finish-status` names, `todo-item finish` fails and a person moves the issue in Jira: the store never walks an issue through other statuses to reach it. lilleman answered on 2026-11-15: "If it is possible to move it to done without a transisiont, that should happen. If not, fail." The implementer read "without a transition" as "without a transition in between". `rank_todo.py` refuses to rank a plan in Jira, and its `--check` passes, since Jira's backlog ranks it; rule-hook's rank check skips it. Each read first asks Jira for the epic itself, since a search under an epic the account cannot see answers no issues, as a finished plan does.
## project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, and a save whose text is the text a read showed leaves the document as Jira or Confluence holds it
2026-11-17, claude, through item 391. Serves goal 1.6: a team's Jira and Confluence keep each block's localId and the empty paragraphs Jira uses for spacing, and agents, the judge and people still read the text easily, which keeps goal 1.4's reading cost down. adf-codec converts markdown and ADF by two pairs of calls, as `bin/adf_codec/convert.py` says. The lossless pair keeps the localIds and spacing paragraphs, but wraps every block in two `!adf:` marker lines: a five-line detail became 17 lines in a trial (larvit/adf-codec#185). The portable pair drops both. So the loop reads by the portable pair, and leaves a document alone where a save's text is the text a read showed. `JiraPlan.shows` compares a new detail with the description converted again. `ConfluencePage` keeps the digest of the text it read in the version a write names, and checks that the page still holds that version. Valid while the lossless markdown wraps every block in marker lines. A save that changes one line still rewrites the whole document by the portable pair, and so drops every localId and spacing paragraph, until `todo.md` item 395 keeps each block the save didn't change.
## project config: `tooling-on-host = true` lets a project with no container setup run its language runtime, package manager, tests and task runner on the host
2026-11-12, lilleman, through todo item 358. Serves goal 1.6: a legacy product may have no container setup, and the loop works in it without its owners adding one. `CLAUDE.md`'s container rule and the stability reviewer follow the setting. rule-hook checks only how a container starts, its memory limits and its image's pin, never where a runtime runs, so it needs no change. A decision may still put one tool on the host where the project has containers, as this repo's `bin/` decision does. Readers seated as the person who writes the config picked the key, 2 of 3. Valid while the loop runs on the machine that holds the checkout.
@@ -392,7 +413,7 @@ Valid while the ladder orders its rungs by competence.
## review: each pass keeps a reviewer of its own
2026-11-09, lilleman asked for the trials. Serves goals 1.1 and 1.4. Merged reviewers were replayed on archived first rounds, beside the separate reviewers (`reviewer-merge-trials/README.md`). Each merge cost a third to a half less, and in each, one absorbed pass lost findings. Stability, product-owner and prose as one reviewer recovered 15 of 34 prose findings, against 22 and 23 for two separate runs. The architect and the principles auditor as one recovered 5 of 17 principles findings, against 10 for one separate run, and lost both principles 🔴. Stability and product-owner as one, run twice, recovered 4 and 5 of 16 product-owner findings, against 7 and 10 for two separate runs. Folding override-reviewer into stability had no data: no archived override round holds a 🔴 or 🟡. Valid until a new model joins the ladder or a reviewer's definition changes: replay the merge then, with the scripts in `reviewer-merge-trials/`.
2026-11-09, lilleman asked for the trials. Serves goals 1.1 and 1.4. Merged reviewers were replayed on archived first rounds, beside the separate reviewers (`reviewer-merge-trials/README.md`). Each merge cost a third to a half less, and in each, one absorbed pass lost findings. Stability, product-owner and prose as one reviewer recovered 15 of 34 prose findings, against 22 and 23 for two separate runs. The architect and the principles auditor as one recovered 5 of 17 principles findings, against 10 for one separate run, and lost both principles 🔴. Stability and product-owner as one, run twice, recovered 4 and 5 of 16 product-owner findings, against 7 and 10 for two separate runs. Folding override-reviewer into stability had no data: no archived override round holds a 🔴 or 🟡. Valid until a reviewer's definition changes, and for a project only while its rung is Opus 5.5 at medium, the rung the trials ran on: replay the merge then, with the scripts in `reviewer-merge-trials/`.
## review: each reviewer under `agents/` writes its full report into the chunk's review folder under `~/.local/state/larv-review/` and returns the worker its verdict and one line per finding; rule-hook refuses its Edit and MultiEdit, and any Write outside `~/.local/state/larv-review/`
@@ -446,7 +467,7 @@ The base is the last measured commit: the default branch's newest merged change
2026-10-28, claude, through todo item 233. Serves goal 1.4. One auditor per round checks the README's goals with code health, and each principles file that applies. It reads the project once, where one auditor per goal set or file ran 3.8 times per chunk before.
A trial checked that it keeps goal 1.1. It replayed the 6 latest first rounds that launched two or more auditors, at most two per project, from this repo, `log`, `visthus` and `adf-codec`: 15 auditors, whose reports held 40 🔴 and 🟡 findings. Each round ran on the project's rung as `review-replay` replays it, once with its separate auditors and once with one auditor handed everything they checked, its new definition passed through `--agents`. A replayed reviewer finds a different part of its findings each time: the separate auditors found only 10 of their own 40 again. The one auditor found 12, among them 8 of those 10. So it finds as much as the separate auditors do. It missed two 🟡s on a missing or weak test, and found 4 findings they missed. No principles file keeps an auditor of its own. Asked whether those two misses should bring back separate auditors for the technical principles and for the goals with code health, lilleman chose to try the one auditor. Valid until the auditor's definition changes or a new model joins the ladder: replay the pass then, as this trial did.
A trial checked that it keeps goal 1.1. It replayed the 6 latest first rounds that launched two or more auditors, at most two per project, from this repo, `log`, `visthus` and `adf-codec`: 15 auditors, whose reports held 40 🔴 and 🟡 findings. Each round ran on the project's rung as `review-replay` replays it, once with its separate auditors and once with one auditor handed everything they checked, its new definition passed through `--agents`. A replayed reviewer finds a different part of its findings each time: the separate auditors found only 10 of their own 40 again. The one auditor found 12, among them 8 of those 10. So it finds as much as the separate auditors do. It missed two 🟡s on a missing or weak test, and found 4 findings they missed. No principles file keeps an auditor of its own. Asked whether those two misses should bring back separate auditors for the technical principles and for the goals with code health, lilleman chose to try the one auditor. Valid until the auditor's definition changes or a project's rung changes from the one this trial replayed it on: replay the pass then, as this trial did.
2026-10-29, lilleman: it runs only in the release review. `review: the architecture and principles passes run only in the release review…` says why.
@@ -509,7 +530,7 @@ The runs, each on the latest 10 rounds of Opus 5.5 projects that hold a 🔴 or
The prose pass went first under the new bar: a missed prose finding breaks no behaviour, and a later change fixes it cheaply. Sonnet 5.5 missed 44 to 55% of what the project's rung found again, in the prose pass as in both stability runs, so it is unlikely to reach the new bar either; and Opus 5.5 at low, the likeliest rung, fell 17 points short on the pass whose misses cost least. The architecture, principles, overrides and product-owner passes are not replayed yet: `todo.md` item 196 keeps their replays in the plan.
Valid until a new model joins the ladder, which calls for a replay of every pass, or a pass's reviewer prompt changes, which calls for a replay of that pass.
Valid until the rungs below the project's rung change, which calls for a replay of every pass, or a pass's reviewer prompt changes, which calls for a replay of that pass.
## routing: a worker delegates a task only where it needs just the task's conclusion, and each subagent runs on the rung its task's L picks from the project's rung, never below the project's rung when it files or scores todo.md items
@@ -557,6 +578,16 @@ The step is one rung up from the rung the worker would otherwise hold: its own,
The archive quotes the judge's passage under `Escalation:` in the unfinished verdict, the interrupt or the quoted answer. The step an unfinished verdict causes is archived after it. The step an interrupt or a quoted answer causes is archived with it, at once: the interrupt or answer names the rung the worker ran on until then, and a step-up archived beside it names the rung it runs on from then. So the routing figures charge each stretch to its own rung, and a loop that dies while the human answers resumes on the stepped rung. Each archived worker line names the mark once there is one, so a resumed loop keeps it. `bin/routing_figures/` counts none of these steps as an escalation by trigger, which it counts only from subagent prompts.
A stepped worker resumes its session, also when the step changes its model. A plan item, since dropped, proposed a fresh session with a brief of the chunk in its place, after arXiv 2608.24358 measured a fresh strong model ahead of one handed the weak model's transcript. `workflow-bench/handoff_bench.py` benched it on 2026-11-17, on L 3 items this repo shipped. Sonnet 5.5 at high effort started each of six and was stopped at $0.36. It stopped itself on item 163 as it reached the budget, which the bench reads as a finished run, so five items went on, each stopped after 9 to 13 turns. Opus 5.5 at medium effort then finished each of the five three ways: resuming the session, resuming it after `/compact`, and in a fresh session with the item's text and a brief, which says that the unfinished change lies uncommitted in the clone. A judge that did not know the arm graded each change. It saw the item, the change that shipped for it, and the result of the tests run on the worker's tree.
| arm | runs | pass | partial | fail | mean cost $ | mean seconds |
|---|---|---|---|---|---|---|
| resume | 5 | 2 | 3 | 0 | 0.68 | 235 |
| compact | 5 | 1 | 3 | 1 | 0.56 | 198 |
| brief | 5 | 1 | 4 | 0 | 0.78 | 310 |
Resume's verdict matched or beat the brief's on every item, and the brief cost less on two of the five. So the bench gives no reason to pay for a fresh session. Its price is more than a change to how the worker starts. The archive files a chunk under its session's first line. For a chunk that spans two sessions, the archive, and each figure read from it, would have to join them. The bench is small: each arm ran once per item, the judge is Opus 5.5 too, and no first run was longer than 13 turns. Run it again before changing this, with longer first runs, since the paper's gap may open only on a long transcript.
Valid while the worker reports in its transcript the triggers that fired, and while a chunk's session transcript only grows.
## routing: loop-claude measures review rounds and unfinished verdicts per rung every 20 cycles, and calls for a routing change when, of two rungs with 5 or more chunks each, the lower took more of either than the one above, or when 5 or more subagents of one type had reports longer than the tool results they read
@@ -591,7 +622,7 @@ On 2026-11-12, `bin/judge-replay claude-opus-5-5` replayed 60 verdicts, 15 of ea
2026-11-12, claude, raising the judge's floor to xhigh, settled by the reruns below. Serves goal 1.1, at a cost to goal 1.4. Today's records still hold each of the 4 archived quotes, so no case drops out. For fejkdata the quote, `Split a milestone into items in todo.md before starting it`, doesn't plainly answer whether a split changes what release 0.5.0 holds, which is lilleman's choice. That case stays, since xhigh quotes it in 3 of 3 replays. Each case was replayed on today's records and the archived transcript copy. Opus 5.5 at medium returned the quoted answer 7 times in 20: adf-codec 4 of 5 (the fifth as finished), claude-config 3 of 5 (2 finished), plainpages 0 of 5 (5 finished) and fejkdata 0 of 5 (5 unfinished). At xhigh it returned it 9 times in 12: adf-codec 1 of 3 (2 interrupts), claude-config 3 of 3, plainpages 2 of 3 (1 finished) and fejkdata 3 of 3. So the misses persist at medium, and xhigh returns the quote more than twice as often. A quoted answer returned as finished closes a chunk whose worker was asking a question, a quality miss (goal 1.1). xhigh costs more per judge run (goal 1.4); the `Judge cost:` lines of three of the four cases, run at medium, read 0.57 to 0.79. xhigh is not free of misses: it returned a quoted answer as finished once, and as an interrupt twice, which only asks lilleman more.
Valid while the archived verdicts show what a judge should return, and until a new model joins the ladder: replay then.
Valid while the archived verdicts show what a judge should return, and until the rungs the replays tested change: the project's rung, those below it and each rung above it that a review climbs to. Replay then.
## routing: one ladder of (model, effort) rungs, ordered by the Artificial Analysis Intelligence Index, shipped as `bin/model_routing/routing-ladder.json`, which a project may replace with a file of that name in its config's folder; a task's L picks its worker's rung and its E + I its reviewers', by fixed steps counted from the cheapest rung of the project's model
+34
View File
@@ -2413,3 +2413,37 @@ Read most (opens and searches):
- `bin/loop_claude/cycle/judge.py` 3
- `bin/rung_replay/test_rung_replay.py` 3
- `bin/claude_code/entries.py` 2
## Reading cost, chunk a683350bf8de, finished 2026-11-17T01:59:09Z
199406 new input tokens before the first commit.
Read most (opens and searches):
- `docs/decisions.md` 10
- `README.md` 7
- `bin/review_replay/test_review_replay.py` 6
- `bin/claude_code/transcript.py` 4
- `bin/judge_replay/test_judge_replay.py` 4
- `bin/reading_cost/main.py` 4
- `bin/judge_replay/main.py` 3
- `bin/reading_cost/test_reading_cost.py` 3
- `bin/review_replay/replay.py` 3
- `bin/routing_figures/archive.py` 3
## Reading cost, chunk 996aefc2924f, finished 2026-11-17T03:46:23Z
73430 new input tokens before the first commit.
Read most (opens and searches):
- `docs/decisions.md` 17
- `bin/touch_predictions/prediction.py` 4
- `bin/touch_predictions/test_touch_predictions.py` 4
- `bin/adf_codec/convert.py` 2
- `bin/adf_codec/test_adf_codec.py` 2
- `README.md` 1
- `bin/plan_store/jira_epic.py` 1
- `bin/touch_predictions/main.py` 1
- `reviewer-merge-trials/README.md` 1
- `rules/todo-items.md` 1
+1 -1
View File
@@ -65,7 +65,7 @@ Give a reviewer without Bash `git ls-files`'s output in its prompt, since some h
`~/.claude/bin/larv-review` holds this section's mechanical rules: the chunk's review folder, the round count, the paired ruling's base and vote count, the scoring run's trigger; the history's format, its latest places and its count of how often seats agree; and the header of each reviewer's prompt. Run it in the chunk's worktree; `~/.claude/bin/larv-review --help` states each rule. On exit 1, repair what its message names and run it again; when you cannot, ask the user.
**Reports.** Run `~/.claude/bin/larv-review reviewer-header <pass>` in the chunk's worktree, and open each prompt to an agent under `~/.claude/agents/` with what it prints; then say in a few lines what the change does and which files it touches. The command makes the chunk's review folder and writes the branch diff and the plan's listing into it. The header names that diff, the reviewer's full report, `<pass>-round-<n>.md`, in the folder, the folders of technical principles and review standards the project's config sets, the listing, and for architecture and stability the places too. For a reviewer whose definition starts it without `CLAUDE.md`, the header's last line names the file that holds the project principles and decision titles, which the reviewer reads first. For a relaunch one rung up, add `--rerun`: the report becomes `<pass>-round-<n>-rerun.md`, and the header names the report that asked. Before you answer a 🔴 or 🟡, read its entry: Grep the report for the finding's place, with the lines after it, rather than Read the whole report. rule-hook refuses a reviewer's Edit and MultiEdit, and any Write outside `~/.local/state/larv-review/`. Keep the folder after the merge: `review-replay` replays a round from the reports its prompt names.
**Reports.** Run `~/.claude/bin/larv-review reviewer-header <pass>` in the chunk's worktree, and open each prompt to an agent under `~/.claude/agents/` with what it prints; then say in a few lines what the change does and which files it touches. The command makes the chunk's review folder and writes the branch diff and the plan's listing into it. The header names that diff, the reviewer's full report, `<pass>-round-<n>.md`, in the folder, the folders of technical principles and review standards the project's config sets, the listing, and for architecture and stability the places too. For a reviewer whose definition starts it without `CLAUDE.md`, the header's last line names the file that holds the project principles and decision titles, which the reviewer reads first. For a relaunch one rung up, add `--rerun`: the report becomes `<pass>-round-<n>-rerun.md`, and the header names the report that asked. Before you answer a 🔴 or 🟡, read its entry: Grep the report for the finding's place, with the lines after it, rather than Read the whole report. rule-hook refuses a reviewer's Edit and MultiEdit, and any Write outside `~/.local/state/larv-review/`. Keep the folder after the merge: `review-replay` replays a round from the reports its prompt names, and `compress-archives` gzips each file 30 days after it last changed.
**Precedent, then ask** — for the full scan:
+6 -89
View File
@@ -6,18 +6,14 @@
`Bar = 0.1`
`Next ID = 395`
`Next ID = 396`
## Items
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 306 | 0.12.0 | | | **Set a strategy that keeps loop-claude's archive and larv-review's review folders backed up and small over the years, compressing older entries without losing any.** | 3 | 2 | 4 | 7 | 5 | 3 by 5 | 3.9 |
| 387 | 0.12.0 | | | **Bind a project's rung to its ladder in one value, so the routing functions and `Cycle` take one argument for both.** | 3 | 1 | 3 | 5 | 5 | code health by 3 | 2.9 |
| 338 | 0.12.0 | | owner | **Script each step that 0.9.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.** | 1 | 2 | 1 | 2 | 2 | 1.4 by 3, 1.5 by 3 | 2.3 |
| 288 | 0.12.0 | | | **Point the two reading-cost decisions at code health and goal 1.4's counter-signal, in place of the goal 1.1 they cite.** | 1 | 2 | 1 | 2 | 1 | code health by 1, 1.4 by 1 | 2.2 |
| 391 | 0.12.0 | | | **Read and write a Jira epic issue's description and a Confluence record page through the lossless pair, so saving an unedited document leaves it unchanged.** | 3 | 2 | 3 | 6 | 5 | 1.6 by 6, 1.1 by 3 | 1.9 |
| 319 | 0.12.0 | | | **Start a fresh session with a brief of the chunk when a rung step changes the worker's model, in place of resuming the old session.** | 3 | 2 | 3 | 6 | 4 | 1.1 by 2, 1.4 by 2 | 1.2 |
| 299 | 0.12.0 | | | **Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3 | 1.0 |
| 311 | 0.12.0 | | | **Hand each worker's claude a Bash output ceiling that saves longer output to a file, and refuse a Read of a session transcript under `~/.claude/projects`.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
| 316 | 0.12.0 | | | **Launch comprehension seats as an agent type of their own under `agents/`, without `CLAUDE.md` and the skill and agent listings a general-purpose subagent carries.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3 | 1.0 |
@@ -26,14 +22,12 @@
| 310 | 0.12.0 | | | **Compact a waiting worker's session shortly before its prompt cache expires, only when the human has not answered an interrupt by then.** | 3 | 2 | 3 | 6 | 4 | 1.4 by 3 | 0.9 |
| 393 | 0.12.0 | | decision | **Step an L 3 worker one rung up and an L 4 worker two up from the project's rung, as lilleman ruled on 2026-11-14.** | 2 | 1 | 3 | 4 | 4 | 1.4 by 3 | 0.9 |
| 298 | 0.12.0 | | | **Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.** | 3 | 3 | 3 | 7 | 4 | 1.4 by 2, 1.5 by 1 | 0.9 |
| 331 | 0.12.0 | | | **Reword four decisions to expire when the rungs their replays tested change, not whenever a model joins the ladder.** | 1 | 1 | 1 | 1 | 2 | 1.4 by 2 | 0.8 |
| 381 | 0.12.0 | | | **Read a `Touches:` list from a plan kept in Jira as from one in todo.md, with a test that converts a description through `adf_codec`.** | 1 | 2 | 1 | 2 | 2 | 1.5 by 2 | 0.7 |
| 327 | 0.12.0 | | | **Have `larv-review` print a round commit's title prefix, counted from the round's report, and have rule-hook refuse a round title whose counts differ from it.** | 3 | 2 | 3 | 6 | 4 | 1.1 by 1, 1.4 by 1 | 0.6 |
| 389 | 0.12.0 | | decision | **Set a Jira issue's "is blocked by" links from `todo-item edit --blocked-by`, on an issue the loop filed.** | 2 | 2 | 2 | 4 | 4 | 1.6 by 2 | 0.5 |
| 365 | 0.13.0 | | | **Have `score-outcomes` read the finished and dropped items of a plan outside the checkout, from the project repo's history or from Jira's issue changelogs.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
| 368 | 0.13.0 | | | **Call a release review due, call a release ready to cut, and tell whether a chunk owes a scoring run, for a plan outside the checkout, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
| 309 | 0.13.0 | | | **Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.** | 2 | 2 | 2 | 4 | 2 | 1.4 by 3 | 1.2 |
| 366 | 0.13.0 | | | **Count the chunks of a project whose plan sits outside the checkout in the routing figures, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.4 by 2 | 1.1 |
| 395 | 0.13.0 | | decision | **Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.** | 3 | 2 | 2 | 5 | 5 | 1.6 by 4 | 0.8 |
| 367 | 0.13.0 | | | **Check that every write to a Jira plan leaves it ranked, as rule-hook's `todo-rank-check` checks `todo.md` today.** | 2 | 2 | 3 | 5 | 3 | 1.6 by 2 | 0.7 |
| 77 | 0.14.0 | | | **Teach `bin/metric-bench` Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.** | 4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 2.1 |
| 196 | 0.14.0 | | decision | **Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.** | 2 | 2 | 3 | 5 | 2 | 1.4 by 3 | 1.2 |
@@ -74,29 +68,6 @@
## Details
### 306. Set a strategy that keeps loop-claude's archive and larv-review's review folders backed up and small over the years, compressing older entries without losing any.
Measured on 2026-11-11: loop-claude's archive is 6.3 GB (12,721 files from October alone, 0.84 GB older than 30 days), and `~/.local/state/larv-review/` is 997 MB, 984 MB of it under `code/`, with no pruning: `skills/larv-review/SKILL.md` → "Reports" keeps every folder after the merge, because `review-replay` replays rounds from them. The largest archived worker transcript, 23.4 MB, gzips to 9.1 MB at level 6. The disk holds 65 GB free today.
Write the strategy as a decision, then build it:
- **Backup.** Item 1 waits on ai-sandbox's data volume backup; check that both `~/.claude/loop-claude-transcripts` and `~/.local/state/larv-review` sit on that volume, since the review folders live outside `~/.claude`.
- **Retention.** Keep everything: the archive is the evidence every routing and replay decision rests on. Compress entries older than a threshold (such as 30 days, or once a chunk is finished and measured) with a format the tools read transparently, such as zstd. Compress, verify the round trip, then remove the original, never the reverse; a failure keeps the original and stops, as a failed archive does.
- **Readers.** `bin/transcript_archive`, `routing_figures`, `score-outcomes`, `review-replay`, `judge-replay` and `reading-cost` must read compressed entries; test each on one.
- Revise `loop-claude: the archive keeps the raw transcript`, whose validity is "while transcripts stay small enough to copy whole".
Touches:
- `bin/transcript_archive/read.py`: `read_threads`, `finished_chunks`, `chunk_outcomes`
- `bin/transcript_archive/write.py`: `copy_for`, `mirror`
- `bin/transcript_archive/test_transcript_archive.py`
- `bin/routing_figures/archive.py`
- `bin/reading_cost/main.py`
- `bin/score_outcomes/reversals.py`
- `bin/review_replay/rounds.py`
- `bin/judge_replay/main.py`: `archived_cases` (small)
- `skills/larv-review/SKILL.md`: `## 6. Wrap up` (small)
- `README.md`: `## Install` (small)
### 387. Bind a project's rung to its ladder in one value, so the routing functions and `Cycle` take one argument for both.
Filed by PR #553, which made the routing ladder a file a project may replace. Its stability round 1 found two places the comprehension seats name as hard, touched and no easier to read: `bin/loop_claude/routing.py` `unfinished_route` took `ladder` as a fourth parameter, and `Cycle` in `bin/loop_claude/cycle/run.py` gained the field `ladder`. The patch threaded the ladder as an argument, since a wider change did not fit a patch cut during an outage.
@@ -126,36 +97,6 @@ The 0.8.0 audit filed four steps, items 334, 335, 336 and 337, and found most ot
The chunk that ships this item files a copy of this audit in release 0.15.0, the release after this item's own, to audit releases 0.10.0 to 0.14.0, rewording "0.9.0" to them, and marks the copy `owner`, so the audit recurs each release until lilleman drops it.
### 288. Point the two reading-cost decisions at code health and goal 1.4's counter-signal, in place of the goal 1.1 they cite.
Both `docs/decisions.md` entries under `reading cost: a rise is …` and `reading cost: larv-review measures each finished chunk …` open "Serves goal 1.1: comprehension …". No goal says that any more: code health (`CLAUDE.md` → "Shared goal") holds comprehension, and since PR #455 a rising reading cost is goal 1.4's first counter-signal. Raised by the stability review of PR #455 as a 🟢 outside that change.
### 391. Read and write a Jira epic issue's description and a Confluence record page through the lossless pair, so saving an unedited document leaves it unchanged.
`plan_store/jira_epic.py` reads an issue's description and writes it back on an edit, and `record_store/confluence_page.py` reads a page and saves it back; both use the portable pair, which drops each block's `localId` and the empty paragraphs Jira uses for spacing. `record_store/jira_description.py` only reads, to show a person: it stays on the portable pair.
Open before building: the lossless markdown is less readable than the portable one (upstream larvit/adf-codec#185), and agents and the judge read the plan's details from it. Decide how a detail stays readable, for example by reading lossless and showing a person a portable view. Test that an unedited issue and page save back identical. The item ships alone because its R is above 5.
Touches:
- `bin/plan_store/jira_epic.py`: `JiraPlan` (small), `description_of` (small)
- `bin/record_store/confluence_page.py`: `ConfluencePage` (small)
- `bin/plan_store/test_jira_epic.py`: `Editing`, `Filing`
- `bin/record_store/test_confluence_page.py`: `ConfluencePageTest`
### 319. Start a fresh session with a brief of the chunk when a rung step changes the worker's model, in place of resuming the old session.
On SWE-bench Verified with Haiku 4.5 and Opus 4.7, handing the strong model the weak model's transcript solved 69.2% at $1.61 a task; starting the strong model fresh solved 79.2% at $0.90 (arXiv 2608.24358, The Handoff Tax). A model switch also rereads the whole context uncached (Claude Code prompt caching docs). A worker that steps from Sonnet 5.5 to Opus 5.5 by `--resume` is the raw handoff; an effort-only step within one model keeps the cache and isn't. The escalated subagent, which starts fresh with the trigger's evidence, already follows the pattern that won.
Bench it first on `workflow-bench`: resume against a fresh start with a brief. The brief is the chunk's item, its branch and diff, open findings and the trigger. lilleman approved on 2026-11-11. Revise the decision `routing: a worker whose run ends asking for a higher rung…`.
Touches:
- `bin/loop_claude/cycle/run.py`: `Cycle`, `run_cycle`, `worker_start`
- `bin/loop_claude/cycle/prompts.py`: `resumed`, `stepped_to`
- `bin/loop_claude/cycle/worker.py`: `worker_args`
- `bin/loop_claude/cycle/states.py`
- `bin/loop_claude/test_loop_claude.py`: `Cycles`, `WorkerArgs`
- `workflow-bench/bench.py`: `run`, `main`
### 299. Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt.
The judge's LLM still decides three things: the verdict kind, the escalation it quotes, and whether a release is ready to cut (`docs/decisions.md` → `routing: loop-claude reads the items a chunk takes …`). This item replays the verdict kind. No single rule found so far separates interrupts: a "?" in the worker's final message appears in 48 interrupts and misses 66. A smaller LLM already failed the replay (`docs/decisions.md` → `routing: on a model judge-replay covered, …`: Sonnet returned another kind on 3 interrupts, Haiku missed 7). Untested are combined code rules, a decision table, and a small classifier trained and run on this machine on the archived verdicts (372 finished, 234 interrupt, 42 unfinished, 20 quoted). Run each offline, nothing sent to an outside service (goal 4), and count disagreements by kind; the bar is zero missed interrupts. A hosted decision model is out of scope, since it would send transcripts to an outside service.
@@ -266,25 +207,6 @@ Touches:
- `bin/loop_claude/test_loop_claude.py`: `JudgePrompt`, `Records`
- `bin/judge_replay/test_judge_replay.py`
### 331. Reword four decisions to expire when the rungs their replays tested change, not whenever a model joins the ladder.
The architect's 🔀 answer on PR #493 (item 292): four decisions expire "when a new model joins the ladder". Fable 5.2 joined above every project model's rung, and none of these replays tested a rung above the project's, so the event fired with nothing new to test. Every later ladder rebuild would fire all four again.
- `review: each pass keeps a reviewer of its own`: the merge trials ran on Opus 5.5 at medium. Reword the "Valid" line to name that rung.
- `review: the principles audit runs only in the release review…`: its trial ran on the project's rung. Reword the line to name that rung.
- `routing: a review pass runs below the project's rung only on a rung whose replay of its rounds finds a share of their 🔴 and 🟡 findings no more than a margin below the project's rung's own replay…`: `review-replay` tests only rungs below the project's rung. Have the line expire when those rungs change, the event item 196 names ("until a cheaper model joins the ladder").
- `routing: on a model judge-replay covered, the judge runs…`: the replays tested the project's rung and below, and each rung above it that a review climbs to. Have the line expire when those rungs change.
lilleman approved rewording all four as above on 2026-11-12.
### 381. Read a `Touches:` list from a plan kept in Jira as from one in todo.md, with a test that converts a description through `adf_codec`.
The product-owner pass of item 373's chunk asked it, 2026-11-15: `bin/touch_predictions/prediction.py` → `predicted` needs a line that is exactly `Touches:` and bullets that open `- `. A Jira issue's description reaches it through `bin/adf_codec/`, whose library chooses the bullet marker and may escape backticks or a trailing colon. No test shows that a list written in Jira comes out in that shape. Where it doesn't, read the shape the codec writes too.
Touches:
- `bin/touch_predictions/prediction.py`: `predicted` (small), `bullet` (small)
- `bin/touch_predictions/test_touch_predictions.py`: `PredictedTest`
### 327. Have `larv-review` print a round commit's title prefix, counted from the round's report, and have rule-hook refuse a round title whose counts differ from it.
The 0.8.0 audit (item 284) measured, over the 35 chunks that shipped 0.7.0: 72 commit calls in 33 chunks titled `Answer <pass> round <n>: R 🔴, Y 🟡 (K misread); <what changed>`, about 11k command tokens, most of them the summary. Each worker counts R, Y and K by hand. All three are in the round's report, `<pass>-round-<n>.md` in the review folder: R and Y are its finding headings, and K counts the findings the reviewer marked `[misread]`, as `bin/review_rules/round_titles.py` says. That module parses the counts for `score-outcomes`, which reports them as goal 1.1's signal, and rule-hook's `round_cap.py` checks only the round number, so nothing checks today that the counts match the report. Of the 8 titles whose report the audit could match, none miscounted; one report wrote its markers at line start rather than in `###` headings, so the reports' format varies by reviewer.
@@ -304,15 +226,6 @@ Touches:
- `skills/larv-review/SKILL.md`: `## 3. Passes` (small)
- `README.md`: `## Tools` (small)
### 389. Set a Jira issue's "is blocked by" links from `todo-item edit --blocked-by`, on an issue the loop filed.
Filed by item 382's chunk on 2026-11-16. `JiraPlan.edit` refuses an edit of Blocked by, saying to link the issue in Jira, so a worker cannot mark an item it filed as blocked once it exists. Read each link's id from the search's `issuelinks`, make a link per new blocker as `add` does, and delete one per blocker removed, after `missing_blocker`'s check. Say in `docs/decisions.md` whether the loop may link an issue a person filed; the decision on the Jira epic as a plan says the loop writes only the scores of such an issue.
Touches:
- `bin/plan_store/jira_epic.py`: `Issue`, `issue_of`, `JiraPlan.edit`, `revision_of`
- `bin/plan_store/test_jira_epic.py`: `Editing`
- `bin/todo_item/main.py`: lines 1-77 (small)
### 365. Have `score-outcomes` read the finished and dropped items of a plan outside the checkout, from the project repo's history or from Jira's issue changelogs.
lilleman's call, 2026-11-12: 0.10.0 and 0.11.0 switch the history tools off for a plan outside the checkout, and 0.13.0 holds only their ports. In Jira a dropped item, which `Drop item` commit titles mark today, is an issue resolved by a resolution the config names.
@@ -354,6 +267,10 @@ Touches:
One of 0.13.0's history ports, per lilleman's call of 2026-11-12.
### 395. Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.
Today a Jira detail edited in one line, or a Confluence record with one entry appended, is written back whole by adf-codec's portable pair, which drops each block's localId and the empty paragraphs Jira uses for spacing. A save whose text is the text a read showed already sends nothing (`docs/decisions.md` → `project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, …`). Match the saved markdown's top-level blocks against the held document's, each as the portable pair reads it, and send the held block where they match. The decision's limit on partial saves ends with this item.
### 367. Check that every write to a Jira plan leaves it ranked, as rule-hook's `todo-rank-check` checks `todo.md` today.
One of 0.13.0's history ports, per lilleman's call of 2026-11-12.
+15 -9
View File
@@ -146,19 +146,25 @@ def run(work, name):
"--no-session-persistence", "--output-format", "stream-json", "--verbose"],
cwd=clone, stdout=out, stderr=subprocess.STDOUT)
read_result((work / f"{name}.jsonl").read_text().splitlines())
git(clone, "add", "--all")
(work / f"{name}.diff").write_text(git(clone, "diff", "--cached", base))
changed = [*git(clone, "diff", "--cached", "--name-only", base).split(), *git(REPO, "diff", "--name-only", base, CASES[case][1]).split()]
targets = [target for target in suite_targets(changed) if (clone / target).exists()] or ["bin"]
tests = subprocess.run([str(RUN_HEAVY), *SUITE, *targets], cwd=clone, capture_output=True, text=True)
(work / f"{name}.tests").write_text(f"exit {tests.returncode}\n{tests.stdout[-1500:]}")
save_change(work, name, clone, CASES[case])
return f"{name}: ran"
def grade(work, name):
"""Have the judge rule on one run's diff, blind to its arm, and save the verdict beside it."""
def save_change(work, name, clone, commits):
"""Save the clone's change since the case's start commit as <name>.diff, and as <name>.tests the result of the tests of the packages that the clone's change or the merged change touches."""
base, merged = commits
git(clone, "add", "--all")
(work / f"{name}.diff").write_text(git(clone, "diff", "--cached", base))
changed = [*git(clone, "diff", "--cached", "--name-only", base).split(), *git(REPO, "diff", "--name-only", base, merged).split()]
targets = [target for target in suite_targets(changed) if (clone / target).exists()] or ["bin"]
tests = subprocess.run([str(RUN_HEAVY), *SUITE, *targets], cwd=clone, capture_output=True, text=True)
(work / f"{name}.tests").write_text(f"exit {tests.returncode}\n{tests.stdout[-1500:]}")
def grade(work, name, cases=None):
"""Have the judge rule on one run's diff, blind to its arm, and save the verdict beside it. cases stands in for CASES where another bench grades its own."""
case = split_name(name)[0]
base, merged = CASES[case]
base, merged = (cases or CASES)[case]
prompt = (BENCH / "advisor-judge-prompt.md").read_text().format(
text=item_text(git(REPO, "show", f"{base}:todo.md"), case), reference=git(REPO, "diff", base, merged, "--", ".", ":!todo.md", ":!docs", ":!CHANGELOG.md"),
diff=(work / f"{name}.diff").read_text(), tests=(work / f"{name}.tests").read_text())
+4
View File
@@ -0,0 +1,4 @@
## Handoff
A worker on a weaker model began this item and stopped before it was done, asking for a stronger model. You start fresh, without its transcript. Its unfinished change is in the clone, uncommitted: read `git status` and `git diff` first. Keep what holds, redo what doesn't, and finish the item.
+1
View File
@@ -0,0 +1 @@
Continue the task; the previous run ended before the item was done. Your request for a higher rung went through: you now run on {rung}. The bench rules still hold.
+200
View File
@@ -0,0 +1,200 @@
#!/usr/bin/env python3
"""Bench how a worker's step to a stronger model hands the chunk over: handoff_bench.py <work dir> start <case>-<n> ..., then run|grade <case>-<arm>-<n> ..., then summary
The question: when loop-claude steps a worker to a rung on another model, which handoff serves the stronger model
best? Each case is an archived chunk of this repo, as in advisor_bench.py, whose judge and files this bench shares.
- `start` runs WEAK on the item in the clone <work dir>/<case>-<n> until it has spent START_BUDGET, which
stands in for a run that ends asking for a higher rung. A worker that finishes within the budget leaves no
handoff, so `start` fails on it. It saves the change so far (<case>-start-<n>.diff), and a copy of the clone
(.tree), which every arm starts from, and a copy of the session's transcript (.session.jsonl), which the resume
and compact arms continue.
- `run` puts the clone and the session back as `start` left them, then continues on STRONG, one arm of a case at a
time. The resume arm resumes WEAK's session with handoff-resume-prompt.md. The compact arm does the same once WEAK
has compacted the session with `/compact`, whose stream it saves as <name>.compact.jsonl. The brief arm starts a
session that has the item and handoff-brief-prompt.md, and no transcript.
- `grade` is advisor_bench.py's, over these cases. `summary` counts an arm's cost and seconds from its own runs,
the compaction included, and leaves out the start, which the arms share (`outcome`).
Run each arm of each case once; the table shows a direction, not a significance.
"""
import collections
import concurrent.futures
import json
import pathlib
import shutil
import subprocess
import sys
import threading
import uuid
import advisor_bench
from advisor_bench import Arm
BENCH = pathlib.Path(__file__).resolve().parent
# The ladder's first step that changes the model: bin/model_routing/routing-ladder.json.
WEAK = Arm("claude-sonnet-5-5", "high")
STRONG = Arm("claude-opus-5-5", "medium")
ARMS = ("resume", "compact", "brief")
# Claude Code keeps a session's transcript at <CLAUDE_PROJECTS>/<the working directory's slug>/<session id>.jsonl.
CLAUDE_PROJECTS = pathlib.Path.home() / ".claude/projects"
# One per case and run number: its arms share one clone, so they run one at a time.
CLONE_LOCKS: collections.defaultdict = collections.defaultdict(threading.Lock)
# L 3 chunks, by todo item. advisor_bench's L 1 and L 2 chunks are too small: Sonnet 5.5 finished five of the six for
# under $0.20, before any handoff.
CASES = {
"143": ("45cb331a4", "545f3c347"),
"163": ("d4cf6e7fe", "d1b0fe3a4"),
"198": ("4a0ba5921", "fd60b6da"),
"210": ("4764d3d52", "dcbdc970"),
"227": ("eb7b50698", "3bde9cd6d"),
"239": ("d583f8d48", "47a40757"),
}
# Dollars WEAK may spend before the handoff. In a trial, $0.36 stopped Sonnet 5.5 on items 163 and 227 after 11 to 14
# turns, as the result line's `num_turns` counts them, with about 250 lines of diff.
START_BUDGET = 0.36
CLAUDE_FLAGS = ["--setting-sources", "project", "--permission-mode", "bypassPermissions", "--output-format", "stream-json", "--verbose"]
def stopped_at_budget(lines):
"""Whether a claude stream's first result line, the worker's own, says the run stopped at --max-budget-usd."""
for line in lines:
try:
entry = json.loads(line)
except ValueError:
continue
if isinstance(entry, dict) and entry.get("type") == "result":
return entry.get("subtype") == "error_max_budget_usd"
return False
def run_prompt(case):
"""advisor_bench's prompt for the case's item, as the item's text stood at the start commit."""
base = CASES[case][0]
text = advisor_bench.item_text(advisor_bench.git(advisor_bench.REPO, "show", f"{base}:todo.md"), case)
return (BENCH / "advisor-run-prompt.md").read_text().format(base=base, item=case, text=text)
def claude(prompt, flags, clone, stream):
with stream.open("w") as out:
subprocess.run(["claude", "-p", prompt, *flags, *CLAUDE_FLAGS], cwd=clone, stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT)
def start(work, name):
"""WEAK's capped run of <case>-<n>, and the tree and session it leaves the arms; the files are in the module docstring."""
case, number = name.split("-")
base = CASES[case][0]
clone, saved = work / name, work / f"{case}-start-{number}"
if clone.exists():
raise ValueError(f"{name} already started: remove {clone}")
advisor_bench.fresh_clone(base, clone)
session = str(uuid.uuid4())
stream = saved.with_suffix(".jsonl")
claude(run_prompt(case), [*WEAK.flags(), "--session-id", session, "--max-budget-usd", str(START_BUDGET)], clone, stream)
if not stopped_at_budget(stream.read_text().splitlines()):
raise ValueError("the weak worker finished within its budget, or its run failed, so there is no handoff to bench")
transcripts = list(CLAUDE_PROJECTS.glob(f"*/{session}.jsonl"))
if len(transcripts) != 1:
raise ValueError(f"expected one transcript of session {session} under {CLAUDE_PROJECTS}, found {len(transcripts)}")
shutil.copy(transcripts[0], saved.with_suffix(".session.jsonl"))
saved.with_suffix(".session").write_text(str(transcripts[0]))
# The worker may have committed against the bench rules, so the arms get its whole change since base, uncommitted.
advisor_bench.git(clone, "add", "--all")
saved.with_suffix(".diff").write_text(advisor_bench.git(clone, "diff", "--cached", base))
advisor_bench.git(clone, "reset", "-q", "--mixed", base)
shutil.copytree(clone, saved.with_suffix(".tree"), symlinks=True)
return f"{name}: started"
def run(work, name):
"""STRONG's run of one arm, <case>-<arm>-<n>, from the tree and session `start` left."""
case, arm, number = advisor_bench.split_name(name)
if arm not in ARMS:
raise ValueError(f"the arm {arm!r} is none of {ARMS}")
clone, saved = work / f"{case}-{number}", work / f"{case}-start-{number}"
if not saved.with_suffix(".tree").exists():
raise ValueError(f"start {case}-{number} first")
stream = work / f"{name}.jsonl"
if stream.exists():
raise ValueError(f"{name} already ran: remove {stream}")
with CLONE_LOCKS[clone]:
shutil.rmtree(clone)
shutil.copytree(saved.with_suffix(".tree"), clone, symlinks=True)
transcript = pathlib.Path(saved.with_suffix(".session").read_text())
shutil.copy(saved.with_suffix(".session.jsonl"), transcript)
resumed = ["--resume", transcript.stem]
if arm == "compact":
claude("/compact", [*WEAK.flags(), *resumed], clone, work / f"{name}.compact.jsonl")
finished(work / f"{name}.compact.jsonl")
if arm == "brief":
claude(run_prompt(case) + "\n" + (BENCH / "handoff-brief-prompt.md").read_text(), [*STRONG.flags(), "--no-session-persistence"], clone, stream)
else:
prompt = (BENCH / "handoff-resume-prompt.md").read_text().format(rung=f"{STRONG.model} at {STRONG.effort} effort")
claude(prompt, [*STRONG.flags(), *resumed], clone, stream)
outcome(work, name)
advisor_bench.save_change(work, name, clone, CASES[case])
return f"{name}: ran"
def results(stream):
"""The result lines of a claude stream. A resumed run may open with one that holds no work, and a background task's notification adds one."""
found = []
for line in stream.read_text().splitlines():
try:
entry = json.loads(line)
except ValueError:
continue
if isinstance(entry, dict) and entry.get("type") == "result":
found.append(entry)
return found
def finished(stream):
"""The result lines of a run that ended well; a run without one, or whose last one is an error, raises."""
ran = results(stream)
if not ran or ran[-1].get("is_error"):
raise ValueError(f"the run ended in an error: {ran[-1].get('result')}" if ran else "the stream has no result line")
return ran
def outcome(work, name):
"""The cost and seconds of an arm's run, its compaction included. A resumed session's cost counts from the session's start, so the start's cost comes off it."""
case, arm, number = advisor_bench.split_name(name)
ran = finished(work / f"{name}.jsonl")
compacted = results(work / f"{name}.compact.jsonl") if arm == "compact" else []
before = 0 if arm == "brief" else results(work / f"{case}-start-{number}.jsonl")[-1]["total_cost_usd"]
return {"cost": ran[-1]["total_cost_usd"] - before, "seconds": sum(entry["duration_ms"] for entry in [*compacted, *ran]) / 1000}
def rows_of(work):
"""One row per graded run: its arm, verdict, cost and seconds."""
rows = []
for verdict_file in sorted(work.glob("*.verdict")):
name = verdict_file.stem
rows.append({"arm": advisor_bench.split_name(name)[1], "verdict": json.loads(verdict_file.read_text())["verdict"], **outcome(work, name)})
return rows
def summary(rows):
"""A markdown table with one row per arm, in ARMS' order: its runs, passes, partials, mean cost and mean seconds."""
lines = ["| arm | runs | pass | partial | mean cost $ | mean seconds |", "|---|---|---|---|---|---|"]
for name in ARMS:
own = [row for row in rows if row["arm"] == name]
if own:
passes, partials = (sum(row["verdict"] == kind for row in own) for kind in ("pass", "partial"))
cost, seconds = (sum(row[key] for row in own) / len(own) for key in ("cost", "seconds"))
lines.append(f"| {name} | {len(own)} | {passes} | {partials} | {cost:.2f} | {seconds:.0f} |")
return "\n".join(lines)
def main(argv):
work, step, names = pathlib.Path(argv[0]), argv[1], argv[2:]
if step == "summary":
print(summary(rows_of(work)))
return
steps = {"start": start, "run": run, "grade": lambda work, name: advisor_bench.grade(work, name, CASES)}
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as pool:
for result in pool.map(lambda name: advisor_bench.attempted(steps[step], work, name), names):
print(result, flush=True)
if __name__ == "__main__":
main(sys.argv[1:])
+258
View File
@@ -0,0 +1,258 @@
import json
import pathlib
import subprocess
import tempfile
import unittest
import unittest.mock
import uuid
import advisor_bench
from test_advisor_bench import commit, TODO
import handoff_bench
SESSION = uuid.UUID("11111111-2222-4333-8444-555555555555")
def result_line(**fields):
return json.dumps({"type": "result", "subtype": "success", "total_cost_usd": 0.5, "duration_ms": 12000, **fields})
class StoppedAtBudget(unittest.TestCase):
def test_holds_where_the_first_result_is_the_budget_stop(self):
self.assertTrue(handoff_bench.stopped_at_budget(["not json", '{"type": "assistant"}', result_line(subtype="error_max_budget_usd", is_error=True)]))
def test_fails_where_the_worker_finished_within_the_budget(self):
self.assertFalse(handoff_bench.stopped_at_budget([result_line()]))
def test_fails_where_the_stream_has_no_result(self):
self.assertFalse(handoff_bench.stopped_at_budget(['{"type": "assistant"}', "[]"]))
class Summary(unittest.TestCase):
def test_prints_one_row_per_arm_with_passes_and_means(self):
rows = [
{"arm": "brief", "verdict": "pass", "cost": 0.5, "seconds": 100.0},
{"arm": "brief", "verdict": "partial", "cost": 1.5, "seconds": 300.0},
{"arm": "resume", "verdict": "fail", "cost": 2.0, "seconds": 50.0},
{"arm": "compact", "verdict": "pass", "cost": 0.25, "seconds": 9.0},
]
self.assertEqual(handoff_bench.summary(rows), "\n".join([
"| arm | runs | pass | partial | mean cost $ | mean seconds |",
"|---|---|---|---|---|---|",
"| resume | 1 | 0 | 0 | 2.00 | 50 |",
"| compact | 1 | 1 | 0 | 0.25 | 9 |",
"| brief | 2 | 1 | 1 | 1.00 | 200 |",
]))
class Orchestration(unittest.TestCase):
"""start, run and main on a throwaway repo, with claude and run-heavy faked. Case 7 starts at a commit holding bin/pkg/mod.py and merges a change to it. The fake claude writes a session's transcript as Claude Code does, and each resumed run adds a line to it."""
def setUp(self):
scratch = tempfile.TemporaryDirectory()
self.addCleanup(scratch.cleanup)
root = pathlib.Path(scratch.name)
self.repo, self.work, self.projects = root / "repo", root / "work", root / "projects"
self.work.mkdir()
subprocess.run(["git", "init", "-q", str(self.repo)], check=True)
(self.repo / "link").symlink_to("todo.md")
self.base = self.commit({"todo.md": TODO, "bin/pkg/mod.py": "x = 1\n", "bin/other/o.py": "o = 1\n"})
merged = self.commit({"bin/pkg/mod.py": "x = 2\n"})
self.commit({"bin/other/o.py": "o = 2\n"})
self.calls, self.heavy = [], []
self.start_result = {"subtype": "error_max_budget_usd", "is_error": True, "total_cost_usd": 1.0}
self.compact_result, self.strong_result = {}, {}
real_run = subprocess.run
for patch in (
unittest.mock.patch.object(advisor_bench, "REPO", self.repo),
unittest.mock.patch.object(handoff_bench, "CASES", {"7": (self.base, merged)}),
unittest.mock.patch.object(handoff_bench, "START_BUDGET", 0.25),
unittest.mock.patch.object(handoff_bench, "CLAUDE_PROJECTS", self.projects),
unittest.mock.patch.object(handoff_bench.uuid, "uuid4", return_value=SESSION),
unittest.mock.patch.object(advisor_bench.subprocess, "run", side_effect=lambda args, **kwargs: self.fake(real_run, args, **kwargs)),
):
patch.start()
self.addCleanup(patch.stop)
self.transcript = self.projects / "slug" / f"{SESSION}.jsonl"
def commit(self, files):
for path, text in files.items():
(self.repo / path).parent.mkdir(parents=True, exist_ok=True)
(self.repo / path).write_text(text)
subprocess.run(["git", "-C", str(self.repo), "add", "."], check=True)
commit(self.repo, "a commit")
return advisor_bench.git(self.repo, "rev-parse", "HEAD").strip()
def fake(self, real_run, args, **kwargs):
if args[0] == "claude":
cwd = pathlib.Path(kwargs["cwd"])
# What the run found as it began: the session's transcript and the files under bin/pkg.
found = (self.transcript.read_text() if self.transcript.exists() else None, sorted(path.name for path in (cwd / "bin/pkg").iterdir()))
self.calls.append((args, kwargs, found))
if "--max-budget-usd" in args:
(cwd / "bin/pkg/mod.py").write_text("x = 3\n")
# Against the bench rules, the worker commits a part of its change.
real_run(["git", "-C", str(cwd), "add", "bin/pkg/mod.py"], check=True)
commit(cwd, "against the rules")
(cwd / "bin/pkg/new.py").write_text("n = 1\n")
self.transcript.parent.mkdir(parents=True)
self.transcript.write_text("weak\n")
kwargs["stdout"].write(result_line(**self.start_result) + "\n")
elif args[2] == "/compact":
self.transcript.write_text(self.transcript.read_text() + "compacted\n")
kwargs["stdout"].write(result_line(**self.compact_result, total_cost_usd=1.25, duration_ms=3000) + "\n")
else:
if "--resume" in args:
self.transcript.write_text(self.transcript.read_text() + "strong\n")
# A resumed session counts its cost from the session's start, and may open with a result that holds no work.
kwargs["stdout"].write(result_line(total_cost_usd=1.0, duration_ms=0) + "\n")
(cwd / "bin/pkg" / f"done_{len(self.calls)}.py").write_text("d = 1\n")
kwargs["stdout"].write("a line that is not JSON\n")
kwargs["stdout"].write(result_line(**self.strong_result, total_cost_usd=3.0 if "--resume" in args else 0.5) + "\n")
return subprocess.CompletedProcess(args, 0)
if args[0] == str(advisor_bench.RUN_HEAVY):
self.heavy.append(args)
return subprocess.CompletedProcess(args, 0, stdout="3 passed\n", stderr="")
return real_run(args, **kwargs)
def started(self):
handoff_bench.start(self.work, "7-1")
self.calls.clear()
def test_start_runs_the_weak_worker_to_its_budget_and_saves_the_tree_and_the_session(self):
self.assertEqual(handoff_bench.start(self.work, "7-1"), "7-1: started")
(claude, kwargs, _), = self.calls
self.assertEqual(kwargs["cwd"], self.work / "7-1")
self.assertEqual((kwargs["stdin"], kwargs["stderr"]), (subprocess.DEVNULL, subprocess.STDOUT))
self.assertEqual(claude[:2] + claude[3:], [
"claude", "-p", *handoff_bench.WEAK.flags(), "--session-id", str(SESSION), "--max-budget-usd", "0.25", "--setting-sources", "project",
"--permission-mode", "bypassPermissions", "--output-format", "stream-json", "--verbose"])
self.assertIn("Do the seventh thing.", claude[2])
self.assertIn(f"a throwaway clone at `{self.base}`. Implement todo item 7 below", claude[2])
self.assertEqual((self.work / "7-start-1.session").read_text(), str(self.transcript))
self.assertEqual((self.work / "7-start-1.session.jsonl").read_text(), "weak\n")
self.assertIn("-x = 1\n+x = 3\n", (self.work / "7-start-1.diff").read_text())
self.assertIn("+n = 1\n", (self.work / "7-start-1.diff").read_text())
tree = self.work / "7-start-1.tree"
self.assertEqual(((tree / "bin/pkg/mod.py").read_text(), (tree / "bin/pkg/new.py").read_text()), ("x = 3\n", "n = 1\n"))
self.assertEqual(advisor_bench.git(tree, "status", "--porcelain"), " M bin/pkg/mod.py\n?? bin/pkg/new.py\n")
self.assertEqual(advisor_bench.git(tree, "rev-parse", "HEAD").strip(), self.base)
self.assertTrue((tree / "link").is_symlink())
def test_start_refuses_a_worker_that_finished_within_its_budget(self):
self.start_result = {}
with self.assertRaisesRegex(ValueError, "^the weak worker finished within its budget, or its run failed, so there is no handoff to bench$"):
handoff_bench.start(self.work, "7-1")
def test_start_refuses_a_name_that_already_started(self):
(self.work / "7-1").mkdir()
with self.assertRaisesRegex(ValueError, "already started"):
handoff_bench.start(self.work, "7-1")
def test_start_refuses_a_session_whose_transcript_it_cannot_find(self):
self.transcript = self.work / "elsewhere" / f"{SESSION}.jsonl"
with self.assertRaisesRegex(ValueError, "found 0"):
handoff_bench.start(self.work, "7-1")
def test_the_resume_arm_resumes_the_weak_workers_session_on_the_strong_rung(self):
self.started()
self.assertEqual(handoff_bench.run(self.work, "7-resume-1"), "7-resume-1: ran")
(claude, kwargs, _), = self.calls
self.assertEqual(kwargs["cwd"], self.work / "7-1")
self.assertEqual(claude[:2] + claude[3:], [
"claude", "-p", *handoff_bench.STRONG.flags(), "--resume", str(SESSION), "--setting-sources", "project",
"--permission-mode", "bypassPermissions", "--output-format", "stream-json", "--verbose"])
self.assertEqual(claude[2], (handoff_bench.BENCH / "handoff-resume-prompt.md").read_text().format(rung="claude-opus-5-5 at medium effort"))
diff = (self.work / "7-resume-1.diff").read_text()
self.assertIn("+x = 3\n", diff)
self.assertIn("+++ b/bin/pkg/done_1.py", diff)
self.assertEqual((self.work / "7-resume-1.tests").read_text(), "exit 0\n3 passed\n")
self.assertEqual(self.heavy[-1][-1], "bin/pkg")
self.assertNotIn("bin/other", self.heavy[-1])
self.assertTrue((self.work / "7-1" / "link").is_symlink())
self.assertEqual(handoff_bench.outcome(self.work, "7-resume-1"), {"cost": 2.0, "seconds": 12.0})
def test_the_compact_arm_has_the_weak_model_compact_the_session_before_the_strong_one_resumes_it(self):
self.started()
handoff_bench.run(self.work, "7-compact-1")
(compact, _, _), (claude, _, found) = self.calls
self.assertEqual(compact[:2] + compact[3:], [
"claude", "-p", *handoff_bench.WEAK.flags(), "--resume", str(SESSION), "--setting-sources", "project",
"--permission-mode", "bypassPermissions", "--output-format", "stream-json", "--verbose"])
self.assertEqual(compact[2], "/compact")
self.assertEqual(claude[3:7], [*handoff_bench.STRONG.flags()][:4])
self.assertIn("--resume", claude)
self.assertEqual(found[0], "weak\ncompacted\n")
self.assertEqual(handoff_bench.outcome(self.work, "7-compact-1"), {"cost": 2.0, "seconds": 15.0})
def test_the_brief_arm_starts_a_fresh_session_with_the_item_and_the_brief(self):
self.started()
self.assertEqual(handoff_bench.run(self.work, "7-brief-1"), "7-brief-1: ran")
(claude, kwargs, _), = self.calls
self.assertEqual(kwargs["cwd"], self.work / "7-1")
self.assertEqual(claude[:2] + claude[3:], [
"claude", "-p", *handoff_bench.STRONG.flags(), "--no-session-persistence", "--setting-sources", "project",
"--permission-mode", "bypassPermissions", "--output-format", "stream-json", "--verbose"])
self.assertIn("### 7. Do the seventh thing.", claude[2])
self.assertTrue(claude[2].endswith("- End with a two-line summary of what you changed.\n\n\n## Handoff\n\n" + (handoff_bench.BENCH / "handoff-brief-prompt.md").read_text().split("## Handoff\n\n")[1]), claude[2][-400:])
self.assertIn("+++ b/bin/pkg/done_1.py", (self.work / "7-brief-1.diff").read_text())
def test_each_arm_begins_on_the_tree_and_the_session_that_start_left(self):
self.started()
for arm in ("resume", "compact", "brief"):
handoff_bench.run(self.work, f"7-{arm}-1")
self.assertEqual([found for _, _, found in self.calls if found[0] != "weak\ncompacted\n"], [("weak\n", ["mod.py", "new.py"])] * 3)
self.assertNotIn("done_1.py", (self.work / "7-brief-1.diff").read_text())
def test_run_refuses_an_arm_that_already_ran_or_never_started(self):
with self.assertRaisesRegex(ValueError, "start 7-1 first"):
handoff_bench.run(self.work, "7-brief-1")
self.started()
handoff_bench.run(self.work, "7-brief-1")
with self.assertRaisesRegex(ValueError, "already ran"):
handoff_bench.run(self.work, "7-brief-1")
def test_run_refuses_an_unknown_arm(self):
self.started()
with self.assertRaisesRegex(ValueError, "arm"):
handoff_bench.run(self.work, "7-sonnet-1")
def test_run_refuses_an_arm_whose_run_ended_in_an_error_before_it_tests(self):
self.started()
self.strong_result = {"is_error": True, "result": "overloaded"}
with unittest.mock.patch.object(advisor_bench, "save_change") as save_change:
with self.assertRaisesRegex(ValueError, "ended in an error: overloaded"):
handoff_bench.run(self.work, "7-brief-1")
save_change.assert_not_called()
def test_run_refuses_a_compact_arm_whose_compaction_failed_before_the_strong_model_runs(self):
self.started()
self.compact_result = {"is_error": True, "result": "no conversation to compact"}
with self.assertRaisesRegex(ValueError, "^the run ended in an error: no conversation to compact$"):
handoff_bench.run(self.work, "7-compact-1")
self.assertEqual([args[2] for args, _, _ in self.calls], ["/compact"])
self.assertFalse((self.work / "7-compact-1.jsonl").exists())
def test_outcome_refuses_a_stream_with_no_result_line(self):
self.started()
(self.work / "7-brief-1.jsonl").write_text("not json\n[]\n")
with self.assertRaisesRegex(ValueError, "^the stream has no result line$"):
handoff_bench.outcome(self.work, "7-brief-1")
def test_main_grades_with_the_handoff_cases_and_prints_the_summary(self):
with unittest.mock.patch("builtins.print") as printed:
handoff_bench.main([str(self.work), "start", "7-1"])
handoff_bench.main([str(self.work), "run", "7-brief-1"])
self.assertEqual([call.args[0] for call in printed.call_args_list], ["7-1: started", "7-brief-1: ran"])
with unittest.mock.patch.object(advisor_bench, "grade", return_value="7-brief-1: pass") as grade, unittest.mock.patch("builtins.print") as printed:
handoff_bench.main([str(self.work), "grade", "7-brief-1"])
grade.assert_called_once_with(self.work, "7-brief-1", handoff_bench.CASES)
printed.assert_called_once_with("7-brief-1: pass", flush=True)
(self.work / "7-brief-1.verdict").write_text(json.dumps({"verdict": "pass"}))
with unittest.mock.patch("builtins.print") as printed:
handoff_bench.main([str(self.work), "summary"])
self.assertIn("| brief | 1 | 1 | 0 | 0.50 | 12 |", printed.call_args.args[0])
if __name__ == "__main__":
unittest.main()