Merge remote-tracking branch 'origin/main' into item-319
CI / tests (pull_request) Successful in 3m24s

# Conflicts:
#	todo.md
This commit was merged in pull request #575.
This commit is contained in:
2026-11-17 04:01:42 +01:00
30 changed files with 670 additions and 74 deletions
+3
View File
@@ -47,6 +47,9 @@
!bin/command_lines/*.py
!bin/command_patterns/
!bin/command_patterns/*.py
!bin/compress-archives
!bin/compress_archives/
!bin/compress_archives/*.py
!bin/deploy-release
!bin/deploy_release/
!bin/deploy_release/*.py
+2
View File
@@ -2,6 +2,8 @@
## Unreleased
- `todo-item edit --detail` and `project-record save` leave a Jira description or a Confluence page exactly as it is, with its block ids and spacing paragraphs, where the text saved is the text a read showed. An unedited save over a page a teammate saved since still refuses.
- Add `compress-archives`, which gzips each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them. The tools that read those folders read a gzipped file by its plain name. A new crontab line in the README's Install section runs it daily.
- review-replay and judge-replay keep each run, with its prompt, claude's exit code and stream, and the report they print, in a `replays/<tool>-<arguments>-<UTC stamp>/` folder of loop-claude's archive, so a replay that decided a rung can be checked and re-priced later; claude's json replies are now stream-json, whose last line is the reply.
- loop-claude's archive keeps each judge run's prompt and json reply in a judge/ folder beside its chunk's transcripts, and names the run on a `Judge run:` line of the outcome it produced; judge-replay replays a verdict on its archived prompt where the archive holds one.
- A plan kept as a Jira epic takes no changelog bullet: `todo-item finish` there needs neither `--changelog` nor `--no-changelog`, and refuses `--changelog`, since the issues' history is the changelog.
+10 -1
View File
@@ -73,6 +73,14 @@ Then pick a model with `/model`.
Link `loop-claude` onto `PATH`: `ln -s ~/.claude/bin/loop-claude ~/.local/bin/loop-claude`. Every claude it starts, the interactive ones included, skips every permission prompt. loop-claude runs each chunk in its own window of a tmux session per project, so it needs tmux 3.2 or later.
Back up two folders: `~/.claude/loop-claude-transcripts`, where loop-claude keeps every transcript, and `~/.local/state/larv-review`, where larv-review keeps each chunk's reports. To keep them small, add a crontab line that gzips each transcript and review file 30 days after it last changed, and logs each run with its date:
```sh
17 4 * * * { date; $HOME/.claude/bin/compress-archives; } >> $HOME/.local/state/compress-archives.log 2>&1
```
Cron runs it with its own `PATH` and environment. Where the `python3` on cron's `PATH` is older than 3.14, or you set `LOOP_CLAUDE_TRANSCRIPTS`, set `PATH` or that variable in the crontab too.
To get loop-claude's questions and stops as DMs from a bot, write the bot's token, your username or member ID and, for Mattermost, the server into `~/.claude-projects/<key>.env`, in the keys `~/.claude/bin/notify --help` names, run `chmod 600` on it, and add `env-file = "~/.claude-projects/<key>.env"` and `notifier = "mattermost"` (or `"slack"`) to the project's config, as Install → "Set up a project" describes.
### Set up a project
@@ -171,6 +179,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `chunk-pr open|sync|ready|merge|clean` opens a chunk's pull request as a draft, brings its branch level with origin's default branch, marks it ready and merges it once its checks pass, with `gh` on a GitHub origin and `tea` elsewhere. The chunks of one project merge one at a time: in its turn, `merge` brings the branch level again, finishes the plan items its `--finish` flags name, and waits for the checks; once the pull request is merged, `clean` removes the chunk's worktree and branch and fast-forwards the main checkout. `chunk-pr --help` states the steps.
- `chunk-preflight` finds the mechanical facts a chunk starts from, and with `--worktree` makes its worktree; `loop-claude` runs it with `--worktree` before it routes each chunk, and `chunk-preflight --help` lists the facts.
- `command-patterns [<version>] [--calls <file>]` prints a table of the shell command patterns a release's archived chunks ran, such as `tea pr create`, for an audit of the release, and with `--calls <file>` saves each call as a JSON line; `command-patterns --help` states the columns and the line's keys.
- `compress-archives` gzips each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them; Install's crontab line runs it daily.
- `deploy-release [--cut] [<checkout>]` moves `~/.claude`, or the checkout named, to the newest release, as Update says; with `--cut`, run in the development clone, it first cuts the next release, as Release says.
- `judge-replay <model>` replays loop-claude's judge on archived verdicts of the projects run on the model, and prints what each rung a review climbs to misses, and the lowest rung below them that, like each candidate above it, returns them; `judge-replay --help` states the replay.
- `larv-review <command>` holds larv-review's mechanical rules: the chunk's review folder, the paired ruling's base and vote count, each pass's round count, the scoring run's trigger and setup; the comprehension history's format, its latest places and its count of how often seats agree; and each reviewer's prompt header. `larv-review --help` states them. `settings.json` → `hooks` runs `larv-review reviewer-context` as each subagent starts. A reviewer under `agents/` whose frontmatter has `omitClaudeMd: true` starts without `CLAUDE.md`, and the command hands it that file's "Shared goal", "Prose" and "Vocabulary" sections.
@@ -221,7 +230,7 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
- `routing_figures/` measures loop-claude's archive, and each of its projects' default branches, for loop-claude's routing calls.
- `rung_replay/` holds what `judge_replay` and `review_replay` share: the parallel replay runner, the argument reader, the candidate rungs and the exits, and `judge_replay`'s rung search. It reads `loop_claude`'s retry count and models file, and finds that file through `loop_state`'s paths. Its `archive.py` keeps each replay's runs and its report in a `replays/` folder of loop-claude's archive.
- `test_env.py` serves the tests.
- `text_files/` reads a Markdown file's `##` sections, quotes a passage and joins names into a list, stamps times in UTC, and replaces a file in one step, private where it is new.
- `text_files/` reads a Markdown file's `##` sections, quotes a passage and joins names into a list, stamps times in UTC, replaces a file in one step, private where it is new, and gzips a file beside its plain name, by which it reads it afterwards.
- `transcript_archive/` stores loop-claude's transcripts. Its `project_key.py` names each project's archive folder, loop-claude session and models-file entry.
## Test
+2 -2
View File
@@ -3,8 +3,8 @@
It runs @larvit/adf-codec, a Node package, in a throwaway container that holds node_shim/shim.mjs; the shim's header says what its exit status means.
adf-codec has two pairs of calls, and each function here names the one it runs:
- The lossless pair, `*_lossless`, for a document that already exists in Jira or Confluence and is edited and saved back. It keeps each block's localId and the empty paragraphs Jira uses for spacing, so a document read and saved unedited comes back identical.
- The portable pair, with no suffix, for new text written from scratch, and for markdown shown to a person. It drops the localId and the empty paragraphs, so saving a document that exists changes it.
- The portable pair, with no suffix, which the loop's tools use. It drops each block's localId and the empty paragraphs Jira uses for spacing, so saving a document read by it changes the document; a tool that saves one unedited sends nothing instead (`docs/decisions.md` → `project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, …`).
- The lossless pair, `*_lossless`, which keeps both, so a document read and saved unedited comes back identical. Its markdown wraps every block in `!adf:` marker lines, so no tool reads by it yet.
"""
import contextlib
import dataclasses
+5 -2
View File
@@ -1,8 +1,11 @@
import io
import json
import pathlib
import typing
from claude_code.entries import AssistantTurn, Entry, HumanTurn, LoopPrompt, parse_entry, Text, ToolCall
from failure import Failure
from text_files.compressed import read_stored
JUDGED_TOOL_INPUT = 400
@@ -14,9 +17,9 @@ class Typed(typing.NamedTuple):
def read_transcript(path: str) -> list[Entry] | Failure:
"""The entries of the transcript at path, or of its gzip where compress-archives gzipped it."""
try:
with open(path, "rb") as handle:
return [entry for raw in handle if (entry := parse_entry(raw))]
return [entry for raw in io.BytesIO(read_stored(pathlib.Path(path))) if (entry := parse_entry(raw))]
except OSError as failure:
return Failure(f"could not read the transcript {path}: {failure.strerror}")
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env python3
import pathlib
import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from compress_archives import main
sys.exit(main.main(sys.argv[1:]))
View File
+65
View File
@@ -0,0 +1,65 @@
"""Gzip each file of loop-claude's archive and the review folders that no tool changes any more, keeping all of them.
usage: compress-archives
It gzips each file that last changed over 30 days ago: in loop-claude's archive,
$LOOP_CLAUDE_TRANSCRIPTS (default ~/.claude/loop-claude-transcripts), each transcript, *.jsonl;
in the review folders, ~/.local/state/larv-review, every file. A file's last change is the later
of its mtime and ctime, so a copied or unpacked file counts from when it landed.
The gzip lands beside the file as <name>.gz, with the same times. The file is removed only once
its gzip reads back the same bytes. The archive's readers and review-replay read a gzipped file by
its plain name. README.md → Install has the crontab line that runs it daily.
It prints how many files it compressed. It exits 1, saying why, on any argument but --help or -h,
and at the first file it cannot compress, leaving that file and every one after it plain.
"""
import os
import pathlib
import stat
import sys
import time
import typing
from transcript_archive.location import archive_root
from text_files.compressed import compress, SUFFIX
from failure import Failure
from review_rules.review_folder import REPORTS
# Why 30 days: docs/decisions.md → `archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them`.
AGE_DAYS = 30
def main(argv: list[str], env: typing.Mapping[str, str] = os.environ, home: pathlib.Path | None = None, now: float | None = None) -> int:
if argv in (["-h"], ["--help"]):
print(__doc__.strip())
return 0
if argv:
print(f"unknown arguments {' '.join(argv)}; usage: compress-archives", file=sys.stderr)
return 1
home = home or pathlib.Path.home()
cutoff = (time.time() if now is None else now) - AGE_DAYS * 24 * 60 * 60
count = 0
for path in [*old_files(archive_root(home, env), "*.jsonl", cutoff), *old_files(home / REPORTS, "*", cutoff)]:
if failed := compress(path):
print(failed.reason, file=sys.stderr)
return 1
count += 1
print(f"Compressed {count} files.")
return 0
def old_files(root: pathlib.Path, pattern: str, cutoff: float) -> list[pathlib.Path]:
"""Each plain file under root that pattern matches and that last changed before the cutoff, sorted; a link is left alone, since its target lies elsewhere."""
found = []
for path in root.rglob(pattern):
if path.name.endswith(SUFFIX) or path.is_symlink():
continue
try:
status = path.stat()
# Removed while the walk ran, such as a file of review-replay's copy of a review folder.
except FileNotFoundError:
continue
if stat.S_ISREG(status.st_mode) and max(status.st_mtime, status.st_ctime) < cutoff:
found.append(path)
return sorted(found)
@@ -0,0 +1,126 @@
import contextlib
import gzip
import io
from unittest import mock
import os
import pathlib
import sys
import tempfile
import time
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from failure import Failure
from compress_archives import main
DAY = 24 * 60 * 60
# A month ahead, so a file's ctime, which is when the test wrote it, lies over 30 days before NOW.
NOW = time.time() + 31 * DAY
OLD = 30 * DAY + 60
NEW = 30 * DAY - 60
class Main(unittest.TestCase):
"""A home whose archive and review folders each hold files that last changed a minute over 30 days ago, and one that changed a minute under."""
def setUp(self):
self.home = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory()))
self.archive = self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1"
self.reviews = self.home / ".local" / "state" / "larv-review" / "code" / "proj-item-1"
for folder in (self.archive / "session" / "s" / "subagents", self.reviews / "probe"):
folder.mkdir(parents=True)
self.write(self.archive / "a.jsonl", OLD)
self.write(self.archive / "a.outcomes.md", OLD)
self.write(self.archive / "session" / "s" / "subagents" / "agent-a.jsonl", OLD)
self.write(self.archive / "session" / "s" / "subagents" / "agent-a.meta.json", OLD)
self.write(self.archive / "b.jsonl", NEW)
self.write(self.reviews / "stability-round-1.md", OLD)
self.write(self.reviews / "probe" / "out.js", OLD)
self.write(self.reviews / "branch-1.diff", NEW)
def write(self, path, seconds_ago):
path.write_bytes(f"{path.name}\n".encode())
os.utime(path, (NOW - seconds_ago, NOW - seconds_ago))
def run_main(self, argv=(), env=None, now=NOW):
out, err = io.StringIO(), io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
code = main.main(list(argv), env or {}, self.home, now)
return code, out.getvalue(), err.getvalue()
def kept(self):
return sorted(str(path.relative_to(self.home)) for path in self.home.rglob("*") if path.is_file())
def test_gzips_the_archives_transcripts_and_every_review_file_last_written_over_30_days_ago(self):
code, out, err = self.run_main()
self.assertEqual((code, err), (0, ""))
self.assertEqual(self.kept(), [
".claude/loop-claude-transcripts/proj/c1/a.jsonl.gz",
".claude/loop-claude-transcripts/proj/c1/a.outcomes.md",
".claude/loop-claude-transcripts/proj/c1/b.jsonl",
".claude/loop-claude-transcripts/proj/c1/session/s/subagents/agent-a.jsonl.gz",
".claude/loop-claude-transcripts/proj/c1/session/s/subagents/agent-a.meta.json",
".local/state/larv-review/code/proj-item-1/branch-1.diff",
".local/state/larv-review/code/proj-item-1/probe/out.js.gz",
".local/state/larv-review/code/proj-item-1/stability-round-1.md.gz",
])
self.assertEqual(gzip.decompress((self.archive / "a.jsonl.gz").read_bytes()), b"a.jsonl\n")
self.assertEqual(out, "Compressed 4 files.\n")
self.assertEqual(self.run_main(), (0, "Compressed 0 files.\n", ""))
def test_keeps_a_file_last_written_exactly_30_days_ago_plain(self):
self.write(self.archive / "b.jsonl", 30 * DAY)
self.run_main()
self.assertTrue((self.archive / "b.jsonl").is_file())
def test_keeps_a_file_copied_today_with_its_old_times_plain(self):
now = time.time()
os.utime(self.archive / "a.jsonl", (now - 31 * DAY, now - 31 * DAY))
self.assertEqual(self.run_main(now=now), (0, "Compressed 0 files.\n", ""))
self.assertTrue((self.archive / "a.jsonl").is_file())
def test_measures_the_30_days_from_the_clock_by_default(self):
with mock.patch.object(main.time, "time", return_value=NOW):
self.assertEqual(self.run_main(now=None), (0, "Compressed 4 files.\n", ""))
def test_walks_on_past_a_gzip_a_link_and_a_file_removed_while_it_walks(self):
gone = mock.Mock(**{"is_symlink.return_value": False, "stat.side_effect": FileNotFoundError})
# Mock(name=...) names the mock itself, so the path's name is set after.
gone.name = "gone.jsonl"
packed = self.archive / "c.jsonl.gz"
packed.write_bytes(b"")
link = self.archive / "d.jsonl"
link.symlink_to(self.archive / "a.jsonl")
kept = self.archive / "a.jsonl"
root = mock.Mock(**{"rglob.return_value": [packed, link, gone, kept]})
self.assertEqual(main.old_files(root, "*.jsonl", NOW), [kept])
def test_reads_the_archive_where_loop_claude_transcripts_puts_it(self):
moved = self.home / "elsewhere"
(self.home / ".claude" / "loop-claude-transcripts").rename(moved)
code, _, err = self.run_main(env={"LOOP_CLAUDE_TRANSCRIPTS": str(moved)})
self.assertEqual((code, err), (0, ""))
self.assertTrue((moved / "proj" / "c1" / "a.jsonl.gz").is_file())
def test_runs_where_neither_folder_exists_yet(self):
with tempfile.TemporaryDirectory() as empty:
self.home = pathlib.Path(empty)
self.assertEqual(self.run_main(), (0, "Compressed 0 files.\n", ""))
def test_stops_at_the_first_failure_keeping_that_file_and_every_later_one(self):
with mock.patch.object(main, "compress", return_value=Failure("could not compress x: full")):
code, out, err = self.run_main()
self.assertEqual((code, out, err), (1, "", "could not compress x: full\n"))
self.assertTrue((self.archive / "a.jsonl").is_file())
def test_prints_its_help_and_refuses_other_arguments(self):
for flag in ("--help", "-h"):
code, out, _ = self.run_main([flag])
self.assertEqual(code, 0)
self.assertTrue(out.startswith("Gzip each file"), out)
self.assertEqual(self.run_main(["x", "y"]), (1, "", "unknown arguments x y; usage: compress-archives\n"))
if __name__ == "__main__":
unittest.main()
+8
View File
@@ -12,6 +12,7 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from transcript_archive.outcomes import ANSWERED, FINISHED, INTERRUPTED, UNFINISHED
from claude_code.entries import AssistantTurn, HumanTurn, LoopPrompt, Text
from judge_replay.replay import Case, case_from, report, reproduced, sample, verdict_kind
from text_files import compressed
from project_config.test_project_config import configured
from loop_effects.effects import Effects
from failure import Failure
@@ -233,6 +234,13 @@ class Main(unittest.TestCase):
self.assertEqual(code, 0)
self.assertIn("replayed failed: could not read the transcript", out)
def test_replays_a_verdict_whose_transcript_is_compressed(self):
self.assertIsNone(compressed.compress(self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1" / "t0.jsonl"))
code, out, err = self.run_main(["claude-opus-5-5"], lambda rung, interrupted: self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}"))
self.assertEqual(code, 0, err)
self.assertNotIn("replayed failed", out)
self.assertIn("The reference judge, claude-opus-5-5 at xhigh effort, reproduced 2.", out)
def test_tries_the_projects_rung_first_and_retries_a_failed_reply_as_the_loop_does(self):
def reply(rung, interrupted):
return self.verdict(interrupted, "not in the transcript" if rung.model != "claude-opus-5-5" else f"said {INTERRUPTED if interrupted else FINISHED}")
+9 -2
View File
@@ -185,6 +185,13 @@ class JiraPlan:
description = self.to_adf(detail)
return Failure(f"cannot convert the item's detail to a Jira description: {description.reason}") if isinstance(description, Failure) else description
def shows(self, issue: Issue, detail: str) -> bool:
"""Whether detail is the issue's description as a read shows it. An edit leaves such a description as Jira holds it, since converting the markdown back would drop each block's localId and Jira's spacing paragraphs."""
if issue.description is None:
return not detail.strip()
texts = self.to_markdown([issue.description])
return isinstance(texts, list) and texts[0].strip() == detail.strip()
def edit(self, id_: int, fields: dict[str, str], detail: str | None, comment: str | None = None) -> Written | Failure:
"""Sets the fields and the detail in one request, so Jira takes all of the edit or none, and then posts comment on the issue. `revision_of` says what an edit may set.
@@ -203,7 +210,7 @@ class JiraPlan:
if isinstance(revision, Failure):
return revision
body = revision.body
if detail is not None:
if detail is not None and not self.shows(issue, detail):
description = self.description_of(detail)
if isinstance(description, Failure):
return description
@@ -217,7 +224,7 @@ class JiraPlan:
return version
body = body | {"fields": body.get("fields", {}) | {"fixVersions": [{"name": version}]}}
url = f"{self.epic.site}/rest/api/3/issue/{key}"
edited = self.http("PUT", url, authorization, body)
edited = self.http("PUT", url, authorization, body) if body else None
if isinstance(edited, Failure):
return Failure(f"cannot edit {key}: {edited.reason}; {PERHAPS_EDITED.format(key=key)}")
if said is None:
+15 -1
View File
@@ -500,7 +500,7 @@ class Editing(unittest.TestCase):
return adf(markdown)
def edit(self, id_: int, fields: dict[str, str], detail: str | None = None, comment: str | None = None):
return jira_epic.JiraPlan(EPIC, self.env, self.http, text_of, self.to_adf, scoring=Stated()).edit(id_, fields, detail, comment)
return jira_epic.JiraPlan(EPIC, self.env, self.http, lambda documents: [text_of(document) for document in documents], self.to_adf, scoring=Stated()).edit(id_, fields, detail, comment)
def writes(self) -> list[tuple[str, str, object]]:
"""Each request that changes Jira, in order."""
@@ -518,7 +518,21 @@ class Editing(unittest.TestCase):
self.edit(34, {"E": "3"})
self.assertEqual([body for _, _, body in self.writes()], [{"fields": {"summary": "Count every run."}}, {"properties": [{"key": "plan-item", "value": PROPERTY | {"E": 3}}]}])
def test_a_detail_unchanged_since_the_read_leaves_the_description_as_jira_holds_it(self):
# Jira's own spacing paragraph and localIds, which a description converted back from markdown would lose.
held = {"type": "doc", "version": 1, "content": [{"type": "paragraph", "attrs": {"localId": "p-1"}, "content": [{"type": "text", "text": "Why it counts."}]}, {"type": "paragraph", "attrs": {"localId": "p-2"}}]}
self.open = [issue("ABC-34", scores=PROPERTY, description=held)]
self.assertEqual(self.edit(34, {"L": "2"}, "Why it counts.\n"), Written(34))
self.assertEqual(self.edit(34, {}, "Why it counts."), Written(34))
self.assertEqual(self.writes(), [("PUT", f"{ISSUES}/ABC-34", {"properties": [{"key": "plan-item", "value": PROPERTY | {"L": 2}}]})])
def test_an_edited_detail_replaces_the_description(self):
self.open = [issue("ABC-34", scores=PROPERTY, description=adf("Why it counts."))]
self.edit(34, {}, "Why it counts twice.")
self.assertEqual(self.writes(), [("PUT", f"{ISSUES}/ABC-34", {"fields": {"description": adf("Why it counts twice.")}})])
def test_an_empty_detail_removes_the_description(self):
self.open = [issue("ABC-34", scores=PROPERTY, description=adf("Why it counts."))]
self.edit(34, {}, " \n")
self.assertEqual(self.writes(), [("PUT", f"{ISSUES}/ABC-34", {"fields": {"description": None}})])
+9
View File
@@ -16,6 +16,7 @@ from transcript_archive.write import archive
from claude_code.entries import AssistantTurn
from project_config.templates import BRANCH
from reading_cost.measure import Checkout, Reading
from text_files import compressed
from failure import Failure
from reading_cost import figures, main, measure
from transcript_archive.read import finished_chunks, FinishedChunk
@@ -173,6 +174,14 @@ class ReadingCostTool(unittest.TestCase):
self.assertEqual((ran.returncode, ran.stdout), (1, ""))
self.assertIn("no config names the checkout", ran.stderr)
def test_measures_a_chunk_whose_archived_transcripts_are_compressed(self):
self.archive_chunk(1790000000)
for copy in self.archive.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
ran = self.run_tool(self.checkout)
self.assertEqual(ran.returncode, 0, ran.stderr)
self.assertIn("33 new input tokens before the first commit.", (self.checkout / "docs" / "reading-cost.md").read_text())
def test_appends_each_finished_chunk_once_in_the_order_they_finished_from_a_worktree(self):
self.archive_chunk(1790000200)
self.archive_chunk(1790000100)
+23 -5
View File
@@ -2,6 +2,7 @@
A write never overwrites a teammate's edit: it sends the version after the one held() read, and Confluence refuses it where someone saved the page since. The human then decides (lilleman, 2026-11-12)."""
import dataclasses
import hashlib
import json
import pathlib
import typing
@@ -20,9 +21,10 @@ SHAPE = "not in the shape Confluence Cloud's REST API version 2 documents"
@dataclasses.dataclass(frozen=True)
class Seen:
"""The page's version number and title as a read found them or a write left them. A write needs both, so a Held's version holds both, as JSON."""
"""The page's version number and title as a read found them or a write left them, and the digest of the markdown it found or sent. A write needs the version and title, and sends nothing where its text has that digest, so a Held's version holds all three, as JSON. The digest is None where the version came from a request that held no text."""
version: int
title: str
digest: str | None = None
@property
def token(self) -> str:
@@ -64,7 +66,7 @@ class ConfluencePage:
if not isinstance(body, str):
return Failure(f"{self.label} is {SHAPE}: it holds no body in atlas_doc_format")
text = self.converted(body)
return text if isinstance(text, Failure) else Held(text, seen.token)
return text if isinstance(text, Failure) else Held(text, Seen(seen.version, seen.title, digest_of(text)).token)
def converted(self, held: str) -> str | Failure:
"""The markdown of the body held, the JSON of an ADF document; empty where held is."""
@@ -86,6 +88,14 @@ class ConfluencePage:
seen = self.seen_of(over)
if isinstance(seen, Failure):
return seen
# A read's markdown leaves out the localIds and spacing paragraphs Confluence holds, so a write of the same text back would drop them: send nothing. Without a write Confluence answers no 409, so read the version to refuse where a teammate saved since.
if over is not None and seen.digest == digest_of(text):
current = self.fetched("")
if isinstance(current, Failure):
return current
if current[0].version != seen.version:
return self.saved_since(seen.version, f"Confluence holds version {current[0].version}")
return Held(text, over)
document = self.to_adf(text)
if isinstance(document, Failure):
return Failure(f"cannot convert the text for {self.label} to ADF: {document.reason}")
@@ -95,10 +105,14 @@ class ConfluencePage:
body = {"id": self.page.page_id, "status": "current", "title": seen.title, "body": {"representation": "atlas_doc_format", "value": json.dumps(document)}, "version": {"number": seen.version + 1}}
answer = self.http("PUT", self.url, authorization, body)
if isinstance(answer, ErrorAnswer) and answer.status == CONFLICT:
return Failure(f"someone saved {self.label} since the loop read version {seen.version} of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do ({answer.reason})")
return self.saved_since(seen.version, answer.reason)
if isinstance(answer, Failure):
return Failure(f"cannot write {self.label}: {answer.reason}")
return Held(text, Seen(seen.version + 1, seen.title).token)
return Held(text, Seen(seen.version + 1, seen.title, digest_of(text)).token)
def saved_since(self, version: int, evidence: str) -> Failure:
"""The refusal of a write over version, which a teammate's save replaced, as evidence shows."""
return Failure(f"someone saved {self.label} since the loop read version {version} of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do ({evidence})")
def fetched(self, query: str) -> tuple[Seen, object] | Failure:
"""The page's version and title, and the value of its body in atlas_doc_format where query asks for it; None where it doesn't."""
@@ -124,6 +138,10 @@ class ConfluencePage:
return fetched if isinstance(fetched, Failure) else fetched[0]
try:
held: typing.Any = json.loads(over)
return Seen(int(held["version"]), str(held["title"]))
return Seen(int(held["version"]), str(held["title"]), held.get("digest"))
except (ValueError, KeyError, TypeError) as failure:
return Failure(f"cannot read the version of {self.label} to write over, {over!r}: {failure}")
def digest_of(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
+37 -5
View File
@@ -80,18 +80,50 @@ class ConfluencePageTest(ConfluenceCase):
store = self.new_store()
held = store.held()
assert isinstance(held, Held), held
self.assertEqual(store.write("- a decision\n", held.version), Held("- a decision\n", '{"version": 8, "title": "Decisions"}'))
written = store.write("- a decision\n", held.version)
assert isinstance(written, Held), written
self.assertEqual(written.text, "- a decision\n")
self.assertEqual(self.confluence.calls[-1], ("PUT", URL, AUTHORIZATION, {"id": PAGE.page_id, "status": "current", "title": "Decisions", "body": {"representation": "atlas_doc_format", "value": json.dumps(to_adf("- a decision\n"))}, "version": {"number": 8}}))
def test_saving_the_text_a_read_found_leaves_the_page_as_confluence_holds_it(self) -> None:
self.confluence.teammate_saves("- a decision\n")
value = self.confluence.value
held = self.new_store().held()
assert isinstance(held, Held), held
self.assertEqual(self.new_store().write(held.text, held.version), held)
self.assertNotIn("PUT", [call[0] for call in self.confluence.calls])
self.assertEqual(self.confluence.calls[-1], ("GET", URL, AUTHORIZATION, None))
self.assertEqual((self.confluence.version, self.confluence.value), (2, value))
def test_refuses_to_save_the_text_a_read_found_where_a_teammate_saved_since(self) -> None:
self.confluence.teammate_saves("- a decision\n")
held = self.new_store().held()
assert isinstance(held, Held), held
self.confluence.teammate_saves("- their decision\n")
written = self.new_store().write(held.text, held.version)
self.assertEqual(written, Failure(f"someone saved {PAGE.spelling} since the loop read version 2 of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do (Confluence holds version 3)"))
self.assertNotIn("PUT", [call[0] for call in self.confluence.calls])
self.assertEqual(self.new_store().read(), "- their decision\n")
def test_an_unedited_save_fails_where_the_pages_version_cannot_be_read(self) -> None:
held = self.new_store().held()
assert isinstance(held, Held), held
store = ConfluencePage(PAGE, self.env, lambda *_: ErrorAnswer("GET /wiki/api/v2/pages/2293761 answered 404: Not Found", 404), to_markdown, to_adf)
self.assertEqual(store.write(held.text, held.version), Failure(f"cannot read {PAGE.spelling}: GET /wiki/api/v2/pages/2293761 answered 404: Not Found"))
def test_saving_an_edited_text_writes_the_page(self) -> None:
self.confluence.teammate_saves("- a decision\n")
held = self.new_store().held()
assert isinstance(held, Held), held
self.assertIsInstance(self.new_store().write("- another decision\n", held.version), Held)
self.assertEqual(self.new_store().read(), "- another decision\n")
def test_refuses_to_overwrite_a_teammates_save_since_the_version_it_is_handed_and_says_to_ask_the_human(self) -> None:
held = self.new_store().held()
assert isinstance(held, Held), held
self.confluence.teammate_saves("their edit\n")
written = self.new_store().write("my edit\n", held.version)
assert isinstance(written, Failure), written
self.assertIn("someone saved", written.reason)
self.assertIn("ask the human", written.reason)
self.assertIn(PAGE.spelling, written.reason)
self.assertEqual(written, Failure(f"someone saved {PAGE.spelling} since the loop read version 1 of it, so the loop didn't write it: never overwrite their edit, but ask the human what to do (PUT /wiki/api/v2/pages/{PAGE.page_id} answered 409: Version must be incremented on update. Current version is: 2)"))
self.assertEqual(self.new_store().read(), "their edit\n")
def test_keeps_no_version_itself_so_a_read_never_moves_the_version_a_write_names(self) -> None:
+8 -1
View File
@@ -16,6 +16,7 @@ from rung_replay.search import first_reply
from record_store.stores import local_file, store
from project_config.main import located
from review_rules.reviewer_context import principles_line, reviewer_context
from text_files.compressed import read_stored, SUFFIX
from review_rules.review_folder import REPORTS, review_folder
from model_routing.ladder import Rung
from claude_code.entries import structured_output
@@ -146,8 +147,14 @@ def copy_reports(found: Round, folder: pathlib.Path, reports: str) -> None:
"""Copies into reports each report written to folder after found's chunk began and before its round did."""
# The round's own report, any later one, and one from an earlier attempt at the item in the same worktree would each hand the reviewer the findings it is to find.
for written in folder.iterdir() if folder.is_dir() else ():
if written.is_file() and found.began <= written.stat().st_mtime < found.seconds:
if not written.is_file() or not found.began <= written.stat().st_mtime < found.seconds:
continue
name = written.name.removesuffix(SUFFIX)
if name == written.name:
shutil.copy2(written, reports)
# A gzipped report. Where a plain report of the same name stands beside it, that one is newer and the gzip is skipped; the plain report is copied only if it was written before the round.
elif not (folder / name).is_file():
(pathlib.Path(reports) / name).write_bytes(read_stored(folder / name))
def redirected(prompt: str, home: pathlib.Path, folder: pathlib.Path, worktree: str, copies: Copies) -> str | Failure:
+3 -2
View File
@@ -11,6 +11,7 @@ from transcript_archive.project_key import checkout_of
from failure import Failure
from claude_code.transcript import first_timestamped_line, read_transcript
from rung_replay.command import projects_on
from text_files.compressed import read_stored, stored_glob
# A line that opens with a 🔴 or 🟡 mark, after any list, table, heading or bold markup, as review-findings.md has reviewers report them.
FINDING = re.compile(r"^[\s#>*+|-]*(?:\d+[.)]\s*)?[*_]*\s*(?:🔴|🟡)")
@@ -126,8 +127,8 @@ def archived_rounds(reviewer: str, model: str, root: pathlib.Path, home: pathlib
def chunk_rounds(reviewer: str, checkout: str, chunk: pathlib.Path) -> list[Round]:
copies = sorted(chunk.glob("*.jsonl"))
began = parse_entry(first_timestamped_line(copies[0].read_bytes())) if copies else None
copies = stored_glob(chunk, "*.jsonl")
began = parse_entry(first_timestamped_line(read_stored(copies[0]))) if copies else None
if not began or not began.timestamp:
return []
found = []
+28 -3
View File
@@ -15,6 +15,7 @@ import unittest.mock
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from review_replay.rounds import authored_by, lost_inputs, recorded_findings, reviewed_path, Round, round_from, sample, stamp_seconds
from text_files import compressed
from project_config.test_project_config import configured, write_config
from loop_effects.effects import Effects, system_effects
from claude_code.entries import entry_from
@@ -375,19 +376,43 @@ class Main(unittest.TestCase):
self.assertIn("claude-opus-5-5 at low effort found 0, 0%, and missed 1 such 🔴: it doesn't qualify.", out)
def review_folder_with_round_1(self):
"""The chunk's review folder, holding round 1's report from before the replayed round began at 10:00, that round's own and a later pass's reports, written after, and an earlier attempt's report, from before the chunk began at 09:15."""
"""The chunk's review folder. The chunk began at 09:15 and the replayed round 2 at 10:00. It holds round 1's report, written at 09:15:00; round 2's report, written after 10:00; the prose pass's report, written at 10:00:00; and an earlier attempt's report, from before 09:15."""
folder = self.home / ".local" / "state" / "larv-review" / "code" / "proj-x"
folder.mkdir(parents=True)
(folder / "stability-round-1.md").write_text("round 1\n")
before = stamp_seconds("2026-10-14T09:50:00Z")
os.utime(folder / "stability-round-1.md", (before, before))
began = stamp_seconds("2026-10-14T09:15:00Z")
os.utime(folder / "stability-round-1.md", (began, began))
(folder / "stability-round-2.md").write_text("🔴 a.txt:1 wrong\n")
(folder / "prose-round-1.md").write_text("later\n")
round_began = stamp_seconds("2026-10-14T10:00:00Z")
os.utime(folder / "prose-round-1.md", (round_began, round_began))
(folder / "stability-round-3.md").write_text("🔴 a.txt:1 wrong\n")
attempt = stamp_seconds("2026-10-14T09:00:00Z")
os.utime(folder / "stability-round-3.md", (attempt, attempt))
return folder
def test_replays_rounds_whose_archived_transcripts_are_compressed(self):
for copy in self.archive.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
code, out, err = self.run_main(["stability", "claude-opus-5-5"], lambda rung: {"found": [1] if rung == LOW else [0, 1]})
self.assertEqual(code, 0, err)
self.assertTrue(out.startswith("Replayed 1 stability rounds of claude-opus-5-5 projects,"), out)
def test_copies_a_compressed_report_from_before_the_round_under_its_plain_name(self):
folder = self.review_folder_with_round_1()
self.assertIsNone(compressed.compress(folder / "stability-round-1.md"))
self.set_prompt(f"Review branch x in {self.home}/code/proj-x. Write your report to ~/.local/state/larv-review/code/proj-x/stability-round-2.md; the last round's is ~/.local/state/larv-review/code/proj-x/stability-round-1.md.")
seen = []
def review(rung):
named = pathlib.Path(re.search(r"report to (\S+)/stability-round-2\.md", self.reviews[-1][4]).group(1))
seen.append((sorted(path.name for path in named.iterdir()), (named / "stability-round-1.md").read_text()))
return {"type": "result", "result": f"review on {rung.model} {rung.effort}"}
code, _, err = self.run_main(["stability", "claude-opus-5-5"], lambda rung: {"found": [0, 1]}, review)
self.assertEqual(code, 0, err)
self.assertEqual(seen[0], (["stability-round-1.md"], "round 1\n"))
def test_points_the_prompts_review_folder_at_a_copy_of_the_reports_from_before_the_round_that_it_removes(self):
folder = self.review_folder_with_round_1()
self.set_prompt(f"Review branch x in {self.home}/code/proj-x. Write your report to ~/.local/state/larv-review/code/proj-x/stability-round-2.md; the last round's is ~/.local/state/larv-review/code/proj-x/stability-round-1.md.")
+3 -2
View File
@@ -3,6 +3,7 @@ import contextlib
import dataclasses
import fcntl
import functools
import io
import json
import pathlib
import typing
@@ -15,6 +16,7 @@ from routing_figures.chunk import Chunk, chunk_figures, ChunkFigures, Subagent
from git_cli.git_repo import default_ref, Git
from claude_code.entries import Entry, parse_entry
from failure import Failure
from text_files.compressed import read_stored
from text_files.file_writes import replace_file
from project_config.main import title_templates
from project_config.templates import Titles
@@ -115,8 +117,7 @@ def branch_history(checkout: str, home: pathlib.Path, titles: Titles, git: Git)
def entries(path: pathlib.Path) -> list[Entry]:
with open(path, "rb") as handle:
return [entry for line in handle if (entry := parse_entry(line))]
return [entry for line in io.BytesIO(read_stored(path)) if (entry := parse_entry(line))]
def subagent(transcript: pathlib.Path) -> Subagent:
@@ -19,6 +19,7 @@ from project_config.templates import BARE, Titles
from branch_chunks.cut import BranchChunk
from routing_figures.calls import Call, calls, totals
from routing_figures.chunk import Chunk, chunk_figures, ChunkFigures, Reports, Subagent
from text_files import compressed
from project_config.test_project_config import configured, write_config
from routing_figures.state import due
from claude_code.entries import entry_from
@@ -352,6 +353,13 @@ class Archive(unittest.TestCase):
self.assertEqual(found.chunks[0].dollars["subagent type"], {"worker": 2.0, "Explore": 2.0})
self.assertEqual(archive.measure_archive(self.root, "/h", "2026-10-14T10:00:00Z", no_checkout), archive.ArchiveChunks([], ()))
def test_measures_a_chunk_whose_transcripts_are_compressed(self):
self.chunk("home/x", "c1", stamped("2026-10-14T10:00:00Z", outcomes.finished(LOW_ROUTE, "m", "j")), [message("m1", "2026-10-14T09:00:00Z", call("Agent", "a1", subagent_type="Explore"))], ({"agentType": "Explore", "toolUseId": "a1"}, [message("s1", "2026-10-14T09:00:01Z")]))
for copy in self.root.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
found = archive.measure_archive(self.root, "/h", "", no_checkout)
self.assertEqual(found.chunks[0].dollars["subagent type"], {"worker": 2.0, "Explore": 2.0})
def test_counts_each_chunks_later_fixes_from_its_projects_checkout(self):
home = self.root / "home"
env = {**isolated_env(), "GIT_CONFIG_GLOBAL": os.devnull, "HOME": str(home)}
+78
View File
@@ -0,0 +1,78 @@
"""A file kept plain, or gzipped beside its plain name as <name>.gz: reading it by its plain name, and compressing it.
Why gzip and when a file is compressed: docs/decisions.md → `archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them`.
"""
import errno
import gzip
import os
import pathlib
import zlib
from failure import Failure
from text_files.file_writes import replace_file
SUFFIX = ".gz"
def stored(path: pathlib.Path) -> pathlib.Path | None:
"""Where the file named path is kept: path itself, else its gzip; None where neither is."""
# A plain file beside its gzip was written after it was compressed, so it holds the newer bytes. Anything else at path, such as a folder, counts as path, so reading it fails naming path.
return next((kept for kept in (path, gzipped(path)) if kept.exists()), None)
def read_stored(path: pathlib.Path) -> bytes:
"""The bytes of the file named path, read from its gzip where it is kept that way. Raises OSError, FileNotFoundError where neither is kept."""
# Tried in turn rather than checked first: compress-archives may remove the plain file once its gzip is whole, between a check and the read.
try:
return path.read_bytes()
except FileNotFoundError:
pass
packed = gzipped(path)
try:
data = packed.read_bytes()
except FileNotFoundError:
raise FileNotFoundError(errno.ENOENT, "No such file or directory", str(path)) from None
try:
return gzip.decompress(data)
except (EOFError, zlib.error, gzip.BadGzipFile) as failure:
raise OSError(errno.EIO, f"not a whole gzip: {failure}", str(packed)) from failure
def stored_glob(folder: pathlib.Path, pattern: str) -> list[pathlib.Path]:
"""The plain name of each file in folder that pattern matches, whether kept plain or gzipped, sorted."""
return sorted({*folder.glob(pattern), *(path.with_name(path.name.removesuffix(SUFFIX)) for path in folder.glob(pattern + SUFFIX))})
def compress(path: pathlib.Path) -> Failure | None:
"""Replaces path with its gzip, which keeps path's times: writes the gzip, reads it back, and removes path only once it reads back path's bytes. A failure keeps path."""
packed = gzipped(path)
try:
before = path.stat()
data = path.read_bytes()
if unwritten := replace_file(packed, gzip.compress(data, mtime=0)):
return Failure(f"could not compress {path}: {unwritten.reason}")
if gzip.decompress(packed.read_bytes()) != data:
return Failure(f"could not compress {path}: {packed} reads back other bytes")
os.utime(packed, ns=(before.st_atime_ns, before.st_mtime_ns))
# On disk before the plain file goes, so a crash cannot leave an empty gzip in its place.
synced(packed)
synced(packed.parent)
after = path.stat()
# Written while it compressed: the plain file keeps the newer bytes, which readers prefer, and a later run compresses them.
if (after.st_size, after.st_mtime_ns) == (before.st_size, before.st_mtime_ns):
path.unlink()
except (OSError, EOFError, zlib.error) as failure:
return Failure(f"could not compress {path}: {failure}")
return None
def synced(path: pathlib.Path) -> None:
handle = os.open(path, os.O_RDONLY)
try:
os.fsync(handle)
finally:
os.close(handle)
def gzipped(path: pathlib.Path) -> pathlib.Path:
return path.with_name(path.name + SUFFIX)
+129
View File
@@ -0,0 +1,129 @@
import errno
import gzip
from unittest import mock
import os
import pathlib
import sys
import tempfile
import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from text_files import compressed
from failure import Failure
class Compress(unittest.TestCase):
def test_replaces_the_file_with_a_gzip_of_the_same_bytes_and_times(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b'{"a": 1}\n')
os.utime(plain, (1000, 2000))
self.assertIsNone(compressed.compress(plain))
self.assertFalse(plain.exists())
packed = pathlib.Path(root, "t.jsonl.gz")
# A gzip header's own time is 0, so compressing the same bytes twice writes the same file.
self.assertEqual(packed.read_bytes(), gzip.compress(b'{"a": 1}\n', mtime=0))
self.assertEqual(packed.stat().st_mtime, 2000)
def test_syncs_the_gzip_and_its_folder_before_removing_the_file(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"a\n")
synced = []
fsync = os.fsync
def record(handle):
synced.append((os.readlink(f"/proc/self/fd/{handle}"), plain.exists()))
fsync(handle)
with mock.patch.object(compressed.os, "fsync", record):
self.assertIsNone(compressed.compress(plain))
self.assertEqual(synced, [(os.path.realpath(f"{plain}.gz"), True), (os.path.realpath(root), True)])
def test_a_gzip_it_cannot_write_keeps_the_original_and_says_why(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"kept\n")
with mock.patch.object(compressed, "replace_file", return_value=Failure("disk full")):
self.assertEqual(compressed.compress(plain), Failure(f"could not compress {plain}: disk full"))
self.assertEqual(plain.read_bytes(), b"kept\n")
def test_a_file_it_cannot_read_fails_naming_it(self):
with tempfile.TemporaryDirectory() as root:
folder = pathlib.Path(root, "t.jsonl")
folder.mkdir()
failed = compressed.compress(folder)
self.assertTrue(failed.reason.startswith(f"could not compress {folder}: [Errno {errno.EISDIR}]"), failed.reason)
def test_a_copy_that_reads_back_other_bytes_keeps_the_original_and_fails(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"kept\n")
with mock.patch.object(compressed.gzip, "compress", return_value=gzip.compress(b"other\n")):
failed = compressed.compress(plain)
self.assertIsInstance(failed, Failure)
self.assertIn(str(plain), failed.reason)
self.assertEqual(plain.read_bytes(), b"kept\n")
def test_a_file_written_while_it_compressed_stays_plain(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "t.jsonl")
plain.write_bytes(b"one\n")
packing = compressed.gzip.compress
def grow_meanwhile(data, **kwargs):
with open(plain, "ab") as handle:
handle.write(b"two\n")
return packing(data, **kwargs)
with mock.patch.object(compressed.gzip, "compress", grow_meanwhile):
self.assertIsNone(compressed.compress(plain))
self.assertEqual(plain.read_bytes(), b"one\ntwo\n")
self.assertEqual(compressed.read_stored(plain), b"one\ntwo\n")
class Read(unittest.TestCase):
def test_reads_a_file_kept_plain_or_gzipped_by_its_plain_name(self):
with tempfile.TemporaryDirectory() as root:
plain = pathlib.Path(root, "a.jsonl")
plain.write_bytes(b"a\n")
pathlib.Path(root, "b.jsonl.gz").write_bytes(gzip.compress(b"b\n"))
self.assertEqual(compressed.read_stored(plain), b"a\n")
self.assertEqual(compressed.read_stored(pathlib.Path(root, "b.jsonl")), b"b\n")
self.assertEqual(compressed.stored(pathlib.Path(root, "b.jsonl")), pathlib.Path(root, "b.jsonl.gz"))
self.assertIsNone(compressed.stored(pathlib.Path(root, "c.jsonl")))
with self.assertRaises(FileNotFoundError) as raised:
compressed.read_stored(pathlib.Path(root, "c.jsonl"))
self.assertEqual((raised.exception.errno, raised.exception.strerror, raised.exception.filename), (errno.ENOENT, "No such file or directory", str(pathlib.Path(root, "c.jsonl"))))
def test_the_plain_file_wins_over_an_older_gzip_beside_it(self):
with tempfile.TemporaryDirectory() as root:
pathlib.Path(root, "a.jsonl").write_bytes(b"a\nmore\n")
pathlib.Path(root, "a.jsonl.gz").write_bytes(gzip.compress(b"a\n"))
self.assertEqual(compressed.read_stored(pathlib.Path(root, "a.jsonl")), b"a\nmore\n")
def test_a_truncated_gzip_fails_as_an_os_error(self):
with tempfile.TemporaryDirectory() as root:
pathlib.Path(root, "a.jsonl.gz").write_bytes(gzip.compress(b"a\n" * 100)[:20])
with self.assertRaises(OSError) as raised:
compressed.read_stored(pathlib.Path(root, "a.jsonl"))
self.assertEqual((raised.exception.errno, raised.exception.filename), (errno.EIO, str(pathlib.Path(root, "a.jsonl.gz"))))
self.assertTrue(raised.exception.strerror.startswith("not a whole gzip: "), raised.exception.strerror)
def test_a_file_that_is_no_gzip_fails_as_an_os_error(self):
with tempfile.TemporaryDirectory() as root:
pathlib.Path(root, "a.jsonl.gz").write_bytes(b"plain text\n")
with self.assertRaises(OSError) as raised:
compressed.read_stored(pathlib.Path(root, "a.jsonl"))
self.assertEqual((raised.exception.errno, raised.exception.filename), (errno.EIO, str(pathlib.Path(root, "a.jsonl.gz"))))
def test_globs_each_file_once_by_its_plain_name(self):
with tempfile.TemporaryDirectory() as root:
for name in ("a.jsonl", "b.jsonl.gz", "c.jsonl", "c.jsonl.gz", "d.md"):
pathlib.Path(root, name).write_bytes(b"")
self.assertEqual(compressed.stored_glob(pathlib.Path(root), "*.jsonl"), [pathlib.Path(root, name) for name in ("a.jsonl", "b.jsonl", "c.jsonl")])
if __name__ == "__main__":
unittest.main()
+3
View File
@@ -22,6 +22,9 @@
- judge/ holds each judge run an outcome names on its `Judge run:` line: the prompt the judge read and claude's
json reply, named by the hash of that content (judge_run.JudgeRun). judge-replay reads the prompt back.
compress-archives gzips each .jsonl 30 days after it last changed, as <name>.jsonl.gz beside its plain name,
and every reader reads it by that name (text_files.compressed); a plain file beside its gzip is the newer.
write.archive writes the layout, at every interrupt and as every cycle ends. read.py and the tools that
measure finished chunks read it.
"""
+1 -1
View File
@@ -29,5 +29,5 @@ def outcomes_of(copy: pathlib.Path) -> pathlib.Path:
def transcript_of(outcomes: pathlib.Path) -> pathlib.Path:
"""The transcript copy an outcomes file sits beside; outcomes_of is its inverse."""
"""The plain name of the transcript copy an outcomes file sits beside, which text_files.compressed reads whether gzipped or not; outcomes_of is its inverse."""
return outcomes.with_name(outcomes.name.removesuffix(FILE_SUFFIX) + ".jsonl")
+8 -2
View File
@@ -4,6 +4,7 @@ import pathlib
from transcript_archive.location import chunk_folder, FILE_SUFFIX, transcript_of
from failure import Failure
from transcript_archive.outcomes import FINISHED_AT, spent_stamps, workers
from text_files.compressed import read_stored, stored_glob
@dataclasses.dataclass(frozen=True)
@@ -32,7 +33,7 @@ def finished_chunks(project_archive: pathlib.Path) -> list[FinishedChunk] | Fail
copies.append((stamps[-1], outcomes))
if copies:
finished, outcomes = max(copies)
found.append(FinishedChunk(folder.name, finished, transcript_of(outcomes), sorted(folder.glob("session/*/subagents/*.jsonl"))))
found.append(FinishedChunk(folder.name, finished, transcript_of(outcomes), subagent_transcripts(folder)))
return sorted(found, key=lambda chunk: (chunk.finished, chunk.folder))
@@ -44,6 +45,11 @@ def archived_outcomes(root: pathlib.Path) -> list[tuple[str, pathlib.Path]] | Fa
return Failure(f"could not read the archive {root}: {failure}")
def subagent_transcripts(folder: pathlib.Path) -> list[pathlib.Path]:
"""The plain name of each subagent's transcript archived in a chunk's folder, kept plain or gzipped, sorted."""
return [path for session in sorted(folder.glob("session/*/subagents")) for path in stored_glob(session, "*.jsonl")]
def chunk_outcomes(folder: pathlib.Path) -> list[pathlib.Path]:
"""The outcomes file of each transcript copy in a chunk's folder, sorted."""
return sorted(folder.glob(f"*{FILE_SUFFIX}"))
@@ -52,7 +58,7 @@ def chunk_outcomes(folder: pathlib.Path) -> list[pathlib.Path]:
def read_threads(chunk: FinishedChunk) -> list[bytes] | Failure:
"""The worker's transcript, then each subagent's."""
try:
return [path.read_bytes() for path in (chunk.transcript, *chunk.subagents)]
return [read_stored(path) for path in (chunk.transcript, *chunk.subagents)]
except OSError as failure:
return Failure(f"could not read {failure.filename}: {failure.strerror}")
@@ -10,6 +10,7 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from transcript_archive.project_key import checkout_of, project_key
from text_files import compressed
from failure import Failure
from transcript_archive.judge_run import JudgeRun
from transcript_archive import outcomes, read, write
@@ -119,6 +120,47 @@ class CopyFor(unittest.TestCase):
self.assertIsNone(write.copy_for(copy, b"second"))
self.assertEqual(write.copy_for(copy, b"first, grown"), copy)
def test_reads_a_compressed_copy_as_the_kept_version(self):
with tempfile.TemporaryDirectory() as temp:
copy = pathlib.Path(temp, "agent-a.jsonl")
copy.write_bytes(b"first\n")
self.assertIsNone(compressed.compress(copy))
self.assertIsNone(write.copy_for(copy, b"first\n"))
self.assertEqual(write.copy_for(copy, b"first\nsecond\n"), copy)
self.assertEqual(write.copy_for(copy, b"other\n"), copy.with_name(f"agent-a.jsonl.{hashlib.sha256(b'other' + bytes([10])).hexdigest()}"))
class Compressed(unittest.TestCase):
"""The archive, its transcript copies and subagents' transcripts gzipped as compress-archives leaves them."""
def archived(self, root: str) -> pathlib.Path:
kept = pathlib.Path(root, "archive")
transcript = pathlib.Path(root, "t.jsonl")
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
subagent = pathlib.Path(root, "t", "subagents", "agent-a.jsonl")
subagent.parent.mkdir(parents=True)
subagent.write_text('{"sub": 1}\n')
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.finished("w", "merged", "j"), NOW))
for copy in kept.rglob("*.jsonl"):
self.assertIsNone(compressed.compress(copy))
return transcript
def test_a_compressed_copy_takes_the_next_outcome_and_is_not_copied_again(self):
with tempfile.TemporaryDirectory() as root:
transcript = self.archived(root)
kept = pathlib.Path(root, "archive")
with contextlib.redirect_stderr(io.StringIO()):
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.stopped("w", "Ctrl-C"), NOW + 1))
self.assertEqual([path.name.endswith(".gz") for path in kept.rglob("*.jsonl*")], [True, True])
self.assertIn("Stopped: Ctrl-C", next(kept.glob("*/*.outcomes.md")).read_text())
def test_reads_a_finished_chunk_whose_copies_are_compressed(self):
with tempfile.TemporaryDirectory() as root:
self.archived(root)
(chunk,) = read.finished_chunks(pathlib.Path(root, "archive"))
self.assertEqual(read.read_threads(chunk), [b'{"timestamp": "2026-10-03T00:00:00Z"}\n', b'{"sub": 1}\n'])
class Outcomes(unittest.TestCase):
def test_the_archive_stamps_what_it_reads_back(self):
+7 -5
View File
@@ -7,6 +7,7 @@ from transcript_archive.location import chunk_folder, outcomes_of
from failure import Failure
from transcript_archive.judge_run import JudgeRun
from text_files.file_writes import make_private, replace_file
from text_files.compressed import read_stored, stored
from claude_code.session import session_files
from transcript_archive.outcomes import TASK
from text_files.utc_stamp import utc
@@ -29,7 +30,7 @@ def archive(project_archive: pathlib.Path, path: str, task: str, outcome: str, a
copy = folder / f"{hashlib.sha256(raw).hexdigest()}.jsonl"
outcomes = outcomes_of(copy)
stamped = f"{utc(at)} {outcome}"
if copy.exists() and outcomes.exists():
if stored(copy) and outcomes.exists():
return add_outcome(outcomes, stamped)
# The copy lands last, so a copy on disk always has its outcomes and its session's files beside it.
for dest, data in ((outcomes, f"{TASK}{task}\n\n{stamped}\n".encode()), (copy, raw)):
@@ -41,7 +42,7 @@ def archive(project_archive: pathlib.Path, path: str, task: str, outcome: str, a
def keep_judge_run(folder: pathlib.Path, run: JudgeRun) -> Failure | None:
kept = folder / run.file
if kept.exists():
if stored(kept):
return None
try:
make_private(kept.parent)
@@ -93,11 +94,12 @@ def files_under(folder: pathlib.Path) -> list[pathlib.Path]:
# Claude Code rewrites some files whole, such as custom-title.json, so a copy is replaced only by a file that grew from it.
# A grown copy lands plain beside its gzip, which compress-archives then replaces without losing anything, since the gzip holds a prefix of it.
def copy_for(copy: pathlib.Path, data: bytes) -> pathlib.Path | None:
if not copy.is_file():
if not stored(copy):
return copy
kept = copy.read_bytes()
kept = read_stored(copy)
if data.startswith(kept):
return copy if data != kept else None
beside = copy.with_name(f"{copy.name}.{hashlib.sha256(data).hexdigest()}")
return None if beside.exists() else beside
return None if stored(beside) else beside
+22 -1
View File
@@ -12,6 +12,7 @@ Each project principle states one rule that the decisions below apply; their ent
# Decisions no project principle covers
- archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them
- bin/: a test parametrizes its cases with `subTest`, since pytest's `parametrize` doesn't reach a unittest `TestCase`
- bin/: the tools run on the host's Python, with no container, and so do their tests outside CI
- bin/: the tools send no OTLP traces, logs or metrics, and loop-claude's archive holds the trace of each cycle
@@ -26,6 +27,7 @@ Each project principle states one rule that the decisions below apply; their ent
- project config: a tool that reads a file's commits says where it does not support that file, and the work goes on
- project config: `language` names the prose's language by its English name, and `commit-title`, `pull-request-title` and `branch` are brace templates over `{id}` and `{summary}`
- project config: `plan` may name a Jira Cloud epic as `jira:` and its address, the plan is its open child issues in backlog order, and an issue's number is the item's ID
- project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, and a save whose text is the text a read showed leaves the document as Jira or Confluence holds it
- project config: `tooling-on-host = true` lets a project with no container setup run its language runtime, package manager, tests and task runner on the host
- review: a release review fixes its findings in the release it reviews, and files under that release only a finding that needs a human answer, adds a feature or outlasts its five rounds
- review: a reviewer or comprehension seat that rules the work below standard may ask to rerun once, one rung up, in the same round; the ruling that stands is an escalation trigger for the worker unless the fix changes only prose, and never for the judge
@@ -58,6 +60,21 @@ Settling a call is the human's. The chunk that settles the calls cites, in `docs
# Decisions
## archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them
2026-11-17, the worker of todo item 306. Serves goal 3, backing up the transcripts, and goal 1, whose measures read them: the routing figures, `reading-cost`, `score-outcomes` and both replays rest on the whole archive, so nothing is pruned. On 2026-11-17 the archive held 7.9 GB and `~/.local/state/larv-review` 1.1 GB, both growing with every chunk.
- **What.** `compress-archives`, run daily from the crontab line in `README.md` → "Install", gzips each file that last changed over 30 days ago. The gzip is `<name>.gz`, beside the plain name, with the same times. In the archive it takes only the transcripts, `*.jsonl`, which held 7.8 GB of the 7.9 GB; the outcomes files stay plain, since loop-claude appends to them every cycle. In the review folders it takes every file, since the probes' build output fills most of them.
- **Last changed.** A file's age is the later of its mtime and its ctime. A copy or an unpacked archive keeps old mtimes, but its ctime is when it landed, so a tree a chunk or a replay unpacks today stays plain while they use it.
- **Safely.** It writes the gzip, reads it back, syncs it to disk, and only then removes the plain file. It removes it only where the gzip read back the same bytes and the plain file didn't change while it was compressed. At the first failure it stops and exits 1, keeping that file plain, as a failed archive stops loop-claude.
- **Readers.** The tools that read old files in either folder, the archive's readers and `review-replay`, read a file by its plain name through `bin/text_files/compressed.py`. A plain file beside its gzip wins, since it was written later. A session file that grows after its copy was compressed lands plain beside the gzip, which holds a prefix of it, so the next run's gzip doesn't lose anything. A report rewritten whole after it was gzipped also lands plain beside its gzip, and the next run's gzip replaces the older one, as rewriting the plain file replaced the older bytes before.
- **Why gzip.** Python 3.14's `compression.zstd` needs a Python built with libzstd, and this machine's lacks `_zstd`; gzip is in every build (goal 2). On 400 archived transcripts older than 30 days, gzip shrank their 609 MB to 23%, at about 29 MB a second.
- **Why 30 days.** A chunk's files stop changing once it merges, and chunks run for hours or days. Since every reader reads both forms, the threshold sets only how long a file stays plain.
- **Backup.** Both folders sit under `$HOME`, on the filesystem that holds `~/code` and `~/.claude` (ai-sandbox's `/dev/vda3` on 2026-11-17), so backing that volume up, as item 1 waits for, covers both. `README.md` → "Install" names them, since the review folders lie outside `~/.claude`.
- **One writer.** Project principle 3 holds: the tool that writes a plain file stays its only writer, and `compress-archives` is the only writer of its gzip.
Valid while every tool that reads old files in the two folders reads through `text_files/compressed.py`.
## bin/: a test parametrizes its cases with `subTest`, since pytest's `parametrize` doesn't reach a unittest `TestCase`
2026-11-12, lilleman, from the release review of 0.9.0. Applies `principles/technical-principles.md` → "Most code should have automated tests", whose "parametrize the cases" asks for one reported result per case. The tests are plain unittest, run by pytest, and pytest's `parametrize` doesn't work on a `TestCase` method; `subTest` is the unittest mechanism that does: each case reports and fails on its own, with its parameters named. A test that loops over its cases itself, outside `subTest`, or holds a branch, is still a breach. Valid while the tests stay in plain unittest.
@@ -184,7 +201,7 @@ Valid while claude takes `--session-id` and resumes by it, and the human runs lo
## loop-claude: the archive keeps the raw transcript and each judge run, a failed archive stops the loop, and a replay keeps its runs and report in a replays/ folder of the archive
2026-09-28, lilleman. Serves goals 1 and 3: saving each transcript locally is the first step of backing it up, and it shows afterwards what an unattended run did. A copy of the `.jsonl`, deduplicated by content hash, with its outcomes beside it; a rendering drops tool results, and a second-resolution stamp names two states within one second as one file. A failed archive exits 1, since a loop that loses its record silently fails the backup goal. The judge's runs are archived too, as `judge/<sha256>.json` in the chunk's folder: the prompt it read and its json reply. Together they are its whole transcript, since it runs with no tools and one turn. A run is kept whole and deduplicated by hash, and the outcome of the verdict, interrupt or rejected reply it produced names it on a `Judge run:` line. Keeping it whole costs little: a prompt is at most `JUDGE_PROMPT_CHARS`, 400,000 characters. Measured 2026-11-17, the archive's 911 judge lines come to at most 0.36 GB of prompts, against 3.5 GB in its 1,211 transcript copies, the largest 23.4 MB. The judge is still priced from its `Judge cost:` line, which is claude's own total for the run, and the Spent block counts the live reply's tokens; the archived reply keeps its `usage` for a later reader, and nothing reads it back. `judge-replay` replays a verdict on its archived prompt where there is one. Valid while transcripts, and the judge's prompts, stay small enough to copy whole.
2026-09-28, lilleman. Serves goals 1 and 3: saving each transcript locally is the first step of backing it up, and it shows afterwards what an unattended run did. A copy of the `.jsonl`, deduplicated by content hash, with its outcomes beside it; a rendering drops tool results, and a second-resolution stamp names two states within one second as one file. A failed archive exits 1, since a loop that loses its record silently fails the backup goal. The judge's runs are archived too, as `judge/<sha256>.json` in the chunk's folder: the prompt it read and its json reply. Together they are its whole transcript, since it runs with no tools and one turn. A run is kept whole and deduplicated by hash, and the outcome of the verdict, interrupt or rejected reply it produced names it on a `Judge run:` line. Keeping it whole costs little: a prompt is at most `JUDGE_PROMPT_CHARS`, 400,000 characters. Measured 2026-11-17, the archive's 911 judge lines come to at most 0.36 GB of prompts, against 3.5 GB in its 1,211 transcript copies, the largest 23.4 MB. The judge is still priced from its `Judge cost:` line, which is claude's own total for the run, and the Spent block counts the live reply's tokens; the archived reply keeps its `usage` for a later reader, and nothing reads it back. `judge-replay` replays a verdict on its archived prompt where there is one. Valid while transcripts, and the judge's prompts, stay small enough to copy whole; `archives: gzip each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them` keeps the copies small on disk.
2026-11-17, lilleman, todo item 304. Serves goals 1.4 and 3. `judge-replay` and `review-replay` each decide which rung a review pass or the judge may run on, so their runs stay checkable and re-priceable once a decision is questioned. They keep their runs in `replays/<tool>-<arguments>-<UTC stamp>/` in the archive's root:
@@ -364,6 +381,10 @@ Messages keep each file's usual name, such as `todo.md`, as its label, since the
- **`todo-item edit --comment` carries the comment, and a plan in a file takes the flag and keeps no comment**, so one command works in every project, as goal 1.3 asks. A bench of readers picked the spelling: three fresh readers, seated as the consultant who runs the loop in a client's project, each picked `--comment` over `--reason`, `--why` and `--because`. Only it says the text is posted where the team reads it, and `--reason` read as `--exempt`'s reason.
- **The store finishes an item with the transition that the config's `jira-finish-transition` names**, which must lead to the status `jira-finish-status` names, since each Jira workflow differs (lilleman's call, 2026-11-12); any open child of the epic may move. It refuses a status outside Jira's done status category: the read counts an issue as finished only in that category, so the item would stay in the plan. Where the issue's present status has no direct transition to the status `jira-finish-status` names, `todo-item finish` fails and a person moves the issue in Jira: the store never walks an issue through other statuses to reach it. lilleman answered on 2026-11-15: "If it is possible to move it to done without a transisiont, that should happen. If not, fail." The implementer read "without a transition" as "without a transition in between". `rank_todo.py` refuses to rank a plan in Jira, and its `--check` passes, since Jira's backlog ranks it; rule-hook's rank check skips it. Each read first asks Jira for the epic itself, since a search under an epic the account cannot see answers no issues, as a finished plan does.
## project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, and a save whose text is the text a read showed leaves the document as Jira or Confluence holds it
2026-11-17, claude, through item 391. Serves goal 1.6: a team's Jira and Confluence keep each block's localId and the empty paragraphs Jira uses for spacing, and agents, the judge and people still read the text easily, which keeps goal 1.4's reading cost down. adf-codec converts markdown and ADF by two pairs of calls, as `bin/adf_codec/convert.py` says. The lossless pair keeps the localIds and spacing paragraphs, but wraps every block in two `!adf:` marker lines: a five-line detail became 17 lines in a trial (larvit/adf-codec#185). The portable pair drops both. So the loop reads by the portable pair, and leaves a document alone where a save's text is the text a read showed. `JiraPlan.shows` compares a new detail with the description converted again. `ConfluencePage` keeps the digest of the text it read in the version a write names, and checks that the page still holds that version. Valid while the lossless markdown wraps every block in marker lines. A save that changes one line still rewrites the whole document by the portable pair, and so drops every localId and spacing paragraph, until `todo.md` item 395 keeps each block the save didn't change.
## project config: `tooling-on-host = true` lets a project with no container setup run its language runtime, package manager, tests and task runner on the host
2026-11-12, lilleman, through todo item 358. Serves goal 1.6: a legacy product may have no container setup, and the loop works in it without its owners adding one. `CLAUDE.md`'s container rule and the stability reviewer follow the setting. rule-hook checks only how a container starts, its memory limits and its image's pin, never where a runtime runs, so it needs no change. A decision may still put one tool on the host where the project has containers, as this repo's `bin/` decision does. Readers seated as the person who writes the config picked the key, 2 of 3. Valid while the loop runs on the machine that holds the checkout.
+1 -1
View File
@@ -65,7 +65,7 @@ Give a reviewer without Bash `git ls-files`'s output in its prompt, since some h
`~/.claude/bin/larv-review` holds this section's mechanical rules: the chunk's review folder, the round count, the paired ruling's base and vote count, the scoring run's trigger; the history's format, its latest places and its count of how often seats agree; and the header of each reviewer's prompt. Run it in the chunk's worktree; `~/.claude/bin/larv-review --help` states each rule. On exit 1, repair what its message names and run it again; when you cannot, ask the user.
**Reports.** Run `~/.claude/bin/larv-review reviewer-header <pass>` in the chunk's worktree, and open each prompt to an agent under `~/.claude/agents/` with what it prints; then say in a few lines what the change does and which files it touches. The command makes the chunk's review folder and writes the branch diff and the plan's listing into it. The header names that diff, the reviewer's full report, `<pass>-round-<n>.md`, in the folder, the folders of technical principles and review standards the project's config sets, the listing, and for architecture and stability the places too. For a reviewer whose definition starts it without `CLAUDE.md`, the header's last line names the file that holds the project principles and decision titles, which the reviewer reads first. For a relaunch one rung up, add `--rerun`: the report becomes `<pass>-round-<n>-rerun.md`, and the header names the report that asked. Before you answer a 🔴 or 🟡, read its entry: Grep the report for the finding's place, with the lines after it, rather than Read the whole report. rule-hook refuses a reviewer's Edit and MultiEdit, and any Write outside `~/.local/state/larv-review/`. Keep the folder after the merge: `review-replay` replays a round from the reports its prompt names.
**Reports.** Run `~/.claude/bin/larv-review reviewer-header <pass>` in the chunk's worktree, and open each prompt to an agent under `~/.claude/agents/` with what it prints; then say in a few lines what the change does and which files it touches. The command makes the chunk's review folder and writes the branch diff and the plan's listing into it. The header names that diff, the reviewer's full report, `<pass>-round-<n>.md`, in the folder, the folders of technical principles and review standards the project's config sets, the listing, and for architecture and stability the places too. For a reviewer whose definition starts it without `CLAUDE.md`, the header's last line names the file that holds the project principles and decision titles, which the reviewer reads first. For a relaunch one rung up, add `--rerun`: the report becomes `<pass>-round-<n>-rerun.md`, and the header names the report that asked. Before you answer a 🔴 or 🟡, read its entry: Grep the report for the finding's place, with the lines after it, rather than Read the whole report. rule-hook refuses a reviewer's Edit and MultiEdit, and any Write outside `~/.local/state/larv-review/`. Keep the folder after the merge: `review-replay` replays a round from the reports its prompt names, and `compress-archives` gzips each file 30 days after it last changed.
**Precedent, then ask** — for the full scan:
+6 -38
View File
@@ -6,18 +6,16 @@
`Bar = 0.1`
`Next ID = 395`
`Next ID = 396`
## Items
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 319 | 0.12.0 | | question | **Start a fresh session with a brief of the chunk when a rung step changes the worker's model, in place of resuming the old session.** | 4 | 2 | 3 | 7 | 5 | 1.1 by 2, 1.4 by 2 | 0.9 |
| 306 | 0.12.0 | | | **Set a strategy that keeps loop-claude's archive and larv-review's review folders backed up and small over the years, compressing older entries without losing any.** | 3 | 2 | 4 | 7 | 5 | 3 by 5 | 3.9 |
| 387 | 0.12.0 | | | **Bind a project's rung to its ladder in one value, so the routing functions and `Cycle` take one argument for both.** | 3 | 1 | 3 | 5 | 5 | code health by 3 | 2.9 |
| 338 | 0.12.0 | | owner | **Script each step that 0.9.0's chunks repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.** | 1 | 2 | 1 | 2 | 2 | 1.4 by 3, 1.5 by 3 | 2.3 |
| 288 | 0.12.0 | | | **Point the two reading-cost decisions at code health and goal 1.4's counter-signal, in place of the goal 1.1 they cite.** | 1 | 2 | 1 | 2 | 1 | code health by 1, 1.4 by 1 | 2.2 |
| 391 | 0.12.0 | | | **Read and write a Jira epic issue's description and a Confluence record page through the lossless pair, so saving an unedited document leaves it unchanged.** | 3 | 2 | 3 | 6 | 5 | 1.6 by 6, 1.1 by 3 | 1.9 |
| 309 | 0.12.0 | | | **Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.** | 2 | 2 | 2 | 4 | 2 | 1.4 by 3 | 1.2 |
| 299 | 0.12.0 | | | **Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3 | 1.0 |
| 311 | 0.12.0 | | | **Hand each worker's claude a Bash output ceiling that saves longer output to a file, and refuse a Read of a session transcript under `~/.claude/projects`.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
@@ -68,6 +66,7 @@
| 193 | 0.14.0 | | | **Narrow `numbered`'s mark to `Mark` by a typed lookup in place of `typing.cast`, so a mark added to the regex but not the lookup fails at run time, and one added to the lookup but not `Mark` fails mypy.** | 1 | 1 | 3 | 3 | 1 | 1.1 by 2 | 0.9 |
| 194 | 0.14.0 | | | **Make `test_waits_out_a_usage_limit_which_ends_a_run_of_failures` expect a usage-limit wait independent of the wall-clock time that passes during the run.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 0.9 |
| 265 | 0.14.0 | | | **Archive each wait that costs a chunk wall time, for a usage limit, low memory, a failed worker start or a heavy slot, with its start and end stamps, so goal 1.5's hours split waiting from work.** | 3 | 2 | 2 | 5 | 4 | 1.5 by 3 | 0.8 |
| 395 | 0.14.0 | | decision | **Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.** | 3 | 2 | 2 | 5 | 5 | 1.6 by 4 | 0.8 |
| 169 | 0.14.0 | | | **Have rule-hook's `todo-rank-check` catch a `todo.md` that the same Bash call edits and then commits.** | 3 | 1 | 3 | 5 | 3 | 1.1 by 2 | 0.7 |
| 197 | 0.14.0 | | | **Let the human edit what they type at loop-claude's terminal questions with the arrow keys, so Left moves the cursor in place of printing `^[[D`.** | 2 | 2 | 3 | 5 | 2 | 1.5 by 1 | 0.4 |
| 1 | 0.15.0 | | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 1 | 2 | 1 | 2 | 1 | | 0.0 |
@@ -90,29 +89,6 @@ Touches:
**Question for lilleman, 2026-11-17: drop this item, or build it despite the bench?** `workflow-bench/handoff_bench.py` benched it on five L 3 items. Resume passed 2 and was partial on 3, at a mean $0.68. The fresh session with a brief passed 1 and was partial on 4, at $0.78. Resume's verdict matched or beat the brief's on every item. The decision `routing: a worker whose run ends asking for a higher rung…` holds the table and the bench's limits. Building it also costs more than the files listed above: the archive files a chunk under its session's first line, so `bin/transcript_archive` and each figure read from it would have to join a chunk's two sessions. Recommendation: drop it with `Drop item 319`, and bench again with longer first runs before filing it anew. Whoever closes the item adds the bench to `CHANGELOG.md`.
### 306. Set a strategy that keeps loop-claude's archive and larv-review's review folders backed up and small over the years, compressing older entries without losing any.
Measured on 2026-11-11: loop-claude's archive is 6.3 GB (12,721 files from October alone, 0.84 GB older than 30 days), and `~/.local/state/larv-review/` is 997 MB, 984 MB of it under `code/`, with no pruning: `skills/larv-review/SKILL.md` → "Reports" keeps every folder after the merge, because `review-replay` replays rounds from them. The largest archived worker transcript, 23.4 MB, gzips to 9.1 MB at level 6. The disk holds 65 GB free today.
Write the strategy as a decision, then build it:
- **Backup.** Item 1 waits on ai-sandbox's data volume backup; check that both `~/.claude/loop-claude-transcripts` and `~/.local/state/larv-review` sit on that volume, since the review folders live outside `~/.claude`.
- **Retention.** Keep everything: the archive is the evidence every routing and replay decision rests on. Compress entries older than a threshold (such as 30 days, or once a chunk is finished and measured) with a format the tools read transparently, such as zstd. Compress, verify the round trip, then remove the original, never the reverse; a failure keeps the original and stops, as a failed archive does.
- **Readers.** `bin/transcript_archive`, `routing_figures`, `score-outcomes`, `review-replay`, `judge-replay` and `reading-cost` must read compressed entries; test each on one.
- Revise `loop-claude: the archive keeps the raw transcript`, whose validity is "while transcripts stay small enough to copy whole".
Touches:
- `bin/transcript_archive/read.py`: `read_threads`, `finished_chunks`, `chunk_outcomes`
- `bin/transcript_archive/write.py`: `copy_for`, `mirror`
- `bin/transcript_archive/test_transcript_archive.py`
- `bin/routing_figures/archive.py`
- `bin/reading_cost/main.py`
- `bin/score_outcomes/reversals.py`
- `bin/review_replay/rounds.py`
- `bin/judge_replay/main.py`: `archived_cases` (small)
- `skills/larv-review/SKILL.md`: `## 6. Wrap up` (small)
- `README.md`: `## Install` (small)
### 387. Bind a project's rung to its ladder in one value, so the routing functions and `Cycle` take one argument for both.
Filed by PR #553, which made the routing ladder a file a project may replace. Its stability round 1 found two places the comprehension seats name as hard, touched and no easier to read: `bin/loop_claude/routing.py` `unfinished_route` took `ladder` as a fourth parameter, and `Cycle` in `bin/loop_claude/cycle/run.py` gained the field `ladder`. The patch threaded the ladder as an argument, since a wider change did not fit a patch cut during an outage.
@@ -146,18 +122,6 @@ The chunk that ships this item files a copy of this audit in release 0.15.0, the
Both `docs/decisions.md` entries under `reading cost: a rise is …` and `reading cost: larv-review measures each finished chunk …` open "Serves goal 1.1: comprehension …". No goal says that any more: code health (`CLAUDE.md` → "Shared goal") holds comprehension, and since PR #455 a rising reading cost is goal 1.4's first counter-signal. Raised by the stability review of PR #455 as a 🟢 outside that change.
### 391. Read and write a Jira epic issue's description and a Confluence record page through the lossless pair, so saving an unedited document leaves it unchanged.
`plan_store/jira_epic.py` reads an issue's description and writes it back on an edit, and `record_store/confluence_page.py` reads a page and saves it back; both use the portable pair, which drops each block's `localId` and the empty paragraphs Jira uses for spacing. `record_store/jira_description.py` only reads, to show a person: it stays on the portable pair.
Open before building: the lossless markdown is less readable than the portable one (upstream larvit/adf-codec#185), and agents and the judge read the plan's details from it. Decide how a detail stays readable, for example by reading lossless and showing a person a portable view. Test that an unedited issue and page save back identical. The item ships alone because its R is above 5.
Touches:
- `bin/plan_store/jira_epic.py`: `JiraPlan` (small), `description_of` (small)
- `bin/record_store/confluence_page.py`: `ConfluencePage` (small)
- `bin/plan_store/test_jira_epic.py`: `Editing`, `Filing`
- `bin/record_store/test_confluence_page.py`: `ConfluencePageTest`
### 309. Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.
Seats cost $1,143 of the $9,154 measured from 2026-09-30 to 2026-11-11 (12.5%), scoring runs $470 of that. A scoring run today is four seats, run twice on one head: eight seat runs. Two seats on the second pass make it six, and two on both passes four.
@@ -579,6 +543,10 @@ Found by review of PR #317. The black-box test, `bin/loop_claude/test_black_box.
No record holds these waits today. An outcomes file holds verdicts, stops, step-ups and the judge's cost (`bin/transcript_archive/outcomes.py`), and `~/.local/state/run-heavy/` holds only the slots' lock files. loop-claude waits out a usage limit and low memory in `bin/loop_claude/cycle/retry.py`; run-heavy waits for a slot in `bin/run_heavy/`. A heavy slot's wait happens inside a worker's Bash call, so loop-claude doesn't see it; run-heavy would write it where the archive can find it.
### 395. Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.
Today a Jira detail edited in one line, or a Confluence record with one entry appended, is written back whole by adf-codec's portable pair, which drops each block's localId and the empty paragraphs Jira uses for spacing. A save whose text is the text a read showed already sends nothing (`docs/decisions.md` → `project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, …`). Match the saved markdown's top-level blocks against the held document's, each as the portable pair reads it, and send the held block where they match. The decision's limit on partial saves ends with this item.
### 169. Have rule-hook's `todo-rank-check` catch a `todo.md` that the same Bash call edits and then commits.
The hook judges a call before it runs, and `changed` in `bin/rule_hook/todo_check.py` compares `todo.md` with the index and HEAD as they stand then. A call that edits `todo.md` and commits it, such as a heredoc script followed by `git commit`, passes unchecked: PR #296's commit 149108e did, and the override marker it carried was never logged. larv-review's gate still runs `rank_todo.py --check`, so an unranked plan stops at merge rather than at commit. A git pre-commit hook would see the staged file.