Key project-record's copies per worktree #555

Merged
lilleman merged 9 commits from item-386 into main 2026-11-15 23:41:30 +01:00
7 changed files with 134 additions and 23 deletions
+1
View File
@@ -2,6 +2,7 @@
## Unreleased
- `project-record open` now writes the copy of a record kept outside a file under a folder of the worktree that ran it, `~/.local/state/project-record/<key>/<worktree>/`, so two chunks of one project no longer overwrite each other's unsaved edit. It also removes the folders of worktrees git no longer lists.
- `chunk-pr merge` merges the chunks of one project on a machine one at a time, and `--finish <ID>` finishes each plan item in the chunk's turn, so `todo.md` and `CHANGELOG.md` no longer conflict between open chunks. A pull request that waits for an approval leaves the queue while it waits, and merge stops where a reviewer's comment waits for an answer.
- Two loop-claude runs in one project now work side by side, each on its own chunk. Each loop holds a slot of the project, a locked file under `~/.local/state/loop-claude/<key>/`, and names its session after it: the first loop's session keeps its name, `A <key> loop-claude`, and the second's is `A <key> loop-claude 2`. A rerun resumes a chunk left open first. A commit in the config's folder now takes only the file it changed, as `git commit -m <title> -- <file>`.
- `todo-item finish` finishes an item of a plan kept in a Jira epic: it moves the item's issue by the workflow transition, and to the status, that the config's new optional settings `jira-finish-transition` and `jira-finish-status` name, both as Jira shows them. It refuses a transition that leads to another status, or to one Jira doesn't count as done, and fails where the config sets neither.
+34 -5
View File
@@ -10,8 +10,10 @@ save refuses them: edit the epic in Jira.
open prints the path of a file holding the record's text, and nothing else. Read and edit that
file. Where the record is kept as a file, the path is that file. Where it is kept elsewhere, the
path is a copy under ~/.local/state/project-record/<key>/, written afresh with the record as it
stands, so open it again before each edit. A record that doesn't exist yet has no file at the path:
path is a copy under ~/.local/state/project-record/<key>/<worktree>/, a folder named for the
worktree that ran open, so two worktrees of one project don't share a copy. The copy is written
afresh with the record as it stands, so open it again before each edit. open also removes the
folders of worktrees git no longer lists. A record that doesn't exist yet has no file at the path:
create it there.
save stores the file's text in the record, and prints one line saying so. Where the record is kept
@@ -24,17 +26,20 @@ writes the copy afresh.
Exit 2 on bad arguments, and 1, saying why, on any other failure.
"""
import hashlib
import pathlib
import shutil
import sys
from failure import Failure
from record_store.stores import FileStore, goals_store, local_file, RECORDS, RecordStore, store, stored
from git_cli.git_repo import git, Git
from project_config.main import located
from project_config.settings import ProjectConfig
# Not git's verbs, such as checkout and commit: those read as acting on the repo.
COMMANDS = ("open", "save")
# Where the copy of a record kept outside files goes, under the user's home, in a folder per project.
# Where the copy of a record kept outside files goes, under the user's home, in a folder per project and worktree.
COPIES = ".local/state/project-record"
# The goals file opens like a record, though it is a setting of its own.
GOALS = "goals-and-audience"
@@ -49,7 +54,7 @@ def main(argv: list[str]) -> int:
return 2
home = pathlib.Path.home()
config = located(None, home)
done = config if isinstance(config, Failure) else act(argv[0], *opened(config, argv[1], home))
done = config if isinstance(config, Failure) else performed(config, argv[0], argv[1], home)
if isinstance(done, Failure):
print(f"project-record: {done.reason}", file=sys.stderr)
return 1
@@ -57,12 +62,36 @@ def main(argv: list[str]) -> int:
return 0
def performed(config: ProjectConfig, command: str, name: str, home: pathlib.Path, run: Git = git) -> str | Failure:
"""What command, one of COMMANDS, prints for the record called name. An open also prunes the copies of removed worktrees."""
if command == "open":
pruned(config, home, run)
return act(command, *opened(config, name, home))
def opened(config: ProjectConfig, name: str, home: pathlib.Path) -> tuple[RecordStore, pathlib.Path]:
"""The store of the record called name, and the copy it gets where it isn't kept as a file. A store that keeps the version it read, for a later save, keeps it beside the copy."""
copy = home / COPIES / config.key / f"{name}.md"
copy = home / COPIES / config.key / worktree_folder(config.top) / f"{name}.md"
return (goals_store(config) if name == GOALS else store(config, name, copy.with_suffix(".version"))), copy
def worktree_folder(top: pathlib.Path) -> str:
"""The name of the folder that holds the copies of the worktree at top: the worktree's own name, for a reader, and a hash of its path, since two worktrees in different folders may share a name."""
return f"{top.name}-{hashlib.sha256(str(top).encode()).hexdigest()[:8]}"
def pruned(config: ProjectConfig, home: pathlib.Path, run: Git = git) -> None:
"""Remove the copies of the worktrees git no longer lists, so the folders don't pile up one per chunk. Best effort: a folder that stays only takes disk, and where git can't list the worktrees nothing is removed."""
listed = run("worktree", "list", "--porcelain")
folders = home / COPIES / config.key
if isinstance(listed, Failure) or not folders.is_dir():
return
worktrees = [config.top, *(pathlib.Path(line.removeprefix("worktree ")) for line in listed.splitlines() if line.startswith("worktree "))]
kept = {worktree_folder(worktree) for worktree in worktrees}
for stale in (folder for folder in folders.iterdir() if folder.is_dir() and folder.name not in kept):
shutil.rmtree(stale, ignore_errors=True)
def act(command: str, record: RecordStore, copy: pathlib.Path) -> str | Failure:
"""What command, one of COMMANDS, prints for record. copy is where a record kept outside files gets its file."""
if command == "open":
+80 -7
View File
@@ -17,19 +17,92 @@ from project_config.settings import Page, ProjectConfig
TOOL = pathlib.Path(__file__).resolve().parents[1] / "project-record"
COPIES = pathlib.Path("/home/me/.local/state/project-record/acme")
def worktree_config(top: str, **kept: Page) -> ProjectConfig:
return ProjectConfig("acme", pathlib.Path("/home/me/.claude-projects/acme"), pathlib.Path(top), {"goals-and-audience": pathlib.Path(top, "README.md")}, pathlib.Path("/home/me/acme.env"), pages=kept)
class Opened(unittest.TestCase):
def test_a_record_on_a_confluence_page_keeps_the_version_it_read_beside_its_copy(self):
page = Page("https://acme.atlassian.net", "2293761", "confluence:https://acme.atlassian.net/wiki/spaces/ENG/pages/2293761")
config = ProjectConfig("acme", pathlib.Path("/home/me/.claude-projects/acme"), pathlib.Path("/code/acme"), {}, pathlib.Path("/home/me/acme.env"), pages={"decisions": page})
record, copy = main.opened(config, "decisions", pathlib.Path("/home/me"))
self.assertEqual(copy, pathlib.Path("/home/me/.local/state/project-record/acme/decisions.md"))
record, copy = main.opened(worktree_config("/code/acme", decisions=page), "decisions", pathlib.Path("/home/me"))
self.assertEqual((copy.parent.parent, copy.name), (COPIES, "decisions.md"))
self.assertRegex(copy.parent.name, r"^acme-[0-9a-f]{8}$")
assert isinstance(record, ConfluencePage), record
self.assertEqual(record.kept, pathlib.Path("/home/me/.local/state/project-record/acme/decisions.version"))
self.assertEqual(record.kept, copy.with_suffix(".version"))
def test_the_goals_file_opens_through_the_goals_store(self):
config = ProjectConfig("acme", pathlib.Path("/home/me/.claude-projects/acme"), pathlib.Path("/code/acme"), {"goals-and-audience": pathlib.Path("/code/acme/README.md")})
record, copy = main.opened(config, "goals-and-audience", pathlib.Path("/home/me"))
self.assertEqual((record.label, copy), ("/code/acme/README.md", pathlib.Path("/home/me/.local/state/project-record/acme/goals-and-audience.md")))
record, copy = main.opened(worktree_config("/code/acme"), "goals-and-audience", pathlib.Path("/home/me"))
self.assertEqual((record.label, copy.parent.parent, copy.name), ("/code/acme/README.md", COPIES, "goals-and-audience.md"))
def test_two_worktrees_of_one_project_get_copies_that_differ(self):
home = pathlib.Path("/home/me")
main_checkout = main.opened(worktree_config("/code/acme"), "goals-and-audience", home)[1]
other = main.opened(worktree_config("/code/acme-item-5"), "goals-and-audience", home)[1]
self.assertNotEqual(main_checkout, other)
def test_one_worktree_always_gets_the_same_copy(self):
home = pathlib.Path("/home/me")
first = main.opened(worktree_config("/code/acme"), "goals-and-audience", home)[1]
self.assertEqual(first, main.opened(worktree_config("/code/acme"), "goals-and-audience", home)[1])
def test_two_worktrees_named_alike_in_different_folders_get_copies_that_differ(self):
home = pathlib.Path("/home/me")
first = main.opened(worktree_config("/code/acme-item-5"), "goals-and-audience", home)[1]
second = main.opened(worktree_config("/other/acme-item-5"), "goals-and-audience", home)[1]
self.assertNotEqual(first, second)
class Pruned(unittest.TestCase):
def setUp(self) -> None:
self.home = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory()))
self.config = worktree_config("/code/acme-item-5")
self.folders = self.home / ".local/state/project-record/acme"
for top in ("/code/acme", "/code/acme-item-5", "/code/acme-item-6"):
copy = main.opened(worktree_config(top), "goals-and-audience", self.home)[1]
copy.parent.mkdir(parents=True)
copy.write_text("edit")
def kept(self) -> set[str]:
return {folder.name for folder in self.folders.iterdir()}
def listed(self, *tops: str):
return lambda *args, **kwargs: "" if args != ("worktree", "list", "--porcelain") else "".join(f"worktree {top}\nHEAD abc\nbranch refs/heads/x\n\n" for top in tops)
def test_removes_the_copies_of_worktrees_git_no_longer_lists_and_keeps_the_others(self):
main.pruned(self.config, self.home, self.listed("/code/acme", "/code/acme-item-5"))
self.assertEqual(self.kept(), {main.worktree_folder(pathlib.Path(top)) for top in ("/code/acme", "/code/acme-item-5")})
def test_always_keeps_the_copy_of_the_worktree_that_asked(self):
main.pruned(self.config, self.home, self.listed("/code/acme"))
self.assertIn(main.worktree_folder(pathlib.Path("/code/acme-item-5")), self.kept())
def test_removes_nothing_where_git_cannot_list_the_worktrees(self):
main.pruned(self.config, self.home, lambda *args, **kwargs: Failure("not a repository"))
self.assertEqual(len(self.kept()), 3)
def test_leaves_files_beside_the_folders_alone(self):
(self.folders / "decisions.md").write_text("from before the folders")
main.pruned(self.config, self.home, self.listed("/code/acme"))
self.assertTrue((self.folders / "decisions.md").exists())
def test_a_copy_that_cannot_be_removed_does_not_fail_the_open(self):
self.folders.chmod(0o500)
self.addCleanup(self.folders.chmod, 0o700)
main.pruned(self.config, self.home, self.listed("/code/acme"))
self.assertEqual(len(self.kept()), 3)
def test_an_open_prunes_and_a_save_does_not(self):
run = self.listed("/code/acme", "/code/acme-item-5")
self.assertEqual(main.performed(self.config, "save", "goals-and-audience", self.home, run), "/code/acme-item-5/README.md is the record itself, so nothing is left to store.")
self.assertEqual(len(self.kept()), 3)
self.assertEqual(main.performed(self.config, "open", "goals-and-audience", self.home, run), "/code/acme-item-5/README.md")
self.assertEqual(self.kept(), {main.worktree_folder(pathlib.Path(top)) for top in ("/code/acme", "/code/acme-item-5")})
def test_does_nothing_where_the_project_has_no_copies(self):
main.pruned(worktree_config("/code/acme"), self.home / "elsewhere", self.listed("/code/acme"))
class Act(unittest.TestCase):
+1 -1
View File
@@ -299,7 +299,7 @@ Valid while the later fixes per finished chunk, which `score-outcomes` and the r
- **A bench of readers picked the spelling.** Three fresh readers, seated as the person who writes a client's `config.toml`, each picked `decisions = "confluence:https://<site>/wiki/spaces/<space>/pages/<page ID>"`, pasted from the browser with or without the title at its end, over the bare address, a page ID beside a `confluence-site` key, and a table. Each named the match with `jira:`. Any of the five records may name a page; the goals file, the plan, the working rules and the principles may not.
- **`decisions` and `comprehension-history` may name a page only where `plan` names a Jira epic.** rule-hook ranks a plan kept as a file before tool calls, inside its 30-second limit, and ranking reads those two records. A Confluence read and two adf-codec container runs would hold up every guarded tool call, and fail it while Confluence is down. `load` refuses goals in Jira beside a plan in a file for the same reason.
- **A write sends the version number the store read, plus one, and Confluence refuses it with 409 where someone saved the page since.** The store then fails, saying to ask the human, and doesn't write the page; the loop never overwrites a teammate's edit on its own (lilleman, 2026-11-12). A write with no read before it replaces the page as it stands, as the record contract asks.
- **`project-record` keeps the version `open` read in a file beside its copy**, `~/.local/state/project-record/<key>/<record>.version`, since `save` runs in a later process. The store keeps the version itself, so `write` still takes only the text. Every other tool reads and writes a record in one process, and its store keeps the version in memory, so a read by another tool, such as larv-review's, never moves the version an edit in `project-record` rests on.
- **`project-record` keeps the version `open` read in a file beside its copy**, `~/.local/state/project-record/<key>/<worktree>/<record>.version`, since `save` runs in a later process. The store keeps the version itself, so `write` still takes only the text. Every other tool reads and writes a record in one process, and its store keeps the version in memory, so a read by another tool, such as larv-review's, never moves the version an edit in `project-record` rests on. The folder `<worktree>` is named for the worktree that ran `open`, and `open` removes the folders of worktrees git no longer lists, so a chunk's copy doesn't outlive its worktree.
- **The page holds ADF, which adf-codec converts both ways**, so a teammate reads the record as Confluence renders it. Where adf-codec cannot convert the markdown, the write fails with adf-codec's error. A node markdown cannot hold reads as text: a mention as its name, a macro as `_(<macro> not included)_`. So the next write replaces it with that text, and a record page should hold only what markdown can.
## project config: a tool that reads a file's commits says where it does not support that file, and the work goes on
+17
View File
@@ -2266,3 +2266,20 @@ Read most (opens and searches):
- `bin/loop_claude/loop/main.py` 2
- `rules/todo-items.md` 2
- `README.md` 1
## Reading cost, chunk 8026e60ea27c, finished 2026-11-15T21:44:03Z
199967 new input tokens before the first commit.
Read most (opens and searches):
- `docs/decisions.md` 11
- `bin/chunk_pr/test_chunk_pr.py` 8
- `skills/larv-review/SKILL.md` 8
- `CLAUDE.md` 5
- `README.md` 5
- `bin/chunk_pr/main.py` 5
- `bin/project_config/templates.py` 4
- `rules/todo-items.md` 4
- `bin/project_config/main.py` 3
- `bin/chunk_pr/approval.py` 2
+1 -1
View File
@@ -2,7 +2,7 @@
paths:
- "**/AGENTS.md"
- "**/docs/decisions.md"
- "**/project-record/*/decisions.md"
- "**/project-record/*/*/decisions.md"
---
# Decisions: one home, each linked to a goal or a technical principle
-9
View File
@@ -12,7 +12,6 @@
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 386 | 0.11.0 | | | **Key project-record's copies per worktree, so two chunks of one project editing the same record kept outside a file don't overwrite each other's unsaved edit.** | 2 | 2 | 2 | 4 | 2 | 1.1 by 2 | 0.8 |
| 359 | 0.11.0 | | | **Tie a project's releases to what its config names: semver tags, Jira Fix Versions, sprints, or none.** | 3 | 2 | 3 | 6 | 4 | 1.6 by 3 | 0.8 |
| 382 | 0.11.0 | | | **Edit a Jira issue the loop filed, and score an unscored issue a teammate filed with a comment saying why.** | 4 | 2 | 3 | 7 | 4 | 1.6 by 3 | 0.8 |
| 384 | 0.11.0 | | | **Rank an issue the loop files in Jira by its score among the issues the loop filed, a security finding first.** | 3 | 2 | 2 | 5 | 3 | 1.6 by 2 | 0.7 |
@@ -78,14 +77,6 @@
## Details
### 386. Key project-record's copies per worktree, so two chunks of one project editing the same record kept outside a file don't overwrite each other's unsaved edit.
Found in item 372's chunk. `project-record open` writes a record kept on a Confluence page to a copy under `~/.local/state/project-record/<key>/`, keyed by the project alone (`bin/project_record/main.py`, `COPIES`). Two chunks of one project, running side by side as item 375 will run them, share that copy: the second `open` rewrites it, and the first chunk's unsaved edit is lost. Key the copy by the worktree as well as the project.
Touches:
- `bin/project_record/main.py`: `(module)` (small)
- `bin/project_record/test_project_record.py`
### 359. Tie a project's releases to what its config names: semver tags, Jira Fix Versions, sprints, or none.
lilleman's call, 2026-11-12: what a release is differs per client. With none, items name no release, and the architecture and principles passes the release review runs need another trigger, which this item settles and records. Readers: `rank_todo.py`'s release order, chunk-preflight's next release, loop-claude's "release ready to cut", larv-review's release review, `CLAUDE.md` → "Working loop". Cutting a release stays the human's.