Key loop-claude's session per slot, so two loops of one project run side by side

This commit is contained in:
2026-11-15 14:09:55 +01:00
parent 510df4b31d
commit cbf007f68c
10 changed files with 111 additions and 11 deletions
+1 -1
View File
@@ -97,7 +97,7 @@ The rules only reviewers apply, for quality, comments, READMEs and prose, are th
- PR summary: short and concise. No test plan (it lives in the ticket), no link to the ticket (it's in the title).
- Branch for a ticket: exactly the ticket ID, nothing more. `ABC-123`, never `ABC-123-some-description`.
- Branch for a plan item without a ticket: `item-<ID>`, in the worktree `<checkout>-item-<ID>` beside the checkout, as `chunk-preflight` makes them. Under a `branch` template, `chunk-preflight` names both by the template.
- Where the project's config keeps a file in the config's folder, `~/.claude-projects/<key>/`, such as the plan or a record, commit each change you make to that file in the folder's own git repo, on its only branch. Title that commit as you would title the change in the checkout: where a commit in the checkout goes with it, give it the same title.
- Where the project's config keeps a file in the config's folder, `~/.claude-projects/<key>/`, such as the plan or a record, commit each change you make to that file in the folder's own git repo, on its only branch. Two chunks may commit there at once, so commit only that file: `git commit -m <title> -- <file>`, after `git add <file>` where it is new. Where git finds `index.lock` held, the other chunk is committing: try again. Title that commit as you would title the change in the checkout: where a commit in the checkout goes with it, give it the same title.
- Use `gh` on GitHub remotes and `tea` on Gitea remotes (e.g. `gitea.larvit.se`) — same command shape: `tea pr list|create`, `tea issue`, `tea release`, `tea api`. Everything in this section applies to both.
- On GitHub and Gitea, open a chunk's pull request as a draft, mark it ready and merge it with `~/.claude/bin/chunk-pr`, which picks `gh` or `tea` from the remote; `chunk-pr --help` says how. Once it has merged, run `chunk-pr clean` in the chunk's worktree: it removes the worktree and the branch and fast-forwards the main checkout.
+31
View File
@@ -0,0 +1,31 @@
"""The slot a loop holds in its project, so that loops of one project, each on its own chunk, run side by side. Each loop names its worker's session after its slot, and the archive and the chunk's route follow the session's transcript (bin/transcript_archive/__init__.py)."""
import os
import pathlib
from failure import Failure
from machine_load.slots import taken_slot
# Under the home folder, a folder per project key holds a file per slot, which a running loop keeps locked with flock.
SLOTS = ".local/state/loop-claude"
# More loops than one project runs at once: todo.md item 375 runs at most the number the project's config allows.
MOST = 64
def held_slot(home: pathlib.Path, key: str) -> int | Failure:
"""The lowest slot no running loop of the project holds, which this loop now holds until it exits, a reload's exec included; or why it could take none."""
folder = home / SLOTS / key
try:
folder.mkdir(parents=True, exist_ok=True)
taken = taken_slot(folder, MOST)
except OSError as failure:
return Failure(f"cannot take a slot in {folder}: {failure.strerror}")
if taken is None:
return Failure(f"{MOST} loops of this project are running already; `fuser {folder}/slot-*` names them")
slot, fd = taken
os.set_inheritable(fd, True)
return slot
def session_name(key: str, slot: int) -> str:
""""A " is the human's cue in the session picker that the session is the loop's. Slot 0 keeps the name every loop had before slots, so a chunk left open then resumes in it."""
return f"A {key} loop-claude" if slot == 0 else f"A {key} loop-claude {slot + 1}"
+8 -2
View File
@@ -3,6 +3,7 @@ import pathlib
import typing
from transcript_archive.location import archive_root, project_archive
from loop_claude.loop.chunk_slot import held_slot, session_name
from loop_claude.loop.chunk_start import archived_route, chunk_start, fetched_todo, routed
from loop_claude.loop.routing_check import check_routing
from loop_claude.project.checkout import checkout_root
@@ -46,8 +47,13 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
if isinstance(loop_run, Stop):
return loop_run
baseline = fingerprint(launcher)
# "A " is the human's cue in the session picker that the session is the loop's.
session = session_named(f"A {project_key(loop_run.root, paths.home)} loop-claude", paths.claude_code_transcripts)
key = project_key(loop_run.root, paths.home)
# A reload's exec keeps the slot's lock open, so the edited script takes the slot it is handed.
slot = held_slot(paths.home, key) if loop_run.slot is None else loop_run.slot
if isinstance(slot, Failure):
return Stop(1, slot.reason)
loop_run = dataclasses.replace(loop_run, slot=slot)
session = session_named(session_name(key, slot), paths.claude_code_transcripts)
loop = Loop(root=loop_run.root, session=session, paths=paths, env=env, launcher=launcher, effects=effects)
while True:
if stop := exec_if_edited(loop, baseline, loop_run):
+3 -1
View File
@@ -25,6 +25,8 @@ class LoopRun:
route: Route | None = None
# How many unfinished verdicts the chunk has had running; a finished chunk and a rerun each set it to 0. It stays apart from route and unfinished_evidence, though all three change together at each verdict: merging or renaming a field stops a running loop at its next reload.
unfinished_run: int = 0
# The project's slot this loop holds (chunk_slot.py); None until it takes one, and from a loop started before slots, which then takes one.
slot: int | None = None
class Unreadable(Exception):
@@ -75,7 +77,7 @@ def decoded_route(value: object) -> Route | None:
# One decoder per field of LoopRun, which a test holds to the dataclass's fields: a field missing here stops a loop at the reload after it joined.
LOOP_RUN_FIELDS = {"cycle_number": exactly(int), "root": exactly(str), "task": exactly(str), "unfinished_evidence": exactly(str), "route": decoded_route, "unfinished_run": exactly(int)}
LOOP_RUN_FIELDS = {"cycle_number": exactly(int), "root": exactly(str), "task": exactly(str), "unfinished_evidence": exactly(str), "route": decoded_route, "unfinished_run": exactly(int), "slot": lambda value: None if value is None else exactly(int)(value)}
def fingerprint(launcher: pathlib.Path) -> str:
+7 -3
View File
@@ -43,8 +43,11 @@ what it prints in the first prompt (prompts.preflight_facts in bin/loop_claude/c
chunk-preflight --help says what it finds and makes. A preflight that fails, or is still running
after 600 s and is stopped, leaves the worker to find the facts itself.
Each project has one session, named "A <key> loop-claude", where <key> is the checkout's path
relative to ~/code, or to / outside it. The session is deleted when its chunk finishes. At every
Each loop runs one chunk at a time, in a session named "A <key> loop-claude", where <key> is the
checkout's path relative to ~/code, or to / outside it. Loops of one project run side by side: each
holds the lowest slot no other holds, a file locked under ~/.local/state/loop-claude/<key>/, and
from the second slot on the session's name ends in the slot's number, as "A <key> loop-claude 2".
The session is deleted when its chunk finishes. At every
interrupt, and as every cycle ends, Ctrl-C and a failure included, the session's transcript is
copied into a folder per chunk under $LOOP_CLAUDE_TRANSCRIPTS/<key> (default root
~/.claude/loop-claude-transcripts), named by its hash, with an outcomes file beside it: the task,
@@ -95,6 +98,7 @@ loop-claude --help succeeds: the loop reloads into the edited script.
Exit 1, saying why, when:
- no task is given
- git cannot start or finds no checkout
- its slot in the project cannot be taken
- the goals file, or AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/*.md or
the project's review standards where they exist, cannot be read or is not UTF-8
- no config names the checkout, its config cannot be read, or the goals file holds no premises
@@ -123,4 +127,4 @@ Exit 2, when the human is needed to go on:
- the claude opened for /talk, or for a release ready to cut, ends without
anything typed
- the judge rules a chunk unfinished 6 cycles running; rerun to give it 6 more
Rerunning resumes any open session."""
Rerunning resumes the open session of the slot the rerun takes."""
+23 -1
View File
@@ -1,5 +1,6 @@
import dataclasses
import datetime
import fcntl
import hashlib
import json
import os
@@ -575,6 +576,27 @@ class Cycles(unittest.TestCase):
self.assertIsNotNone(argument(run.calls[0], "--resume"))
self.assertEqual((argument(run.calls[0], "--model"), argument(run.calls[0], "--effort")), ("claude-fable-5-2", "medium"))
def test_a_second_loop_in_the_project_runs_its_own_session_and_leaves_the_first_loops_open_chunk_alone(self):
checkout = Checkout(self)
first = checkout.run([worker(say="first chunk"), judge(evidence="first chunk")])
session = argument(first.calls[0], "--session-id")
transcript = checkout.transcripts()[0]
kept, archived = transcript.read_bytes(), checkout.archived()
slot = checkout.home / ".local" / "state" / "loop-claude" / "proj" / "slot-0"
slot.parent.mkdir(parents=True, exist_ok=True)
# The test holds the first loop's slot, as the first loop does while it runs.
with open(slot, "a") as running:
fcntl.flock(running, fcntl.LOCK_EX | fcntl.LOCK_NB)
second = checkout.run([worker(say="second chunk"), judge(evidence="second chunk")])
self.assertEqual(argument(second.calls[0], "--name"), "A proj loop-claude 2")
self.assertNotIn(session, [argument(call, "--session-id") or argument(call, "--resume") for call in second.calls if call["role"] == "worker"])
self.assertNotIn("first chunk", second.calls[1]["stdin"])
self.assertEqual(transcript.read_bytes(), kept)
self.assertLessEqual(archived.items(), checkout.archived().items())
self.assertEqual(len(checkout.archived()), len(archived) + 1)
rerun = checkout.run([worker()])
self.assertEqual(argument(rerun.calls[0], "--resume"), session)
def test_exits_1_when_the_transcript_cannot_be_archived_and_resumes_on_the_next_run(self):
checkout = Checkout(self)
checkout.archive().parent.mkdir(parents=True)
@@ -917,7 +939,7 @@ class Reload(unittest.TestCase):
self.assertEqual(run.roles(), ["worker", "judge"] * 5)
self.assertIn("keeping the running script", run.stderr)
reloads = [json.loads(argv[1]) for argv in run.starts if argv[:1] == ["--reloaded"]]
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working", "route": OFF_LADDER_ROUTE, "unfinished_run": cycle} for cycle in (2, 3, 4)])
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working", "route": OFF_LADDER_ROUTE, "unfinished_run": cycle, "slot": 0} for cycle in (2, 3, 4)])
self.assertIn("=== cycle 5,", run.stdout)
outside = checkout.temp / "outside"
+1
View File
@@ -1951,6 +1951,7 @@ class ExitContract(unittest.TestCase):
("cycle/run.py", "close", "1", "failure.reason", read),
("cycle/run.py", "close", "1", "failure.reason", "a finished chunk's session cannot be deleted"),
("loop/close.py", "closed_cycle", "1", "failure.reason", read),
("loop/main.py", "run_loop", "1", "slot.reason", "its slot in the project cannot be taken"),
("loop/main.py", "next_cycle", "1", "premises.reason", premises_),
("loop/main.py", "next_cycle", "1", "records.reason", unreadable),
("loop/main.py", "loop_run_from", "0 if task else 1", "HELP", "no task is given"),
+10 -2
View File
@@ -17,7 +17,7 @@ Each project principle states one rule that the decisions below apply; their ent
- bin/: the tools support only the latest stable Python, and the tests, written in plain unittest, run on it in four pytest-xdist workers
- deploy: this repo keeps no `AGENTS.md`, since `~/.claude` is its checkout; its principles and decision titles head `docs/decisions.md`, its working rules follow them there, and its `.claude/settings.json` excludes every `.claude/CLAUDE.md`, so a session in the clone loads only the clone's `CLAUDE.md`
- loop-claude: each verdict records the claude-config release and commit that ran it, the raw tokens each thread spent by model and kind since the previous outcome that recorded a spend, and the human's typed answer the judge quotes as reversing a choice the worker took on its own recommendation
- loop-claude: one session per project, its id derived from the checkout's path relative to `~/code`, or to `/` outside it
- loop-claude: one session per running loop of a project, named after the lowest slot no other loop of the project holds, its id derived from that name; a commit in the config's folder commits only its own file
- machine load: a heavy command waits for one of the machine's heavy slots, one for every 4 CPUs, which every session shares, and loop-claude starts a worker only while at least a quarter of the machine's memory is available
- principles: about seven things per level, four or five where they act on each other, more where a reader finds them by name; crossing a limit asks for a grouping or a recorded reason
- project config: a record may name a Confluence Cloud page as `confluence:` and its address, the loop reads and writes it as markdown, and a write never overwrites a save made since the loop's read
@@ -142,10 +142,18 @@ Valid while `~/.claude` is a checkout of this repo.
2026-09-30, lilleman. Applies "Always grant explicit access to data": the sandbox `loop-claude` runs in is where access is granted, so the worker and the claude an interrupt opens run with `bypassPermissions`. The worker may read `~/.claude/.credentials.json`: it holds Claude Code's own credentials, which claude reads to run anyway. Its one seam to production is the forge: what it pushes or merges can publish through CI. Valid while `loop-claude` runs only in the sandbox README → Install names.
## loop-claude: one session per project, its id derived from the checkout's path relative to `~/code`, or to `/` outside it
## loop-claude: one session per running loop of a project, named after the lowest slot no other loop of the project holds, its id derived from that name; a commit in the config's folder commits only its own file
2026-09-21, lilleman; revised 2026-09-30 by lilleman. Serves goal 1. One fixed id per project keeps the session picker at one entry per project and makes any transcript under that id loop-claude's own to delete. A checkout outside `~/code` mirroring a path inside it shares that id, and a second loop on it is not refused. Once `todo.md` item 5 lands, the id comes from the full path, so no two checkouts share one, and `loop-claude` refuses to start while another session on that id is active. Moving the checkout orphans its open sessions, archive folder and models entry: sessions stay in the picker until the human deletes them, migration code not being worth a one-human audience. Valid while claude takes `--session-id` and resumes by it.
2026-11-15, lilleman's call of 2026-11-13 in item 372, which split item 361: loops of one project, each on its own chunk, run side by side without touching each other's state. The choices the call left open:
- **A slot, not the chunk, names the session.** A loop names its session before it knows its chunk, since a rerun resumes the open session in place of running `chunk-preflight`. So each loop holds the lowest free slot, a file it keeps locked with flock under `~/.local/state/loop-claude/<key>/` (`bin/loop_claude/loop/chunk_slot.py`). A session lives one chunk, so one slot holds one chunk at a time. Slot 0 keeps the name every loop had before slots, so a chunk left open then still resumes. A reload's exec keeps the lock and hands on the slot.
- **The archive and the route needed no change.** The archive keys each chunk by its own transcript's first line, and a rerun's route comes from the latest verdict on that transcript, so both follow the session. The routing figures are measured across all projects, under their own lock.
- **git's own lock serializes the config folder's commits.** `git commit` holds `.git/index.lock`, so of two commits at once the second fails, and the worker tries again. A wrapper lock would only make it wait, and a command spelled `flock … git commit` would pass rule-hook unchecked, since it reads only a command that starts with `git`. What two chunks could still break is one committing the file the other staged, so each commits only its own file, as `git commit -m <title> -- <file>` (`CLAUDE.md` → "Git, commits, and PRs").
Valid while a loop names its session before it knows its chunk.
## loop-claude: the archive keeps the raw transcript, and a failed archive stops the loop
2026-09-28, lilleman. Serves goals 1 and 3: saving each transcript locally is the first step of backing it up, and it shows afterwards what an unattended run did. A copy of the `.jsonl`, deduplicated by content hash, with its outcomes beside it; a rendering drops tool results, and a second-resolution stamp names two states within one second as one file. A failed archive exits 1, since a loop that loses its record silently fails the backup goal. Valid while transcripts stay small enough to copy whole.
+17
View File
@@ -2232,3 +2232,20 @@ Read most (opens and searches):
- `bin/loop_claude/project/records.py` 3
- `bin/project_config/main.py` 3
- `bin/project_config/settings.py` 3
## Reading cost, chunk 09532b58c32b, finished 2026-11-15T13:01:57Z
152048 new input tokens before the first commit.
Read most (opens and searches):
- `README.md` 5
- `bin/project_config/settings.py` 5
- `bin/plan_store/jira_epic.py` 4
- `bin/project_config/main.py` 4
- `bin/todo_item/main.py` 4
- `bin/plan_store/test_jira_epic.py` 3
- `bin/project_config/test_project_config.py` 3
- `bin/plan_store/stores.py` 2
- `docs/decisions.md` 2
- `CHANGELOG.md` 1
+10 -1
View File
@@ -6,7 +6,7 @@
`Bar = 0.1`
`Next ID = 386`
`Next ID = 387`
## Items
@@ -14,6 +14,7 @@
|---|---|---|---|---|---|---|---|---|---|---|---|
| 372 | 0.11.0 | | | **Key loop-claude's session, archive, routing state and config-folder commits per chunk, so two chunks of one project run side by side without touching each other's state.** | 3 | 2 | 3 | 6 | 6 | 1.5 by 3 | 0.4 |
| 374 | 0.11.0 | 372 | decision | **Merge open chunks one at a time: rebase, finish the item in todo.md and CHANGELOG.md, rerun the gate and the paired ruling, and keep a chunk waiting for approval in the queue without blocking the next.** | 4 | 2 | 3 | 7 | 6 | 1.5 by 4, 1.6 by 3 | 1.0 |
| 386 | 0.11.0 | | | **Key project-record's copies per worktree, so two chunks of one project editing the same record kept outside a file don't overwrite each other's unsaved edit.** | 2 | 2 | 2 | 4 | 2 | 1.1 by 2 | 0.8 |
| 359 | 0.11.0 | | | **Tie a project's releases to what its config names: semver tags, Jira Fix Versions, sprints, or none.** | 3 | 2 | 3 | 6 | 4 | 1.6 by 3 | 0.8 |
| 382 | 0.11.0 | | | **Edit a Jira issue the loop filed, and score an unscored issue a teammate filed with a comment saying why.** | 4 | 2 | 3 | 7 | 4 | 1.6 by 3 | 0.8 |
| 384 | 0.11.0 | | | **Rank an issue the loop files in Jira by its score among the issues the loop filed, a security finding first.** | 3 | 2 | 2 | 5 | 3 | 1.6 by 2 | 0.7 |
@@ -93,6 +94,14 @@ Where the rebase conflicts, the chunk's worker resolves it, and only the review
`bin/chunk_preflight/choices.py:80` `forge_of` is a place the comprehension seats found hard: forge code split across tools. Where this item touches it, build on item 287's forge module if that has shipped.
### 386. Key project-record's copies per worktree, so two chunks of one project editing the same record kept outside a file don't overwrite each other's unsaved edit.
Found in item 372's chunk. `project-record open` writes a record kept on a Confluence page to a copy under `~/.local/state/project-record/<key>/`, keyed by the project alone (`bin/project_record/main.py`, `COPIES`). Two chunks of one project, running side by side as item 375 will run them, share that copy: the second `open` rewrites it, and the first chunk's unsaved edit is lost. Key the copy by the worktree as well as the project.
Touches:
- `bin/project_record/main.py`: `(module)` (small)
- `bin/project_record/test_project_record.py`
### 359. Tie a project's releases to what its config names: semver tags, Jira Fix Versions, sprints, or none.
lilleman's call, 2026-11-12: what a release is differs per client. With none, items name no release, and the architecture and principles passes the release review runs need another trigger, which this item settles and records. Readers: `rank_todo.py`'s release order, chunk-preflight's next release, loop-claude's "release ready to cut", larv-review's release review, `CLAUDE.md` → "Working loop". Cutting a release stays the human's.