Plan task-bench: the MVP and Go in mutate-changes in 0.15.0, follow-ups in 0.16.0 #676
+1
-1
@@ -392,7 +392,7 @@ Valid while the later fixes per finished chunk, which `score-outcomes` and the r
|
||||
- A file outside the checkout is read as it stands, never as the default branch held it: release standing, chunk-preflight and loop-claude's routing read such a plan or changelog from disk.
|
||||
- chunk-preflight and loop-claude's routing learn that a release's items are all done from the plan's commits. A plan outside the checkout no longer names such a release, so the release after it reads as next, and only the last release of all gets a release review called due and is called ready to cut. Both tools say so on every run, and that the release reviews and the cuts are the human's to call (`bin/release_plan/release_standing.py`, `UNSEEN_RELEASES`).
|
||||
|
||||
Messages keep each file's usual name, such as `todo.md`, as its label, since the config keeps the names. Revisit at 0.22.0, which plans the support for `score-outcomes`, the routing figures, the release standing and the owed scoring run.
|
||||
Messages keep each file's usual name, such as `todo.md`, as its label, since the config keeps the names. Revisit at 0.23.0, which plans the support for `score-outcomes`, the routing figures, the release standing and the owed scoring run.
|
||||
|
||||
## project config: `language` names the prose's language by its English name, and `commit-title`, `pull-request-title` and `branch` are brace templates over `{id}` and `{summary}`
|
||||
|
||||
|
||||
@@ -6,99 +6,212 @@
|
||||
|
||||
`Bar = 0.1`
|
||||
|
||||
`Next ID = 500`
|
||||
`Next ID = 511`
|
||||
|
||||
## Items
|
||||
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 479 | 0.15.0 | | owner | **Record in a decision which of `CLAUDE.md`, path-scoped rules, agents, skills, hooks and plugins Claude Code loads from a `CLAUDE_CONFIG_DIR` home, whether a long-lived token logs in there, and where its transcripts and model catalogue land.** | 1 | 2 | 1 | 2 | 3 | 1.6 by 4 | 1.3 |
|
||||
| 500 | 0.15.0 | | owner | **Mutate the Go lines a branch changes with Gremlins in `mutate-changes`, as it mutates Python, Rust, TypeScript and PHP, and print each mutant the tests do not kill.** | 2 | 2 | 2 | 4 | 4 | 1.1 by 4 | 1.2 |
|
||||
| 501 | 0.15.0 | 479, 500 | owner | **Ship `task-bench`, which plants a bug the tests catch at the places the comprehension seats found hardest, has Haiku fix it in a vanilla Claude Code at a base and a head, and prints for each side how often the fix passes, with its median tokens, turns, wall time and reads before it opens a file the bug is in.** | 4 | 2 | 1 | 5 | 9 | 1.1 by 5, code health by 4 | 0.6 |
|
||||
| 502 | 0.15.0 | 501 | owner | **Distil each `task-bench` run into a digest of short traces and decoys, the files most failing runs read but no fix touches, and have a stronger model turn it into plan items before the transcripts are deleted.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, code health by 4 | 4.8 |
|
||||
| 469 | 0.15.0 | | | **Move a do-epic parent issue whose sub-tasks a person filed to Release Ready once every sub-task has landed, and to Done once every sub-task is done, by transitions its config names.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 4, 1.5 by 4, 1.6 by 3 | 3.1 |
|
||||
| 503 | 0.15.0 | 502 | owner | **Calibrate `task-bench` on fejkdata: bench a head against itself for the noise floor, then one finished chunk at its base and head, and record in a decision how many tasks and runs the bench needs.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3 | 1.2 |
|
||||
| 472 | 0.15.0 | | owner | **Let a project's config name a working branch per epic, such as `TW-12345`, that chunks branch off, merge into and keep level with `main`, so a person merges the finished epic into `main` in one pull request.** | 4 | 2 | 3 | 7 | 7 | 1.5 by 6, 1.6 by 7 | 1.1 |
|
||||
| 467 | 0.15.0 | 472 | question | **Cut 0.15.0 as soon as every other item in it has merged, before its release review, and run that review in 0.15.1.** | 1 | 1 | 1 | 1 | 1 | | 0.0 |
|
||||
| 469 | 0.15.1 | | | **Move a do-epic parent issue whose sub-tasks a person filed to Release Ready once every sub-task has landed, and to Done once every sub-task is done, by transitions its config names.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 4, 1.5 by 4, 1.6 by 3 | 3.1 |
|
||||
| 477 | 0.16.0 | | question | **Rewrite `## Goals` and `## Audience` for a setup others install and run apart from their own `~/.claude`.** | 1 | 1 | 2 | 2 | 2 | | 0.0 |
|
||||
| 478 | 0.16.0 | | question | **Name the project, its repo and its two commands, today proposed as `do-release` and `do-epic`.** | 1 | 1 | 2 | 2 | 1 | | 0.0 |
|
||||
| 479 | 0.16.0 | | owner | **Record in a decision which of `CLAUDE.md`, path-scoped rules, agents, skills, hooks and plugins Claude Code loads from a `CLAUDE_CONFIG_DIR` home, whether a long-lived token logs in there, and where its transcripts and model catalogue land.** | 1 | 2 | 1 | 2 | 3 | 1.6 by 4 | 1.3 |
|
||||
| 480 | 0.17.0 | | owner | **Resolve every path the tools use in one module: the release root (the folder the running code ships in), the config folder, the state folder and the Claude home. Then only that module names `~/.claude`, and the loop still runs from `~/.claude`.** | 3 | 2 | 3 | 6 | 6 | 1.6 by 3, code health by 3 | 2.4 |
|
||||
| 481 | 0.17.0 | 480 | owner | **Put the release's tools on `PATH` for every session the loop starts and every hook it runs, name them bare in the prose in place of `~/.claude/bin/<tool>`, and refuse the old spelling in new prose.** | 3 | 2 | 3 | 6 | 6 | 1.4 by 2, code health by 2 | 1.6 |
|
||||
| 482 | 0.18.0 | 478, 479, 481 | owner | **Install the setup with `uv tool install` from a release tag, with `do-release` and `do-epic` on `PATH`, and replace `deploy-release` with `do-release self-update`.** | 3 | 2 | 3 | 6 | 7 | 1.6 by 4, 1.3 by 3 | 1.3 |
|
||||
| 485 | 0.18.0 | 478, 482 | owner | **Rename `loop-claude` to `do-release`, and keep `loop-claude` as an alias that warns, for one release.** | 2 | 2 | 3 | 5 | 4 | 1.3 by 2 | 2.1 |
|
||||
| 483 | 0.18.0 | 479, 482 | owner | **Run every claude the loop starts in a Claude home the install generates from the release, with hooks that point into it and `do-release login` to sign in there, so the human's `~/.claude` stays theirs.** | 4 | 2 | 3 | 7 | 7 | 1.6 by 6 | 0.5 |
|
||||
| 486 | 0.18.0 | 483 | owner | **Open an interactive claude in the loop's home with `do-release claude`, for a manual `/larv-review` or a talk with the setup loaded.** | 2 | 1 | 1 | 2 | 3 | 1.6 by 2 | 0.7 |
|
||||
| 484 | 0.18.0 | 482 | owner | **Keep project configs, this machine's rules, transcripts and review reports in `~/.config/do-release/` and `~/.local/state/do-release/`, move them there with `do-release migrate`, and have rule-hook guard the new places.** | 3 | 2 | 4 | 7 | 6 | 1.6 by 3 | 0.4 |
|
||||
| 487 | 0.19.0 | 483 | owner | **Run larv-review's passes from a program that picks them from the diff and the item, counts rounds against the cap and decides each rerun, with each reviewer returning its findings as JSON against a schema.** | 4 | 2 | 3 | 7 | 8 | 1.1 by 5, 1.4 by 5 | 0.5 |
|
||||
| 498 | 0.19.0 | 487 | owner | **Drive each chunk as a state machine in `do-release`, from preflight through implementing, review, the pull request and the merge to finishing its items, with the worker as one model step that ends in a result that code checks.** | 4 | 2 | 3 | 7 | 8 | 1.1 by 4, 1.4 by 4 | 0.4 |
|
||||
| 488 | 0.19.0 | 498 | owner | **Cut the larv-review skill, `CLAUDE.md` and rule-hook down to what `do-release` doesn't own: the judgement the model makes and the safety guards.** | 2 | 2 | 3 | 5 | 5 | 1.4 by 4, code health by 2 | 2.8 |
|
||||
| 489 | 0.20.0 | 498 | owner | **Run every model step through a harness interface whose operations are a headless run, a one-shot reply, an interactive run, resume and reading a transcript, with each harness declaring what it supports and Claude Code as the first adapter.** | 4 | 2 | 3 | 7 | 8 | 1.3 by 4, code health by 3 | 1.3 |
|
||||
| 492 | 0.20.0 | 489 | owner | **Refuse to run a harness without pre-tool hooks in a project that sets an `env-file`, unless it runs in a sandbox that cannot read that file.** | 3 | 2 | 4 | 7 | 5 | 4 by 5 | 3.9 |
|
||||
| 491 | 0.20.0 | 489 | owner | **Move the harness detail out of the shipped instructions, rules, standards, principles and reviewer briefs, and have each adapter render its own form, such as Claude Code's agent frontmatter, skills and settings.** | 3 | 2 | 3 | 6 | 7 | 1.3 by 3 | 0.9 |
|
||||
| 490 | 0.20.0 | 489 | owner | **Name a harness on each rung of the routing ladder, as `{"harness": "claude-code", "model": …, "effort": …}`, so the worker, the judge and each reviewer pick a harness as they pick a model.** | 3 | 2 | 3 | 6 | 5 | 1.4 by 3 | 0.6 |
|
||||
| 493 | 0.20.0 | 490, 491 | owner | **Add a second harness adapter and run one chunk of this repo on it end to end.** | 3 | 2 | 2 | 5 | 7 | 1.4 by 3, 1.3 by 2 | 0.9 |
|
||||
| 495 | 0.21.0 | | owner | **Fail the gate where shipped content names a person, a client, a private host or a private URL.** | 2 | 1 | 3 | 4 | 3 | 4 by 5 | 6.2 |
|
||||
| 497 | 0.21.0 | 498 | decision | **Compare the first 10 chunks `do-release`'s state machine drove with the 10 before them on tokens, 🔴 and 🟡 findings and interrupts per chunk, and move back to prose each step whose figures got worse.** | 1 | 2 | 1 | 2 | 3 | 1.1 by 3, 1.4 by 3 | 2.1 |
|
||||
| 496 | 0.21.0 | 478, 491 | owner | **Rename the repo to the name item 478 picks, and rename what says Claude where it means any harness, such as `CLAUDE.md`.** | 3 | 2 | 3 | 6 | 6 | code health by 2 | 1.3 |
|
||||
| 494 | 0.21.0 | 477, 483 | owner | **Let a newcomer install, log in and run a first project from the README alone, with an example config, and test a fresh install in a container in the gate.** | 2 | 2 | 2 | 4 | 5 | 1.6 by 4 | 0.8 |
|
||||
| 446 | 0.22.0 | | question | **Kill the process a stuck check names where its verdict is `stuck`, once the human says the recorded verdicts match what they would have done.** | 2 | 2 | 3 | 5 | 3 | 1.5 by 3 | 1.0 |
|
||||
| 462 | 0.22.0 | | | **Run under `mutate-changes` the tests that today fail in mutmut's copy of the tree, so a mutant of a line only they cover is killed.** | 2 | 1 | 2 | 3 | 3 | code health by 3 | 4.7 |
|
||||
| 77 | 0.22.0 | | | **Teach `bin/metric-bench` Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.** | 4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 2.1 |
|
||||
| 196 | 0.22.0 | | decision | **Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.** | 2 | 2 | 3 | 5 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 321 | 0.22.0 | 196 | decision | **Evaluate the review passes that run on a rung `review-replay` qualified, against the chunks before them, and revert any pass whose findings or later fixes worsened.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3, 1.4 by 2 | 2.0 |
|
||||
| 273 | 0.22.0 | | decision | **Re-tune the E + I steps of `REVIEW_STEPS` for 6 to 8 once 10 finished chunks have scored an E + I of 6 or more, from how well E + I orders their review rounds and 🔴 findings.** | 1 | 2 | 2 | 3 | 2 | 1.1 by 2, 1.4 by 3 | 2.0 |
|
||||
| 268 | 0.22.0 | | | **End a run whose release is ready to cut with a notice on loop-claude's terminal, in place of an interrupt that waits for an answer.** | 2 | 2 | 3 | 5 | 3 | 1 by 5 | 1.9 |
|
||||
| 499 | 0.22.0 | | defect | **Stop `todo-item add` and `edit` from waiting on a stdin that stays open without data, so a call without a heredoc ends at once.** | 2 | 2 | 2 | 4 | 2 | 1.5 by 3, 1.4 by 2 | 1.9 |
|
||||
| 445 | 0.22.0 | | | **Refuse a plan that holds an item a `Finish item <ID>` commit already removed, so a merge resolution cannot bring a finished item back.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3, 1.1 by 2 | 1.8 |
|
||||
| 171 | 0.22.0 | | decision | **Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 3, 1.4 by 3 | 1.8 |
|
||||
| 300 | 0.22.0 | | owner | **Reach the comprehension floor: restructure the places the comprehension seats found hardest to follow, so the project reads at 7.0 or above on every dimension, Locality and Self-sufficiency first; today the scoring run at a357cc7b scored Navigation 7.00, Locality 6.00, Shape 6.13, Self-sufficiency 6.00 and Overall 6.38.** | 4 | 2 | 3 | 7 | 8 | code health by 8 | 1.7 |
|
||||
| 264 | 0.22.0 | | | **Report the comprehension seats and the escalated subagents as subagent types of their own in `routing-figures.md`, with their dollars.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 5 | 1.7 |
|
||||
| 323 | 0.22.0 | | | **Count a release review as landed once its pull request has merged, whatever its commits are titled.** | 3 | 2 | 3 | 6 | 3 | 1.4 by 3, 1.5 by 2 | 1.7 |
|
||||
| 342 | 0.22.0 | | | **Research the tooling agents read, search and run commands with once more, and file an item for each tool that cuts tokens a replay shows keep quality.** | 1 | 2 | 1 | 2 | 3 | 1.4 by 3, 1.5 by 2 | 1.7 |
|
||||
| 471 | 0.22.0 | | | **Ship `jira-rest METHOD PATH [BODY]`, which makes one Jira Cloud REST call with the credentials of the project's config, or do-epic's, reading the env file itself, so an agent calls Jira without naming a file rule-hook guards.** | 2 | 2 | 4 | 6 | 3 | 1.6 by 3, 1.4 by 2 | 1.7 |
|
||||
| 149 | 0.22.0 | | decision | **Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.** | 4 | 2 | 3 | 7 | 5 | 1.1 by 7 | 1.6 |
|
||||
| 165 | 0.22.0 | | | **Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, 1.4 by 3 | 1.6 |
|
||||
| 392 | 0.22.0 | | owner | **Group bin/'s packages by concern into seven folders (plan, records, integrations, loop, review, forge and host), so a newcomer finds a tool's neighbours in one place.** | 3 | 1 | 3 | 5 | 7 | code health by 4 | 1.6 |
|
||||
| 456 | 0.22.0 | | | **Make the child-shell stop test pass under load, so a full test run doesn't fail on a race.** | 2 | 2 | 2 | 4 | 2 | 1.1 by 2, 1.5 by 2 | 1.6 |
|
||||
| 151 | 0.22.0 | | decision | **Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes.** | 3 | 2 | 2 | 5 | 5 | 1.2 by 7 | 1.6 |
|
||||
| 68 | 0.22.0 | | principle | **Test `workflow-bench/compare.py`'s blinding and `bench.py`'s run plan.** | 2 | 1 | 1 | 2 | 4 | 1.1 by 5 | 1.5 |
|
||||
| 102 | 0.22.0 | | | **Give loop-claude's retry delay, and the file it reads the machine's memory from, environment overrides, so the black-box test stops replacing `time.sleep` and `machine_load.memory.MEMINFO` through `sitecustomize`.** | 1 | 1 | 2 | 2 | 3 | 1.1 by 4 | 1.5 |
|
||||
| 135 | 0.22.0 | | | **Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop.** | 2 | 2 | 3 | 5 | 3 | 1.1 by 4 | 1.5 |
|
||||
| 443 | 0.22.0 | | | **Measure whether compacting a waiting worker's session cut the cost of 20 chunks with an interrupt, with quality holding.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 4 | 1.4 |
|
||||
| 5 | 0.22.0 | | decision | **Key a project on its checkout's full path, and refuse to start while another session uses that key.** | 4 | 2 | 3 | 7 | 5 | 1.1 by 6 | 1.4 |
|
||||
| 291 | 0.22.0 | | owner | **Let `principles/technical-principles.md` → "Sane defaults" allow a library whose README goals ask that it work with no config to default the clock and randomness in its public constructor, where a documented option overrides each.** | 1 | 2 | 3 | 4 | 1 | 1.2 by 3 | 1.3 |
|
||||
| 156 | 0.22.0 | | decision | **Drop `rank_todo.py`'s ranking of an old-shape table once no project under `~/code` holds one.** | 2 | 1 | 3 | 4 | 4 | 1.1 by 4 | 1.2 |
|
||||
| 388 | 0.22.0 | | | **Refuse an agent's Write or Edit of a routing ladder file in rule-hook, since a change to the ladder is the human's.** | 1 | 1 | 3 | 3 | 2 | 1.2 by 3 | 1.2 |
|
||||
| 365 | 0.22.0 | | | **Have `score-outcomes` read the finished and dropped items of a plan outside the checkout, from the project repo's history or from Jira's issue changelogs.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
|
||||
| 368 | 0.22.0 | | | **Call a release review due, call a release ready to cut, and tell whether a chunk owes a scoring run, for a plan outside the checkout, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
|
||||
| 260 | 0.22.0 | | decision | **Compare the first 20 chunks whose worker got the reading rule with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and drop the rule where any rose.** | 1 | 2 | 1 | 2 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 309 | 0.22.0 | | | **Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.** | 2 | 2 | 2 | 4 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 51 | 0.22.0 | | principle | **Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints.** | 2 | 1 | 3 | 4 | 5 | 1.1 by 5 | 1.1 |
|
||||
| 419 | 0.22.0 | | | **Run the external auto-reviewer until its review of the PR's head has no findings, and answer each of its threads on the PR.** | 3 | 2 | 3 | 6 | 5 | 1.1 by 5 | 1.1 |
|
||||
| 444 | 0.22.0 | | | **Rank an epic item that is split by repo, by the goals of each repo its sub-tasks name.** | 2 | 2 | 2 | 4 | 3 | 1 by 3 | 1.1 |
|
||||
| 366 | 0.22.0 | | | **Count the chunks of a project whose plan sits outside the checkout in the routing figures, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.4 by 2 | 1.1 |
|
||||
| 201 | 0.22.0 | | owner | **Save the human's tmux setups and restore them after a reboot, so work is up and running again quickly.** | 3 | 2 | 1 | 4 | 4 | 1.5 by 4 | 1.1 |
|
||||
| 267 | 0.22.0 | | | **Keep waiting when an interrupt gets an empty answer, at the terminal or in the claude opened for it, in place of stopping the loop.** | 2 | 1 | 3 | 4 | 2 | 1.5 by 3 | 1.1 |
|
||||
| 4 | 0.22.0 | | | **Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.** | 3 | 2 | 3 | 6 | 3 | 1.1 by 3 | 1.1 |
|
||||
| 324 | 0.22.0 | | | **Count in the routing figures the finished chunks whose worker wrote no `Chunk items:` line.** | 2 | 2 | 1 | 3 | 3 | 1.1 by 2, 1.4 by 1 | 1.1 |
|
||||
| 187 | 0.22.0 | | | **Count the tokens a worker spends reading reviewers' reports, so goal 1.4's signal shows whether report files lowered what a review costs the worker's context.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 266 | 0.22.0 | | | **Move the chunks archived under the key `home/lilleman/.claude` to the key `claude-config`, so the routing figures and `score-outcomes` count this repo's chunks as one project.** | 3 | 2 | 2 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 339 | 0.22.0 | | | **Lower the reading cost of the files a chunk reads most before its first commit.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 341 | 0.22.0 | | | **Hand each worker's claude a lower Read limit through `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS`, kept only where replays of the archived review rounds find what two unchanged replays find.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 181 | 0.22.0 | | principle | **Have the test of a quoted answer's step-up assert the stepped rung's facts as literals, not through `prompts.rungs_held`.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 193 | 0.22.0 | | | **Narrow `numbered`'s mark to `Mark` by a typed lookup in place of `typing.cast`, so a mark added to the regex but not the lookup fails at run time, and one added to the lookup but not `Mark` fails mypy.** | 1 | 1 | 3 | 3 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 194 | 0.22.0 | | | **Make `test_waits_out_a_usage_limit_which_ends_a_run_of_failures` expect a usage-limit wait independent of the wall-clock time that passes during the run.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 298 | 0.22.0 | | | **Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.** | 3 | 3 | 3 | 7 | 4 | 1.4 by 2, 1.5 by 1 | 0.9 |
|
||||
| 265 | 0.22.0 | | | **Archive each wait that costs a chunk wall time, for a usage limit, low memory, a failed worker start or a heavy slot, with its start and end stamps, so goal 1.5's hours split waiting from work.** | 3 | 2 | 2 | 5 | 4 | 1.5 by 3 | 0.8 |
|
||||
| 475 | 0.22.0 | | | **Have stability-reviewer report, as a finding, each lookup that an input from outside the repo can make fail without an error naming what was missing.** | 1 | 1 | 2 | 2 | 2 | 1.1 by 2 | 0.8 |
|
||||
| 395 | 0.22.0 | | decision | **Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.** | 3 | 2 | 2 | 5 | 5 | 1.6 by 4 | 0.8 |
|
||||
| 425 | 0.22.0 | | | **Show on loop-claude's terminal why it doesn't start a chunk beside the next item, as chunk-preflight prints it.** | 1 | 1 | 2 | 2 | 2 | 1.5 by 2 | 0.7 |
|
||||
| 169 | 0.22.0 | | | **Have rule-hook's `todo-rank-check` catch a `todo.md` that the same Bash call edits and then commits.** | 3 | 1 | 3 | 5 | 3 | 1.1 by 2 | 0.7 |
|
||||
| 447 | 0.22.0 | | | **Stop a headless claude call that runs past a time limit, and report it as a failure, so a hung judge or stuck check can't hold the loop.** | 2 | 2 | 2 | 4 | 3 | 1.5 by 2 | 0.7 |
|
||||
| 367 | 0.22.0 | | | **Check that every write to a Jira plan leaves it ranked, as rule-hook's `todo-rank-check` checks `todo.md` today.** | 2 | 2 | 3 | 5 | 3 | 1.6 by 2 | 0.7 |
|
||||
| 423 | 0.22.0 | | | **Skip the judge's model call on a worker's final message that the offline classifier decides is finished.** | 3 | 3 | 3 | 7 | 5 | 1.4 by 2 | 0.4 |
|
||||
| 197 | 0.22.0 | | | **Let the human edit what they type at loop-claude's terminal questions with the arrow keys, so Left moves the cursor in place of printing `^[[D`.** | 2 | 2 | 3 | 5 | 2 | 1.5 by 1 | 0.4 |
|
||||
| 1 | 0.23.0 | | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 1 | 2 | 1 | 2 | 1 | | 0.0 |
|
||||
| 397 | 0.23.0 | | owner | **Script each step that the chunks of releases 0.10.0 to 0.22.0 repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.** | 1 | 2 | 1 | 2 | 2 | 1.4 by 3, 1.5 by 3 | 2.3 |
|
||||
| 509 | 0.16.0 | 503 | owner | **Run `task-bench` beside the scoring run in larv-review comprehension check, and record its figures in the comprehension history beside the panel scores.** | 3 | 2 | 3 | 6 | 5 | code health by 5, 1.1 by 3 | 5.6 |
|
||||
| 504 | 0.16.0 | 503 | owner | **Bench new features in `task-bench`, each a spec for Haiku with acceptance tests it never sees and a reference patch that proves it can be built, kept in a `bench-tasks` folder the project config names, starting with six for fejkdata.** | 3 | 2 | 1 | 4 | 6 | 1.1 by 4, code health by 3 | 2.6 |
|
||||
| 505 | 0.16.0 | 504 | owner | **Bench adding source data in `task-bench`: hand Haiku a fake register that a seeded script generates for a made-up country, and check the importer, the data and the licence entry it adds, starting with five tasks for fejkdata.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3, code health by 2 | 2.6 |
|
||||
| 508 | 0.16.0 | 501 | owner | **Plant bugs for `task-bench` in Python, Rust, TypeScript and PHP, from the mutants `mutate-changes` runs for each, so this repo benches itself.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3, code health by 2 | 2.6 |
|
||||
| 506 | 0.16.0 | 504 | owner | **Go over the bench features in each release review: reapply each reference patch, drop a feature that no longer breaks a goal, and add features until they cover the places the seats find hardest.** | 2 | 2 | 2 | 4 | 3 | 1.1 by 3 | 1.1 |
|
||||
| 507 | 0.16.0 | 504 | owner | **Bench rebuilding a feature the project shipped: revert its commit at the head, hand Haiku its item tagline, and check with the tests that commit added.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 3 | 0.7 |
|
||||
| 477 | 0.17.0 | | question | **Rewrite `## Goals` and `## Audience` for a setup others install and run apart from their own `~/.claude`.** | 1 | 1 | 2 | 2 | 2 | | 0.0 |
|
||||
| 478 | 0.17.0 | | question | **Name the project, its repo and its two commands, today proposed as `do-release` and `do-epic`.** | 1 | 1 | 2 | 2 | 1 | | 0.0 |
|
||||
| 480 | 0.18.0 | | owner | **Resolve every path the tools use in one module: the release root (the folder the running code ships in), the config folder, the state folder and the Claude home. Then only that module names `~/.claude`, and the loop still runs from `~/.claude`.** | 3 | 2 | 3 | 6 | 6 | 1.6 by 3, code health by 3 | 2.4 |
|
||||
| 481 | 0.18.0 | 480 | owner | **Put the release's tools on `PATH` for every session the loop starts and every hook it runs, name them bare in the prose in place of `~/.claude/bin/<tool>`, and refuse the old spelling in new prose.** | 3 | 2 | 3 | 6 | 6 | 1.4 by 2, code health by 2 | 1.6 |
|
||||
| 482 | 0.19.0 | 478, 479, 481 | owner | **Install the setup with `uv tool install` from a release tag, with `do-release` and `do-epic` on `PATH`, and replace `deploy-release` with `do-release self-update`.** | 3 | 2 | 3 | 6 | 7 | 1.6 by 4, 1.3 by 3 | 1.3 |
|
||||
| 485 | 0.19.0 | 478, 482 | owner | **Rename `loop-claude` to `do-release`, and keep `loop-claude` as an alias that warns, for one release.** | 2 | 2 | 3 | 5 | 4 | 1.3 by 2 | 2.1 |
|
||||
| 483 | 0.19.0 | 479, 482 | owner | **Run every claude the loop starts in a Claude home the install generates from the release, with hooks that point into it and `do-release login` to sign in there, so the human's `~/.claude` stays theirs.** | 4 | 2 | 3 | 7 | 7 | 1.6 by 6 | 0.5 |
|
||||
| 486 | 0.19.0 | 483 | owner | **Open an interactive claude in the loop's home with `do-release claude`, for a manual `/larv-review` or a talk with the setup loaded.** | 2 | 1 | 1 | 2 | 3 | 1.6 by 2 | 0.7 |
|
||||
| 484 | 0.19.0 | 482 | owner | **Keep project configs, this machine's rules, transcripts and review reports in `~/.config/do-release/` and `~/.local/state/do-release/`, move them there with `do-release migrate`, and have rule-hook guard the new places.** | 3 | 2 | 4 | 7 | 6 | 1.6 by 3 | 0.4 |
|
||||
| 487 | 0.20.0 | 483 | owner | **Run larv-review's passes from a program that picks them from the diff and the item, counts rounds against the cap and decides each rerun, with each reviewer returning its findings as JSON against a schema.** | 4 | 2 | 3 | 7 | 8 | 1.1 by 5, 1.4 by 5 | 0.5 |
|
||||
| 498 | 0.20.0 | 487 | owner | **Drive each chunk as a state machine in `do-release`, from preflight through implementing, review, the pull request and the merge to finishing its items, with the worker as one model step that ends in a result that code checks.** | 4 | 2 | 3 | 7 | 8 | 1.1 by 4, 1.4 by 4 | 0.4 |
|
||||
| 488 | 0.20.0 | 498 | owner | **Cut the larv-review skill, `CLAUDE.md` and rule-hook down to what `do-release` doesn't own: the judgement the model makes and the safety guards.** | 2 | 2 | 3 | 5 | 5 | 1.4 by 4, code health by 2 | 2.8 |
|
||||
| 489 | 0.21.0 | 498 | owner | **Run every model step through a harness interface whose operations are a headless run, a one-shot reply, an interactive run, resume and reading a transcript, with each harness declaring what it supports and Claude Code as the first adapter.** | 4 | 2 | 3 | 7 | 8 | 1.3 by 4, code health by 3 | 1.3 |
|
||||
| 492 | 0.21.0 | 489 | owner | **Refuse to run a harness without pre-tool hooks in a project that sets an `env-file`, unless it runs in a sandbox that cannot read that file.** | 3 | 2 | 4 | 7 | 5 | 4 by 5 | 3.9 |
|
||||
| 510 | 0.21.0 | 489 | owner | **Run `task-bench`, its gather step included, through the harness interface, in place of calling `claude -p` directly.** | 2 | 2 | 1 | 3 | 3 | code health by 2 | 3.1 |
|
||||
| 491 | 0.21.0 | 489 | owner | **Move the harness detail out of the shipped instructions, rules, standards, principles and reviewer briefs, and have each adapter render its own form, such as Claude Code's agent frontmatter, skills and settings.** | 3 | 2 | 3 | 6 | 7 | 1.3 by 3 | 0.9 |
|
||||
| 490 | 0.21.0 | 489 | owner | **Name a harness on each rung of the routing ladder, as `{"harness": "claude-code", "model": …, "effort": …}`, so the worker, the judge and each reviewer pick a harness as they pick a model.** | 3 | 2 | 3 | 6 | 5 | 1.4 by 3 | 0.6 |
|
||||
| 493 | 0.21.0 | 490, 491 | owner | **Add a second harness adapter and run one chunk of this repo on it end to end.** | 3 | 2 | 2 | 5 | 7 | 1.4 by 3, 1.3 by 2 | 0.9 |
|
||||
| 495 | 0.22.0 | | owner | **Fail the gate where shipped content names a person, a client, a private host or a private URL.** | 2 | 1 | 3 | 4 | 3 | 4 by 5 | 6.2 |
|
||||
| 497 | 0.22.0 | 498 | decision | **Compare the first 10 chunks `do-release`'s state machine drove with the 10 before them on tokens, 🔴 and 🟡 findings and interrupts per chunk, and move back to prose each step whose figures got worse.** | 1 | 2 | 1 | 2 | 3 | 1.1 by 3, 1.4 by 3 | 2.1 |
|
||||
| 496 | 0.22.0 | 478, 491 | owner | **Rename the repo to the name item 478 picks, and rename what says Claude where it means any harness, such as `CLAUDE.md`.** | 3 | 2 | 3 | 6 | 6 | code health by 2 | 1.3 |
|
||||
| 494 | 0.22.0 | 477, 483 | owner | **Let a newcomer install, log in and run a first project from the README alone, with an example config, and test a fresh install in a container in the gate.** | 2 | 2 | 2 | 4 | 5 | 1.6 by 4 | 0.8 |
|
||||
| 446 | 0.23.0 | | question | **Kill the process a stuck check names where its verdict is `stuck`, once the human says the recorded verdicts match what they would have done.** | 2 | 2 | 3 | 5 | 3 | 1.5 by 3 | 1.0 |
|
||||
| 462 | 0.23.0 | | | **Run under `mutate-changes` the tests that today fail in mutmut's copy of the tree, so a mutant of a line only they cover is killed.** | 2 | 1 | 2 | 3 | 3 | code health by 3 | 4.7 |
|
||||
| 77 | 0.23.0 | | | **Teach `bin/metric-bench` Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.** | 4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 2.1 |
|
||||
| 196 | 0.23.0 | | decision | **Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.** | 2 | 2 | 3 | 5 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 321 | 0.23.0 | 196 | decision | **Evaluate the review passes that run on a rung `review-replay` qualified, against the chunks before them, and revert any pass whose findings or later fixes worsened.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3, 1.4 by 2 | 2.0 |
|
||||
| 273 | 0.23.0 | | decision | **Re-tune the E + I steps of `REVIEW_STEPS` for 6 to 8 once 10 finished chunks have scored an E + I of 6 or more, from how well E + I orders their review rounds and 🔴 findings.** | 1 | 2 | 2 | 3 | 2 | 1.1 by 2, 1.4 by 3 | 2.0 |
|
||||
| 268 | 0.23.0 | | | **End a run whose release is ready to cut with a notice on loop-claude's terminal, in place of an interrupt that waits for an answer.** | 2 | 2 | 3 | 5 | 3 | 1 by 5 | 1.9 |
|
||||
| 499 | 0.23.0 | | defect | **Stop `todo-item add` and `edit` from waiting on a stdin that stays open without data, so a call without a heredoc ends at once.** | 2 | 2 | 2 | 4 | 2 | 1.5 by 3, 1.4 by 2 | 1.9 |
|
||||
| 445 | 0.23.0 | | | **Refuse a plan that holds an item a `Finish item <ID>` commit already removed, so a merge resolution cannot bring a finished item back.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3, 1.1 by 2 | 1.8 |
|
||||
| 171 | 0.23.0 | | decision | **Group the routing figures' review rounds by the reviewers' rung as well as the worker's, so a call blames a rung only for rounds its own reviews raised.** | 3 | 2 | 2 | 5 | 4 | 1.2 by 3, 1.4 by 3 | 1.8 |
|
||||
| 300 | 0.23.0 | | owner | **Reach the comprehension floor: restructure the places the comprehension seats found hardest to follow, so the project reads at 7.0 or above on every dimension, Locality and Self-sufficiency first; today the scoring run at a357cc7b scored Navigation 7.00, Locality 6.00, Shape 6.13, Self-sufficiency 6.00 and Overall 6.38.** | 4 | 2 | 3 | 7 | 8 | code health by 8 | 1.7 |
|
||||
| 264 | 0.23.0 | | | **Report the comprehension seats and the escalated subagents as subagent types of their own in `routing-figures.md`, with their dollars.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 5 | 1.7 |
|
||||
| 323 | 0.23.0 | | | **Count a release review as landed once its pull request has merged, whatever its commits are titled.** | 3 | 2 | 3 | 6 | 3 | 1.4 by 3, 1.5 by 2 | 1.7 |
|
||||
| 342 | 0.23.0 | | | **Research the tooling agents read, search and run commands with once more, and file an item for each tool that cuts tokens a replay shows keep quality.** | 1 | 2 | 1 | 2 | 3 | 1.4 by 3, 1.5 by 2 | 1.7 |
|
||||
| 471 | 0.23.0 | | | **Ship `jira-rest METHOD PATH [BODY]`, which makes one Jira Cloud REST call with the credentials of the project's config, or do-epic's, reading the env file itself, so an agent calls Jira without naming a file rule-hook guards.** | 2 | 2 | 4 | 6 | 3 | 1.6 by 3, 1.4 by 2 | 1.7 |
|
||||
| 149 | 0.23.0 | | decision | **Fill each release by priority up to a capacity it states, in place of the bar, and move what does not fit to a later release.** | 4 | 2 | 3 | 7 | 5 | 1.1 by 7 | 1.6 |
|
||||
| 165 | 0.23.0 | | | **Make this repo's tests assert the facts a prompt or help text carries in place of its sentences, and drop the tests that scan source or assert a constant.** | 3 | 2 | 1 | 4 | 5 | 1.1 by 4, 1.4 by 3 | 1.6 |
|
||||
| 392 | 0.23.0 | | owner | **Group bin/'s packages by concern into seven folders (plan, records, integrations, loop, review, forge and host), so a newcomer finds a tool's neighbours in one place.** | 3 | 1 | 3 | 5 | 7 | code health by 4 | 1.6 |
|
||||
| 456 | 0.23.0 | | | **Make the child-shell stop test pass under load, so a full test run doesn't fail on a race.** | 2 | 2 | 2 | 4 | 2 | 1.1 by 2, 1.5 by 2 | 1.6 |
|
||||
| 151 | 0.23.0 | | decision | **Keep 10 to 20 settled decisions per project as its goals' test set, and rerun them after a goal changes.** | 3 | 2 | 2 | 5 | 5 | 1.2 by 7 | 1.6 |
|
||||
| 68 | 0.23.0 | | principle | **Test `workflow-bench/compare.py`'s blinding and `bench.py`'s run plan.** | 2 | 1 | 1 | 2 | 4 | 1.1 by 5 | 1.5 |
|
||||
| 102 | 0.23.0 | | | **Give loop-claude's retry delay, and the file it reads the machine's memory from, environment overrides, so the black-box test stops replacing `time.sleep` and `machine_load.memory.MEMINFO` through `sitecustomize`.** | 1 | 1 | 2 | 2 | 3 | 1.1 by 4 | 1.5 |
|
||||
| 135 | 0.23.0 | | | **Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop.** | 2 | 2 | 3 | 5 | 3 | 1.1 by 4 | 1.5 |
|
||||
| 443 | 0.23.0 | | | **Measure whether compacting a waiting worker's session cut the cost of 20 chunks with an interrupt, with quality holding.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 4 | 1.4 |
|
||||
| 5 | 0.23.0 | | decision | **Key a project on its checkout's full path, and refuse to start while another session uses that key.** | 4 | 2 | 3 | 7 | 5 | 1.1 by 6 | 1.4 |
|
||||
| 291 | 0.23.0 | | owner | **Let `principles/technical-principles.md` → "Sane defaults" allow a library whose README goals ask that it work with no config to default the clock and randomness in its public constructor, where a documented option overrides each.** | 1 | 2 | 3 | 4 | 1 | 1.2 by 3 | 1.3 |
|
||||
| 156 | 0.23.0 | | decision | **Drop `rank_todo.py`'s ranking of an old-shape table once no project under `~/code` holds one.** | 2 | 1 | 3 | 4 | 4 | 1.1 by 4 | 1.2 |
|
||||
| 388 | 0.23.0 | | | **Refuse an agent's Write or Edit of a routing ladder file in rule-hook, since a change to the ladder is the human's.** | 1 | 1 | 3 | 3 | 2 | 1.2 by 3 | 1.2 |
|
||||
| 365 | 0.23.0 | | | **Have `score-outcomes` read the finished and dropped items of a plan outside the checkout, from the project repo's history or from Jira's issue changelogs.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
|
||||
| 368 | 0.23.0 | | | **Call a release review due, call a release ready to cut, and tell whether a chunk owes a scoring run, for a plan outside the checkout, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
|
||||
| 260 | 0.23.0 | | decision | **Compare the first 20 chunks whose worker got the reading rule with the 20 before them, on review rounds past each pass's first, unfinished verdicts, 🔴, 🟡 and misread findings, dollars and hours per chunk, and drop the rule where any rose.** | 1 | 2 | 1 | 2 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 309 | 0.23.0 | | | **Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.** | 2 | 2 | 2 | 4 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 51 | 0.23.0 | | principle | **Hand loop-claude's terminal output to an injected effect, so no storage or retry function prints.** | 2 | 1 | 3 | 4 | 5 | 1.1 by 5 | 1.1 |
|
||||
| 419 | 0.23.0 | | | **Run the external auto-reviewer until its review of the PR's head has no findings, and answer each of its threads on the PR.** | 3 | 2 | 3 | 6 | 5 | 1.1 by 5 | 1.1 |
|
||||
| 444 | 0.23.0 | | | **Rank an epic item that is split by repo, by the goals of each repo its sub-tasks name.** | 2 | 2 | 2 | 4 | 3 | 1 by 3 | 1.1 |
|
||||
| 366 | 0.23.0 | | | **Count the chunks of a project whose plan sits outside the checkout in the routing figures, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.4 by 2 | 1.1 |
|
||||
| 201 | 0.23.0 | | owner | **Save the human's tmux setups and restore them after a reboot, so work is up and running again quickly.** | 3 | 2 | 1 | 4 | 4 | 1.5 by 4 | 1.1 |
|
||||
| 267 | 0.23.0 | | | **Keep waiting when an interrupt gets an empty answer, at the terminal or in the claude opened for it, in place of stopping the loop.** | 2 | 1 | 3 | 4 | 2 | 1.5 by 3 | 1.1 |
|
||||
| 4 | 0.23.0 | | | **Name each chunk's archive folder by the chunk's start time, so `<key>/` lists chunks in the order they ran.** | 3 | 2 | 3 | 6 | 3 | 1.1 by 3 | 1.1 |
|
||||
| 324 | 0.23.0 | | | **Count in the routing figures the finished chunks whose worker wrote no `Chunk items:` line.** | 2 | 2 | 1 | 3 | 3 | 1.1 by 2, 1.4 by 1 | 1.1 |
|
||||
| 187 | 0.23.0 | | | **Count the tokens a worker spends reading reviewers' reports, so goal 1.4's signal shows whether report files lowered what a review costs the worker's context.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 266 | 0.23.0 | | | **Move the chunks archived under the key `home/lilleman/.claude` to the key `claude-config`, so the routing figures and `score-outcomes` count this repo's chunks as one project.** | 3 | 2 | 2 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 339 | 0.23.0 | | | **Lower the reading cost of the files a chunk reads most before its first commit.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 341 | 0.23.0 | | | **Hand each worker's claude a lower Read limit through `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS`, kept only where replays of the archived review rounds find what two unchanged replays find.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 181 | 0.23.0 | | principle | **Have the test of a quoted answer's step-up assert the stepped rung's facts as literals, not through `prompts.rungs_held`.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 193 | 0.23.0 | | | **Narrow `numbered`'s mark to `Mark` by a typed lookup in place of `typing.cast`, so a mark added to the regex but not the lookup fails at run time, and one added to the lookup but not `Mark` fails mypy.** | 1 | 1 | 3 | 3 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 194 | 0.23.0 | | | **Make `test_waits_out_a_usage_limit_which_ends_a_run_of_failures` expect a usage-limit wait independent of the wall-clock time that passes during the run.** | 1 | 1 | 1 | 1 | 1 | 1.1 by 2 | 0.9 |
|
||||
| 298 | 0.23.0 | | | **Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.** | 3 | 3 | 3 | 7 | 4 | 1.4 by 2, 1.5 by 1 | 0.9 |
|
||||
| 265 | 0.23.0 | | | **Archive each wait that costs a chunk wall time, for a usage limit, low memory, a failed worker start or a heavy slot, with its start and end stamps, so goal 1.5's hours split waiting from work.** | 3 | 2 | 2 | 5 | 4 | 1.5 by 3 | 0.8 |
|
||||
| 475 | 0.23.0 | | | **Have stability-reviewer report, as a finding, each lookup that an input from outside the repo can make fail without an error naming what was missing.** | 1 | 1 | 2 | 2 | 2 | 1.1 by 2 | 0.8 |
|
||||
| 395 | 0.23.0 | | decision | **Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.** | 3 | 2 | 2 | 5 | 5 | 1.6 by 4 | 0.8 |
|
||||
| 425 | 0.23.0 | | | **Show on loop-claude's terminal why it doesn't start a chunk beside the next item, as chunk-preflight prints it.** | 1 | 1 | 2 | 2 | 2 | 1.5 by 2 | 0.7 |
|
||||
| 169 | 0.23.0 | | | **Have rule-hook's `todo-rank-check` catch a `todo.md` that the same Bash call edits and then commits.** | 3 | 1 | 3 | 5 | 3 | 1.1 by 2 | 0.7 |
|
||||
| 447 | 0.23.0 | | | **Stop a headless claude call that runs past a time limit, and report it as a failure, so a hung judge or stuck check can't hold the loop.** | 2 | 2 | 2 | 4 | 3 | 1.5 by 2 | 0.7 |
|
||||
| 367 | 0.23.0 | | | **Check that every write to a Jira plan leaves it ranked, as rule-hook's `todo-rank-check` checks `todo.md` today.** | 2 | 2 | 3 | 5 | 3 | 1.6 by 2 | 0.7 |
|
||||
| 423 | 0.23.0 | | | **Skip the judge's model call on a worker's final message that the offline classifier decides is finished.** | 3 | 3 | 3 | 7 | 5 | 1.4 by 2 | 0.4 |
|
||||
| 197 | 0.23.0 | | | **Let the human edit what they type at loop-claude's terminal questions with the arrow keys, so Left moves the cursor in place of printing `^[[D`.** | 2 | 2 | 3 | 5 | 2 | 1.5 by 1 | 0.4 |
|
||||
| 1 | 0.24.0 | | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 1 | 2 | 1 | 2 | 1 | | 0.0 |
|
||||
| 397 | 0.24.0 | | owner | **Script each step that the chunks of releases 0.10.0 to 0.22.0 repeated by hand and a program can compute, filing one item per step, and file this audit again for the next release.** | 1 | 2 | 1 | 2 | 2 | 1.4 by 3, 1.5 by 3 | 2.3 |
|
||||
|
||||
## Details
|
||||
|
||||
### 479. Record in a decision which of `CLAUDE.md`, path-scoped rules, agents, skills, hooks and plugins Claude Code loads from a `CLAUDE_CONFIG_DIR` home, whether a long-lived token logs in there, and where its transcripts and model catalogue land.
|
||||
|
||||
Part of the standalone-install plan lilleman approved on 2026-11-20. `do-release` and `do-epic` stand for the names item 478 picks. The plan runs every claude the loop starts in a Claude home of its own, so the human's `~/.claude` stays theirs. Find out by testing on a throwaway home, never the human's. The result decides whether the loop gets a home of its own, or loads the setup with `--plugin-dir`, `--settings` and `--append-system-prompt-file` instead.
|
||||
|
||||
lilleman moved it into 0.15.0 on 2026-11-20, ahead of item 501: `task-bench` runs Haiku in a container, which is a fresh Claude home signed in by a long-lived token, so the decision also says how that container signs in.
|
||||
|
||||
Touches:
|
||||
|
||||
- `docs/decisions.md`: `## Claude Code in a home of its own: what loads from CLAUDE_CONFIG_DIR, and how a long-lived token signs in`
|
||||
|
||||
### 500. Mutate the Go lines a branch changes with Gremlins in `mutate-changes`, as it mutates Python, Rust, TypeScript and PHP, and print each mutant the tests do not kill.
|
||||
|
||||
fejkdata runs Gremlins v0.6.0 from a compose service of its own today, so a Go project's review mutates outside the shared tool. Run `gremlins unleash --diff <merge base> --output <file>` through `--prefix`, as the other tools run, read its JSON report, and print each mutant that `LIVED` or is `NOT COVERED` as `<path>:<line>:<column>: <status>: <mutator>`. Leave out `*_test.go`. Gremlins v0.6.0 mutates one run of lines per hunk, so its git runs with `diff.context=0`, as fejkdata's compose file sets. A worktree keeps git's data outside the folder a container mounts: `--help` shows the prefix that mounts it, as fejkdata's README does with `GIT_COMMON_DIR`.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/mutate_changes/main.py`: `(module)` (small), `mutate` (small)
|
||||
- `bin/mutate_changes/changed_lines.py`: `TOOLS` (small), `TEST_NAME` (small)
|
||||
- `bin/mutate_changes/in_diff_tools.py`: `run_gremlins`
|
||||
- `bin/mutate_changes/test_mutate_changes.py`
|
||||
- `README.md`: `## Tools` (small)
|
||||
|
||||
### 501. Ship `task-bench`, which plants a bug the tests catch at the places the comprehension seats found hardest, has Haiku fix it in a vanilla Claude Code at a base and a head, and prints for each side how often the fix passes, with its median tokens, turns, wall time and reads before it opens a file the bug is in.
|
||||
|
||||
The panel's scores barely move between chunks, and `reading-cost` measures chunks whose tasks differ. A planted task stays fixed, so a change in its cost comes from the code. The bench runs only programs; no model judges anything in it.
|
||||
|
||||
- **Plant.** Run Gremlins, as item 500 runs it, without `--diff`, over the files that hold the places `larv-review places` prints. Draw 5 of its `KILLED` mutants by a seeded pick, and re-apply each at its file, line and column. The task's text is the failing assertion's message, without the test's name, file or line, so it says what goes wrong and not where. Go is the first language; item 508 adds the others.
|
||||
- **Copy.** The tree at the commit, without `.git`, committed once with the bug in it, so `git diff` and `git log` show Haiku nothing.
|
||||
- **Run.** Inside a container that holds only the copy: the project's development image with Claude Code added, such as `golang` for fejkdata, signed in by a long-lived token in an environment variable, with a memory limit and the network only for the API. In it, `claude -p --bare --strict-mcp-config --disable-slash-commands --no-session-persistence --model claude-haiku-5-5 --effort low --output-format stream-json` runs with every tool, Bash included, so Haiku greps, builds, runs the CLI and runs single tests as an agent normally does. A narrower tool list would raise the figures for reasons outside the code, and an allowlist can't fence Bash anyway: a rule that allows `cat` allows it on any path, such as the real checkout, whose diff would give away the bug. `--bare` drops the human's instructions, memories, hooks, skills and plugins, which would skew the figures and change with claude-config. The repo's README, `AGENTS.md` and `docs/` stay, since they are part of what the bench measures; a project's `CLAUDE.md`, which `--bare` skips, is passed in, so every project is measured alike.
|
||||
- **Pair.** 5 runs per task at the base and at the head, in one bench, so a model or CLI update moves both sides. Each result names the Claude Code version and the model id.
|
||||
- **Check.** The project's test command over Haiku's changes to files other than tests, with the tests restored from the commit, so deleting a test doesn't pass.
|
||||
- **Measure.** From the stream: passed or not, tokens split into input, cache write, cache read and output, turns, wall time, and the reads before the first open of the file the bug is in.
|
||||
- **Print.** One row per task and side, then the head's change against the base. Results and transcripts go under the chunk's review folder.
|
||||
|
||||
It runs under `run-heavy`. The image, the test command and the mutator's prefix are flags, as in `mutate-changes`. Item 503 sets the task and run counts.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/task-bench`
|
||||
- `bin/task_bench/main.py`
|
||||
- `bin/task_bench/plant.py`
|
||||
- `bin/task_bench/runs.py`
|
||||
- `bin/task_bench/test_task_bench.py`
|
||||
- `README.md`: `## Tools` (small), `` ### Packages under `bin/` `` (small)
|
||||
|
||||
### 502. Distil each `task-bench` run into a digest of short traces and decoys, the files most failing runs read but no fix touches, and have a stronger model turn it into plan items before the transcripts are deleted.
|
||||
|
||||
The thought process of the runs shows why Haiku failed, but whole transcripts would flood the model that reads them. A program distils them; only the last step uses a model.
|
||||
|
||||
- **Trace.** One line per tool call, such as `Read pick.go:120-180` or `Grep "readName" → 4 hits`, with the model's text between calls cut to 3 lines.
|
||||
- **Decoys.** Per task, the files more than half its failing runs read that the fix doesn't change. They name the code that misled the runs.
|
||||
- **Digest.** The figures, the decoys, and per task two traces picked by rule: the failing run that spent the most tokens and the passing run with the median tokens. A few kilobytes per task. The raw transcripts stay beside it, for the model to open one by its run id where the digest leaves a question open.
|
||||
- **Gather.** `task-bench gather` starts a claude on the project's rung with the digest and a prompt kept beside the code. It files a plan item with `todo-item add` for each place to make easier, citing the decoys and traces that show it.
|
||||
- **Delete.** Once gather ends, the bench run's folder goes, transcripts and traces with it. Each bench run also deletes the folders earlier runs left, so a session that crashed doesn't leave data growing. The figures alone outlive the run, in what item 509 records.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/task_bench/digest.py`
|
||||
- `bin/task_bench/gather.py`
|
||||
- `bin/task_bench/gather-prompt.md`
|
||||
- `bin/task_bench/main.py`: `main` (small)
|
||||
- `bin/task_bench/test_task_bench.py`
|
||||
|
||||
### 469. Move a do-epic parent issue whose sub-tasks a person filed to Release Ready once every sub-task has landed, and to Done once every sub-task is done, by transitions its config names.
|
||||
|
||||
do-epic moves to done only a parent it split itself (`split_items_closed`). A parent whose sub-tasks a person filed stays as Jira holds it, and `--status` says "its sub-tasks are done; close it in Jira". Until someone moves it, the parent blocks every issue it blocks, and through it their sub-tasks. On TW-13503, TW-13505 blocks most of the critical path this way, and TW-13547's sub-tasks are all Release Ready while the parent stays In Progress. The human asked, on 2026-11-20, for deterministic code to move these parents, not a worker or a person.
|
||||
|
||||
- A parent whose sub-tasks have all landed, each in one of `jira-unblocking-statuses` or done, with at least one not yet done, moves by a transition the config names, such as `jira-landed-transition = "Release Ready"`.
|
||||
- A parent whose sub-tasks are all done moves by `jira-finish-transition`, as a split item does today.
|
||||
- do-epic moves such parents as each run plans, and after each chunk finishes, where it now closes split items. A parent already at or past the target status stays: one in a done status never moves back.
|
||||
- A parent with no sub-tasks, or with one sub-task still open, is left alone.
|
||||
- Without `jira-landed-transition`, only the move to Done happens, and only where `jira-finish-transition` is set, as today.
|
||||
- `--dry-run` prints each parent move it would make.
|
||||
|
||||
Choice, made 2026-11-20: a sub-task in any of `jira-unblocking-statuses` counts as landed, so TW's "Teach & Learn" counts as "Release Ready" does. The human named Release Ready and Done; Teach & Learn also means the work merged.
|
||||
|
||||
Release: lilleman moved it back into 0.15.0 on 2026-11-20, after dropping the plan to cut 0.15.0 early. A chunk had started it in 0.15.0 and was stopped before it committed; its naming panel picked `jira-sub-tasks-landed-transition` for the setting, 2 to 1.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/do_epic/main.py`: `split_items_closed`, `planned_and_started` (small)
|
||||
- `bin/do_epic/planning.py`: `planned` (small)
|
||||
- `bin/do_epic/scheduler.py`: `chunks` (small)
|
||||
- `bin/do_epic/split.py`: `finished` (small)
|
||||
- `bin/do_epic/config.py`: `DoEpicConfig`, `load` (small)
|
||||
- `bin/do_epic/backend/protocol.py`: `EpicStore`
|
||||
- `bin/do_epic/backend/jira.py`: `JiraEpic`
|
||||
- `bin/do_epic/backend/fake.py`: `MemoryEpic`
|
||||
- `bin/do_epic/backend/contract.py`
|
||||
- `bin/do_epic/dry_run.py`
|
||||
- `bin/do_epic/test_do_epic.py`
|
||||
- `bin/do_epic/backend/test_jira.py`
|
||||
- `README.md`: `### Set up a project` (small)
|
||||
|
||||
### 503. Calibrate `task-bench` on fejkdata: bench a head against itself for the noise floor, then one finished chunk at its base and head, and record in a decision how many tasks and runs the bench needs.
|
||||
|
||||
The decision records the spread of the pass rate and of the median tokens when both sides are the same commit, the task and run counts at which a chunk's change rises above that spread, and what one bench costs in tokens and hours. Where no affordable count rises above the noise, it says so, and files what to try instead. The bench's defaults take the counts it picks.
|
||||
|
||||
Touches:
|
||||
|
||||
- `docs/decisions.md`: `## task-bench: how many tasks and runs a bench takes`
|
||||
- `bin/task_bench/main.py`: `main` (small)
|
||||
|
||||
### 472. Let a project's config name a working branch per epic, such as `TW-12345`, that chunks branch off, merge into and keep level with `main`, so a person merges the finished epic into `main` in one pull request.
|
||||
|
||||
Today every chunk branches off the default branch and its pull request merges into it. A team whose rules keep merges into `main` with a person, such as Timewave's, then sets `merge = "human"`, and the loop stops at every chunk until someone merges. lilleman asked on 2026-11-20 for a working branch per epic instead: agents merge each chunk into a branch named as the epic, such as `TW-12345`, and when the whole epic is done and reviewed, a person merges that branch into `main` in one pull request.
|
||||
@@ -129,44 +242,48 @@ Touches:
|
||||
- `docs/decisions.md`
|
||||
- `README.md`: `### Set up a project` (small)
|
||||
|
||||
### 467. Cut 0.15.0 as soon as every other item in it has merged, before its release review, and run that review in 0.15.1.
|
||||
### 509. Run `task-bench` beside the scoring run in larv-review comprehension check, and record its figures in the comprehension history beside the panel scores.
|
||||
|
||||
lilleman's call on 2026-11-20: they need what 0.15.0 holds to work, so 0.15.0 is cut without a release review. This item holds the loop back from starting 0.15.0's release review once its last item merges: it waits as a question, so the human, or the session that cuts, drops it at the cut. Item 469 moved to 0.15.1 by lilleman's call the same day, so 0.15.0 ships sooner. After the cut, 0.15.1's release review, which runs once 469 has merged, covers what 0.15.0 shipped too, and its follow-ups land in 0.15.1.
|
||||
It stays a report beside the panel until about 10 chunks show whether the two agree; whether it can replace the scoring run or gate a merge is the human's call then.
|
||||
|
||||
Touches:
|
||||
### 504. Bench new features in `task-bench`, each a spec for Haiku with acceptance tests it never sees and a reference patch that proves it can be built, kept in a `bench-tasks` folder the project config names, starting with six for fejkdata.
|
||||
|
||||
- `CHANGELOG.md`: `## Unreleased` (small)
|
||||
Each feature breaks one of the project's goals while it would suit another audience, so none ever turns into a real plan item, lies half-built in the code, or tempts a merge. A stronger model writes the tasks once. A run counts only where the acceptance tests fail at both the base and the head before Haiku starts, and pass with the reference patch. Haiku's copy leaves the folder out.
|
||||
|
||||
### 469. Move a do-epic parent issue whose sub-tasks a person filed to Release Ready once every sub-task has landed, and to Done once every sub-task is done, by transitions its config names.
|
||||
fejkdata's first six, each with the goal it breaks:
|
||||
|
||||
do-epic moves to done only a parent it split itself (`split_items_closed`). A parent whose sub-tasks a person filed stays as Jira holds it, and `--status` says "its sub-tasks are done; close it in Jira". Until someone moves it, the parent blocks every issue it blocks, and through it their sub-tasks. On TW-13503, TW-13505 blocks most of the critical path this way, and TW-13547's sub-tasks are all Release Ready while the parent stays In Progress. The human asked, on 2026-11-20, for deterministic code to move these parents, not a worker or a person.
|
||||
1. `{p!}` picks the name `p` again (4.3, a name keeps one pick); it exercises `pick.go`, the seats' hardest place.
|
||||
2. `{frist}` renders as `{first}`, with a warning on stderr (4.7, fejkdata does not guess at mistakes).
|
||||
3. `--typos 0.1` misspells one value in ten, for testing deduplication (3.3, values look real).
|
||||
4. `{@define full = {first} {last}}` defines a macro that `{@full}` prints (4, more syntax).
|
||||
5. `--table` prints a boxed, aligned table (the bulk author, who pipes CSV).
|
||||
6. `--unique-forever` keeps a state file, so no value repeats across runs (seeded output reproduces).
|
||||
|
||||
- A parent whose sub-tasks have all landed, each in one of `jira-unblocking-statuses` or done, with at least one not yet done, moves by a transition the config names, such as `jira-landed-transition = "Release Ready"`.
|
||||
- A parent whose sub-tasks are all done moves by `jira-finish-transition`, as a split item does today.
|
||||
- do-epic moves such parents as each run plans, and after each chunk finishes, where it now closes split items. A parent already at or past the target status stays: one in a done status never moves back.
|
||||
- A parent with no sub-tasks, or with one sub-task still open, is left alone.
|
||||
- Without `jira-landed-transition`, only the move to Done happens, and only where `jira-finish-transition` is set, as today.
|
||||
- `--dry-run` prints each parent move it would make.
|
||||
### 505. Bench adding source data in `task-bench`: hand Haiku a fake register that a seeded script generates for a made-up country, and check the importer, the data and the licence entry it adds, starting with five tasks for fejkdata.
|
||||
|
||||
Choice, made 2026-11-20: a sub-task in any of `jira-unblocking-statuses` counts as landed, so TW's "Teach & Learn" counts as "Release Ready" does. The human named Release Ready and Done; Teach & Learn also means the work merged.
|
||||
Adding a source is fejkdata's recurring work, so it stays easy only while the bench shows it does. The country code comes from ISO 3166-1's user-assigned range (AA, QM–QZ, XA–XZ, ZZ), leaving out XK, which the EU and others use for Kosovo, so no task meets real data. Each register has the quirks real ones publish, and its script regenerates it byte for byte.
|
||||
|
||||
Release: lilleman moved it to 0.15.1 on 2026-11-20 so 0.15.0 ships sooner, although it adds a feature, which semver would put in a minor release. A chunk had started it in 0.15.0 and was stopped before it committed; its naming panel picked `jira-sub-tasks-landed-transition` for the setting, 2 to 1.
|
||||
The checker runs a template such as `{/person[QZ]}` with a fixed seed, fejkdata's full test suite with the data-licence tests, and the importer again, which must reproduce the committed data.
|
||||
|
||||
Touches:
|
||||
fejkdata's five:
|
||||
|
||||
- `bin/do_epic/main.py`: `split_items_closed`, `planned_and_started` (small)
|
||||
- `bin/do_epic/planning.py`: `planned` (small)
|
||||
- `bin/do_epic/scheduler.py`: `chunks` (small)
|
||||
- `bin/do_epic/split.py`: `finished` (small)
|
||||
- `bin/do_epic/config.py`: `DoEpicConfig`, `load` (small)
|
||||
- `bin/do_epic/backend/protocol.py`: `EpicStore`
|
||||
- `bin/do_epic/backend/jira.py`: `JiraEpic`
|
||||
- `bin/do_epic/backend/fake.py`: `MemoryEpic`
|
||||
- `bin/do_epic/backend/contract.py`
|
||||
- `bin/do_epic/dry_run.py`
|
||||
- `bin/do_epic/test_do_epic.py`
|
||||
- `bin/do_epic/backend/test_jira.py`
|
||||
- `README.md`: `### Set up a project` (small)
|
||||
- D1: names weighted by frequency, from a semicolon CSV in Latin-1.
|
||||
- D2: addresses, with postcodes mapped to towns, from nested JSON; values that belong together come from one pick.
|
||||
- D3: a valid national id, from a spec page giving the checksum; the checker validates every id the template prints.
|
||||
- D4: companies with legal-form suffixes, from an XML register.
|
||||
- D5: D1's register in its next edition, with a renamed column and a new code list, so Haiku updates an importer that exists.
|
||||
|
||||
### 508. Plant bugs for `task-bench` in Python, Rust, TypeScript and PHP, from the mutants `mutate-changes` runs for each, so this repo benches itself.
|
||||
|
||||
Each tool reports a killed mutant by file and line, and some by column; the planter re-applies it from that report, as it does Gremlins'.
|
||||
|
||||
### 506. Go over the bench features in each release review: reapply each reference patch, drop a feature that no longer breaks a goal, and add features until they cover the places the seats find hardest.
|
||||
|
||||
A reference patch that no longer applies is rewritten by the review's model. A feature whose goal changed may have become real work, so it goes. New features follow item 504's rule: plausible for another audience, against one of this project's goals.
|
||||
|
||||
### 507. Bench rebuilding a feature the project shipped: revert its commit at the head, hand Haiku its item tagline, and check with the tests that commit added.
|
||||
|
||||
Such tasks cost nothing to write and are realistic. A commit whose revert doesn't apply cleanly at both the base and the head is skipped.
|
||||
|
||||
### 477. Rewrite `## Goals` and `## Audience` for a setup others install and run apart from their own `~/.claude`.
|
||||
|
||||
@@ -176,10 +293,6 @@ Part of the standalone-install plan lilleman approved on 2026-11-20. `do-release
|
||||
|
||||
Part of the standalone-install plan lilleman approved on 2026-11-20. Later items say `do-release` and `do-epic` for the names this item picks. The names set the install folders, such as `~/.config/do-release/`, so items 482, 485 and 496, and every item they block, wait for them.
|
||||
|
||||
### 479. Record in a decision which of `CLAUDE.md`, path-scoped rules, agents, skills, hooks and plugins Claude Code loads from a `CLAUDE_CONFIG_DIR` home, whether a long-lived token logs in there, and where its transcripts and model catalogue land.
|
||||
|
||||
Part of the standalone-install plan lilleman approved on 2026-11-20. `do-release` and `do-epic` stand for the names item 478 picks. The plan runs every claude the loop starts in a Claude home of its own, so the human's `~/.claude` stays theirs. Find out by testing on a throwaway home, never the human's. The result decides whether the loop gets a home of its own, or loads the setup with `--plugin-dir`, `--settings` and `--append-system-prompt-file` instead.
|
||||
|
||||
### 480. Resolve every path the tools use in one module: the release root (the folder the running code ships in), the config folder, the state folder and the Claude home. Then only that module names `~/.claude`, and the loop still runs from `~/.claude`.
|
||||
|
||||
Part of the standalone-install plan lilleman approved on 2026-11-20. `do-release` and `do-epic` stand for the names item 478 picks. Today about 160 references in some 40 modules name `~/.claude`; `bin/loop_state/paths.py` holds a few. This item changes no behaviour.
|
||||
|
||||
Reference in New Issue
Block a user