Answer stability round 1: 0 🔴, 2 🟡 (0 misread); inject home into the goals file lookup and rename readme in ranking
This commit is contained in:
+2
-2
@@ -2,9 +2,9 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- A project reads its goals and audience from the file its config sets as `goals-and-audience`, in place of `README.md`: loop-claude's prompts and judge, `rank_todo.py`, `todo-item` and rule-hook all read that file. `larv-review reviewer-header` names it on a `Goals and audience:` line, and the reviewers read it from there. A client's project can keep the file, named `goals.md`, in its config's folder, and void code health there as a README can. A checkout no config names now gets no premises and no ranking.
|
||||
- loop-claude, `rank_todo.py`, `todo-item` and rule-hook's `todo.md` checks now fail in a checkout no config names: write a config for each project, with `goals-and-audience = "checkout:README.md"` where its goals stay in its README. They read the goals and audience from the file that setting names, in place of `README.md`. `larv-review reviewer-header` names it on a `Goals and audience:` line, and the reviewers read it from there. A client's project can keep the file, named `goals.md`, in its config's folder, and void code health there as a README can.
|
||||
- Reviewers read the technical principles from the folder a project's config sets as `principles`, in place of `~/.claude/principles/`. `larv-review reviewer-header` names the folder on a `Technical principles:` line and fails where no config sets it, or the folder is missing or empty. Write a config for each project before its next review; a project that keeps today's principles sets `principles = "~/.claude/principles"`.
|
||||
- A project can now have a config, `~/.claude-projects/<key>/config.toml`, that names its checkout and the path of every file the loop keeps for it. `project-config` prints those paths, and fails naming each setting a config leaves out. README → Install → "Set up a project" shows how to write one. Only larv-review's `reviewer-header` reads it yet, so a project without one can't run a review; a later release will need one for every project.
|
||||
- A project can now have a config, `~/.claude-projects/<key>/config.toml`, that names its checkout and the path of every file the loop keeps for it. `project-config` prints those paths, and fails naming each setting a config leaves out. README → Install → "Set up a project" shows how to write one. Every project needs one, as the first bullet says.
|
||||
- larv-review takes an item as a floor item only where its tagline opens with `Reach the comprehension floor:`. A tagline that quotes the floor elsewhere, such as in a decision title, no longer makes its chunk due a scoring run. Reword each open floor item's tagline in every project's `todo.md` to open with the phrase.
|
||||
|
||||
## 0.9.0
|
||||
|
||||
@@ -122,8 +122,8 @@ Claude Code reads this section by its heading when it compacts a conversation. K
|
||||
|
||||
- **chunk** — the unit of work, shipped as one PR; "Working loop" says what it holds.
|
||||
- **cycle** — in `loop-claude`, worker runs and the judge's verdict on each, until the chunk closes or is left for the next cycle. In between, the human answers any interrupt, or the worker resumes with an answer the judge quoted from the premises or the judge's records.
|
||||
- **goals file** — the file holding a project's `## Goals` and `## Audience`: the one its config names as `goals-and-audience`, which `~/.claude/bin/project-config goals-and-audience` prints. It is `README.md`, unless a client's project keeps it in the config's folder.
|
||||
- **gate** — the merge gate: the checks a PR passes before it merges.
|
||||
- **goals file** — the file holding a project's `## Goals` and `## Audience`: the one its config names as `goals-and-audience`, which `~/.claude/bin/project-config goals-and-audience` prints. It is `README.md`, unless a client's project keeps it in the config's folder.
|
||||
- **interrupt** — `loop-claude` asking the human, on its terminal and in `loop-inbox`, for a decision that is theirs. They type a one-line answer in either, or `/talk` to open the worker's session with `claude --resume` where they typed it. A release ready to cut opens that session at once, on loop-claude's terminal. A routing call opens claude on that terminal too, in `~/code/claude-config`, the checkout where this setup is developed.
|
||||
- **judge** — the headless claude that reads a worker's transcript each cycle and decides whether to interrupt and whether the chunk is finished.
|
||||
- **judge's records** — the files the judge reads beside the premises: the project's `AGENTS.md` and `docs/decisions.md`, then the human's `~/.claude/CLAUDE.md` and the `.md` files in `~/.claude/rules/` and `~/.claude/skills/larv-review/standards/`.
|
||||
|
||||
@@ -79,7 +79,7 @@ To get loop-claude's questions and stops as Mattermost DMs from a bot, write the
|
||||
|
||||
Each project the loop runs in has a folder, `~/.claude-projects/<key>/`, where `<key>` is the project's name, and a config in it, `config.toml`. The folder is a git repo of its own, so the config has a history and a client's files never enter this repo. A project is a whole git checkout, never a subfolder of one.
|
||||
|
||||
The config names the project's main checkout and the path of every file the loop keeps for it. No setting has a default: `project-config` fails and names each setting a config leaves out. A path is `checkout:<path>` inside the checkout, `folder:<path>` inside the config's folder, or one that starts with `/` or `~/`. Today the tools read only two settings from the config. `principles` reaches the reviewers through `larv-review reviewer-header`. `goals-and-audience` is read by loop-claude, `rank_todo.py`, `todo-item` and rule-hook, and reaches the reviewers the same way. Every other tool still reads each file at the place this example gives it. A project of your own keeps every file but the principles in its checkout:
|
||||
The config names the project's main checkout and the path of every file the loop keeps for it. No setting has a default: `project-config` fails and names each setting a config leaves out. A path is `checkout:<path>` inside the checkout, `folder:<path>` inside the config's folder, or one that starts with `/` or `~/`. Today the tools read only two settings from the config. `principles` reaches the reviewers through `larv-review reviewer-header`. `goals-and-audience` is read by loop-claude, `rank_todo.py`, `todo-item` and rule-hook, and reaches the reviewers the same way. The tools still read every other file at the place this example gives it. A project of your own keeps every file but the principles in its checkout:
|
||||
|
||||
```toml
|
||||
checkout = "~/code/log"
|
||||
|
||||
@@ -4,7 +4,7 @@ description: "Use this agent for larv-review's overrides pass."
|
||||
tools: Read, Grep, Glob, Write
|
||||
---
|
||||
|
||||
You rule on each override of a rule `~/.claude/bin/rule-hook` enforces. The prompt gives you the lines `rule-hook overrides` printed, each with its rule and reason, and the branch diff. Read each rule and its stated exceptions in `~/.claude/bin/rule_hook/rules.py` → `RULES`, how an override is written and logged in the docstring of `~/.claude/bin/rule_hook/main.py`, and the project's `README.md` → "Goals" and "Audience", with code health (`~/.claude/CLAUDE.md` → "Shared goal") among the goals unless the README voids it.
|
||||
You rule on each override of a rule `~/.claude/bin/rule-hook` enforces. The prompt gives you the lines `rule-hook overrides` printed, each with its rule and reason, and the branch diff. Read each rule and its stated exceptions in `~/.claude/bin/rule_hook/rules.py` → `RULES`, how an override is written and logged in the docstring of `~/.claude/bin/rule_hook/main.py`, and the project's goals and audience, `## Goals` and `## Audience` in the file your prompt's `Goals and audience:` line names, with code health (`~/.claude/CLAUDE.md` → "Shared goal") among the goals unless that file voids it.
|
||||
|
||||
For each override:
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ Before anything else, **read the technical principles** in the folder your promp
|
||||
|
||||
## Your Output Format
|
||||
|
||||
Report only a defect a persona hits, a requirement of the plan, the project's goals, in the file your prompt's `Goals and audience:` line names, or code health (`~/.claude/CLAUDE.md` → "Shared goal", unless that file voids it) that the change breaks, or a breach of a technical principle. Friction that is none of these is not a finding.
|
||||
Report only a defect a persona hits; a requirement that the change breaks, of the plan, of the goals in the file your prompt's `Goals and audience:` line names, or of code health (`~/.claude/CLAUDE.md` → "Shared goal", unless that file voids it); or a breach of a technical principle. Friction that is none of these is not a finding.
|
||||
|
||||
Verdict, severities and rounds: ~/.claude/skills/larv-review/review-findings.md. After the verdict:
|
||||
|
||||
|
||||
@@ -130,7 +130,7 @@ Every PR gets at least this sweep.
|
||||
|
||||
## Output Format
|
||||
|
||||
Report only a defect, a requirement of the plan, the project's goals, in the file your prompt's `Goals and audience:` line names, or code health (`~/.claude/CLAUDE.md` → "Shared goal", unless that file voids it) that the change breaks, a breach of a technical principle, and the places the prompt hands you. A finding that is none of these is noise, however true: leave it out.
|
||||
Report only a defect; a requirement that the change breaks, of the plan, of the goals in the file your prompt's `Goals and audience:` line names, or of code health (`~/.claude/CLAUDE.md` → "Shared goal", unless that file voids it); a breach of a technical principle; and the places the prompt hands you. A finding that is none of these is noise, however true: leave it out.
|
||||
|
||||
Verdict, severities and rounds: ~/.claude/skills/larv-review/review-findings.md. After the verdict, a short **Coverage & Verification** block: what you ran (tests, static analysis, mutation — and their results), what you could not verify and why, and any files that got a lighter pass — never let silence imply coverage. Then the findings, most severe first, each with:
|
||||
- **Location**: File and line/function
|
||||
|
||||
@@ -52,7 +52,7 @@ You work in distinct stages. Never skip stages. Never collapse them. Goals, audi
|
||||
### Stage 0: Establish goals, audience, scale & domain language (do this first)
|
||||
You cannot judge whether an implementation is the best one without a yardstick. The yardstick is five artifacts: **the audience** (who this is for — concrete roles, not "users"), **the problem statement** (what hurts without this), **the goals and their priority order** (including which failure classes are intolerable — the correctness targets the test machinery is priced against), **the scale & horizon** (expected load and data volume in orders of magnitude, team and consumer count, and the intended lifetime — throwaway experiment / MVP / production / long-lived platform — with a promotion event, the one that graduates it and forces re-review), and **the domain glossary** — the ubiquitous language: each core domain concept, one name, one line of meaning. Every name in the design must come from it.
|
||||
|
||||
- **Gather** them — read the README, the goals and audience in the file your prompt's `Goals and audience:` line names, else in the README, CLAUDE.md, AGENTS.md, the technical principles (constraints every design must satisfy), the relevant ticket, and any design docs. What isn't written down is a question for the user.
|
||||
- **Gather** them — read the goals and audience in the file your prompt's `Goals and audience:` line names, or in the README where the prompt has no such line; then the README, CLAUDE.md, AGENTS.md, the technical principles (constraints every design must satisfy), the relevant ticket, and any design docs. What isn't written down is a question for the user.
|
||||
- **Test** them: Is the audience concrete? Does the problem statement name the pain, not the solution? Are the goals explicit, prioritized so you know which wins when two collide, and mutually consistent? Is the horizon stated — spike or platform? Nothing is fit for a purpose whose size and lifespan are unknown. Does the glossary give each concept exactly one name?
|
||||
- **Fix first if broken.** Unless the purpose and goals are stated and consistent, surfacing the gap IS this run's deliverable: you execute one-shot and cannot pause for answers, so return a short draft of the missing artifacts plus the questions the user must answer, and stop there — design or review continues in a later invocation, once the agreed result is recorded. A missing audience, problem statement, scale & horizon, or glossary is the same fix-first gap. Never design against guessed goals, a guessed audience, or invented vocabulary — surface the ambiguity instead.
|
||||
|
||||
|
||||
@@ -67,7 +67,7 @@ def main(argv: list[str], env: typing.Mapping[str, str], home: pathlib.Path, eff
|
||||
cases = sample(cases, PER_KIND)
|
||||
reference = Rung(project.model, REFERENCE_EFFORT)
|
||||
# The candidates stop at the lowest rung a worker routes to: no route uses a rung below it for the judge.
|
||||
replay = replayed(cases, reference, down_from(project, lowest_worker_rung(project)), lambda found, rung: rule(found, rung, home / ".claude", effects), reproduced, "verdicts", climbs(project, reference))
|
||||
replay = replayed(cases, reference, down_from(project, lowest_worker_rung(project)), lambda found, rung: rule(found, rung, home, effects), reproduced, "verdicts", climbs(project, reference))
|
||||
if isinstance(replay, Limited):
|
||||
return limited_exit(TOOL, replay)
|
||||
print(report(project.model, reference, cases, replay))
|
||||
@@ -101,8 +101,8 @@ def archived_cases(model: str, root: pathlib.Path, home: pathlib.Path) -> list[C
|
||||
return found
|
||||
|
||||
|
||||
def rule(case: Case, rung: Rung, claude_dir: pathlib.Path, effects: Effects) -> Kind | Limited:
|
||||
premises, records, entries = read_premises(case.root), read_records(case.root, claude_dir), read_transcript(str(case.transcript))
|
||||
def rule(case: Case, rung: Rung, home: pathlib.Path, effects: Effects) -> Kind | Limited:
|
||||
premises, records, entries = read_premises(case.root, home), read_records(case.root, home / ".claude"), read_transcript(str(case.transcript))
|
||||
if isinstance(premises, Failure):
|
||||
return verdict_kind(premises)
|
||||
if isinstance(records, Failure):
|
||||
|
||||
@@ -157,7 +157,7 @@ class Main(unittest.TestCase):
|
||||
with self.subTest(said=said):
|
||||
self.assertIn(said, self.prompts[0])
|
||||
# The premises head the prompt and the records follow them.
|
||||
self.assertRegex(self.prompts[0], r"(?s)From the goals file:.*1\. Ship\..*The records:.*Be kind\.")
|
||||
self.assertRegex(self.prompts[0], r"(?s)From the goals file, \S*/README\.md:\n## Goals\n1\. Ship\..*The records:.*Be kind\.")
|
||||
# The replay shows the judge the whole transcript, with no step-up marked in it.
|
||||
self.assertNotIn("[step]", self.prompts[0])
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ def reviewer_header(default: str, pass_: str, rerun: bool) -> Failure | None:
|
||||
folder = made_review_folder(worktree)
|
||||
if isinstance(folder, Failure):
|
||||
return folder
|
||||
config = located()
|
||||
config = located(None, pathlib.Path.home())
|
||||
if isinstance(config, Failure):
|
||||
return config
|
||||
principles = config.paths["principles"]
|
||||
|
||||
@@ -252,7 +252,7 @@ def answer_checked(verdict: Verdict, quotable: str) -> Verdict:
|
||||
def judge_prompt(brief: Brief) -> tuple[str, Grounds]:
|
||||
"""The prompt, and what the judge's quotes are checked against."""
|
||||
head = (
|
||||
f"From the goals file:\n{brief.premises}\n\n"
|
||||
f"From the goals file, {brief.premises}\n\n"
|
||||
f"The records: the project's working rules and decisions, then the human's rules for every project:\n{brief.records or 'none'}\n\n"
|
||||
f"The task the worker was given:\n{brief.task}\n\n"
|
||||
"What the worker did and said, what the human typed on [human] lines and what loop-claude told it on [loop] lines, latest last:\n"
|
||||
|
||||
@@ -33,7 +33,7 @@ def fresh(task: str, premises: str, route: Route, project: Rung | None, facts: s
|
||||
"""A chunk's first prompt; facts is preflight_facts' or preflight_failed's text."""
|
||||
routed = f"{routed_by(route, project)} {recount(route)}" if route.item else " ".join(rung_sentences(route, project))
|
||||
opening = f"{routed}\n\n" if routed else ""
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one. {READING_RULE} {NAME_THE_ITEMS}\n\n{opening}{facts}\n\nThe chunk's todo.md items, or else the task above, set its scope. The goals below, {AFTER_CODE_HEALTH}, choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file:\n{premises}"
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one. {READING_RULE} {NAME_THE_ITEMS}\n\n{opening}{facts}\n\nThe chunk's todo.md items, or else the task above, set its scope. The goals below, {AFTER_CODE_HEALTH}, choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file, {premises}"
|
||||
|
||||
|
||||
def routed_by(route: Route, project: Rung | None) -> str:
|
||||
|
||||
@@ -72,7 +72,7 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
|
||||
def next_cycle(loop_run: LoopRun, loop: Loop) -> Cycle | Stop:
|
||||
"""The cycle loop_run runs next, from the premises, the records and the project's models read afresh, and the route the chunk runs on; the stop where one of them cannot be read."""
|
||||
premises = read_premises(loop.root)
|
||||
premises = read_premises(loop.root, loop.paths.home)
|
||||
if isinstance(premises, Failure):
|
||||
return Stop(1, premises.reason)
|
||||
records = read_records(loop.root, loop.paths.claude_dir)
|
||||
|
||||
@@ -3,13 +3,13 @@ HELP = """Run a task as headless claude cycles; a judge interrupts only when a d
|
||||
usage: loop-claude <task>
|
||||
|
||||
Run it anywhere inside a project checkout whose goals file has ## Goals and ## Audience, the
|
||||
premises: the file its config, ~/.claude-projects/<key>/config.toml, sets as goals-and-audience,
|
||||
which `project-config goals-and-audience` prints. The task runs at the checkout root. Each cycle a
|
||||
worker runs the task. A chunk's first worker prompt also quotes the premises. They, with code health
|
||||
(~/.claude/CLAUDE.md → "Shared goal") ranked first unless the goals file voids it, choose between
|
||||
ways to deliver the chunk's todo.md items, or else the task, and never widen them. A judge then
|
||||
reads the transcript's end, the premises, and the records: the project's AGENTS.md and
|
||||
docs/decisions.md, and the human's ~/.claude/CLAUDE.md, ~/.claude/rules/*.md and
|
||||
premises. The goals file is the one the project's config, ~/.claude-projects/<key>/config.toml, sets
|
||||
as goals-and-audience; `project-config goals-and-audience` prints it. The task runs at the checkout
|
||||
root. Each cycle a worker runs the task. A chunk's first worker prompt also quotes the premises.
|
||||
They, with code health (~/.claude/CLAUDE.md → "Shared goal") ranked first unless the goals file
|
||||
voids it, choose between ways to deliver the chunk's todo.md items, or else the task, and never
|
||||
widen them. A judge then reads the transcript's end, the premises, and the records: the project's
|
||||
AGENTS.md and docs/decisions.md, and the human's ~/.claude/CLAUDE.md, ~/.claude/rules/*.md and
|
||||
~/.claude/skills/larv-review/standards/*.md, where they exist. It rules whether the chunk, the work
|
||||
of one PR, is finished. A chunk stays unfinished until each answer the human typed in it that a
|
||||
later chunk could need is recorded: the premises or the records hold it, or the transcript shows the
|
||||
@@ -100,7 +100,7 @@ Exit 1, saying why, when:
|
||||
- git cannot start or finds no checkout
|
||||
- the goals file, or AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/*.md or
|
||||
~/.claude/skills/larv-review/standards/*.md where they exist, cannot be read or is not UTF-8
|
||||
- no config names the checkout, or the goals file holds no premises
|
||||
- no config names the checkout, its config cannot be read, or the goals file holds no premises
|
||||
- the models file cannot be read or written
|
||||
- claude leaves no transcript
|
||||
- a transcript cannot be read or archived
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
import pathlib
|
||||
|
||||
from failure import Failure
|
||||
from project_config.main import goals_file
|
||||
from loop_claude.project.text_file import read_utf8
|
||||
from records.markdown_text import sections
|
||||
|
||||
|
||||
def read_premises(root: str) -> str | Failure:
|
||||
"""## Goals and ## Audience in the file root's project config names for them; a Failure where no config names it, it cannot be read, or it lacks either, or holds one twice or empty."""
|
||||
path = goals_file(root)
|
||||
def read_premises(root: str, home: pathlib.Path) -> str | Failure:
|
||||
"""The goals file's path, then its ## Goals and ## Audience; the goals file is the one that the config under home of root's project names. A Failure where no config names root's checkout, the file cannot be read, or the file lacks a section, holds one twice or leaves one empty."""
|
||||
path = goals_file(root, home)
|
||||
if isinstance(path, Failure):
|
||||
return Failure(f"{path.reason} - no premises.")
|
||||
text = read_utf8(str(path.parent), path.name, "no premises")
|
||||
@@ -16,9 +18,9 @@ def read_premises(root: str) -> str | Failure:
|
||||
for name in ("Goals", "Audience"):
|
||||
bodies = found.get(name, [])
|
||||
if not bodies:
|
||||
return Failure(f"{path.name} has no ## {name} heading - no premises.")
|
||||
return Failure(f"{path} has no ## {name} heading - no premises.")
|
||||
if len(bodies) > 1:
|
||||
return Failure(f"{path.name} has ## {name} more than once - no premises.")
|
||||
return Failure(f"{path} has ## {name} more than once - no premises.")
|
||||
if not bodies[0]:
|
||||
return Failure(f"{path.name} has no text under ## {name} - no premises.")
|
||||
return f"## Goals\n{found['Goals'][0]}\n\n## Audience\n{found['Audience'][0]}"
|
||||
return Failure(f"{path} has no text under ## {name} - no premises.")
|
||||
return f"{path}:\n## Goals\n{found['Goals'][0]}\n\n## Audience\n{found['Audience'][0]}"
|
||||
|
||||
@@ -342,10 +342,10 @@ class Premises(unittest.TestCase):
|
||||
readmes = {
|
||||
None: "README.md in {root}: No such file or directory - no premises.",
|
||||
b"# P\n## Goals\n1. g\xff\n## Audience\nme\n": "README.md in {root} is not UTF-8 on line 3 - no premises.",
|
||||
b"# P\n## Audience\nme\n": "README.md has no ## Goals heading - no premises.",
|
||||
b"# P\n## Goals\n1. g\n": "README.md has no ## Audience heading - no premises.",
|
||||
b"# P\n## Goals\n\n## Audience\nme\n": "README.md has no text under ## Goals - no premises.",
|
||||
b"# P\n## Goals\n1. g\n## Audience\nme\n## Audience\nyou\n": "README.md has ## Audience more than once - no premises.",
|
||||
b"# P\n## Audience\nme\n": "{root}/README.md has no ## Goals heading - no premises.",
|
||||
b"# P\n## Goals\n1. g\n": "{root}/README.md has no ## Audience heading - no premises.",
|
||||
b"# P\n## Goals\n\n## Audience\nme\n": "{root}/README.md has no text under ## Goals - no premises.",
|
||||
b"# P\n## Goals\n1. g\n## Audience\nme\n## Audience\nyou\n": "{root}/README.md has ## Audience more than once - no premises.",
|
||||
}
|
||||
(checkout.root / "sub").mkdir()
|
||||
for text, said in readmes.items():
|
||||
@@ -424,7 +424,7 @@ class Cycles(unittest.TestCase):
|
||||
self.assertIsNone(argument(workers[1], "--name"))
|
||||
self.assertEqual(argument(workers[2], "--session-id"), session)
|
||||
self.assertTrue(argument(workers[0], "-p").startswith(f"{TASK}\n\nContinue any open draft PR"))
|
||||
self.assertTrue(argument(workers[0], "-p").endswith("The chunk's todo.md items, or else the task above, set its scope. The goals below, " 'which rank after code health (~/.claude/CLAUDE.md → "Shared goal") unless the goals file voids it' ", choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file:\n## Goals\n1. Work.\n\n## Audience\nMe."))
|
||||
self.assertTrue(argument(workers[0], "-p").endswith("The chunk's todo.md items, or else the task above, set its scope. The goals below, " 'which rank after code health (~/.claude/CLAUDE.md → "Shared goal") unless the goals file voids it' ", choose between ways to deliver that scope, and never widen it.\n\nFrom the goals file, " f"{checkout.root}/README.md:\n## Goals\n1. Work.\n\n## Audience\nMe."))
|
||||
self.assertIn("main or master", argument(workers[0], "-p"))
|
||||
self.assertNotIn(TASK, argument(workers[1], "-p"))
|
||||
self.assertIn("Even then, it is finished only once every answer the human typed in it", argument(workers[1], "-p"))
|
||||
|
||||
@@ -520,7 +520,7 @@ Spent since 2026-09-21T14:13:20Z, in tokens of input, cache write 5m, cache writ
|
||||
self.assertEqual(len(judge_prompts), 2)
|
||||
self.assertNotIn("the second", judge_prompts[0])
|
||||
self.assertIn("[human] the second", judge_prompts[1])
|
||||
self.assertIn("From the goals file:\n## Goals", judge_prompts[0])
|
||||
self.assertIn("From the goals file, ## Goals", judge_prompts[0])
|
||||
self.assertIn("The task the worker was given:\nDo it.", judge_prompts[0])
|
||||
|
||||
def test_a_question_the_records_answer_resumes_the_worker_with_the_answer_three_times_a_cycle(self):
|
||||
@@ -2026,7 +2026,7 @@ class HelpLimits(unittest.TestCase):
|
||||
|
||||
class ExitContract(unittest.TestCase):
|
||||
def test_every_exit_1_stop_has_its_clause_in_help(self):
|
||||
read, git, premises_ = "a transcript cannot be read or archived", "git cannot start or finds no checkout", "no config names the checkout, or the goals file holds no premises"
|
||||
read, git, premises_ = "a transcript cannot be read or archived", "git cannot start or finds no checkout", "no config names the checkout, its config cannot be read, or the goals file holds no premises"
|
||||
models_ = "the models file cannot be read or written"
|
||||
names = (*records.RECORD_FILES, records.HUMAN_RULE_NAMES[0], *(f"{folder}*.md" for folder in records.HUMAN_RULE_NAMES[1:]))
|
||||
unreadable = f"the goals file, or {', '.join(names[:-1])} or {names[-1]} where they exist, cannot be read or is not UTF-8"
|
||||
|
||||
@@ -39,7 +39,7 @@ def main(argv: list[str]) -> int:
|
||||
if len(argv) > 1 or (argv and argv[0] not in SETTINGS):
|
||||
print(f"usage: project-config [<setting>], where <setting> is one of {', '.join(SETTINGS)}", file=sys.stderr)
|
||||
return 1
|
||||
config = located()
|
||||
config = located(None, pathlib.Path.home())
|
||||
if isinstance(config, Failure):
|
||||
print(config.reason, file=sys.stderr)
|
||||
return 1
|
||||
@@ -47,18 +47,18 @@ def main(argv: list[str]) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def located(directory: str | None = None) -> ProjectConfig | Failure:
|
||||
"""The config of the checkout that holds directory, else the current directory."""
|
||||
def located(directory: str | None, home: pathlib.Path) -> ProjectConfig | Failure:
|
||||
"""The config under home of the checkout that holds directory, else the current directory."""
|
||||
found = git("rev-parse", "--path-format=absolute", "--show-toplevel", "--git-common-dir", cwd=directory)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
if len(lines := found.splitlines()) != 2:
|
||||
return Failure(f"git rev-parse printed {found!r}, not a checkout's top folder and git directory on two lines")
|
||||
# The common dir is the main checkout's .git, whichever worktree runs this.
|
||||
return load(pathlib.Path(lines[0]), pathlib.Path(lines[1]).parent, pathlib.Path.home())
|
||||
return load(pathlib.Path(lines[0]), pathlib.Path(lines[1]).parent, home)
|
||||
|
||||
|
||||
def goals_file(directory: str) -> pathlib.Path | Failure:
|
||||
"""The file holding the premises, `## Goals` and `## Audience`, of the project whose checkout holds directory."""
|
||||
config = located(directory)
|
||||
def goals_file(directory: str, home: pathlib.Path) -> pathlib.Path | Failure:
|
||||
"""The file holding the premises, `## Goals` and `## Audience`, of the project whose checkout holds directory, as its config under home names it."""
|
||||
config = located(directory, home)
|
||||
return config if isinstance(config, Failure) else config.paths["goals-and-audience"]
|
||||
|
||||
+11
-10
@@ -3,14 +3,14 @@
|
||||
|
||||
usage: rank_todo.py [--check] [todo.md]
|
||||
|
||||
Reads todo.md (default: ./todo.md); the goals file, which its project's config sets as
|
||||
Reads todo.md (default: ./todo.md); the goals file, which todo.md's project config sets as
|
||||
goals-and-audience; and docs/decisions.md and docs/comprehension-history.md where they exist.
|
||||
Rewrites each item's Score from the formula and sorts the items by release; within a release,
|
||||
question rows come first, then security rows, then the rest, each group by score from highest,
|
||||
then by ID. Then it moves each blocker, an item another's Blocked by cell names, that ranks below
|
||||
an item it blocks to just above the first such item.
|
||||
Orders the details to match. Writes each item's R, and adds the Blocked by and R columns to a
|
||||
table without them. --check doesn't write anything and fails when the file is not ranked as it would be.
|
||||
an item it blocks to just above the first such item. Orders the details to match. Writes each
|
||||
item's R, and adds the Blocked by and R columns to a table without them. --check doesn't write
|
||||
anything and fails when the file is not ranked as it would be.
|
||||
|
||||
A valid todo.md; ## Details and each item's detail are optional:
|
||||
|
||||
@@ -39,8 +39,8 @@ A valid todo.md; ## Details and each item's detail are optional:
|
||||
- The formula holds numbers, + - * / ** and parentheses over L, E, I, R, S and V.
|
||||
- V sums, over the goals in an item's Moves cell, each goal's weight times how far the item moves it.
|
||||
The goals file marks each goal **Maintain:** or **Achieve:**. Code health, the achieve goal
|
||||
every project shares, ranks first; a Moves cell names it as in `code health by 6`. Its achieve
|
||||
goals follow in reading order, a sub-goal such as 1.1 included. Each weighs 0.97 times the one
|
||||
every project shares, ranks first; a Moves cell names it as in `code health by 6`. The goals
|
||||
file's achieve goals follow in reading order, a sub-goal such as 1.1 included. Each weighs 0.97 times the one
|
||||
before, and the weights sum to 1. With 5 achieve goals the first weighs 0.212 and the last 0.188.
|
||||
With 15 they weigh 0.082 and 0.053. The overall is the mean of the overall means of every scoring
|
||||
run on docs/comprehension-history.md's newest scored commit. While it is under 7.0, or the history
|
||||
@@ -81,10 +81,11 @@ A valid todo.md; ## Details and each item's detail are optional:
|
||||
reader. No item may sit under a marked release, and a line goes once a tag at or past its first
|
||||
release is cut. A release whose items all moved elsewhere without a line reads as finished.
|
||||
|
||||
Exit 1, saying why, on bad arguments, a checkout no config names, a file that cannot be read or
|
||||
written, a todo.md, goals file or docs/decisions.md that breaks the rules above, a scoring run on
|
||||
docs/comprehension-history.md's newest scored commit with no Mean row ending in its overall, an
|
||||
item that is not exempt scoring under the bar, or --check finding the file unranked.
|
||||
Exit 1, saying why, on bad arguments, a checkout no config names or a config that cannot be read, a
|
||||
file that cannot be read or written, a todo.md, goals file or docs/decisions.md that breaks the
|
||||
rules above, a scoring run on docs/comprehension-history.md's newest scored commit with no Mean row
|
||||
ending in its overall, an item that is not exempt scoring under the bar, or --check finding the file
|
||||
unranked.
|
||||
"""
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
@@ -43,22 +43,33 @@ def check_failures(todo: pathlib.Path) -> list[str]:
|
||||
|
||||
def read_all(todo: pathlib.Path) -> tuple[str, str, str, str] | Failure:
|
||||
"""todo.md, the goals file its project's config names, and its docs/decisions.md and docs/comprehension-history.md. The last two are empty when missing; a missing todo.md, config or goals file is a Failure."""
|
||||
goals = goals_file(str(todo.parent))
|
||||
plan = read(todo, "every project keeps a todo.md, and rank_todo.py --help shows one")
|
||||
if isinstance(plan, Failure):
|
||||
return plan
|
||||
# Looked up after todo.md is read, so a mistyped path fails naming todo.md, not in git's words.
|
||||
goals = goals_file(str(todo.parent), pathlib.Path.home())
|
||||
if isinstance(goals, Failure):
|
||||
return goals
|
||||
texts = []
|
||||
for path, missing in ((todo, "every project keeps a todo.md, and rank_todo.py --help shows one"), (goals, "rank_todo.py weighs goals from the file the project's config names as goals-and-audience"), (todo.parent / "docs" / "decisions.md", None), (todo.parent / HISTORY, None)):
|
||||
try:
|
||||
texts.append(path.read_text(encoding="utf-8"))
|
||||
except FileNotFoundError:
|
||||
if missing:
|
||||
return Failure(f"{path}: not found; {missing}")
|
||||
texts.append("")
|
||||
except OSError as failure:
|
||||
return Failure(f"{path}: {failure.strerror}")
|
||||
except UnicodeDecodeError:
|
||||
return Failure(f"{path}: not UTF-8")
|
||||
return texts[0], texts[1], texts[2], texts[3]
|
||||
goals_text, decisions, history = read(goals, "rank_todo.py weighs goals from the file the project's config names as goals-and-audience"), read(todo.parent / "docs" / "decisions.md", None), read(todo.parent / HISTORY, None)
|
||||
if isinstance(goals_text, Failure):
|
||||
return goals_text
|
||||
if isinstance(decisions, Failure):
|
||||
return decisions
|
||||
if isinstance(history, Failure):
|
||||
return history
|
||||
return plan, goals_text, decisions, history
|
||||
|
||||
|
||||
def read(path: pathlib.Path, missing: str | None) -> str | Failure:
|
||||
"""path's text; where it is missing, a Failure saying missing, or "" where missing is None."""
|
||||
try:
|
||||
return path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError:
|
||||
return Failure(f"{path}: not found; {missing}") if missing else ""
|
||||
except OSError as failure:
|
||||
return Failure(f"{path}: {failure.strerror}")
|
||||
except UnicodeDecodeError:
|
||||
return Failure(f"{path}: not UTF-8")
|
||||
|
||||
|
||||
def write(path: pathlib.Path, text: str) -> Failure | None:
|
||||
|
||||
+10
-10
@@ -49,17 +49,17 @@ class Scoring:
|
||||
class Ranked:
|
||||
text: str
|
||||
under_bar: list[str]
|
||||
# What ranking prints and --check does not: an old-shape table's call to reshape, a README that voids code health and marks no goal **Achieve:**, and each dropped blocker ID, since a mistyped ID drops the same way.
|
||||
# What ranking prints and --check does not: an old-shape table's call to reshape, a goals file that voids code health and marks no goal **Achieve:**, and each dropped blocker ID, since a mistyped ID drops the same way.
|
||||
notes: list[str]
|
||||
|
||||
|
||||
def rank(text: str, readme: str, decisions: str, history: str) -> Ranked | Failure:
|
||||
def rank(text: str, goals_text: str, decisions: str, history: str) -> Ranked | Failure:
|
||||
lines = text.split("\n")
|
||||
table = read_table(lines)
|
||||
if isinstance(table, Failure):
|
||||
return table
|
||||
shape = shape_of(table.columns)
|
||||
scoring = read_scoring(text, lines, readme, decisions, history, shape)
|
||||
scoring = read_scoring(text, lines, goals_text, decisions, history, shape)
|
||||
if isinstance(scoring, Failure):
|
||||
return scoring
|
||||
items = read_items(table, scoring)
|
||||
@@ -121,8 +121,8 @@ def under_the_bar(item: Item, value: float, bar: float, items: list[Item]) -> st
|
||||
return f"item {item.id} scores {value:.1f}, under the bar {bar:g}; delete it{blocked}, or mark it with a reason the bar does not apply"
|
||||
|
||||
|
||||
def read_scoring(text: str, lines: list[str], readme: str, decisions: str, history: str, shape: Shape) -> Scoring | Failure:
|
||||
goals = ranked_goals(readme, decisions)
|
||||
def read_scoring(text: str, lines: list[str], goals_text: str, decisions: str, history: str, shape: Shape) -> Scoring | Failure:
|
||||
goals = ranked_goals(goals_text, decisions)
|
||||
if isinstance(goals, Failure):
|
||||
return goals
|
||||
formula = parse_formula(stated(text, "Score"), variables(shape))
|
||||
@@ -140,12 +140,12 @@ def read_scoring(text: str, lines: list[str], readme: str, decisions: str, histo
|
||||
return Scoring(shape, formula, bar, int(next_id), dict(goals), weights)
|
||||
|
||||
|
||||
def ranked_goals(readme: str, decisions: str) -> list[tuple[str, Mark]] | Failure:
|
||||
def ranked_goals(goals_text: str, decisions: str) -> list[tuple[str, Mark]] | Failure:
|
||||
"""Every goal in rank order, each with its mark: code health first, unless the goals file voids it, then the goals file's goals in reading order. A **Void:** line other than `**Void:** code health, since <reason>.` fails, and so does a docs/decisions.md that opts out of the comprehension check where the goals file keeps code health, since the goal then has no signal."""
|
||||
goals = readme_goals(readme)
|
||||
goals = numbered_goals(goals_text)
|
||||
if isinstance(goals, Failure):
|
||||
return goals
|
||||
voids = VOID.findall("".join(sections(readme).get("Goals", [])))
|
||||
voids = VOID.findall("".join(sections(goals_text).get("Goals", [])))
|
||||
if other := [void for void in voids if not void.startswith(f"{CODE_HEALTH}, since ")]:
|
||||
if other[0].lower().startswith(CODE_HEALTH):
|
||||
return Failure(f"the goals file's ## Goals voids code health as `{other[0]}`; spell the void exactly `**Void:** {CODE_HEALTH}, since <reason>.`")
|
||||
@@ -248,9 +248,9 @@ def check_citations(items: list[Item], decisions: str) -> Failure | None:
|
||||
return None
|
||||
|
||||
|
||||
def readme_goals(readme: str) -> list[tuple[str, Mark]] | Failure:
|
||||
def numbered_goals(goals_text: str) -> list[tuple[str, Mark]] | Failure:
|
||||
"""The goals file's goals in reading order: each one's number, a sub-goal as 4.1, and its mark, Maintain, Achieve or None."""
|
||||
found = sections(readme).get("Goals", [])
|
||||
found = sections(goals_text).get("Goals", [])
|
||||
if len(found) != 1:
|
||||
return Failure(f"the goals file has ## Goals {len(found)} times; it needs it once")
|
||||
goals: list[tuple[str, Mark]] = []
|
||||
|
||||
@@ -121,12 +121,12 @@ class Rank(unittest.TestCase):
|
||||
|
||||
def test_weighs_each_achieve_goal_97_percent_of_the_one_above_and_no_maintain_goal(self):
|
||||
readme = "## Goals\n\n1. **Achieve:** A.\n 1. **Maintain:** B.\n 2. **Achieve:** C.\n2. **Achieve:** D.\n"
|
||||
self.assertEqual({goal: round(weight, 4) for goal, weight in ranking.goal_weights(ranking.readme_goals(readme), None).items()}, {"1": 0.3435, "1.2": 0.3332, "2": 0.3232})
|
||||
self.assertEqual({goal: round(weight, 4) for goal, weight in ranking.goal_weights(ranking.numbered_goals(readme), None).items()}, {"1": 0.3435, "1.2": 0.3332, "2": 0.3232})
|
||||
|
||||
def test_weighs_5_achieve_goals_from_0212_to_0188_and_15_from_0082_to_0053(self):
|
||||
for count, first, last in ((5, 0.212, 0.188), (15, 0.082, 0.053)):
|
||||
with self.subTest(count=count):
|
||||
goals = ranking.readme_goals("## Goals\n\n" + "".join(f"{number}. **Achieve:** G.\n" for number in range(1, count + 1)))
|
||||
goals = ranking.numbered_goals("## Goals\n\n" + "".join(f"{number}. **Achieve:** G.\n" for number in range(1, count + 1)))
|
||||
weights = list(ranking.goal_weights(goals, None).values())
|
||||
self.assertEqual((round(weights[0], 3), round(weights[-1], 3), round(sum(weights), 6)), (first, last, 1))
|
||||
|
||||
@@ -155,15 +155,15 @@ class Rank(unittest.TestCase):
|
||||
self.assertEqual(self.ranked(ranked.text.replace("| 2 | 0.1.0 | | |", "| 2 | 0.1.0 | | owner |")).under_bar, [])
|
||||
|
||||
def test_reads_goals_and_sub_goals_in_reading_order(self):
|
||||
self.assertEqual(ranking.readme_goals(README), [("1", "Achieve"), ("2", "Maintain"), ("2.1", "Achieve")])
|
||||
self.assertEqual(ranking.readme_goals(README.replace(" 1.", "\t1.")), [("1", "Achieve"), ("2", "Maintain"), ("2.1", "Achieve")])
|
||||
self.assertEqual(ranking.readme_goals(README.replace("2. **Maintain:** Correct.", "```\n## Goals\n```\n2. Correct.")), [("1", "Achieve"), ("2", None), ("2.1", "Achieve")])
|
||||
self.assertEqual(ranking.readme_goals(README.replace("2. **Maintain:**", "1. **Maintain:**")), Failure("the goals file's ## Goals numbers 1, 1, 1.1; number each goal by its position"))
|
||||
self.assertEqual(ranking.numbered_goals(README), [("1", "Achieve"), ("2", "Maintain"), ("2.1", "Achieve")])
|
||||
self.assertEqual(ranking.numbered_goals(README.replace(" 1.", "\t1.")), [("1", "Achieve"), ("2", "Maintain"), ("2.1", "Achieve")])
|
||||
self.assertEqual(ranking.numbered_goals(README.replace("2. **Maintain:** Correct.", "```\n## Goals\n```\n2. Correct.")), [("1", "Achieve"), ("2", None), ("2.1", "Achieve")])
|
||||
self.assertEqual(ranking.numbered_goals(README.replace("2. **Maintain:**", "1. **Maintain:**")), Failure("the goals file's ## Goals numbers 1, 1, 1.1; number each goal by its position"))
|
||||
self.assertEqual(ranking.parse_weights(["| Goal | W |", "|---|---|"], ["1"]), Failure("the goal weights list no goal, but the goals file's goals are 1; by position they weigh:\n| Goal | W |\n|---|---|\n| 1 | 1.00 |"))
|
||||
self.assertEqual(ranking.readme_goals("# P\n"), Failure("the goals file has ## Goals 0 times; it needs it once"))
|
||||
self.assertEqual(ranking.readme_goals("## Goals\n\nFast.\n"), Failure("the goals file's ## Goals has no numbered goal"))
|
||||
self.assertEqual(ranking.numbered_goals("# P\n"), Failure("the goals file has ## Goals 0 times; it needs it once"))
|
||||
self.assertEqual(ranking.numbered_goals("## Goals\n\nFast.\n"), Failure("the goals file's ## Goals has no numbered goal"))
|
||||
self.assertEqual(ranking.parse_weights(["| Goal | W |", "|---|---|", "| 1 | 0.5 |", "| 3 | 0.5 |"], ["1", "2"]), Failure("the goal weights list 1, 3, but the goals file's goals are 1, 2; by position they weigh:\n| Goal | W |\n|---|---|\n| 1 | 1.00 |\n| 2 | 0.50 |"))
|
||||
self.assertEqual(ranking.readme_goals("## Goals\n\n1. **Achieve:** A.\n 1. **Achieve:** B.\n\t2. **Achieve:** C.\n"), [("1", "Achieve"), ("1.1", "Achieve"), ("1.2", "Achieve")])
|
||||
self.assertEqual(ranking.numbered_goals("## Goals\n\n1. **Achieve:** A.\n 1. **Achieve:** B.\n\t2. **Achieve:** C.\n"), [("1", "Achieve"), ("1.1", "Achieve"), ("1.2", "Achieve")])
|
||||
|
||||
def test_refuses_a_goal_it_cannot_weigh(self):
|
||||
cases = {
|
||||
|
||||
@@ -6,7 +6,7 @@ usage: todo-item add --release <version> --l <L> --e <E> --i <I> --s <S> [--move
|
||||
todo-item show <ID>
|
||||
|
||||
All three work on the todo.md in the current folder. add and finish also use the CHANGELOG.md beside it,
|
||||
and read the goals file its project's config names and the docs/decisions.md beside todo.md.
|
||||
and read the docs/decisions.md beside it and the goals file that todo.md's project config names.
|
||||
The cells take the format rank_todo.py --help states; the letter flags take upper case too, as --L.
|
||||
|
||||
add takes Next ID as the item's ID and raises Next ID by one. It writes the row, with the tagline
|
||||
|
||||
+3
-3
@@ -35,7 +35,7 @@ Each project principle states one rule that the decisions below apply; their ent
|
||||
- routing: the worker changes rung only as a chunk starts, after every second unfinished verdict, when it names a chunk item that picks a higher rung, or when it cites an escalation trigger the judge finds
|
||||
- tests: each test earns its keep as the technical principles' test bullet says, and the worker takes a test's expected values from its item, never from the code
|
||||
- the repo may name bender, bender-infra and ai-sandbox
|
||||
- todo.md: one table of items, scored by the formula it states, held to its bar and ranked by `bin/rank_todo.py`, which ranks each blocker above the items it blocks; each achieve goal weighs by its position, code health first and four times as heavy while the comprehension panel's overall is under 7.0 or unmeasured, then README.md's goals, and a maintain goal weighs 0.5 per point of the confidence an item's test, gate leg or probe adds to it; the table's order is the plan's only order
|
||||
- todo.md: one table of items, scored by the formula it states, held to its bar and ranked by `bin/rank_todo.py`, which ranks each blocker above the items it blocks; each achieve goal weighs by its position, code health first and four times as heavy while the comprehension panel's overall is under 7.0 or unmeasured, then the goals file's goals, and a maintain goal weighs 0.5 per point of the confidence an item's test, gate leg or probe adds to it; the table's order is the plan's only order
|
||||
- todo.md: only a session or subagent on the project's rung or above scores or reshapes the plan; one below it drafts new items and changes to filed ones for a subagent at or above that rung to apply and score, and removes finished ones
|
||||
- worker: a chunk's worker writes the simplest change that solves its item, test first wherever a test can check the change, and reads a technical principle only where the item, a finding or its own record names it; larv-review's reviewers hold the change to the principles and to the standards in `skills/larv-review/standards/`
|
||||
- workflow-bench/: the remote may show the bench cases' names and commit SHAs
|
||||
@@ -108,7 +108,7 @@ Valid while `~/.claude` is a checkout of this repo.
|
||||
|
||||
2026-09-28, larv-review, declining a product-owner nit to exit at once when `claude` is missing. Serves goal 1. Claude Code's updater repoints `~/.local/bin/claude` at each new version under running loops, and a start that meets it mid-update fails with the same error as a broken install; the retry costs two minutes on the second and saves an unattended loop on the first. Valid while Claude Code updates by repointing its link.
|
||||
|
||||
## loop-claude: a judge decides each interrupt and each quoted answer from the transcript, the README's premises, `AGENTS.md`, `docs/decisions.md`, `~/.claude/CLAUDE.md`, `~/.claude/rules/` and larv-review's standards, and holds a chunk open while an answer the human typed in it is recorded nowhere; it interrupts for a reserved choice those don't settle, or an ask that carries no recommendation
|
||||
## loop-claude: a judge decides each interrupt and each quoted answer from the transcript, the premises, `AGENTS.md`, `docs/decisions.md`, `~/.claude/CLAUDE.md`, `~/.claude/rules/` and larv-review's standards, and holds a chunk open while an answer the human typed in it is recorded nowhere; it interrupts for a reserved choice those don't settle, or an ask that carries no recommendation
|
||||
|
||||
2026-09-21, lilleman. Serves goal 1. Asked to name the goal behind each choice, the worker cites a real goal for a choice that goal does not settle, and under-reports its own decisions; the judge's bar is whether the README's premises or the files in the title discriminate between the alternatives. Valid while the worker's self-report stays unreliable. Since 2026-11-08, below, that bar holds only for a reserved choice and an ask without a recommendation.
|
||||
|
||||
@@ -621,7 +621,7 @@ Valid while the worker writes the tests that larv-review's reviewers hold to the
|
||||
|
||||
2026-10-04, lilleman, answering the principles audit of PR #215. Serves goal 4, which keeps credentials, transcripts and memory files off the remote; the names of the host, its infrastructure project and the sandbox VM are none of those. Valid while the repo names them without their addresses or credentials.
|
||||
|
||||
## todo.md: one table of items, scored by the formula it states, held to its bar and ranked by `bin/rank_todo.py`, which ranks each blocker above the items it blocks; each achieve goal weighs by its position, code health first and four times as heavy while the comprehension panel's overall is under 7.0 or unmeasured, then README.md's goals, and a maintain goal weighs 0.5 per point of the confidence an item's test, gate leg or probe adds to it; the table's order is the plan's only order
|
||||
## todo.md: one table of items, scored by the formula it states, held to its bar and ranked by `bin/rank_todo.py`, which ranks each blocker above the items it blocks; each achieve goal weighs by its position, code health first and four times as heavy while the comprehension panel's overall is under 7.0 or unmeasured, then the goals file's goals, and a maintain goal weighs 0.5 per point of the confidence an item's test, gate leg or probe adds to it; the table's order is the plan's only order
|
||||
|
||||
2026-10-02, lilleman. Serves goal 1: findings were filed faster than chunks closed them, so the plan grew without telling what mattered. Each item is scored as `~/.claude/rules/todo-items.md` says; this repo's formula counts value highest and size least. The format is mandatory in every project, as the premises are (goal 1.3); the formula and bar are each project's own. A script ranks the table so a re-rank after each addition or edit is deterministic and cheap. Valid while one human sets the bar and reads the ranking, and while the items marked `principle` stay few enough to rank.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ paths:
|
||||
|
||||
# Goals and Audience
|
||||
|
||||
They live in the project's goals file (`~/.claude/CLAUDE.md` → "Vocabulary"): `README.md`, or a client project's `goals.md` in its config's folder.
|
||||
A project's goals and audience live in its goals file (`~/.claude/CLAUDE.md` → "Vocabulary"): `README.md`, or a client project's `goals.md` in its config's folder.
|
||||
|
||||
- **Goals** state the project's aim and intent, not promises: one numbered line each, like slogans, in priority order; a sub-goal is indented under its goal — "1. Blazing fast performance. 2. Correctness is more important than many features." A goal names no step of the work, such as a check to run first, and rarely an implementation. How a goal is met goes in a method section further down the goals file and names the goal it serves. BANNED as a goal: `Kill weak candidates early, with the cheap checks first.` GOOD: the goal `Keep research on a budget, so it scales in both time and cost.`, with the cheap checks moved to the method section, naming that goal.
|
||||
- **Mark each goal `**Maintain:**` or `**Achieve:**`.** A maintain goal is a state to hold: a change that breaks it is redesigned or dropped, whatever its value to other goals. An achieve goal is a direction to pursue, weighed against the other achieve goals. Under each achieve goal, three indented bullets: `Signal:` what moving towards it looks like, `Counter-signal:` what shows it is bought at a wrong price, and `Why:` one line on how the two connect. A goal with sub-goals states its own signal. A maintain goal has no bullets. BANNED: an unmarked goal; an achieve goal with no signal; `**Achieve:** Delivery speed.` with the signal `faster`. GOOD: `**Achieve:** Delivery speed.` with `Signal: hours from a chunk's start to its merge fall.`, `Counter-signal: 🔴 findings per chunk rise.` and `Why: speed bought by skipping review shows up as defects.`
|
||||
|
||||
+1
-1
@@ -21,7 +21,7 @@ paths:
|
||||
| 5 | — | — | irreversible: data lost for good, or a one-way door |
|
||||
|
||||
- **I's reach counts every project and session that runs the change.** A change to a hook or a tool that every session runs is I level 3, every consumer affected, even where one person uses it.
|
||||
- **Name in Moves each goal in the goals file's `## Goals` the item moves, and how far,** from 1 to 10, judged by its value to the audience in `## Audience`: `1.1 by 8, 1.2 by 5`. Where the item moves code health, the goal in `CLAUDE.md` → "Shared goal", name it as in `code health by 6`, unless the README voids it. Name the most specific goal it moves, and a parent goal only for a move none of its sub-goals covers, never beside its own sub-goal. Name a maintain goal only where the item adds a test, a gate leg or a probe that fails where the goal breaks, or is a `defect` that restores it. For the test, gate leg or probe, the move is how much more confident the project becomes that the goal holds, judged against the confidence it already has: 9 or 10 where nothing yet shows the goal holds, about 5 where the item guards something new in an area mostly covered, 1 where it adds another way to check what is already checked. After one such item lands, judge the next against the new confidence. For a defect, the move is how much of the goal is broken. A change that only claims the goal doesn't move it. `rank_todo.py --help` says how it weighs each goal.
|
||||
- **Name in Moves each goal in the goals file's `## Goals` the item moves, and how far,** from 1 to 10, judged by its value to the audience in `## Audience`: `1.1 by 8, 1.2 by 5`. Where the item moves code health, the goal in `CLAUDE.md` → "Shared goal", name it as in `code health by 6`, unless the goals file voids it. Name the most specific goal it moves, and a parent goal only for a move none of its sub-goals covers, never beside its own sub-goal. Name a maintain goal only where the item adds a test, a gate leg or a probe that fails where the goal breaks, or is a `defect` that restores it. For the test, gate leg or probe, the move is how much more confident the project becomes that the goal holds, judged against the confidence it already has: 9 or 10 where nothing yet shows the goal holds, about 5 where the item guards something new in an area mostly covered, 1 where it adds another way to check what is already checked. After one such item lands, judge the next against the new confidence. For a defect, the move is how much of the goal is broken. A change that only claims the goal doesn't move it. `rank_todo.py --help` says how it weighs each goal.
|
||||
- **A table with C, A, G and Goals in place of L, E, I and Moves is in the old shape.** Reshaping it is the project's only work until done, as `CLAUDE.md` → "Working loop" says, and `rank_todo.py` says how. Until then, A is the item's value to the audience and G its value to the goals it cites.
|
||||
- **Score L, E and I against the project's anchors, and check every score against finished chunks.** Guess the scores, then run `~/.claude/bin/score-outcomes <L> <E> <I> <S>` in the project. It lists the finished items guessed nearest and the project's anchors, the shipped items its `docs/score-anchors.md` names for each level, each with what its chunk measured; then the median outcome at each guessed R and S, and how well L, E and I each order the chunks by review rounds and 🔴 findings, L by the share of chunks with a 🔴 instead where most hold none. Compare the item with each anchor in turn: is it more or less likely to hold a defect, to let one escape, or to do damage? Give it the level whose anchors it matches; where a level has no anchor, judge by the level's words in the table above. Set S to the value guessed for the finished items whose outcome matches what you expect of this one. Keep your guess when no listed item resembles the one you are scoring.
|
||||
- **Anchor each level of L, E and I with two or three items the project shipped,** in `docs/score-anchors.md`: a table whose Level column names a level, such as `L 1`, and whose Items column lists the anchors' IDs, such as `12, 14`. Choose an L anchor whose chunk, as `score-outcomes` measured it, fits the level's words: its lines changed and the components it touched. Choose an E anchor by the level's words and the later fixes `score-outcomes` counts for its chunk, the defects that escaped it. Choose an I anchor by the level's words and what followed its release, since no tool measures I yet. Leave out a level no shipped item shows. Replace a level's anchors when the items scored at it turn out about ten times more likely, or ten times less likely, than its words say: to hold a defect, to let one escape, or to do damage.
|
||||
|
||||
@@ -75,7 +75,7 @@ Give a reviewer without Bash `git ls-files`'s output in its prompt, since some h
|
||||
|
||||
**Opt-out** — for the architect, the product-owner pass and the comprehension check:
|
||||
|
||||
- **Entry.** A project opts out only through a `docs/decisions.md` entry, held before the chunk began, whose title opens `review opt-out:` and names one or more of the three. The entry names the project goal that skipping them serves. A project that opts out of the comprehension check voids code health in its README, as `~/.claude/CLAUDE.md` → "Shared goal" says; `rank_todo.py` refuses its plan until it does.
|
||||
- **Entry.** A project opts out only through a `docs/decisions.md` entry, held before the chunk began, whose title opens `review opt-out:` and names one or more of the three. The entry names the project goal that skipping them serves. A project that opts out of the comprehension check voids code health in its goals file, as `~/.claude/CLAUDE.md` → "Shared goal" says; `rank_todo.py` refuses its plan until it does.
|
||||
- **Skip.** Every chunk skips each reviewer the entry names, even where its trigger, a 🔀 or a due scoring run calls for it, and the chunk's summary cites the entry. Unlike a suspension, an opt-out needs no plan item to end it.
|
||||
- **Malformed.** A title naming another reviewer, or an entry naming no goal, is a finding of the pass the chunk runs, and the chunk runs every pass its trigger fires.
|
||||
|
||||
@@ -134,7 +134,7 @@ The release review is a release's last chunk, run once the release holds no item
|
||||
|
||||
### Architecture
|
||||
|
||||
Launch systems-architect: review the whole project as the release leaves it; the decision log carries previously accepted decisions to honor, the goals file the goals and audience, the README the Stage 0 artifacts. Give it the places per "Comprehension check" → "Places". Each of its other findings names a concrete later change the PR makes harder, such as "a second forge needs edits in four modules"; drop one that names none.
|
||||
Launch systems-architect: review the whole project as the release leaves it; the decision log carries previously accepted decisions to honor, the goals file the goals and audience, and the README the other Stage 0 artifacts. Give it the places per "Comprehension check" → "Places". Each of its other findings names a concrete later change the PR makes harder, such as "a second forge needs edits in four modules"; drop one that names none.
|
||||
|
||||
- **Stage 0 questions returned**: relay them to the user, record the goals and audience in the goals file and the rest in the README, re-run once.
|
||||
- **BLOCK** standing after any relaunch ("Rung up"): confirm with the user first when the restructure materially reshapes their PR; until confirmed, no later pass runs.
|
||||
|
||||
Reference in New Issue
Block a user