Merge remote-tracking branch 'origin/main' into item-311
CI / tests (pull_request) Successful in 4m1s
CI / tests (pull_request) Successful in 4m1s
# Conflicts: # docs/reading-cost.md # todo.md
This commit is contained in:
@@ -1,6 +1,12 @@
|
||||
name: CI
|
||||
on: pull_request
|
||||
|
||||
# A push to a pull request cancels that pull request's older runs; the merge gate reads only the head commit's status.
|
||||
# The group is the ref: refs/pull/<number>/head for a pull request, the branch for a push, so no two refs cancel each other's runs.
|
||||
concurrency:
|
||||
group: ci-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
runs-on: docker-host
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
!.gitignore
|
||||
!CHANGELOG.md
|
||||
!CLAUDE.md
|
||||
!LICENSE.md
|
||||
!README.md
|
||||
!ci.sh
|
||||
!settings.json
|
||||
@@ -53,6 +54,10 @@
|
||||
!bin/deploy-release
|
||||
!bin/deploy_release/
|
||||
!bin/deploy_release/*.py
|
||||
!bin/do_epic/
|
||||
!bin/do_epic/*.py
|
||||
!bin/do_epic/backend/
|
||||
!bin/do_epic/backend/*.py
|
||||
!bin/failure.py
|
||||
!bin/forge/
|
||||
!bin/forge/*.py
|
||||
@@ -74,6 +79,9 @@
|
||||
!bin/loop_state/*.py
|
||||
!bin/machine_load/
|
||||
!bin/machine_load/*.py
|
||||
!bin/markdown-section
|
||||
!bin/markdown_section/
|
||||
!bin/markdown_section/*.py
|
||||
!bin/metric-bench
|
||||
!bin/metric_bench/
|
||||
!bin/metric_bench/*.py
|
||||
|
||||
@@ -2,6 +2,14 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- judge-replay <model> --offline replays every archived verdict through a "?" rule, a decision table and a classifier trained on the archive, with no judge call, and counts the interrupts each misses.
|
||||
- do-epic finds an epic repo's checkout and project config by the repo's name, and clones a missing checkout from the host and owner of the epic's other repos, asking the human where they differ or the clone fails.
|
||||
- do-epic reads its machine-wide settings from ~/.claude-projects/do-epic/config.toml and its repo rule from repo-rule.md beside it, and refuses to start while either is missing or misspelt.
|
||||
- `project-record save` now writes a goals file kept as a Jira epic's description, except a change to `## Goals` or `## Audience`, which stays an edit in Jira.
|
||||
- Add do-epic's epic backend: one interface that reads a Jira epic's child issues and their sub-tasks, with labels, assignee, status and parent, and files new ones, met by a Jira backend and an in-memory fake that pass the same contract tests.
|
||||
- Fix for an outside reviewer's comment on a PR now ends with a change to the setup that would have caught it, in claude-config first, then the principles and standards, then the project.
|
||||
- larv-review reruns the pass that raised a 🟡 when the fix removes or loosens an assertion, changes an expected value, or skips, deletes or guards a test. Other 🟡 fixes still rerun no pass.
|
||||
- Add `markdown-section <file> <fragment>`, which prints the section of a markdown file under the heading the fragment picks, to the next heading of its level or a higher one. It fails, listing the headings, where several hold the fragment.
|
||||
- loop-claude and subagent-rung run an L 3 task one rung above the project's rung and an L 4 task two above it: on an Opus 5.5 project, Fable 5.2 at low and at medium.
|
||||
- Add `larv-review round-title <pass>`, which prints a review round commit's title prefix with the 🔴, 🟡 and misread counts taken from the round's report. rule-hook refuses a round commit whose counts differ from that report, where the report exists and can be counted; a report must head each finding `### 🔴` or `### 🟡`, optionally followed by ` [misread]`.
|
||||
- Add `workflow-bench/handoff_bench.py`, a bench of how a worker's step to a stronger model hands over the chunk, on six archived L 3 chunks. Sonnet 5.5 left five unfinished at its budget. On those, Opus 5.5 resuming Sonnet 5.5's session passed two and was partial on three, for a mean $0.68; a fresh session with a brief passed one and was partial on four, for $0.78. So a stepped worker still resumes its session.
|
||||
|
||||
@@ -13,11 +13,17 @@ How work proceeds in any repo, unless the repo's `AGENTS.md` says otherwise. The
|
||||
- **Open every chunk with one line:** the last version cut, how many changes are unreleased (the entries under `CHANGELOG.md` → `## Unreleased`, or else the commits since the last tag), and the release this chunk ships into. A repo that doesn't publish anything says so once.
|
||||
- **A client project's `CLAUDE.md` and `AGENTS.md` in its repo belong to its team.** Read both at the start of each chunk there. Propose each change that aligns them with this setup in a pull request of its own, never inside a chunk's; the one edit a chunk makes there is a decision's title in `AGENTS.md`. Until the team accepts a proposal, and after it declines or doesn't answer, follow the team's rule unless it breaks one of the project's goals. Record each such conflict as a decision in the `decisions` record, as `rules/decisions.md` asks: which rule you follow, why, and the open proposal's PR. Then nobody is asked twice.
|
||||
- **Read the goals file's `## Goals` and `## Audience` before starting a chunk, unless your prompt quotes them.** The chunk's items set its scope. The goals choose between ways to deliver those items, and never add to them.
|
||||
- **Read the plan and the decisions only as far as the chunk needs.** Where your prompt quotes `chunk-preflight`'s facts, they hold the next item's L, E + I, tagline and detail, and each item that may join it with its L, E + I, S, tagline and detail; run `todo-item` only for what they leave out. Otherwise run `~/.claude/bin/todo-item list`, which prints every item's row in the plan's order, without details. Then read the details of only the items the chunk takes, with `~/.claude/bin/todo-item show <ID>`, which prints an item's row and detail. `AGENTS.md` lists the project principles and the title of each decision no project principle covers. Read an entry in `docs/decisions.md` only where the project principle or the title leaves open what the chunk or the review needs.
|
||||
- **Read the plan and the decisions only as far as the chunk needs.** Where your prompt quotes `chunk-preflight`'s facts, they hold the next item's L, E + I, tagline and detail, and each item that may join it with its L, E + I, S, tagline and detail; run `todo-item` only for what they leave out. Otherwise run `~/.claude/bin/todo-item list`, which prints every item's row in the plan's order, without details. Then read the details of only the items the chunk takes, with `~/.claude/bin/todo-item show <ID>`, which prints an item's row and detail. `AGENTS.md` lists the project principles and the title of each decision no project principle covers. Read an entry in `docs/decisions.md` only where the project principle or the title leaves open what the chunk or the review needs. Print one entry with `~/.claude/bin/markdown-section <file> <fragment of its title>`, where `<file>` is what `~/.claude/bin/project-record open decisions` prints.
|
||||
- **Work one chunk at a time, and put every item that can share its review into it.** Every review has a fixed cost, so a chunk takes the next plan item plus every other item for which three things hold: they name one release; they fire the same review passes (`larv-review` §3); and no item outside the chunk blocks one of them. The items need not share a purpose. An item whose risk R, in the plan, is above 5 ships alone: it joins no chunk, and none joins it. The next plan item is the first that does not wait: an item waits while it is marked `question`, until the human answers, or while an item that blocks it is still in the plan. Tiny items, a few lines of change each, need not fire the same passes: bundle up to ten that name one release, and that no item outside the chunk blocks, into one chunk. Split a milestone into items in the plan before starting it. Fix a finding in the chunk's own change within the chunk. File a finding about anything else in `todo.md`.
|
||||
- **Ask, don't guess.** Three choices are always the human's: a goal; a release, meaning the items planned into it and when it is cut; and a change to loop-claude's routing ladder, its rungs included. Any other rule that asks the human's approval reserves that choice too. Ask about a reserved choice. For any other choice, first check whether the goals already answer it. If they nearly do, propose a change to the goals and show how it answers this case: an answer settles one question, a goal settles every question like it. Otherwise take the option you would recommend, and record it with its reason: in the item's detail, or, where it governs later work, as the next sentence says. A recommendation you can give a reason for is not a guess. Ask only where you have no recommendation. Write every answer once, where it belongs, never only in chat: a goal in the goals file, a working rule in `AGENTS.md`, a standing choice in a comment at the one place it governs, or in `docs/decisions.md` once made in more than one place. When an answer holds a goal and a choice, write the goal in the goals file, and record as a choice only what the goal leaves open.
|
||||
- **Bench a syntax with the people who write it.** Before a new or changed spelling of something users read or write ships — markdown, HTML, a config key, a CLI flag — a panel of fresh-context readers, seated as those users from the goals file's `## Audience`, picks it.
|
||||
- **Implement the simplest change that solves the item, and ship it with `/larv-review`.** Whenever a test can check the change, write that test first and see it fail. Take its expected values from the item, never from the code. larv-review's reviewers hold the change to the technical principles, and a finding against one names it. Read a file in the principles folder the project's config sets (`~/.claude/bin/project-config principles` prints it) only when the item changes it, a finding or a record you write names it, or you decline a finding.
|
||||
- **Change the setup for each reasonable fix an outside reviewer asked for on a PR.** An outside reviewer is a forge's auto-reviewer or a human, not one of larv-review's reviewer agents. When you fix a comment of theirs, ask why our review didn't catch the fault first, and change the setup so it would; a fix that only answers the comment leaves the same gap open for the next chunk. Put the change in the first place that fits:
|
||||
1. claude-config's core: the reviewers' agents, `larv-review`, the hooks and tools, so every project gains.
|
||||
2. claude-config's technical principles or review standards, where a rule was missing or too vague.
|
||||
3. The project's own settings, principles folder or `AGENTS.md`, only where the gap is that project's alone. A client repo's `AGENTS.md` changes by a proposal of its own, per "A client project's `CLAUDE.md` and `AGENTS.md`" above.
|
||||
|
||||
In claude-config, the change is a fix of the chunk. In another project, places 1 and 2 are an item in claude-config's plan: start a worktree of claude-config off `origin/main`, run `todo-item add` in it, commit the plan file, and ship it as a pull request of its own with `chunk-pr`, since claude-config's default branch may not take a direct push. Describe the gap without the client's name, code or comment text. File a change too big for the chunk as its own item. Where no place holds the gap, say so in the chunk's summary rather than inventing a change.
|
||||
- **Remove a finished item from the plan, and revise any decision that cites it.** If the repo publishes something others consume — a package, an image, an API — reword the item for them and move it to `CHANGELOG.md` under `## Unreleased`. A changelog is for the public, never for the next agent. Otherwise delete it; git history is the record. `~/.claude/bin/todo-item finish <ID>` removes the item with its detail, and adds the bullet you pass with `--changelog <bullet>`; it refuses while a decision cites the item. A plan kept in Jira takes no bullet, since the issues' history is its changelog: `finish` there refuses `--changelog`. A chunk finishes its items as it merges, through `chunk-pr merge --finish <ID>`, which takes the same flags (`larv-review` §5).
|
||||
- **Releases are the human's.** The version bump and the publishing credentials are theirs, whatever another rule says. Prepare everything else, so cutting a release is one decision.
|
||||
- **Open each plan item or ticket with a bold one-sentence tagline**, so the taglines alone say what a release holds. The tagline states the target state, checkable when done: an imperative verb plus what holds afterwards. A present-tense fact reads as the current state and doesn't say anything about what to do. Add "today X" only where the direction is otherwise ambiguous. GOOD: `**Accept undefined in MetadataValue and drop such keys on output.**` BANNED: `MetadataValue accepts undefined; such keys are dropped on output`.
|
||||
@@ -128,7 +134,7 @@ Claude Code reads this section by its heading when it compacts a conversation. K
|
||||
- **chunk** — the unit of work, shipped as one PR; "Working loop" says what it holds.
|
||||
- **cycle** — in `loop-claude`, worker runs and the judge's verdict on each, until the chunk closes or is left for the next cycle. In between, the human answers any interrupt, or the worker resumes with an answer the judge quoted from the premises or the judge's records.
|
||||
- **gate** — the merge gate: the checks a PR passes before it merges.
|
||||
- **goals file** — the file holding a project's `## Goals` and `## Audience`: the one its config names as `goals-and-audience`. Read it in the file `~/.claude/bin/project-record open goals-and-audience` prints. It is `README.md`, unless a client's project keeps it in the config's folder, or in a Jira epic's description, which the loop only reads.
|
||||
- **goals file** — the file holding a project's `## Goals` and `## Audience`: the one its config names as `goals-and-audience`. Read it in the file `~/.claude/bin/project-record open goals-and-audience` prints. It is `README.md`, unless a client's project keeps it in the config's folder, or in a Jira epic's description, whose `## Goals` and `## Audience` the loop only reads.
|
||||
- **interrupt** — `loop-claude` asking the human, on its terminal, for a decision that is theirs. They type a one-line answer there, or `/talk` to open the worker's session there with `claude --resume`. A release ready to cut opens that session at once. A routing call opens claude on that terminal too, in `~/code/claude-config`, the checkout where this setup is developed.
|
||||
- **judge** — the headless claude that reads a worker's transcript each cycle and decides whether to interrupt and whether the chunk is finished.
|
||||
- **judge's records** — the files the judge reads beside the premises: the project's `AGENTS.md` and `docs/decisions.md`, then the human's `~/.claude/CLAUDE.md` and the `.md` files in `~/.claude/rules/`, then those in the project's review standards.
|
||||
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
# MIT License
|
||||
|
||||
Copyright (c) 2026 Larv IT AB
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -134,7 +134,7 @@ The loop reads such a plan and files items into it, each as a Task under the epi
|
||||
|
||||
`todo-item edit` and `move` change only an issue the loop filed. Of an issue a person filed, `edit` sets only the scores, and posts its `--comment` on the issue to tell the team why; the summary, description and Fix Version stay the team's to change in Jira, as do its links. On an issue the loop filed, `edit --blocked-by` makes and deletes the links to the epic's open items. For `todo-item finish`, the config names the workflow transition that finishes an item and the status it leads to, both as Jira shows them: `jira-finish-transition = "Mark as done"` and `jira-finish-status = "Done"`. Finish then moves the item's issue by that transition. It refuses a transition that leads to another status, or to one Jira doesn't count as done, and fails where the config sets neither. The four things not supported for a plan outside the checkout, above, are not supported for a plan in Jira either.
|
||||
|
||||
A client that keeps its goals and audience in an epic's description names that epic as the goals file: `goals-and-audience = "jira:https://acme.atlassian.net/browse/ABC-12"`, which may be the plan's own epic. Such a config keeps its plan in Jira too, and reads both with the same `env-file`. The description needs `## Goals` and `## Audience` as headings, level-2 headings in Jira's editor. The loop only reads the description, so edit it in Jira; `project-record open goals-and-audience` prints a file holding its text. Whoever may edit that description sets the premises every worker and judge follows, so keep its edit rights to the people who own the project's goals.
|
||||
A client that keeps its goals and audience in an epic's description names that epic as the goals file: `goals-and-audience = "jira:https://acme.atlassian.net/browse/ABC-12"`, which may be the plan's own epic. Such a config keeps its plan in Jira too, and reads both with the same `env-file`. The description needs `## Goals` and `## Audience` as headings, level-2 headings in Jira's editor. `project-record open goals-and-audience` prints a file holding the description's text, and `project-record save` writes it back, except a change to `## Goals` or `## Audience`: edit those two in Jira. A save rewrites the whole description from markdown, which keeps no mention, macro or block id that markdown can't hold. Whoever may edit that description sets the premises every worker and judge follows, so keep its edit rights to the people who own the project's goals.
|
||||
|
||||
To name a chunk's branch, worktree, commits and pull request by the issue key, as in `ABC-123` and `ABC-123 - …`, set these three templates, with your Jira project's key in place of `ABC`: `branch = "ABC-{id}"`, `commit-title = "ABC-{id} - {summary}"` and `pull-request-title = "ABC-{id} - {summary}"`. `project-config --help` says how the templates work.
|
||||
|
||||
@@ -174,16 +174,17 @@ It renames `## Unreleased` in `CHANGELOG.md` to the version and merges that thro
|
||||
|
||||
## Tools
|
||||
|
||||
Install puts only `loop-claude` on `PATH`; call the others by their path, such as `~/.claude/bin/metric-bench`, from the project's checkout. Those under `~/.claude` are the release's; test a change to one from `~/code/claude-config/bin`.
|
||||
Install puts only `loop-claude` on `PATH`; call the others by their path, such as `~/.claude/bin/metric-bench`, from the project's checkout. Those under `~/.claude` are the release's; test a change to one from `~/code/claude-config/bin`. The tools are listed in alphabetical order, and the packages under `bin/` follow under their own heading.
|
||||
|
||||
- `chunk-pr open|sync|ready|merge|clean` opens a chunk's pull request as a draft, brings its branch level with origin's default branch, marks it ready and merges it once its checks pass, with `gh` on a GitHub origin and `tea` elsewhere. The chunks of one project merge one at a time: in its turn, `merge` brings the branch level again, finishes the plan items its `--finish` flags name, and waits for the checks; once the pull request is merged, `clean` removes the chunk's worktree and branch and fast-forwards the main checkout. `chunk-pr --help` states the steps.
|
||||
- `chunk-preflight` finds the mechanical facts a chunk starts from, and with `--worktree` makes its worktree; `loop-claude` runs it with `--worktree` before it routes each chunk, and `chunk-preflight --help` lists the facts.
|
||||
- `command-patterns [<version>] [--calls <file>]` prints a table of the shell command patterns a release's archived chunks ran, such as `tea pr create`, for an audit of the release, and with `--calls <file>` saves each call as a JSON line; `command-patterns --help` states the columns and the line's keys.
|
||||
- `compress-archives` gzips each transcript in loop-claude's archive and each file in the review folders 30 days after it last changed, keeping all of them; Install's crontab line runs it daily.
|
||||
- `deploy-release [--cut] [<checkout>]` moves `~/.claude`, or the checkout named, to the newest release, as Update says; with `--cut`, run in the development clone, it first cuts the next release, as Release says.
|
||||
- `judge-replay <model>` replays loop-claude's judge on archived verdicts of the projects run on the model, and prints what each rung a review climbs to misses, and the lowest rung below them that, like each candidate above it, returns them; `judge-replay --help` states the replay.
|
||||
- `judge-replay <model> [--offline]` replays loop-claude's judge on archived verdicts of the projects run on the model. It prints what each rung a review climbs to misses, and the lowest rung below those that returns every verdict, as each candidate above it does. With `--offline` it doesn't call a judge: it counts what a "?" rule, a decision table and a classifier trained on the archive decide of each verdict's kind from the worker's final message, and how many interrupts each misses. `judge-replay --help` states both replays.
|
||||
- `larv-review <command>` holds larv-review's mechanical rules: the chunk's review folder, the paired ruling's base and vote count, each pass's round count and the opening of its round commit's title, the scoring run's trigger and setup; the comprehension history's format, its latest places and its count of how often seats agree; and each reviewer's prompt header. `larv-review --help` states them. `settings.json` → `hooks` runs `larv-review reviewer-context` as each subagent starts. A reviewer under `agents/` whose frontmatter has `omitClaudeMd: true` starts without `CLAUDE.md`, and the command hands it that file's "Shared goal", "Prose" and "Vocabulary" sections.
|
||||
- `loop-claude <task>` runs a project's next chunks, each in its own tmux window, cycle after cycle; `loop-claude --help` states its contract.
|
||||
- `markdown-section <file> <fragment>` prints the section of a markdown file under the first heading that holds the fragment, to the next heading of its level or a higher one, instead of a `grep -n` and a guessed `sed -n` range; `markdown-section --help` says what it does where several headings match.
|
||||
- `metric-bench [history]` benches cheap static metrics of a project's Python against the places its comprehension history names as hardest; `metric-bench --help` states the metrics and the scores.
|
||||
- `mutate-changes [--base <ref>] [--prefix <shell text>] [--jobs <n>]` mutation-tests the lines a branch changes and prints each mutant its tests don't kill; `mutate-changes --help` says which tool it runs for each language.
|
||||
- `notify [<message>]` sends a message, or else stdin, as a DM from the bot of the project's `notifier`, Mattermost or Slack, to the human; the project's `env-file` holds the bot's token, the recipient and, for Mattermost, the server, and `notify --help` names its keys. `loop-claude --help` says when loop-claude sends one.
|
||||
@@ -203,6 +204,8 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `touch-predictions record <ID>...` appends what a chunk's items were predicted to touch, under `Touches:` in their details, beside the files and parts its branch touched, to the record the config names as `touch-predictions`; `touch-predictions --help` shows the spelling and how a part is named. `chunk-preflight` reads the same predictions to name the items that may start chunks at once.
|
||||
- `wait-for <file> [<pattern>] [--timeout <seconds>]` waits until a file a background job writes holds a line matching the pattern, and prints it.
|
||||
|
||||
### Packages under `bin/`
|
||||
|
||||
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests. Beside the packages, `bin/`'s root holds only the launchers, the tools of one file with their tests, `test_env.py`, `failure.py` and `setup.cfg`. `failure.py`'s `Failure`, which modules across `bin/` return for a failed step, belongs to no single package. `atlassian/`, `branch_chunks/`, `claude_code/`, `command_lines/`, `forge/`, `git_cli/`, `loop_effects/`, `loop_state/`, `machine_load/`, `model_routing/`, `notifier/`, `plan/`, `plan_store/`, `record_store/`, `release_plan/`, `review_rules/`, `rung_replay/`, `text_files/` and `transcript_archive/` serve more than one tool. No package imports one that imports it back, which `repo_checks/test_package_cycles.py` checks.
|
||||
|
||||
- `adf_codec/` converts markdown to ADF, the rich text of Jira Cloud and Confluence, and back, in a throwaway container running a pinned `@larvit/adf-codec`. It builds the container's image inside a heavy slot.
|
||||
@@ -210,7 +213,8 @@ Install puts only `loop-claude` on `PATH`; call the others by their path, such a
|
||||
- `branch_chunks/` cuts a default branch into chunks and finds, for each, the items it shipped and the later chunks that fixed its code, for `score_outcomes` and `routing_figures`. It imports `reading_cost`'s record path.
|
||||
- `command_lines/` reads a shell command: its simple commands, and a docker call's subcommand and memory limits, for `command_patterns`, `reading_cost`, `routing_figures`, `rule_hook` and `run_heavy`.
|
||||
- `deploy_release` imports `release_plan`'s `release_standing`.
|
||||
- `forge/` tells a GitHub origin from a Gitea one and runs that forge's CLI (`commands.py`), and finds a pull request and waits for its checks (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release` and `larv_review`.
|
||||
- `do_epic/` will hold the tool that drives one Jira epic. So far `config.py` reads its machine-wide settings, importing `project_config`'s settings and `model_routing`'s ladder file, and `epic_record.py` reads and writes an epic's description by section and language line, through `record_store`'s `EpicDescription`. `repos.py` finds a repo's checkout and project config by its name, and clones a missing checkout from the host and owner of the epic's other repos, through `forge`'s `url_beside`.
|
||||
- `forge/` tells a GitHub origin from a Gitea one, names a repo beside an origin's and runs that forge's CLI (`commands.py`), and finds a pull request and waits for its checks (`pulls.py`), for `chunk_preflight`, `chunk_pr`, `deploy_release`, `do_epic` and `larv_review`.
|
||||
- `git_cli/` runs git for every tool (`git_repo.py`), and reads the git calls in a shell command (`git_command.py`).
|
||||
- `judge_replay` imports `loop_claude`'s judge, verdicts, premises and records.
|
||||
- `loop_claude` runs `chunk-preflight` as a process of its own.
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
"""What every backend of an epic's plan must hold, as tests. A backend's test case inherits EpicStoreContract and unittest.TestCase, and builds an epic with no issues in new_store. Its assign and close change an issue as a person would in the tracker."""
|
||||
import re
|
||||
import typing
|
||||
|
||||
from do_epic.backend.protocol import EpicIssue, EpicStore
|
||||
from failure import Failure
|
||||
|
||||
|
||||
class EpicStoreContract:
|
||||
assertEqual: typing.Callable[..., None]
|
||||
assertIsInstance: typing.Callable[..., None]
|
||||
subTest: typing.Callable[..., typing.ContextManager[None]]
|
||||
|
||||
def new_store(self) -> EpicStore:
|
||||
raise NotImplementedError
|
||||
|
||||
def assign(self, store: EpicStore, key: str, name: str) -> None:
|
||||
"""Assigns the issue named key to the person whose display name is name."""
|
||||
raise NotImplementedError
|
||||
|
||||
def close(self, store: EpicStore, key: str, status: str) -> None:
|
||||
"""Moves the issue named key to the done status named status."""
|
||||
raise NotImplementedError
|
||||
|
||||
def filed(self, store: EpicStore, parent: str, summary: str, detail: str = "", labels: tuple[str, ...] = ()) -> str:
|
||||
key = store.file(parent, summary, detail, labels)
|
||||
assert isinstance(key, str), key
|
||||
return key
|
||||
|
||||
def issues(self, store: EpicStore) -> list[EpicIssue]:
|
||||
read = store.read()
|
||||
assert isinstance(read, list), read
|
||||
return read
|
||||
|
||||
def test_an_epic_with_no_issues_reads_empty(self) -> None:
|
||||
self.assertEqual(self.new_store().read(), [])
|
||||
|
||||
def test_reads_back_a_child_issue_it_filed(self) -> None:
|
||||
store = self.new_store()
|
||||
key = self.filed(store, store.epic_key, "Count the runs.", "Why it counts.\n\nA second paragraph.", ("security", "api"))
|
||||
self.assertEqual(self.issues(store), [EpicIssue(key, "Count the runs.", "Why it counts.\n\nA second paragraph.", ("api", "security"), None, "To Do", False, store.epic_key)])
|
||||
|
||||
def test_reads_back_a_sub_task_it_filed_after_its_parent(self) -> None:
|
||||
store = self.new_store()
|
||||
first = self.filed(store, store.epic_key, "Count the runs.")
|
||||
second = self.filed(store, store.epic_key, "Ship the later thing.")
|
||||
task = self.filed(store, first, "Count them in the api.", labels=("api",))
|
||||
read = self.issues(store)
|
||||
self.assertEqual([(issue.key, issue.parent) for issue in read], [(first, store.epic_key), (task, first), (second, store.epic_key)])
|
||||
self.assertEqual((read[1].summary, read[1].labels, read[1].detail), ("Count them in the api.", ("api",), ""))
|
||||
|
||||
def test_files_each_issue_under_its_own_key_in_the_epics_project(self) -> None:
|
||||
store = self.new_store()
|
||||
child = self.filed(store, store.epic_key, "Count the runs.")
|
||||
keys = [child, self.filed(store, store.epic_key, "Count the runs."), self.filed(store, child, "Count them in the api.")]
|
||||
self.assertEqual(len(set(keys)), 3)
|
||||
project = store.epic_key.rpartition("-")[0]
|
||||
self.assertEqual([re.fullmatch(f"{project}-[1-9][0-9]*", key) is not None for key in keys], [True] * 3)
|
||||
|
||||
def test_reads_the_assignee_and_a_done_status(self) -> None:
|
||||
store = self.new_store()
|
||||
first = self.filed(store, store.epic_key, "Count the runs.")
|
||||
second = self.filed(store, store.epic_key, "Ship the later thing.")
|
||||
self.assign(store, first, "Ada Lovelace")
|
||||
self.close(store, second, "Done")
|
||||
read = {issue.key: issue for issue in self.issues(store)}
|
||||
self.assertEqual((read[first].assignee, read[first].status, read[first].done), ("Ada Lovelace", "To Do", False))
|
||||
self.assertEqual((read[second].assignee, read[second].status, read[second].done), (None, "Done", True))
|
||||
|
||||
def test_refuses_to_file_under_a_sub_task(self) -> None:
|
||||
store = self.new_store()
|
||||
child = self.filed(store, store.epic_key, "Count the runs.")
|
||||
task = self.filed(store, child, "Count them in the api.")
|
||||
self.assertEqual(store.file(task, "Count more.", "", ()), Failure(f"{task} is neither the epic {store.epic_key} nor one of its child issues, so nothing can be filed under it"))
|
||||
self.assertEqual(len(self.issues(store)), 2)
|
||||
|
||||
def test_refuses_to_file_under_an_issue_outside_the_epic(self) -> None:
|
||||
store = self.new_store()
|
||||
for parent in ("ZZZ-1", "not a key"):
|
||||
with self.subTest(parent=parent):
|
||||
self.assertIsInstance(store.file(parent, "Count more.", "", ()), Failure)
|
||||
self.assertEqual(self.issues(store), [])
|
||||
|
||||
def test_refuses_a_label_with_white_space_and_a_summary_of_two_lines(self) -> None:
|
||||
store = self.new_store()
|
||||
self.assertEqual(store.file(store.epic_key, "Count the runs.", "", ("two words",)), Failure("the label 'two words' is empty or holds white space"))
|
||||
self.assertEqual(store.file(store.epic_key, "Count\nthe runs.", "", ()), Failure("the summary must be one line of text"))
|
||||
self.assertEqual(self.issues(store), [])
|
||||
@@ -0,0 +1,33 @@
|
||||
"""An epic held in memory, for do-epic's tests, held to contract.py like every backend."""
|
||||
import dataclasses
|
||||
|
||||
from do_epic.backend.protocol import EpicIssue, no_parent, unfit
|
||||
from failure import Failure
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class MemoryEpic:
|
||||
"""An epic whose issues rank in the order filed. Where fails is set, every read and filing fails with it as the reason."""
|
||||
epic_key: str
|
||||
issues: list[EpicIssue] = dataclasses.field(default_factory=list)
|
||||
fails: str | None = None
|
||||
|
||||
def read(self) -> list[EpicIssue] | Failure:
|
||||
if self.fails:
|
||||
return Failure(self.fails)
|
||||
return [issue for child in self.issues if child.parent == self.epic_key for issue in (child, *(task for task in self.issues if task.parent == child.key))]
|
||||
|
||||
def file(self, parent: str, summary: str, detail: str, labels: tuple[str, ...]) -> str | Failure:
|
||||
if self.fails:
|
||||
return Failure(self.fails)
|
||||
if refused := unfit(summary, labels):
|
||||
return refused
|
||||
if parent != self.epic_key and parent not in {issue.key for issue in self.issues if issue.parent == self.epic_key}:
|
||||
return no_parent(self.epic_key, parent)
|
||||
key = f"{self.epic_key.rpartition('-')[0]}-{len(self.issues) + 100}"
|
||||
self.issues.append(EpicIssue(key, summary, detail.strip(), tuple(sorted(labels)), None, "To Do", False, parent))
|
||||
return key
|
||||
|
||||
def change(self, key: str, **fields: object) -> None:
|
||||
"""Changes the issue named key as a person would in the tracker, such as assignee="Ada Lovelace"."""
|
||||
self.issues = [dataclasses.replace(issue, **fields) if issue.key == key else issue for issue in self.issues] # type: ignore[arg-type]
|
||||
@@ -0,0 +1,148 @@
|
||||
"""An epic's plan kept as a Jira Cloud epic's child issues and their sub-tasks, read and filed over Jira's REST API version 3."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from adf_codec.convert import adfs_to_markdown, markdown_to_adf, Refused
|
||||
from atlassian.client import basic_auth, Http, jira_json, ToAdf, ToMarkdown
|
||||
from project_config.settings import Epic
|
||||
from do_epic.backend.protocol import EpicIssue, no_parent, unfit
|
||||
from failure import Failure
|
||||
from plan_store.jira_epic import ISSUE_KEY, ISSUE_TYPE, project_of, searched
|
||||
|
||||
FIELDS = ["summary", "description", "labels", "assignee", "status", "parent"]
|
||||
# How many child issues one search for sub-tasks names, which keeps its JQL well inside the length Jira takes.
|
||||
PARENTS_PER_SEARCH = 50
|
||||
PERHAPS_FILED = "unless Jira refused the request, the issue may exist: look under {parent} before filing it again"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Found:
|
||||
"""An issue as Jira answered it, its description still in Jira's document format."""
|
||||
issue: EpicIssue
|
||||
description: dict[str, typing.Any] | None
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class JiraEpic:
|
||||
"""The issues of a Jira epic, read and filed with the credentials env_file holds."""
|
||||
epic: Epic
|
||||
env_file: pathlib.Path
|
||||
http: Http = jira_json
|
||||
to_markdown: ToMarkdown = adfs_to_markdown
|
||||
to_adf: ToAdf = markdown_to_adf
|
||||
|
||||
@property
|
||||
def epic_key(self) -> str:
|
||||
return self.epic.key
|
||||
|
||||
def read(self) -> list[EpicIssue] | Failure:
|
||||
authorization = basic_auth(self.env_file)
|
||||
if isinstance(authorization, Failure):
|
||||
return authorization
|
||||
found = self.found(authorization)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
# A container run converts every description at once. It comes after every other check, since a read that fails doesn't need it.
|
||||
described = [each for each in found if each.description is not None]
|
||||
texts = self.to_markdown([each.description for each in described if each.description is not None])
|
||||
if isinstance(texts, Refused):
|
||||
return Failure(f"{described[texts.index].issue.key}'s description: {texts.failure.reason}")
|
||||
if isinstance(texts, Failure):
|
||||
return Failure(f"cannot convert the descriptions of {self.epic_key}'s issues to markdown: {texts.reason}")
|
||||
details = {each.issue.key: text.strip() for each, text in zip(described, texts)}
|
||||
return [dataclasses.replace(each.issue, detail=details.get(each.issue.key, "")) for each in found]
|
||||
|
||||
def file(self, parent: str, summary: str, detail: str, labels: tuple[str, ...]) -> str | Failure:
|
||||
authorization = basic_auth(self.env_file)
|
||||
if isinstance(authorization, Failure):
|
||||
return authorization
|
||||
if refused := unfit(summary, labels) or self.not_a_parent(parent, authorization):
|
||||
return refused
|
||||
fields: dict[str, typing.Any] = {"project": {"key": project_of(parent)}, "parent": {"key": parent}, "summary": summary, "labels": list(labels)}
|
||||
kind = {"name": ISSUE_TYPE} if parent == self.epic_key else self.sub_task_type(project_of(parent), authorization)
|
||||
if isinstance(kind, Failure):
|
||||
return kind
|
||||
if detail.strip():
|
||||
description = self.to_adf(detail)
|
||||
if isinstance(description, Failure):
|
||||
return Failure(f"cannot convert the detail to a Jira description: {description.reason}")
|
||||
fields["description"] = description
|
||||
created = self.http("POST", f"{self.epic.site}/rest/api/3/issue", authorization, {"fields": fields | {"issuetype": kind}})
|
||||
if isinstance(created, Failure):
|
||||
return Failure(f"cannot file the issue under {parent}: {created.reason}; {PERHAPS_FILED.format(parent=parent)}")
|
||||
key = created.get("key") if isinstance(created, dict) else None
|
||||
if not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
|
||||
return Failure(f"Jira answered the new issue without an issue key; {PERHAPS_FILED.format(parent=parent)}")
|
||||
return key
|
||||
|
||||
def not_a_parent(self, parent: str, authorization: str) -> Failure | None:
|
||||
"""Why nothing can be filed under parent; None where it is the epic or one of its child issues."""
|
||||
if parent == self.epic_key:
|
||||
return None
|
||||
if not ISSUE_KEY.fullmatch(parent):
|
||||
return no_parent(self.epic_key, parent)
|
||||
# A search may not show an issue filed a moment ago, so the parent is asked for by its key.
|
||||
answered = self.http("GET", f"{self.epic.site}/rest/api/3/issue/{parent}?fields=parent", authorization, None)
|
||||
if isinstance(answered, Failure):
|
||||
return Failure(f"cannot read {parent} to file under it: {answered.reason}")
|
||||
fields = answered.get("fields") if isinstance(answered, dict) else None
|
||||
above = fields.get("parent") if isinstance(fields, dict) else None
|
||||
return None if isinstance(above, dict) and above.get("key") == self.epic_key else no_parent(self.epic_key, parent)
|
||||
|
||||
def found(self, authorization: str) -> list[Found] | Failure:
|
||||
"""Each child issue, each followed by its sub-tasks."""
|
||||
# The search answers no issues for an epic the account cannot see; asking for the epic itself tells that from an epic without children.
|
||||
seen = self.http("GET", f"{self.epic.site}/rest/api/3/issue/{self.epic_key}?fields=summary", authorization, None)
|
||||
if isinstance(seen, Failure):
|
||||
return Failure(f"cannot read the epic {self.epic_key}: {seen.reason}")
|
||||
children = self.searched(f"{self.epic_key}'s child issues", (self.epic_key,), authorization)
|
||||
if isinstance(children, Failure):
|
||||
return children
|
||||
keys = [each.issue.key for each in children]
|
||||
tasks: list[Found] = []
|
||||
for start in range(0, len(keys), PARENTS_PER_SEARCH):
|
||||
batch = self.searched(f"the sub-tasks of {self.epic_key}'s child issues", tuple(keys[start:start + PARENTS_PER_SEARCH]), authorization)
|
||||
if isinstance(batch, Failure):
|
||||
return batch
|
||||
tasks += batch
|
||||
return [each for child in children for each in (child, *(task for task in tasks if task.issue.parent == child.issue.key))]
|
||||
|
||||
def searched(self, what: str, parents: tuple[str, ...], authorization: str) -> list[Found] | Failure:
|
||||
"""The issues whose parent is one of parents, in rank order."""
|
||||
named = ", ".join(f'"{key}"' for key in parents)
|
||||
answered = searched(self.epic.site, what, f"parent in ({named}) ORDER BY Rank ASC", {"fields": FIELDS}, authorization, self.http)
|
||||
if isinstance(answered, Failure):
|
||||
return answered
|
||||
found = [found_of(raw) for raw in answered]
|
||||
return next((each for each in found if isinstance(each, Failure)), None) or [each for each in found if isinstance(each, Found)]
|
||||
|
||||
def sub_task_type(self, project: str, authorization: str) -> dict[str, str] | Failure:
|
||||
"""The project's sub-task issue type, by id, since sites name it differently, such as Subtask or Sub-task."""
|
||||
answered = self.http("GET", f"{self.epic.site}/rest/api/3/issue/createmeta/{project}/issuetypes", authorization, None)
|
||||
if isinstance(answered, Failure):
|
||||
return Failure(f"cannot read {project}'s issue types: {answered.reason}")
|
||||
types = answered.get("issueTypes") if isinstance(answered, dict) else None
|
||||
if not isinstance(types, list):
|
||||
return Failure(f"Jira answered {project}'s issue types in a shape its REST API version 3 doesn't document")
|
||||
found = next((kind["id"] for kind in types if isinstance(kind, dict) and kind.get("subtask") is True and isinstance(kind.get("id"), str)), None)
|
||||
return Failure(f"{project} has no sub-task issue type; turn sub-tasks on in Jira") if found is None else {"id": found}
|
||||
|
||||
|
||||
def found_of(raw: object) -> Found | Failure:
|
||||
"""The issue in one entry of a search's answer, or which issue is not in the shape Jira documents."""
|
||||
key = raw.get("key") if isinstance(raw, dict) else None
|
||||
if not isinstance(raw, dict) or not isinstance(key, str) or not ISSUE_KEY.fullmatch(key):
|
||||
return Failure("the search answered an issue without an issue key")
|
||||
# The answer's nested fields are read as Jira documents them; one missing or of another type ends the read here.
|
||||
try:
|
||||
fields = raw["fields"]
|
||||
summary, labels, assignee, status, parent, description = fields["summary"], fields["labels"], fields["assignee"], fields["status"], fields["parent"]["key"], fields["description"]
|
||||
name = assignee["displayName"] if assignee is not None else None
|
||||
state, done = status["name"], status["statusCategory"]["key"] == "done"
|
||||
except (AttributeError, KeyError, TypeError) as failure:
|
||||
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: {failure!r}")
|
||||
texts = (summary, *labels, state, parent) if isinstance(labels, list) else ()
|
||||
if not texts or not all(isinstance(text, str) for text in texts) or not isinstance(name, str | None) or not ISSUE_KEY.fullmatch(parent) or not isinstance(description, dict | None):
|
||||
return Failure(f"{key} is not in the shape Jira Cloud's REST API version 3 documents: its summary, labels, assignee, status, parent or description holds another kind of value")
|
||||
return Found(EpicIssue(key, summary.strip(), "", tuple(sorted(labels)), name, state, done, parent), description)
|
||||
@@ -0,0 +1,58 @@
|
||||
"""What every backend of an epic's plan reads and files: an EpicIssue, and the EpicStore protocol they all meet. do-epic reads an epic only through EpicStore, so another tracker can stand behind it; contract.py holds the tests every backend passes.
|
||||
|
||||
EpicStore is not plan_store's PlanStore. A PlanStore item names a release, but an epic item has none (`docs/decisions.md` → `do-epic: drives one Jira epic …`), and it may be a sub-task."""
|
||||
import dataclasses
|
||||
import re
|
||||
import typing
|
||||
|
||||
from failure import Failure
|
||||
|
||||
# What a tracker accepts as a label: Jira refuses a label holding white space.
|
||||
LABEL = re.compile(r"\S+")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class EpicIssue:
|
||||
"""One of an epic's child issues, or a sub-task of one, as a read found it."""
|
||||
key: str
|
||||
summary: str
|
||||
# The description, as markdown; empty where the issue has none.
|
||||
detail: str
|
||||
# Sorted by name.
|
||||
labels: tuple[str, ...]
|
||||
# The assignee's display name; None where nobody is assigned.
|
||||
assignee: str | None
|
||||
# The status's name, such as "In Progress".
|
||||
status: str
|
||||
# Whether the status is one the tracker counts as done.
|
||||
done: bool
|
||||
# The epic's key for a child issue, the child's key for a sub-task.
|
||||
parent: str
|
||||
|
||||
|
||||
class EpicStore(typing.Protocol):
|
||||
"""The issues of one epic. A write that fails doesn't file anything, except where its Failure says the tracker may have taken it all the same."""
|
||||
|
||||
@property
|
||||
def epic_key(self) -> str:
|
||||
"""The parent of each child issue."""
|
||||
|
||||
def read(self) -> list[EpicIssue] | Failure:
|
||||
"""Each child issue, open or done, in the tracker's rank order, each followed by its sub-tasks in theirs."""
|
||||
|
||||
def file(self, parent: str, summary: str, detail: str, labels: tuple[str, ...]) -> str | Failure:
|
||||
"""Files an open issue under parent and answers its key: a child issue where parent is the epic, a sub-task where parent is one of its child issues. detail is markdown, and may be empty."""
|
||||
|
||||
|
||||
def unfit(summary: str, labels: tuple[str, ...]) -> Failure | None:
|
||||
"""Why no tracker can file an issue with this summary and these labels; None where one can."""
|
||||
if not summary.strip() or "\n" in summary:
|
||||
return Failure("the summary must be one line of text")
|
||||
if bad := next((label for label in labels if not LABEL.fullmatch(label)), None):
|
||||
return Failure(f"the label {bad!r} is empty or holds white space")
|
||||
return None
|
||||
|
||||
|
||||
def no_parent(epic_key: str, parent: str) -> Failure:
|
||||
"""What every backend says when asked to file under parent, which is neither the epic nor one of its child issues."""
|
||||
return Failure(f"{parent} is neither the epic {epic_key} nor one of its child issues, so nothing can be filed under it")
|
||||
@@ -0,0 +1,30 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from do_epic.backend.contract import EpicStoreContract
|
||||
from failure import Failure
|
||||
from do_epic.backend.fake import MemoryEpic
|
||||
|
||||
|
||||
class MemoryEpicContract(EpicStoreContract, unittest.TestCase):
|
||||
def new_store(self) -> MemoryEpic:
|
||||
return MemoryEpic("ABC-12")
|
||||
|
||||
def assign(self, store: MemoryEpic, key: str, name: str) -> None: # type: ignore[override]
|
||||
store.change(key, assignee=name)
|
||||
|
||||
def close(self, store: MemoryEpic, key: str, status: str) -> None: # type: ignore[override]
|
||||
store.change(key, status=status, done=True)
|
||||
|
||||
|
||||
class Failing(unittest.TestCase):
|
||||
def test_every_read_and_filing_fails_with_the_reason_given(self) -> None:
|
||||
store = MemoryEpic("ABC-12", fails="Jira is down")
|
||||
self.assertEqual((store.read(), store.file("ABC-12", "Count the runs.", "", ())), (Failure("Jira is down"), Failure("Jira is down")))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,190 @@
|
||||
import base64
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
|
||||
|
||||
from project_config.settings import Epic
|
||||
from do_epic.backend.contract import EpicStoreContract
|
||||
from failure import Failure
|
||||
from plan_store import jira_epic
|
||||
from do_epic.backend.jira import JiraEpic
|
||||
from adf_codec.convert import Refused
|
||||
|
||||
SITE = "https://acme.atlassian.net"
|
||||
EPIC = Epic(SITE, "ABC-12")
|
||||
AUTHORIZATION = "Basic " + base64.b64encode(b"me@example.com:s3cret").decode()
|
||||
EPIC_URL = f"{SITE}/rest/api/3/issue/ABC-12?fields=summary"
|
||||
ISSUE_TYPES = f"{SITE}/rest/api/3/issue/createmeta/ABC/issuetypes"
|
||||
SEARCH = f"{SITE}/rest/api/3/search/jql"
|
||||
ISSUES = f"{SITE}/rest/api/3/issue"
|
||||
TO_DO = {"name": "To Do", "statusCategory": {"key": "new"}}
|
||||
TYPES = {"issueTypes": [{"id": "10001", "name": "Task", "subtask": False}, {"id": "10003", "name": "Subtask", "subtask": True}]}
|
||||
|
||||
|
||||
def to_adf(markdown: str) -> dict[str, object]:
|
||||
"""Stands in for the adf-codec shim: one paragraph holding the markdown."""
|
||||
return {"type": "doc", "version": 1, "content": [{"type": "paragraph", "content": [{"type": "text", "text": markdown}]}]}
|
||||
|
||||
|
||||
def to_markdown(documents: list[dict[str, object]]) -> list[str]:
|
||||
return [document["content"][0]["content"][0]["text"] + "\n" for document in documents] # type: ignore[index]
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class FakeJira:
|
||||
"""Jira Cloud's REST API as JiraEpic calls it. It holds issues as Jira answers them, in rank order. A search answers two issues a page, with only the fields it asks for. A call whose method and URL are in answers gets the answer held there instead."""
|
||||
issues: list[dict[str, object]] = dataclasses.field(default_factory=list)
|
||||
answers: dict[tuple[str, str], object] = dataclasses.field(default_factory=dict)
|
||||
calls: list[tuple[str, str]] = dataclasses.field(default_factory=list)
|
||||
|
||||
def __call__(self, method: str, url: str, authorization: str, body: object) -> object:
|
||||
self.calls.append((method, url))
|
||||
if authorization != AUTHORIZATION:
|
||||
return Failure("Jira answered 401")
|
||||
if (method, url) in self.answers:
|
||||
return self.answers[method, url]
|
||||
if method == "GET" and url == EPIC_URL:
|
||||
return {"key": "ABC-12"}
|
||||
if method == "GET" and url == ISSUE_TYPES:
|
||||
return TYPES
|
||||
if method == "POST" and url == SEARCH:
|
||||
return self.page(body)
|
||||
if method == "GET" and (held := next((issue for issue in self.issues if url == f"{ISSUES}/{issue['key']}?fields=parent"), None)):
|
||||
return {"key": held["key"], "fields": {"parent": held["fields"]["parent"]}} # type: ignore[index]
|
||||
if method == "POST" and url == ISSUES:
|
||||
return self.created(body)
|
||||
return Failure(f"Jira answered 404 for {method} {url}")
|
||||
|
||||
def page(self, body: object) -> object:
|
||||
assert isinstance(body, dict)
|
||||
matched = re.fullmatch(r'parent in \(("[A-Z]+-[0-9]+"(?:, "[A-Z]+-[0-9]+")*)\) ORDER BY Rank ASC', body["jql"])
|
||||
if matched is None:
|
||||
return Failure("Jira answered 400: the JQL is not valid")
|
||||
parents = re.findall(r'"([A-Z]+-[0-9]+)"', matched[1])
|
||||
found = [{"key": issue["key"], "fields": {name: value for name, value in issue["fields"].items() if name in body.get("fields", [])}} for issue in self.issues if issue["fields"]["parent"]["key"] in parents] # type: ignore[index, attr-defined]
|
||||
start = int(body.get("nextPageToken", 0))
|
||||
return {"issues": found[start:start + 2]} | ({"nextPageToken": str(start + 2)} if start + 2 < len(found) else {})
|
||||
|
||||
def created(self, body: object) -> object:
|
||||
assert isinstance(body, dict)
|
||||
fields = body["fields"]
|
||||
parent = fields["parent"]["key"]
|
||||
kind = {"name": "Task"} if parent == "ABC-12" else {"id": "10003"} if any(issue["key"] == parent for issue in self.issues) else None
|
||||
if fields["project"] != {"key": "ABC"} or kind is None or fields["issuetype"] != kind:
|
||||
return Failure("Jira answered 400: the project, parent or issue type is not valid")
|
||||
key = f"ABC-{len(self.issues) + 100}"
|
||||
self.issues.append({"key": key, "fields": {"summary": fields["summary"], "description": fields.get("description"), "labels": fields["labels"], "assignee": None, "status": TO_DO, "parent": {"key": parent}}})
|
||||
return {"id": "1", "key": key}
|
||||
|
||||
def change(self, key: str, **fields: object) -> None:
|
||||
next(issue for issue in self.issues if issue["key"] == key)["fields"].update(fields) # type: ignore[union-attr]
|
||||
|
||||
|
||||
class Case(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
folder = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(folder.cleanup)
|
||||
self.env_file = pathlib.Path(folder.name) / "jira.env"
|
||||
self.env_file.write_text("ATLASSIAN_EMAIL=me@example.com\nATLASSIAN_API_TOKEN=s3cret\n")
|
||||
self.env_file.chmod(0o600)
|
||||
self.jira = FakeJira()
|
||||
|
||||
def new_store(self) -> JiraEpic:
|
||||
return JiraEpic(EPIC, self.env_file, self.jira, to_markdown, to_adf)
|
||||
|
||||
|
||||
class JiraEpicContract(Case, EpicStoreContract):
|
||||
def assign(self, store: JiraEpic, key: str, name: str) -> None: # type: ignore[override]
|
||||
self.jira.change(key, assignee={"accountId": "5b10a2844c20165700ede21g", "displayName": name})
|
||||
|
||||
def close(self, store: JiraEpic, key: str, status: str) -> None: # type: ignore[override]
|
||||
self.jira.change(key, status={"name": status, "statusCategory": {"key": "done"}})
|
||||
|
||||
|
||||
class Jira(Case):
|
||||
def test_reads_the_sub_tasks_of_more_children_than_one_search_names(self) -> None:
|
||||
store = self.new_store()
|
||||
children = [store.file("ABC-12", f"Item {n}.", "", ()) for n in range(60)]
|
||||
tasks = [store.file(child, "A task.", "", ()) for child in children]
|
||||
read = store.read()
|
||||
assert isinstance(read, list), read
|
||||
self.assertEqual([issue.key for issue in read], [key for pair in zip(children, tasks) for key in pair])
|
||||
|
||||
def test_files_a_child_without_asking_for_issue_types(self) -> None:
|
||||
self.new_store().file("ABC-12", "Count the runs.", "", ())
|
||||
self.assertNotIn(("GET", ISSUE_TYPES), self.jira.calls)
|
||||
|
||||
def test_a_filing_jira_fails_or_answers_without_a_key_says_the_issue_may_exist(self) -> None:
|
||||
for answer, reason in ((Failure("Jira answered 503"), "cannot file the issue under ABC-12: Jira answered 503"), ({"id": "1"}, "Jira answered the new issue without an issue key"), (["ABC-100"], "Jira answered the new issue without an issue key")):
|
||||
with self.subTest(answer=answer):
|
||||
self.jira.answers[("POST", ISSUES)] = answer
|
||||
self.assertEqual(self.new_store().file("ABC-12", "Count the runs.", "", ()), Failure(f"{reason}; unless Jira refused the request, the issue may exist: look under ABC-12 before filing it again"))
|
||||
|
||||
def test_a_detail_the_codec_refuses_leaves_the_epic_as_it_was(self) -> None:
|
||||
store = JiraEpic(EPIC, self.env_file, self.jira, to_markdown, lambda markdown: Failure("adf-codec refused it"))
|
||||
self.assertEqual(store.file("ABC-12", "Count the runs.", "Why.", ()), Failure("cannot convert the detail to a Jira description: adf-codec refused it"))
|
||||
self.assertEqual(self.jira.issues, [])
|
||||
|
||||
def test_files_no_sub_task_where_the_project_has_no_sub_task_type_or_its_types_cannot_be_read(self) -> None:
|
||||
store = self.new_store()
|
||||
child = store.file("ABC-12", "Count the runs.", "", ())
|
||||
for answer, reason in ((Failure("Jira answered 403"), "cannot read ABC's issue types: Jira answered 403"), ({"issueTypes": [{"id": "10001", "subtask": False}]}, "ABC has no sub-task issue type; turn sub-tasks on in Jira"), ({}, "Jira answered ABC's issue types in a shape its REST API version 3 doesn't document"), ([], "Jira answered ABC's issue types in a shape its REST API version 3 doesn't document")):
|
||||
with self.subTest(answer=answer):
|
||||
self.jira.answers[("GET", ISSUE_TYPES)] = answer
|
||||
self.assertEqual(store.file(str(child), "A task.", "", ()), Failure(reason))
|
||||
self.assertEqual(len(self.jira.issues), 1)
|
||||
|
||||
def test_an_issue_in_another_shape_fails_the_read(self) -> None:
|
||||
valid = {"summary": "Count the runs.", "description": None, "labels": [], "assignee": None, "status": TO_DO, "parent": {"key": "ABC-12"}}
|
||||
shape = "ABC-34 is not in the shape Jira Cloud's REST API version 3 documents"
|
||||
for raw, reason in (
|
||||
("ABC-34", "the search answered an issue without an issue key"),
|
||||
({"fields": valid}, "the search answered an issue without an issue key"),
|
||||
({"key": "abc-34", "fields": valid}, "the search answered an issue without an issue key"),
|
||||
({"key": "ABC-34", "fields": {"summary": "Count the runs.", "parent": {"key": "ABC-12"}}}, shape + ": KeyError('labels')"),
|
||||
*((({"key": "ABC-34", "fields": valid | changed}, shape + ": its summary, labels, assignee, status, parent or description holds another kind of value") for changed in (
|
||||
{"summary": 3}, {"labels": "api"}, {"labels": [3]}, {"assignee": {"displayName": 0}}, {"status": {"name": 0, "statusCategory": {"key": "new"}}}, {"parent": {"key": "ABC"}}, {"parent": {"key": 12}}, {"description": "text"},
|
||||
))),
|
||||
):
|
||||
with self.subTest(raw=raw):
|
||||
self.jira.answers[("POST", SEARCH)] = {"issues": [raw]}
|
||||
self.assertEqual(self.new_store().read(), Failure(reason))
|
||||
|
||||
def test_a_description_the_codec_cannot_convert_fails_the_read(self) -> None:
|
||||
self.new_store().file("ABC-12", "Count the runs.", "Why.", ())
|
||||
for converted, reason in ((Refused(0, Failure("a table cell holds a list")), "ABC-100's description: a table cell holds a list"), (Failure("docker is not running"), "cannot convert the descriptions of ABC-12's issues to markdown: docker is not running")):
|
||||
with self.subTest(converted=converted):
|
||||
self.assertEqual(JiraEpic(EPIC, self.env_file, self.jira, lambda documents: converted, to_adf).read(), Failure(reason))
|
||||
|
||||
def test_a_search_that_does_not_end_names_the_issues_it_sought(self) -> None:
|
||||
store = self.new_store()
|
||||
children = [store.file("ABC-12", "Count the runs.", "", ()) for _ in range(2)]
|
||||
for _ in range(3):
|
||||
store.file(str(children[0]), "A task.", "", ())
|
||||
with unittest.mock.patch.object(jira_epic, "MAX_PAGES", 1):
|
||||
self.assertEqual(store.read(), Failure("the search for the sub-tasks of ABC-12's child issues did not end within 1 pages"))
|
||||
store.file("ABC-12", "Count the runs.", "", ())
|
||||
self.assertEqual(store.read(), Failure("the search for ABC-12's child issues did not end within 1 pages"))
|
||||
|
||||
def test_refuses_to_file_under_another_epics_child_or_an_issue_it_cannot_read(self) -> None:
|
||||
self.jira.issues.append({"key": "ABC-5", "fields": {"parent": {"key": "ABC-1"}}})
|
||||
store = self.new_store()
|
||||
self.assertEqual(store.file("ABC-5", "Count more.", "", ()), Failure("ABC-5 is neither the epic ABC-12 nor one of its child issues, so nothing can be filed under it"))
|
||||
self.assertEqual(store.file("ABC-77", "Count more.", "", ()), Failure(f"cannot read ABC-77 to file under it: Jira answered 404 for GET {ISSUES}/ABC-77?fields=parent"))
|
||||
self.jira.answers[("GET", f"{ISSUES}/ABC-78?fields=parent")] = ["ABC-12"]
|
||||
self.assertEqual(store.file("ABC-78", "Count more.", "", ()), Failure("ABC-78 is neither the epic ABC-12 nor one of its child issues, so nothing can be filed under it"))
|
||||
self.assertEqual([call for call in self.jira.calls if call[0] == "POST" and call[1] == ISSUES], [])
|
||||
|
||||
def test_an_epic_the_account_cannot_see_fails_the_read(self) -> None:
|
||||
store = JiraEpic(Epic(SITE, "ABC-99"), self.env_file, self.jira, to_markdown, to_adf)
|
||||
self.assertEqual(store.read(), Failure(f"cannot read the epic ABC-99: Jira answered 404 for GET {SITE}/rest/api/3/issue/ABC-99?fields=summary"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,82 @@
|
||||
"""do-epic's machine-wide settings: `~/.claude-projects/do-epic/config.toml`, and `repo-rule.md` beside it.
|
||||
|
||||
One pair per machine, the same for every epic (docs/decisions.md → `do-epic: drives one Jira epic …`). The config is TOML:
|
||||
|
||||
jira-url = "https://acme.atlassian.net"
|
||||
env-file = "~/.secrets/acme-jira.env"
|
||||
repoless-model = "claude-opus-5-5"
|
||||
clone-folder = "~/code"
|
||||
language = "Swedish"
|
||||
|
||||
- `jira-url` is the Jira site's address: https, a host, and no path.
|
||||
- `env-file` is the file of Jira credentials: a path that starts with / or ~/, outside every folder of `~/.claude-projects` and outside the clone folder, which hold git checkouts. `~/.claude-projects/<key>.env` is fine.
|
||||
- `repoless-model` is the model that works an epic item with no repo: a model on the routing ladder, `model_routing/routing-ladder.json`.
|
||||
- `clone-folder` is the folder a missing repo is cloned into: a path that starts with / or ~/. `~/code` where left out.
|
||||
- `language` is the default language of Jira text, by its English name. English where left out.
|
||||
- `repo-rule.md` is prose that says how to tell an item's repo, and mustn't be blank."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import re
|
||||
|
||||
from project_config.settings import anywhere, CONFIG, DO_EPIC, ENV_FILE, LANGUAGE, language_fault, lies_in, parsed, PROJECTS, SITE
|
||||
from failure import Failure
|
||||
from model_routing.ladder_file import read_ladder, SHIPPED
|
||||
|
||||
REPO_RULE = "repo-rule.md"
|
||||
# Three readers, seated as the human who writes this config, picked these spellings from four candidates each, 3-0: `jira-url` over `site`, `clone-folder` over `repos-folder` (which reads as the folder of repos that already exist) and `repoless-model` over `no-repo-model`. `env-file` and `language` keep the names a project's config gives them.
|
||||
JIRA_URL = "jira-url"
|
||||
REPOLESS_MODEL = "repoless-model"
|
||||
CLONE_FOLDER = "clone-folder"
|
||||
SETTINGS = (JIRA_URL, ENV_FILE, REPOLESS_MODEL)
|
||||
OPTIONAL = (CLONE_FOLDER, LANGUAGE)
|
||||
LISTED = f"The settings are {', '.join(SETTINGS)}, and optionally {' and '.join(OPTIONAL)}."
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class DoEpicConfig:
|
||||
"""folder is `~/.claude-projects/do-epic`; repo_rule is `repo-rule.md`'s text."""
|
||||
folder: pathlib.Path
|
||||
jira_url: str
|
||||
env_file: pathlib.Path
|
||||
clone_folder: pathlib.Path
|
||||
language: str
|
||||
repoless_model: str
|
||||
repo_rule: str
|
||||
|
||||
|
||||
def load(home: pathlib.Path, ladder_file: pathlib.Path = SHIPPED) -> DoEpicConfig | Failure:
|
||||
"""The settings and the repo rule under home; a Failure naming the file it can't read, else each missing or misspelt setting it finds, so that do-epic refuses to start."""
|
||||
folder = home / PROJECTS / DO_EPIC
|
||||
values = parsed(folder / CONFIG)
|
||||
if isinstance(values, Failure):
|
||||
return Failure(f"{values.reason}. {LISTED}")
|
||||
ladder = read_ladder(ladder_file)
|
||||
if isinstance(ladder, Failure):
|
||||
return ladder
|
||||
models = sorted({rung.model for rung in ladder.rungs})
|
||||
rule: str | Failure
|
||||
try:
|
||||
rule = (folder / REPO_RULE).read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
rule = Failure(f"cannot read {folder / REPO_RULE}: {failure}")
|
||||
config_folders = [path for path in (home / PROJECTS).iterdir() if path.is_dir()]
|
||||
clone_folder = anywhere(values.get(CLONE_FOLDER, "~/code"), home)
|
||||
env_file = anywhere(values.get(ENV_FILE), home)
|
||||
language = values.get(LANGUAGE, "English")
|
||||
url, model = values.get(JIRA_URL), values.get(REPOLESS_MODEL)
|
||||
faults = [
|
||||
*(f"`{name}` is missing" for name in SETTINGS if name not in values),
|
||||
*(f"`{name}` is not a setting" for name in values if name not in (*SETTINGS, *OPTIONAL)),
|
||||
*([f"`{JIRA_URL}` is {url!r}, not a site's https address without a path, such as \"https://acme.atlassian.net\""] if JIRA_URL in values and not (isinstance(url, str) and re.fullmatch(SITE, url)) else []),
|
||||
*([f"`{CLONE_FOLDER}` is {values[CLONE_FOLDER]!r}, not a path that starts with / or ~/"] if clone_folder is None else []),
|
||||
*([f"`{ENV_FILE}` is {values[ENV_FILE]!r}, not a path that starts with / or ~/"] if ENV_FILE in values and env_file is None else []),
|
||||
*([f"`{ENV_FILE}` is {values[ENV_FILE]!r}, inside a folder of ~/{PROJECTS} or the clone folder, which hold git checkouts"] if env_file and lies_in(env_file, *config_folders, *([clone_folder] if clone_folder else [])) else []),
|
||||
*([f"`{LANGUAGE}` is {language!r}, {fault}"] if LANGUAGE in values and (fault := language_fault(language)) else []),
|
||||
*([f"`{REPOLESS_MODEL}` is {model!r}, not a model on the routing ladder: {', '.join(models)}"] if REPOLESS_MODEL in values and model not in models else []),
|
||||
*([rule.reason] if isinstance(rule, Failure) else [f"`{REPO_RULE}` is blank"] if not rule.strip() else []),
|
||||
]
|
||||
if faults:
|
||||
return Failure(f"{folder}: {'; '.join(faults)}. {LISTED}")
|
||||
# Without a fault, each value above is present and of the type asserted here.
|
||||
assert clone_folder and env_file and isinstance(url, str) and isinstance(model, str) and isinstance(language, str) and isinstance(rule, str)
|
||||
return DoEpicConfig(folder, url, env_file, clone_folder, language, model, rule)
|
||||
@@ -0,0 +1,101 @@
|
||||
"""A Jira epic's description as the record of do-epic, the tool that will drive one epic.
|
||||
|
||||
The description holds a language line and the sections Goals, Audience, Scoring (the formula and the bar) and Decisions. They are read and written one at a time through `EpicDescription`, and a write leaves every other line of the markdown as it was. Goals and Audience change only where the human approved in an interrupt, which `write_section` takes as `approved`; `EpicDescription` refuses the change otherwise."""
|
||||
import dataclasses
|
||||
import re
|
||||
import typing
|
||||
|
||||
from record_store.jira_description import EpicDescription
|
||||
from failure import Failure
|
||||
from text_files.markdown_text import fenced_lines
|
||||
from record_store.held import Held
|
||||
from project_config.settings import language_fault
|
||||
|
||||
SECTIONS = ("Goals", "Audience", "Scoring", "Decisions")
|
||||
LANGUAGE_LINE = re.compile(r"^\*\*Language:\*\* (.*?)\s*$")
|
||||
|
||||
|
||||
def read_section(text: str, name: str) -> str | Failure:
|
||||
"""The text under the one ## name heading, up to the next heading of level 1 or 2, without the blank lines around it; empty where there is no such heading."""
|
||||
span = span_of(text, name)
|
||||
if isinstance(span, Failure):
|
||||
return span
|
||||
return "" if span is None else "\n".join(text.splitlines()[span[0] + 1 : span[1]]).strip("\n")
|
||||
|
||||
|
||||
def with_section(text: str, name: str, body: str) -> str | Failure:
|
||||
"""text with the one ## name section holding body, put after the others where there is none. A section that already holds body leaves text as it is."""
|
||||
span = span_of(text, name)
|
||||
if isinstance(span, Failure):
|
||||
return span
|
||||
if any(line.startswith(("# ", "## ")) and not fenced for line, fenced in fenced_lines(body)):
|
||||
return Failure(f"the body of ## {name} holds a level 1 or 2 heading, which would start another section")
|
||||
if span is None:
|
||||
return text.rstrip("\n") + "\n\n" * bool(text.strip()) + f"## {name}\n\n{body.strip()}\n"
|
||||
lines = text.splitlines()
|
||||
if read_section(text, name) == body.strip():
|
||||
return text
|
||||
last = span[1] == len(lines)
|
||||
return "\n".join([*lines[: span[0] + 1], "", *body.strip().splitlines(), *([] if last else [""]), *lines[span[1] :]]) + "\n"
|
||||
|
||||
|
||||
def span_of(text: str, name: str) -> tuple[int, int] | None | Failure:
|
||||
"""The index of the one ## name heading's line and of the line after its section; None where there is no such heading. A heading inside a code fence is ignored."""
|
||||
if name not in SECTIONS:
|
||||
return Failure(f"{name} is not a section of an epic's description; the sections are {', '.join(SECTIONS)}")
|
||||
lines = fenced_lines(text)
|
||||
headings = [index for index, (line, fenced) in enumerate(lines) if not fenced and (line.startswith("# ") or line.startswith("## "))]
|
||||
starts = [index for index in headings if lines[index][0].startswith("## ") and lines[index][0][3:].strip() == name]
|
||||
if len(starts) > 1:
|
||||
return Failure(f"the epic's description has ## {name} more than once")
|
||||
return (starts[0], next((index for index in headings if index > starts[0]), len(lines))) if starts else None
|
||||
|
||||
|
||||
def language_in(text: str) -> str | None:
|
||||
"""The language the description's language line names; None where it has none."""
|
||||
for line, fenced in fenced_lines(text):
|
||||
if not fenced and (found := LANGUAGE_LINE.match(line)):
|
||||
return found.group(1)
|
||||
return None
|
||||
|
||||
|
||||
def with_language(text: str, language: str) -> str | Failure:
|
||||
"""text with its language line naming language, put first where there is none."""
|
||||
if fault := language_fault(language):
|
||||
return Failure(f"cannot write the language {language!r}: {fault}")
|
||||
line = f"**Language:** {language}"
|
||||
lines = fenced_lines(text)
|
||||
for index, (old, fenced) in enumerate(lines):
|
||||
if not fenced and LANGUAGE_LINE.match(old):
|
||||
return "\n".join([*(old for old, _ in lines[:index]), line, *(old for old, _ in lines[index + 1 :])]) + "\n"
|
||||
return line + "\n" + "\n" * bool(text.strip()) + text
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class EpicRecord:
|
||||
"""One epic's record, kept in its description."""
|
||||
description: EpicDescription
|
||||
|
||||
def section(self, name: str) -> str | Failure:
|
||||
text = self.description.read()
|
||||
return text if isinstance(text, Failure) else read_section(text, name)
|
||||
|
||||
def language(self) -> str | None | Failure:
|
||||
text = self.description.read()
|
||||
return text if isinstance(text, Failure) else language_in(text)
|
||||
|
||||
def write_section(self, name: str, body: str, approved: bool = False) -> Held | Failure:
|
||||
"""Writes body as the ## name section. approved says the human approved the change in an interrupt, which Goals and Audience need."""
|
||||
return self.written(lambda text: with_section(text, name, body), frozenset({name}) if approved else frozenset())
|
||||
|
||||
def write_language(self, language: str) -> Held | Failure:
|
||||
return self.written(lambda text: with_language(text, language), frozenset())
|
||||
|
||||
def written(self, change: typing.Callable[[str], str | Failure], approved: frozenset[str]) -> Held | Failure:
|
||||
held = self.description.held()
|
||||
if isinstance(held, Failure):
|
||||
return held
|
||||
text = change(held.text)
|
||||
if isinstance(text, Failure):
|
||||
return text
|
||||
return dataclasses.replace(self.description, approved=approved).write(text, held.version)
|
||||
@@ -0,0 +1,96 @@
|
||||
"""Where an epic's repo is on this machine: its checkout and its project config, found by the repo's name.
|
||||
|
||||
A missing checkout is cloned from the host and owner of the epic's other repos (docs/decisions.md → `do-epic: drives one Jira epic …`). Where that is unclear, the answer is an Unsure."""
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import re
|
||||
import shutil
|
||||
|
||||
from project_config.settings import by_name, CONFIG, ProjectConfig, PROJECTS
|
||||
from do_epic.config import DoEpicConfig
|
||||
from failure import Failure
|
||||
from git_cli.git_repo import Git
|
||||
from forge.commands import url_beside
|
||||
|
||||
# One folder name, which can't be a path or a git option.
|
||||
REPO_NAME = r"\w[\w.-]*"
|
||||
# The addresses a clone may use: https, ssh, or scp-style `user@host:path`. Not ext:: or file:, which run a program or read this machine's files, and an origin can spell either.
|
||||
CLONE_ADDRESS = rf"https://[\w.-]+(:\d+)?/\S+|ssh://[\w.@-]+(:\d+)?/\S+|\w[\w.-]*@[\w.-]+:[\w./~-]+"
|
||||
# A clone that waits longer is stopped.
|
||||
CLONE_SECONDS = 600
|
||||
# git's options that make a password or a host key fail at once instead of prompting: an unattended run never waits on a prompt (docs/decisions.md → principle 6).
|
||||
NO_PROMPTS = ("-c", "core.askPass=true", "-c", "core.sshCommand=ssh -o BatchMode=yes")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Located:
|
||||
"""checkout is the repo's git checkout, and config the project config that names it."""
|
||||
checkout: pathlib.Path
|
||||
config: ProjectConfig
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Unsure(Failure):
|
||||
"""reason is a question for the human."""
|
||||
|
||||
|
||||
def locate(name: str, others: list[pathlib.Path], config: DoEpicConfig, home: pathlib.Path, git: Git) -> Located | Failure:
|
||||
"""The checkout and project config of the repo called name; others are the checkouts of the epic's other repos.
|
||||
|
||||
A project config names the repo's checkout, and it is cloned where missing. Where no config names the repo, it is cloned into the clone folder, if it isn't there, and the human is asked to write the config, since the repo's plan, goals and records are theirs to place.
|
||||
A name that is no repo's name, or a config that can't be read, is a plain Failure. Any other fault is an Unsure."""
|
||||
if not re.fullmatch(REPO_NAME, name):
|
||||
return Failure(f"{name!r} is not a repo's name: one folder name that starts with a letter, digit or underscore")
|
||||
found = by_name(name, home)
|
||||
if isinstance(found, Failure):
|
||||
return found
|
||||
checkout = found.top if found else config.clone_folder / name
|
||||
cloned = ensure(name, checkout, others, git)
|
||||
if isinstance(cloned, Failure):
|
||||
return cloned
|
||||
if found is None:
|
||||
return Unsure(f"{checkout} {'was cloned' if cloned else 'holds the repo'}, but no project config names it. Write {home / PROJECTS}/<key>/{CONFIG} with `checkout = \"{checkout}\"`, as `project-config --help` shows, and run do-epic again.")
|
||||
return Located(checkout, found)
|
||||
|
||||
|
||||
def ensure(name: str, checkout: pathlib.Path, others: list[pathlib.Path], git: Git) -> bool | Failure:
|
||||
"""Whether checkout was cloned just now: False where it is a git checkout already. Else it is cloned there from the host and owner of the others, or an Unsure says why not."""
|
||||
if (checkout / ".git").exists():
|
||||
return False
|
||||
if checkout.exists() and (not checkout.is_dir() or any(checkout.iterdir())):
|
||||
return Unsure(f"{checkout} exists and is not a git checkout, so {name} cannot be cloned there. Where should it be cloned?")
|
||||
url = clone_url(name, others, git)
|
||||
if isinstance(url, Failure):
|
||||
return url
|
||||
cloned = git(*NO_PROMPTS, "clone", "--", url, str(checkout), timeout=CLONE_SECONDS)
|
||||
if isinstance(cloned, Failure):
|
||||
# What the clone began, and nothing else: checkout was absent or empty. A kept half-clone would pass for the repo on the next run.
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
return Unsure(f"cannot clone {url} to {checkout}: {cloned.reason}. Where does {name} live?")
|
||||
return True
|
||||
|
||||
|
||||
def without_credentials(url: str) -> str:
|
||||
"""url without a `user:token@` after its scheme, so that a clone and a question for the team never carry one. The clone then uses git's own credentials."""
|
||||
return re.sub(r"^(https?://)[^/]*@", r"\1", url)
|
||||
|
||||
|
||||
def clone_url(name: str, others: list[pathlib.Path], git: Git) -> str | Failure:
|
||||
"""The address of the repo called name, under the host and owner all the others share. An Unsure where they share none or an address can't be built."""
|
||||
if not others:
|
||||
return Unsure(f"{name} is missing and no other repo of the epic shows what host to clone it from. Where does it live?")
|
||||
urls = set()
|
||||
for other in others:
|
||||
origin = git("remote", "get-url", "origin", cwd=str(other))
|
||||
if isinstance(origin, Failure):
|
||||
return Unsure(f"cannot read the origin of {other}: {origin.reason}. Where does {name} live?")
|
||||
beside = url_beside(without_credentials(origin.strip()), name)
|
||||
if beside is None:
|
||||
return Unsure(f"{other} has an origin, {without_credentials(origin.strip())}, with no host and owner to put another repo under. Where does {name} live?")
|
||||
urls.add(beside)
|
||||
if len(urls) > 1:
|
||||
return Unsure(f"The epic's other repos sit under more than one host or owner, so {name} could be at {' or '.join(sorted(urls))}. Which holds it?")
|
||||
url = next(iter(urls))
|
||||
if not re.fullmatch(CLONE_ADDRESS, url):
|
||||
return Unsure(f"{url} is not an https, ssh or scp-style address. Where does {name} live?")
|
||||
return url
|
||||
@@ -0,0 +1,132 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from do_epic.config import DoEpicConfig, load
|
||||
from failure import Failure
|
||||
|
||||
SETTINGS = """jira-url = "https://acme.atlassian.net"
|
||||
env-file = "~/jira.env"
|
||||
repoless-model = "claude-opus-5-5"
|
||||
"""
|
||||
|
||||
|
||||
class ConfigTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
made = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(made.cleanup)
|
||||
self.home = pathlib.Path(made.name).resolve()
|
||||
self.folder = self.home / ".claude-projects" / "do-epic"
|
||||
self.folder.mkdir(parents=True)
|
||||
|
||||
def write(self, settings: str | None = SETTINGS, rule: str | None = "A label that equals a repo's name names that repo.\n") -> None:
|
||||
if settings is not None:
|
||||
(self.folder / "config.toml").write_text(settings, encoding="utf-8")
|
||||
if rule is not None:
|
||||
(self.folder / "repo-rule.md").write_text(rule, encoding="utf-8")
|
||||
|
||||
def failure(self) -> str:
|
||||
found = load(self.home)
|
||||
assert isinstance(found, Failure), found
|
||||
return found.reason
|
||||
|
||||
def test_reads_the_settings_and_the_rule_and_defaults_the_clone_folder_and_language(self) -> None:
|
||||
self.write()
|
||||
self.assertEqual(load(self.home), DoEpicConfig(self.folder, "https://acme.atlassian.net", self.home / "jira.env", self.home / "code", "English", "claude-opus-5-5", "A label that equals a repo's name names that repo.\n"))
|
||||
|
||||
def test_reads_a_clone_folder_and_language_the_config_sets(self) -> None:
|
||||
self.write(SETTINGS + 'clone-folder = "/srv/repos"\nlanguage = "Brazilian Portuguese"\n')
|
||||
found = load(self.home)
|
||||
assert isinstance(found, DoEpicConfig), found
|
||||
self.assertEqual((found.clone_folder, found.language), (pathlib.Path("/srv/repos"), "Brazilian Portuguese"))
|
||||
|
||||
def test_refuses_while_the_config_is_missing(self) -> None:
|
||||
self.write(None)
|
||||
self.assertIn(f"cannot read {self.folder / 'config.toml'}", self.failure())
|
||||
|
||||
def test_lists_the_settings_where_the_config_is_missing(self) -> None:
|
||||
self.write(None)
|
||||
self.assertIn("The settings are jira-url, env-file, repoless-model", self.failure())
|
||||
|
||||
def test_refuses_while_the_rule_is_missing(self) -> None:
|
||||
self.write(rule=None)
|
||||
self.assertIn(f"cannot read {self.folder / 'repo-rule.md'}", self.failure())
|
||||
|
||||
def test_refuses_while_the_rule_is_blank(self) -> None:
|
||||
self.write(rule=" \n")
|
||||
self.assertIn("`repo-rule.md` is blank", self.failure())
|
||||
|
||||
def test_a_missing_rule_does_not_hide_the_settings_faults(self) -> None:
|
||||
self.write(SETTINGS.replace("claude-opus-5-5", "x"), rule=None)
|
||||
self.assertIn("`repoless-model` is 'x'", self.failure())
|
||||
|
||||
def test_names_each_missing_setting(self) -> None:
|
||||
self.write("")
|
||||
reason = self.failure()
|
||||
for name in ("jira-url", "env-file", "repoless-model"):
|
||||
self.assertIn(f"`{name}` is missing", reason)
|
||||
|
||||
def test_names_a_setting_it_does_not_know(self) -> None:
|
||||
self.write(SETTINGS + 'checkout = "~/code/acme"\n')
|
||||
self.assertIn("`checkout` is not a setting", self.failure())
|
||||
|
||||
def test_names_each_misspelt_value(self) -> None:
|
||||
for line, name in (
|
||||
('jira-url = "http://acme.atlassian.net"', "jira-url"),
|
||||
('jira-url = "https://acme.atlassian.net/"', "jira-url"),
|
||||
('env-file = "jira.env"', "env-file"),
|
||||
('clone-folder = "code"', "clone-folder"),
|
||||
('language = "sv_SE"', "language"),
|
||||
('repoless-model = "opus"', "repoless-model"),
|
||||
('repoless-model = "claude-opsu-5-5"', "repoless-model"),
|
||||
):
|
||||
with self.subTest(line):
|
||||
self.write("\n".join([ln for ln in SETTINGS.splitlines() if not ln.startswith(name)] + [line]))
|
||||
self.assertIn(f"`{name}` is", self.failure())
|
||||
|
||||
def test_keeps_the_credentials_outside_every_config_folder_and_the_clone_folder(self) -> None:
|
||||
(self.home / ".claude-projects" / "acme").mkdir()
|
||||
for env_file in ("~/.claude-projects/do-epic/jira.env", "~/.claude-projects/acme/jira.env", "~/code/acme/jira.env"):
|
||||
with self.subTest(env_file):
|
||||
self.write(SETTINGS.replace("~/jira.env", env_file))
|
||||
self.assertIn("`env-file` is", self.failure())
|
||||
|
||||
def test_keeps_the_credentials_outside_a_clone_folder_the_config_sets(self) -> None:
|
||||
self.write(SETTINGS.replace("~/jira.env", "~/clones/jira.env") + 'clone-folder = "~/clones"\n')
|
||||
self.assertIn("`env-file` is", self.failure())
|
||||
|
||||
def test_accepts_credentials_beside_the_config_folders(self) -> None:
|
||||
self.write(SETTINGS.replace("~/jira.env", "~/.claude-projects/do-epic.env"))
|
||||
found = load(self.home)
|
||||
assert isinstance(found, DoEpicConfig), found
|
||||
self.assertEqual(found.env_file, self.home / ".claude-projects" / "do-epic.env")
|
||||
|
||||
def test_refuses_while_the_routing_ladder_cannot_be_read(self) -> None:
|
||||
self.write()
|
||||
found = load(self.home, self.home / "no-ladder.json")
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("no-ladder.json", found.reason)
|
||||
|
||||
def test_reports_every_fault_at_once(self) -> None:
|
||||
self.write('jira-url = "acme"\nrepoless-model = "x"\n')
|
||||
reason = self.failure()
|
||||
for part in ("`jira-url` is", "`env-file` is missing", "`repoless-model` is"):
|
||||
self.assertIn(part, reason)
|
||||
self.assertIn("`env-file` is missing; `jira-url` is", reason, "joined by semicolons")
|
||||
|
||||
def test_names_the_models_on_the_ladder_where_the_model_is_off_it(self) -> None:
|
||||
ladder = self.home / "ladder.json"
|
||||
ladder.write_text('[{"model": "claude-sonnet-5-5", "effort": "low"}, {"model": "claude-opus-5-5", "effort": "low"}]', encoding="utf-8")
|
||||
self.write(SETTINGS.replace("claude-opus-5-5", "claude-haiku-5-5"))
|
||||
found = load(self.home, ladder)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("not a model on the routing ladder: claude-opus-5-5, claude-sonnet-5-5.", found.reason)
|
||||
|
||||
def test_reads_the_rule_and_the_settings_as_utf8(self) -> None:
|
||||
self.write(SETTINGS + 'language = "Svenska"\n', rule="Etiketten är repots namn.\n")
|
||||
found = load(self.home)
|
||||
assert isinstance(found, DoEpicConfig), found
|
||||
self.assertEqual(found.repo_rule, "Etiketten är repots namn.\n")
|
||||
@@ -0,0 +1,163 @@
|
||||
import dataclasses
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from project_config.settings import Epic
|
||||
from record_store.jira_description import EpicDescription
|
||||
from do_epic.epic_record import EpicRecord, language_in, read_section, with_language, with_section
|
||||
from failure import Failure
|
||||
from record_store.fakes import fake_adf, fake_markdown, FakeJira
|
||||
from record_store.held import Held
|
||||
|
||||
DESCRIPTION = """**Language:** Swedish
|
||||
|
||||
## Goals
|
||||
1. **Achieve:** Ship.
|
||||
|
||||
More about shipping.
|
||||
|
||||
## Audience
|
||||
Me.
|
||||
|
||||
## Scoring
|
||||
Value.
|
||||
|
||||
### Bar
|
||||
7.
|
||||
|
||||
## Decisions
|
||||
- A choice.
|
||||
"""
|
||||
|
||||
|
||||
class SectionsTest(unittest.TestCase):
|
||||
def test_reads_a_section_up_to_the_next_heading_of_its_level_keeping_its_blank_lines_and_subsections(self) -> None:
|
||||
self.assertEqual(read_section(DESCRIPTION, "Goals"), "1. **Achieve:** Ship.\n\n More about shipping.")
|
||||
self.assertEqual(read_section(DESCRIPTION, "Scoring"), "Value.\n\n### Bar\n7.")
|
||||
self.assertEqual(read_section(DESCRIPTION, "Decisions"), "- A choice.")
|
||||
|
||||
def test_a_missing_section_reads_empty_and_a_repeated_one_fails(self) -> None:
|
||||
self.assertEqual(read_section("## Goals\nShip.\n", "Scoring"), "")
|
||||
found = read_section("## Scoring\nA\n\n## Scoring\nB\n", "Scoring")
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("Scoring", found.reason)
|
||||
|
||||
def test_a_heading_inside_a_code_fence_starts_no_section(self) -> None:
|
||||
text = "## Scoring\n```\n## Decisions\n```\n"
|
||||
self.assertEqual(read_section(text, "Decisions"), "")
|
||||
self.assertEqual(read_section(text, "Scoring"), "```\n## Decisions\n```")
|
||||
|
||||
def test_a_top_level_heading_ends_a_section_and_indented_text_stays_indented(self) -> None:
|
||||
self.assertEqual(read_section("## Scoring\n code\n# Title\nB\n", "Scoring"), " code")
|
||||
|
||||
def test_appending_keeps_the_trailing_spaces_of_the_last_line(self) -> None:
|
||||
self.assertEqual(with_section("## Goals\nShip. \n", "Scoring", "Value."), "## Goals\nShip. \n\n## Scoring\n\nValue.\n")
|
||||
|
||||
def test_replaces_one_section_and_leaves_every_other_line_as_it_was(self) -> None:
|
||||
changed = with_section(DESCRIPTION, "Scoring", "Risk.\n")
|
||||
self.assertEqual(changed, DESCRIPTION.replace("Value.\n\n### Bar\n7.\n", "\nRisk.\n"))
|
||||
self.assertEqual(read_section(changed, "Scoring"), "Risk.") # type: ignore[arg-type]
|
||||
|
||||
def test_replaces_the_last_section_without_a_blank_line_after_it(self) -> None:
|
||||
self.assertEqual(with_section("## Scoring\nValue.\n", "Scoring", "Risk."), "## Scoring\n\nRisk.\n")
|
||||
|
||||
def test_a_section_with_the_text_it_holds_leaves_the_description_untouched(self) -> None:
|
||||
self.assertEqual(with_section(DESCRIPTION, "Goals", "1. **Achieve:** Ship.\n\n More about shipping.\n"), DESCRIPTION)
|
||||
|
||||
def test_appends_a_missing_section_after_the_others(self) -> None:
|
||||
self.assertEqual(with_section("## Goals\nShip.\n", "Scoring", "Value."), "## Goals\nShip.\n\n## Scoring\n\nValue.\n")
|
||||
self.assertEqual(with_section("", "Scoring", "Value."), "## Scoring\n\nValue.\n")
|
||||
|
||||
def test_refuses_a_repeated_section_and_a_name_that_is_no_section(self) -> None:
|
||||
self.assertIsInstance(with_section("## Scoring\nA\n\n## Scoring\nB\n", "Scoring", "C"), Failure)
|
||||
for found in (read_section(DESCRIPTION, "Language"), with_section(DESCRIPTION, "Language", "Swedish")):
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("Language is not a section of an epic's description; the sections are Goals, Audience, Scoring, Decisions", found.reason)
|
||||
|
||||
def test_refuses_a_body_that_would_start_another_section(self) -> None:
|
||||
for body in ("A\n## Goals\nB", "# Title"):
|
||||
with self.subTest(body):
|
||||
found = with_section(DESCRIPTION, "Scoring", body)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("level 1 or 2 heading", found.reason)
|
||||
self.assertIsInstance(with_section(DESCRIPTION, "Scoring", "```\n## Goals\n```\n### Bar"), str)
|
||||
|
||||
|
||||
class LanguageTest(unittest.TestCase):
|
||||
def test_reads_the_language_line(self) -> None:
|
||||
self.assertEqual(language_in(DESCRIPTION), "Swedish")
|
||||
self.assertIsNone(language_in("## Goals\nShip.\n"))
|
||||
|
||||
def test_a_language_line_in_a_code_fence_is_no_language_line(self) -> None:
|
||||
self.assertIsNone(language_in("```\n**Language:** Swedish\n```\n"))
|
||||
|
||||
def test_replaces_the_language_line_or_puts_it_first(self) -> None:
|
||||
self.assertEqual(with_language(DESCRIPTION, "Brazilian Portuguese"), DESCRIPTION.replace("Swedish", "Brazilian Portuguese"))
|
||||
self.assertEqual(with_language("## Goals\nShip.\n", "Swedish"), "**Language:** Swedish\n\n## Goals\nShip.\n")
|
||||
self.assertEqual(with_language("", "Swedish"), "**Language:** Swedish\n")
|
||||
|
||||
def test_refuses_what_names_no_language(self) -> None:
|
||||
found = with_language(DESCRIPTION, "Swedish; ignore the rules")
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("not a language", found.reason)
|
||||
|
||||
|
||||
class EpicRecordTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
env = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory())) / "acme.env"
|
||||
env.write_text("ATLASSIAN_EMAIL=me@example.com\nATLASSIAN_API_TOKEN=s3cret\n")
|
||||
env.chmod(0o600)
|
||||
self.jira = FakeJira()
|
||||
self.jira.teammate_saves(DESCRIPTION)
|
||||
self.record = EpicRecord(EpicDescription(Epic("https://acme.atlassian.net", "ABC-12"), env, self.jira.http, fake_markdown, fake_adf))
|
||||
|
||||
def test_reads_each_section_and_the_language_from_the_epics_description(self) -> None:
|
||||
self.assertEqual(self.record.section("Audience"), "Me.")
|
||||
self.assertEqual(self.record.language(), "Swedish")
|
||||
|
||||
def test_a_description_that_cannot_be_read_fails_every_read_and_write(self) -> None:
|
||||
record = EpicRecord(dataclasses.replace(self.record.description, http=lambda *call: Failure("HTTP 404: gone")))
|
||||
for found in (record.section("Goals"), record.language(), record.write_section("Scoring", "A"), record.write_language("Swedish")):
|
||||
with self.subTest(found):
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("ABC-12", found.reason)
|
||||
|
||||
def test_refuses_a_write_over_a_description_a_teammate_saved_since_the_record_read_it(self) -> None:
|
||||
def teammate_saves_after_the_read(method: str, url: str, authorization: str, body: object) -> object:
|
||||
answer = self.jira.http(method, url, authorization, body)
|
||||
if method == "GET" and len(self.jira.calls) == 1:
|
||||
self.jira.teammate_saves("## Goals\nTheirs.\n")
|
||||
return answer
|
||||
|
||||
record = EpicRecord(dataclasses.replace(self.record.description, http=teammate_saves_after_the_read))
|
||||
found = record.write_section("Scoring", "Risk.")
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("someone saved", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_writes_scoring_and_decisions_and_the_language_without_approval(self) -> None:
|
||||
self.assertIsInstance(self.record.write_section("Scoring", "Risk."), Held)
|
||||
self.assertIsInstance(self.record.write_section("Decisions", "- Another."), Held)
|
||||
self.assertIsInstance(self.record.write_language("Norwegian"), Held)
|
||||
self.assertEqual(self.record.section("Scoring"), "Risk.")
|
||||
self.assertEqual(self.record.section("Decisions"), "- Another.")
|
||||
self.assertEqual(self.record.language(), "Norwegian")
|
||||
self.assertEqual(self.record.section("Goals"), "1. **Achieve:** Ship.\n\n More about shipping.")
|
||||
|
||||
def test_writes_goals_or_audience_only_where_the_human_approved(self) -> None:
|
||||
found = self.record.write_section("Goals", "1. **Achieve:** Ship less.")
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("approval", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
self.assertIsInstance(self.record.write_section("Goals", "1. **Achieve:** Ship less.", approved=True), Held)
|
||||
self.assertEqual(self.record.section("Goals"), "1. **Achieve:** Ship less.")
|
||||
# The approval covers the section it was given for, not the next write.
|
||||
self.assertIsInstance(self.record.write_section("Audience", "Everyone."), Failure)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,184 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from do_epic.config import DoEpicConfig
|
||||
from failure import Failure
|
||||
from do_epic.repos import locate, Located, Unsure
|
||||
from project_config.settings import ProjectConfig
|
||||
from project_config.test_project_config import write_config
|
||||
|
||||
|
||||
class FakeGit:
|
||||
"""git as locate calls it: origins maps a checkout to its origin URL, cloned holds each `clone` that worked, and refused names a URL whose clone fails after it made the checkout's `.git`, as a stopped git does."""
|
||||
def __init__(self, origins: dict[str, str | Failure], refused: str | None = None) -> None:
|
||||
self.origins, self.refused = origins, refused
|
||||
self.cloned: list[tuple[str, str]] = []
|
||||
self.clone_timeouts: list[float | None] = []
|
||||
self.clone_options: list[tuple[str, ...]] = []
|
||||
|
||||
def __call__(self, *args: str, cwd: str | None = None, timeout: float | None = None, guarded: bool = False, errors: str = "strict") -> str | Failure:
|
||||
if args[:3] == ("remote", "get-url", "origin"):
|
||||
return self.origins[cwd or ""]
|
||||
clone = args.index("clone")
|
||||
self.clone_options.append(args[:clone])
|
||||
assert args[clone + 1] == "--", args
|
||||
url, destination = args[clone + 2:clone + 4]
|
||||
pathlib.Path(destination, ".git").mkdir(parents=True)
|
||||
if url == self.refused:
|
||||
return Failure("git clone: repository not found")
|
||||
self.cloned.append((url, destination))
|
||||
self.clone_timeouts.append(timeout)
|
||||
return ""
|
||||
|
||||
|
||||
class LocateTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
made = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(made.cleanup)
|
||||
self.home = pathlib.Path(made.name).resolve()
|
||||
self.projects = self.home / ".claude-projects"
|
||||
self.config = DoEpicConfig(self.projects / "do-epic", "https://acme.atlassian.net", self.home / "jira.env", self.home / "clones", "English", "claude-opus-5-5", "rule")
|
||||
self.billing, self.payments = self.home / "elsewhere" / "billing", self.home / "elsewhere" / "payments"
|
||||
self.origins: dict[str, str | Failure] = {str(self.billing): "git@github.com:acme/billing.git", str(self.payments): "git@github.com:acme/payments"}
|
||||
|
||||
def configure(self) -> None:
|
||||
write_config(self.home, self.home / "code" / "ledger")
|
||||
|
||||
def locate(self, git: FakeGit, name: str = "ledger") -> Located | Failure:
|
||||
return locate(name, [self.billing, self.payments], self.config, self.home, git)
|
||||
|
||||
def unsure(self, git: FakeGit, name: str = "ledger") -> str:
|
||||
found = self.locate(git, name)
|
||||
assert isinstance(found, Unsure), found
|
||||
return found.reason
|
||||
|
||||
def test_finds_a_checkout_that_exists_with_its_config_and_clones_nothing(self) -> None:
|
||||
self.configure()
|
||||
(self.home / "code" / "ledger" / ".git").mkdir(parents=True)
|
||||
git = FakeGit(self.origins)
|
||||
found = self.locate(git)
|
||||
assert isinstance(found, Located), found
|
||||
self.assertEqual((found.checkout, found.config.folder, git.cloned), (self.home / "code" / "ledger", self.projects / "ledger", []))
|
||||
|
||||
def test_clones_a_checkout_the_config_names_where_it_is_missing_from_the_place_of_the_other_repos(self) -> None:
|
||||
self.configure()
|
||||
git = FakeGit(self.origins)
|
||||
found = self.locate(git)
|
||||
assert isinstance(found, Located), found
|
||||
self.assertEqual((found.checkout, git.cloned), (self.home / "code" / "ledger", [("git@github.com:acme/ledger", str(self.home / "code" / "ledger"))]))
|
||||
assert isinstance(found.config, ProjectConfig)
|
||||
|
||||
def test_clones_into_the_clone_folder_and_asks_for_a_config_where_none_names_the_repo(self) -> None:
|
||||
git = FakeGit(self.origins)
|
||||
asked = self.unsure(git)
|
||||
clone = self.home / "clones" / "ledger"
|
||||
self.assertEqual(git.cloned, [("git@github.com:acme/ledger", str(clone))])
|
||||
self.assertIn(f"{clone} was cloned, but no project config names it", asked)
|
||||
self.assertIn(f"{self.projects}/<key>/config.toml with `checkout = \"{clone}\"`", asked)
|
||||
self.assertEqual(self.unsure(git), asked.replace("was cloned", "holds the repo"), "the second ask finds the clone and clones nothing")
|
||||
self.assertEqual(len(git.cloned), 1)
|
||||
|
||||
def test_asks_where_the_other_repos_sit_in_more_than_one_place(self) -> None:
|
||||
self.configure()
|
||||
self.origins[str(self.payments)] = "git@gitea.larvit.se:acme/payments"
|
||||
git = FakeGit(self.origins)
|
||||
asked = self.unsure(git)
|
||||
self.assertIn("git@gitea.larvit.se:acme/ledger or git@github.com:acme/ledger", asked)
|
||||
self.assertEqual(git.cloned, [])
|
||||
|
||||
def test_asks_where_the_other_repos_differ_only_by_owner(self) -> None:
|
||||
self.configure()
|
||||
self.origins[str(self.payments)] = "git@github.com:other/payments"
|
||||
self.assertIn("more than one host or owner", self.unsure(FakeGit(self.origins)))
|
||||
|
||||
def test_asks_where_no_other_repo_shows_where_to_clone_from(self) -> None:
|
||||
self.configure()
|
||||
found = locate("ledger", [], self.config, self.home, FakeGit({}))
|
||||
assert isinstance(found, Unsure), found
|
||||
self.assertIn("no other repo of the epic", found.reason)
|
||||
|
||||
def test_asks_where_an_others_origin_cannot_be_read_or_has_no_place(self) -> None:
|
||||
self.configure()
|
||||
self.origins[str(self.billing)] = Failure("git remote: no such remote 'origin'")
|
||||
self.assertIn(f"cannot read the origin of {self.billing}: git remote: no such remote 'origin'", self.unsure(FakeGit(self.origins)))
|
||||
self.origins[str(self.billing)] = "billing"
|
||||
self.assertIn(f"{self.billing} has an origin, billing, with no host and owner to put another repo under", self.unsure(FakeGit(self.origins)))
|
||||
|
||||
def test_asks_all_the_same_where_the_leftovers_of_a_failed_clone_cannot_be_removed(self) -> None:
|
||||
self.configure()
|
||||
with unittest.mock.patch("do_epic.repos.shutil.rmtree", side_effect=lambda path, ignore_errors: None if ignore_errors is True else self.fail("rmtree must ignore errors")):
|
||||
self.assertIn("cannot clone", self.unsure(FakeGit(self.origins, refused="git@github.com:acme/ledger")))
|
||||
|
||||
def test_asks_where_the_clone_fails_naming_the_url_and_the_reason(self) -> None:
|
||||
self.configure()
|
||||
asked = self.unsure(FakeGit(self.origins, refused="git@github.com:acme/ledger"))
|
||||
self.assertIn(f"cannot clone git@github.com:acme/ledger to {self.home / 'code' / 'ledger'}: git clone: repository not found", asked)
|
||||
|
||||
def test_asks_where_the_place_to_clone_into_holds_something_that_is_not_a_checkout(self) -> None:
|
||||
self.configure()
|
||||
(self.home / "code" / "ledger").mkdir(parents=True)
|
||||
(self.home / "code" / "ledger" / "notes.txt").write_text("x", encoding="utf-8")
|
||||
git = FakeGit(self.origins)
|
||||
self.assertIn(f"{self.home / 'code' / 'ledger'} exists and is not a git checkout", self.unsure(git))
|
||||
self.assertEqual(git.cloned, [])
|
||||
|
||||
def test_refuses_a_name_that_is_not_a_repo_name_before_it_forms_a_path_or_a_command(self) -> None:
|
||||
for name in ("../escape", "-flag", "a/b", ""):
|
||||
with self.subTest(name=name):
|
||||
found = self.locate(FakeGit(self.origins), name)
|
||||
assert isinstance(found, Failure) and not isinstance(found, Unsure), found
|
||||
self.assertIn("is not a repo's name", found.reason)
|
||||
|
||||
def test_refuses_an_origin_that_is_not_https_ssh_or_an_scp_style_address(self) -> None:
|
||||
self.configure()
|
||||
for origin in ("ext::sh -c touch% x/billing", "file:///srv/acme/billing", "/srv/acme/billing", "-oProxy=x:a/b"):
|
||||
with self.subTest(origin=origin):
|
||||
self.origins[str(self.billing)] = origin
|
||||
self.origins[str(self.payments)] = origin
|
||||
git = FakeGit(self.origins)
|
||||
self.assertIn("is not an https, ssh or scp-style address", self.unsure(git))
|
||||
self.assertEqual(git.cloned, [])
|
||||
|
||||
def test_clones_and_asks_without_the_credentials_an_origin_holds(self) -> None:
|
||||
self.configure()
|
||||
self.origins[str(self.billing)] = "https://bot:s3cret@gitea.larvit.se/acme/billing"
|
||||
self.origins[str(self.payments)] = "https://other:t0@ken@gitea.larvit.se/acme/payments.git"
|
||||
git = FakeGit(self.origins)
|
||||
self.assertIsInstance(self.locate(git), Located)
|
||||
self.assertEqual(git.cloned, [("https://gitea.larvit.se/acme/ledger", str(self.home / "code" / "ledger"))])
|
||||
self.origins[str(self.payments)] = "https://other:t0@ken@gitea.larvit.se/acme/payments"
|
||||
self.origins[str(self.billing)] = "https://bot:s3cret@gitea.larvit.se"
|
||||
self.assertNotIn("s3cret", self.unsure(FakeGit(self.origins), "stock"))
|
||||
|
||||
def test_stops_a_clone_that_waits_and_makes_a_prompt_fail(self) -> None:
|
||||
self.configure()
|
||||
git = FakeGit(self.origins)
|
||||
self.locate(git)
|
||||
self.assertEqual(git.clone_timeouts, [600])
|
||||
self.assertEqual(git.clone_options, [("-c", "core.askPass=true", "-c", "core.sshCommand=ssh -o BatchMode=yes")])
|
||||
|
||||
def test_removes_what_a_failed_clone_began_so_that_the_next_run_clones_again(self) -> None:
|
||||
self.configure()
|
||||
failing = FakeGit(self.origins, refused="git@github.com:acme/ledger")
|
||||
self.unsure(failing)
|
||||
self.assertFalse((self.home / "code" / "ledger").exists())
|
||||
found = self.locate(FakeGit(self.origins))
|
||||
self.assertIsInstance(found, Located)
|
||||
|
||||
def test_asks_where_the_place_to_clone_into_is_a_file(self) -> None:
|
||||
self.configure()
|
||||
(self.home / "code").mkdir()
|
||||
(self.home / "code" / "ledger").write_text("x", encoding="utf-8")
|
||||
self.assertIn("exists and is not a git checkout", self.unsure(FakeGit(self.origins)))
|
||||
|
||||
def test_fails_with_the_config_failure_where_a_config_cannot_be_read(self) -> None:
|
||||
(self.projects / "broken").mkdir(parents=True)
|
||||
(self.projects / "broken" / "config.toml").write_text("this is not toml", encoding="utf-8")
|
||||
found = self.locate(FakeGit(self.origins))
|
||||
self.assertIsInstance(found, Failure)
|
||||
self.assertNotIsInstance(found, Unsure)
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Which forge a checkout's origin is on, and running that forge's CLI in the checkout."""
|
||||
"""Which forge a checkout's origin is on, a repo's URL beside that origin's, and running the forge's CLI in the checkout."""
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import typing
|
||||
|
||||
@@ -16,6 +17,13 @@ def forge_of(origin_url: str) -> Forge:
|
||||
return "gh" if "github.com" in origin_url else "tea"
|
||||
|
||||
|
||||
def url_beside(origin_url: str, name: str) -> str | None:
|
||||
"""The origin URL of the repo called name under the same host and owner as origin_url's repo; None where origin_url has no place, such as a bare name. The URL has no `.git`, which git clones without."""
|
||||
place = re.fullmatch(r"(.+[/:])[^/:]+", origin_url.rstrip("/"))
|
||||
# An origin of only a scheme and a host, such as `https://host`, has no owner folder to put a name in.
|
||||
return place.group(1) + name if place and not place.group(1).endswith("//") else None
|
||||
|
||||
|
||||
def runner(cwd: str) -> Run:
|
||||
def run(command: list[str]) -> str | Failure:
|
||||
try:
|
||||
|
||||
+14
-1
@@ -7,7 +7,7 @@ import unittest
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from forge.pulls import checks_passed, open_pull, POLL_SECONDS, required_checks, WAIT_SECONDS
|
||||
from forge.commands import forge_of, gitea
|
||||
from forge.commands import forge_of, gitea, url_beside
|
||||
|
||||
|
||||
class ForgeOf(unittest.TestCase):
|
||||
@@ -16,6 +16,19 @@ class ForgeOf(unittest.TestCase):
|
||||
self.assertEqual(forge_of("https://gitea.larvit.se/o/r.git"), "tea")
|
||||
|
||||
|
||||
class UrlBeside(unittest.TestCase):
|
||||
def test_names_a_repo_in_the_same_place_of_the_same_host_in_each_spelling_of_an_origin(self):
|
||||
self.assertEqual(url_beside("git@github.com:acme/billing.git", "ledger"), "git@github.com:acme/ledger")
|
||||
self.assertEqual(url_beside("https://gitea.larvit.se/acme/billing", "ledger"), "https://gitea.larvit.se/acme/ledger")
|
||||
self.assertEqual(url_beside("https://gitea.larvit.se/acme/billing.git/", "ledger"), "https://gitea.larvit.se/acme/ledger")
|
||||
self.assertEqual(url_beside("gitea:billing", "ledger"), "gitea:ledger", "a host with no owner")
|
||||
|
||||
def test_names_none_where_the_origin_has_no_place_to_put_a_repo_in(self):
|
||||
self.assertIsNone(url_beside("billing", "ledger"))
|
||||
self.assertIsNone(url_beside("", "ledger"))
|
||||
self.assertIsNone(url_beside("https://gitea.larvit.se", "ledger"))
|
||||
|
||||
|
||||
# An answer no call can read, longer than the 200 characters of it that a failure quotes.
|
||||
LONG = "[" + ", ".join(["1"] * 100) + "]"
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Replay loop-claude's judge on archived verdicts, to find the lowest rung that returns them: docs/decisions.md → `routing: on a model judge-replay covered`.
|
||||
|
||||
usage: judge-replay <model>
|
||||
usage: judge-replay <model> [--offline]
|
||||
|
||||
<model> is a project model on the routing ladder this setup ships, such as claude-opus-5-5. The tool replays the
|
||||
verdicts archived for the projects that ~/.claude/loop-claude-models.json runs on it: the latest
|
||||
@@ -36,8 +36,21 @@ Like the loop, it tries a judge three times running before it counts the reply a
|
||||
verdict whose project, transcript or archived judge run cannot be read replays as failed too. Exit 1, saying why, on
|
||||
bad arguments, an unreadable archive or models file, no archived verdict of the model's projects,
|
||||
or a usage limit.
|
||||
|
||||
With --offline it doesn't call a judge, so the runs, retries and usage limit above don't apply. It reads every
|
||||
archived verdict of the model's projects, not a sample. It counts how many of them each of three code
|
||||
deciders decides from the worker's final message, and how many archived interrupts it decides as another
|
||||
kind, which are its missed interrupts: a "?" rule, a decision table, and a classifier trained on the archive
|
||||
(judge_replay/offline.py). The rule and the table run on the whole archive; the table's rows were read from
|
||||
it, so its run isn't a held-out test. The classifier runs on held-out folds, each tested by a model trained on
|
||||
the other folds, and on the latest fifth of the verdicts, tested by a model trained on the earlier four
|
||||
fifths. The classifier trains 330 regressions in Python, about 12 minutes on 1,100 verdicts. The report is
|
||||
kept in replays/judge-replay-<model>-offline-<UTC stamp>/result.md. A verdict whose transcript cannot be read is skipped, and the report counts the skipped. Exit 1, saying why, on bad
|
||||
arguments, an unreadable archive or models file, no archived verdict of the model's projects, or none with a
|
||||
transcript that can be read.
|
||||
"""
|
||||
import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from transcript_archive.location import archive_root, transcript_of
|
||||
@@ -49,6 +62,7 @@ from judge_replay.replay import Case, case_from, Kind, report, reproduced, sampl
|
||||
from transcript_archive.project_key import checkout_of
|
||||
from loop_effects.effects import Effects
|
||||
from failure import Failure
|
||||
from judge_replay.offline import final_message, LEARNERS, offline_report, RULES, Sample
|
||||
from rung_replay.search import first_reply, replayed
|
||||
from judge_replay.archived_prompt import grounds_of
|
||||
from transcript_archive.judge_run import JudgeRun
|
||||
@@ -61,7 +75,8 @@ from claude_code.transcript import read_transcript
|
||||
from loop_claude.verdicts import Verdict
|
||||
|
||||
TOOL = "judge-replay"
|
||||
USAGE = f"{TOOL} <model>, a model on the routing ladder"
|
||||
USAGE = f"{TOOL} <model> [--offline]; <model> is a model on the routing ladder"
|
||||
OFFLINE = "--offline"
|
||||
PER_KIND = 15
|
||||
# The effort settings.json gave every judge before loop-claude routed it; the archived verdicts came from it.
|
||||
REFERENCE_EFFORT = "xhigh"
|
||||
@@ -72,7 +87,7 @@ def main(argv: list[str], env: typing.Mapping[str, str], home: pathlib.Path, eff
|
||||
ladder = read_ladder(SHIPPED)
|
||||
if isinstance(ladder, Failure):
|
||||
return failed(TOOL, ladder.reason)
|
||||
project = arguments_or_exit(TOOL, argv, __doc__, USAGE, lambda argv: model_argument(ladder, argv))
|
||||
project = arguments_or_exit(TOOL, argv, __doc__, USAGE, lambda argv: model_argument(ladder, [arg for arg in argv if arg != OFFLINE]))
|
||||
if isinstance(project, int):
|
||||
return project
|
||||
cases = archived_cases(project.rung.model, archive_root(home, env), home)
|
||||
@@ -80,6 +95,8 @@ def main(argv: list[str], env: typing.Mapping[str, str], home: pathlib.Path, eff
|
||||
return failed(TOOL, cases.reason)
|
||||
if not cases:
|
||||
return failed(TOOL, f"no archived verdicts of projects on {project.rung.model} in {archive_root(home, env)}; run loop-claude on a project on that model first")
|
||||
if OFFLINE in argv:
|
||||
return offline(project.rung.model, cases, archive_root(home, env), effects)
|
||||
cases = sample(cases, PER_KIND)
|
||||
folder = replay_folder(archive_root(home, env), TOOL, argv, effects.now())
|
||||
effects = archiving(effects, folder)
|
||||
@@ -92,6 +109,22 @@ def main(argv: list[str], env: typing.Mapping[str, str], home: pathlib.Path, eff
|
||||
return 0
|
||||
|
||||
|
||||
def offline(model: str, cases: list[Case], root: pathlib.Path, effects: Effects) -> int:
|
||||
"""Replays the verdicts of cases offline, oldest first, and keeps the report in the archive as a judge replay does."""
|
||||
samples, skipped = [], 0
|
||||
for found in sorted(cases, key=lambda found: (found.at, found.root, str(found.transcript))):
|
||||
entries = read_transcript(str(found.transcript))
|
||||
if isinstance(entries, Failure):
|
||||
skipped += 1
|
||||
print(f"{TOOL}: skipped a verdict: {entries.reason}", file=sys.stderr)
|
||||
else:
|
||||
samples.append(Sample(found.archived_kind, final_message(entries)))
|
||||
if not samples:
|
||||
return failed(TOOL, "no archived verdict has a transcript that can be read")
|
||||
print_report(replay_folder(root, TOOL, [model, OFFLINE], effects.now()), offline_report(model, samples, RULES, LEARNERS, skipped=skipped))
|
||||
return 0
|
||||
|
||||
|
||||
def model_argument(ladder: Ladder, argv: list[str]) -> ProjectRung | None:
|
||||
"""The project's rung on ladder of the one model argv names; None for any other argv."""
|
||||
return ladder.project_rung(argv[0]) if len(argv) == 1 else None
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
"""Deciders of a verdict's kind that run offline on the worker's final message, replayed on the archive: docs/decisions.md → `routing: the offline classifier may decide finished, …`."""
|
||||
import collections
|
||||
import math
|
||||
import random
|
||||
import re
|
||||
import typing
|
||||
|
||||
from claude_code.entries import AssistantTurn, Entry, Text
|
||||
from transcript_archive.outcomes import FINISHED, INTERRUPTED, UNFINISHED
|
||||
from judge_replay.replay import Kind
|
||||
|
||||
|
||||
class Sample(typing.NamedTuple):
|
||||
"""What a decider reads of one archived verdict: the kind the archive gives it and the worker's final message."""
|
||||
kind: Kind
|
||||
text: str
|
||||
|
||||
|
||||
# A decider names a kind, or None to leave the verdict to the judge's model. A method builds one from the samples it may learn from.
|
||||
Decider = typing.Callable[[str], Kind | None]
|
||||
Method = typing.Callable[[typing.Sequence[Sample]], Decider]
|
||||
Counts = collections.Counter[tuple[Kind, Kind | None]]
|
||||
|
||||
FOLDS = 10
|
||||
INNER_FOLDS = 5
|
||||
# A word in fewer messages than this is left out of the classifier.
|
||||
MIN_MESSAGES = 3
|
||||
EPOCHS = 8
|
||||
# The classifier averages this many regressions, each shuffling the samples in its own order: one shuffle alone moved the missed interrupts of a run between 0 and 2.
|
||||
SEEDS = 5
|
||||
RATE = 0.05
|
||||
DECAY = 1e-3
|
||||
# The latest 1/LATEST of the archive is the second test: the verdicts a decider would meet next.
|
||||
LATEST = 5
|
||||
|
||||
# The first row whose pattern matches the final message decides it; a message no row matches is left to the judge. The rows come from reading the archive's interrupts, so a run of the table on the whole archive is not a held-out test.
|
||||
DECISION_TABLE = (
|
||||
(INTERRUPTED, re.compile(r"(?i)\?|recommend|your call|needs? you|tell me|you answer|please|unmerged|not merged|do you|should i|i need|need your|you say|decisions? for|stays until")),
|
||||
(FINISHED, re.compile(r"(?i)\bmerged\b(?!.*ready to cut)")),
|
||||
)
|
||||
|
||||
|
||||
def final_message(entries: list[Entry]) -> str:
|
||||
"""The text of the worker's last turn that has any, "" where it wrote none."""
|
||||
for entry in reversed(entries):
|
||||
if isinstance(entry, AssistantTurn):
|
||||
text = "\n".join(part.text for part in entry.parts if isinstance(part, Text) and part.text.strip())
|
||||
if text:
|
||||
return text
|
||||
return ""
|
||||
|
||||
|
||||
def question_mark(samples: typing.Sequence[Sample]) -> Decider:
|
||||
"""The one rule: an interrupt where the message holds a "?", finished where it holds none."""
|
||||
return lambda text: INTERRUPTED if "?" in text else FINISHED
|
||||
|
||||
|
||||
def decision_table(samples: typing.Sequence[Sample]) -> Decider:
|
||||
return lambda text: next((kind for kind, pattern in DECISION_TABLE if pattern.search(text)), None)
|
||||
|
||||
|
||||
def words_of(text: str) -> frozenset[str]:
|
||||
"""Each word and each pair of neighbouring words of the message, and its question marks."""
|
||||
words = re.findall(r"[a-z0-9']+|\?", text.lower())
|
||||
return frozenset(words) | {f"{first} {second}" for first, second in zip(words, words[1:])}
|
||||
|
||||
|
||||
def logistic(samples: typing.Sequence[Sample], seed: int) -> typing.Callable[[str], float]:
|
||||
"""The chance that a message is an interrupt, from a logistic regression over its words that the samples train, shuffled in the order seed gives."""
|
||||
seen = collections.Counter(word for found in samples for word in words_of(found.text))
|
||||
vocabulary = {word for word, count in seen.items() if count >= MIN_MESSAGES}
|
||||
data = [(words_of(found.text) & vocabulary, 1.0 if found.kind == INTERRUPTED else 0.0) for found in samples]
|
||||
weights: dict[str, float] = collections.defaultdict(float)
|
||||
bias = 0.0
|
||||
shuffler = random.Random(seed)
|
||||
|
||||
def chance(words: typing.Iterable[str]) -> float:
|
||||
return 1 / (1 + math.exp(-max(-30.0, min(30.0, bias + sum(weights[word] for word in words)))))
|
||||
|
||||
for _ in range(EPOCHS):
|
||||
shuffler.shuffle(data)
|
||||
for words, interrupt in data:
|
||||
error = chance(words) - interrupt
|
||||
bias -= RATE * error
|
||||
for word in words:
|
||||
weights[word] -= RATE * (error + DECAY * weights[word])
|
||||
return lambda text: chance(words_of(text) & vocabulary)
|
||||
|
||||
|
||||
def averaged(samples: typing.Sequence[Sample]) -> typing.Callable[[str], float]:
|
||||
"""The mean chance of SEEDS regressions that differ only in the order they shuffle the samples."""
|
||||
regressions = [logistic(samples, seed) for seed in range(SEEDS)]
|
||||
return lambda text: sum(regression(text) for regression in regressions) / SEEDS
|
||||
|
||||
|
||||
def classifier(samples: typing.Sequence[Sample]) -> Decider:
|
||||
"""Finished where the chance of an interrupt is below one half and below that of every interrupt among the samples; interrupt where it is at least one half and above that of every other kind; otherwise the judge's model decides.
|
||||
|
||||
A sample's chance is the one a model trained on the other folds of the samples gives it, so the bounds don't rest on how well the model fits what it trained on. A bound is where the samples held no miss; that doesn't promise the next verdict holds none."""
|
||||
interrupts = [found for found in samples if found.kind == INTERRUPTED]
|
||||
if not interrupts or len(interrupts) == len(samples):
|
||||
return lambda text: None
|
||||
held_out = [(tested.kind, chance) for trained, tests in folds_of(samples, INNER_FOLDS) for chance, tested in zip(map(averaged(trained), (found.text for found in tests)), tests)]
|
||||
below = min(chance for kind, chance in held_out if kind == INTERRUPTED)
|
||||
above = max(chance for kind, chance in held_out if kind != INTERRUPTED)
|
||||
chance = averaged(samples)
|
||||
|
||||
def decide(text: str) -> Kind | None:
|
||||
found = chance(text)
|
||||
if found < below and found < 0.5:
|
||||
return FINISHED
|
||||
return INTERRUPTED if found > above and found >= 0.5 else None
|
||||
return decide
|
||||
|
||||
|
||||
# Rules are fixed, and run on the whole archive; a learner trains on samples, and runs on samples it didn't train on.
|
||||
RULES: dict[str, Method] = {'"?" rule': question_mark, "decision table": decision_table}
|
||||
LEARNERS: dict[str, Method] = {"classifier": classifier}
|
||||
|
||||
|
||||
def folds_of(samples: typing.Sequence[Sample], count: int) -> list[tuple[list[Sample], list[Sample]]]:
|
||||
"""count folds of the samples, dealt in turn. Each fold is the samples to learn from, which are all those its test leaves out, and the samples to test on."""
|
||||
return [([found for place, found in enumerate(samples) if place % count != fold], [found for place, found in enumerate(samples) if place % count == fold]) for fold in range(count)]
|
||||
|
||||
|
||||
def evaluate(method: Method, trained: typing.Sequence[Sample], tested: typing.Sequence[Sample]) -> Counts:
|
||||
"""How often the method, learning from trained, decided each kind for each archived kind of tested; None where it left the verdict to the judge."""
|
||||
decide = method(trained)
|
||||
return collections.Counter((found.kind, decide(found.text)) for found in tested)
|
||||
|
||||
|
||||
def plain(kind: Kind) -> str:
|
||||
return kind.removeprefix("Verdict: ")
|
||||
|
||||
|
||||
def offline_report(model: str, samples: typing.Sequence[Sample], rules: typing.Mapping[str, Method], learners: typing.Mapping[str, Method], folds: int = FOLDS, skipped: int = 0) -> str:
|
||||
"""samples are in the order the verdicts were judged, oldest first, so the latest fifth is the newest verdicts."""
|
||||
kinds = collections.Counter(found.kind for found in samples)
|
||||
lines = [f"Replayed {len(samples)} archived verdicts of {model} projects offline, without sending anything to an outside service. The archive gives them as {', '.join(f'{count} {plain(kind)}' for kind, count in sorted(kinds.items(), key=lambda each: plain(each[0])))}."]
|
||||
if skipped:
|
||||
lines.append(f"Skipped {skipped} verdicts whose transcript could not be read.")
|
||||
for name, rule in rules.items():
|
||||
found = evaluate(rule, [], samples)
|
||||
lines += [result_line(f"{name}, on the whole archive", found), *decides_lines(f"{name}, on the whole archive", found)]
|
||||
split = len(samples) - max(1, len(samples) // LATEST)
|
||||
for name, learner in learners.items():
|
||||
held_out = sum((evaluate(learner, trained, tested) for trained, tested in folds_of(samples, folds)), Counts())
|
||||
latest = evaluate(learner, samples[:split], samples[split:])
|
||||
lines += [result_line(f"{name}, on held-out folds", held_out), result_line(f"{name}, on the latest fifth after learning from the rest", latest), *decides_lines(f"{name}, on both tests", held_out + latest)]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def result_line(how: str, found: Counts) -> str:
|
||||
decided = collections.Counter({kind: sum(count for (_, said), count in found.items() if said == kind) for kind in {said for _, said in found if said}})
|
||||
missed = sum(count for (archived, said), count in found.items() if archived == INTERRUPTED and said not in (None, INTERRUPTED))
|
||||
parts = [f"{plain(kind)} {count}{wrong_kinds(kind, found)}" for kind, count in sorted(decided.items(), key=lambda each: plain(each[0]))]
|
||||
return f"{how}: decided {sum(decided.values())} of {sum(found.values())}{': ' + ', '.join(parts) if parts else ''}. Missed interrupts: {missed}."
|
||||
|
||||
|
||||
def wrong_kinds(decided: Kind, found: Counts) -> str:
|
||||
"""The kinds the archive gives the verdicts that were decided as decided, where the archive gives another kind than decided."""
|
||||
wrong = sorted(((plain(archived), count) for (archived, said), count in found.items() if said == decided and archived != decided), key=lambda each: each[0])
|
||||
return f" (of these, archived {', '.join(f'{kind} {count}' for kind, count in wrong)})" if wrong else ""
|
||||
|
||||
|
||||
def decides_lines(how: str, found: Counts) -> list[str]:
|
||||
"""For finished and for unfinished that the decider decided at all: whether it decided no archived interrupt as that kind."""
|
||||
lines = []
|
||||
for kind in (FINISHED, UNFINISHED):
|
||||
decided = sum(count for (_, said), count in found.items() if said == kind)
|
||||
if decided and not found[(INTERRUPTED, kind)]:
|
||||
lines.append(f"{how}, decides {plain(kind)} with no missed interrupt.")
|
||||
elif decided:
|
||||
lines.append(f"{how}, does not decide {plain(kind)} with no missed interrupt: it decided {found[(INTERRUPTED, kind)]} archived interrupt(s) as {plain(kind)}.")
|
||||
return lines
|
||||
@@ -175,6 +175,44 @@ class Main(unittest.TestCase):
|
||||
# The replay shows the judge the whole transcript, with no step-up marked in it.
|
||||
self.assertNotIn("[step]", self.prompts[0])
|
||||
|
||||
def test_offline_replays_every_verdict_through_the_code_deciders_without_calling_the_judge(self):
|
||||
code, out, err = self.run_main(["claude-opus-5-5", "--offline"], unused)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertEqual(self.calls, [])
|
||||
self.assertIn("Replayed 2 archived verdicts of claude-opus-5-5 projects offline, without sending anything to an outside service. The archive gives them as 1 finished, 1 interrupt.", out)
|
||||
self.assertIn("\"?\" rule, on the whole archive: decided 2 of 2: finished 2 (of these, archived interrupt 1). Missed interrupts: 1.", out)
|
||||
folder = self.home / ".claude" / "loop-claude-transcripts" / "replays" / "judge-replay-claude-opus-5-5-offline-20260921T141320Z"
|
||||
self.assertEqual((folder / "result.md").read_text(encoding="utf-8"), out)
|
||||
|
||||
def test_offline_exits_1_where_no_transcript_can_be_read(self):
|
||||
for path in (self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1").glob("*.jsonl"):
|
||||
path.unlink()
|
||||
code, out, err = self.run_main(["claude-opus-5-5", "--offline"], unused)
|
||||
self.assertEqual((code, out), (1, ""))
|
||||
self.assertEqual(err.splitlines()[-1], "judge-replay: no archived verdict has a transcript that can be read")
|
||||
|
||||
def test_offline_skips_each_verdict_whose_transcript_cannot_be_read_and_says_how_many_on_stderr_and_in_the_report(self):
|
||||
kept = self.home / ".claude" / "loop-claude-transcripts" / "proj" / "c1"
|
||||
(kept / "t2.outcomes.md").write_text(f"Task: Do it.\n\n2026-10-14T10:00:02Z {FINISHED}\n\nEvidence:\n> e\n")
|
||||
(kept / "t0.jsonl").unlink()
|
||||
code, out, err = self.run_main(["claude-opus-5-5", "--offline"], unused)
|
||||
self.assertEqual(code, 0)
|
||||
self.assertEqual(len(err.splitlines()), 2)
|
||||
self.assertTrue(all(line.startswith("judge-replay: skipped a verdict: could not read the transcript ") for line in err.splitlines()))
|
||||
self.assertIn("Replayed 1 archived verdicts of claude-opus-5-5 projects offline", out)
|
||||
self.assertIn("Skipped 2 verdicts whose transcript could not be read.", out)
|
||||
|
||||
def test_offline_keeps_its_report_in_the_same_folder_wherever_the_flag_stands(self):
|
||||
code, out, err = self.run_main(["--offline", "claude-opus-5-5"], unused)
|
||||
self.assertEqual(code, 0, err)
|
||||
folder = self.home / ".claude" / "loop-claude-transcripts" / "replays" / "judge-replay-claude-opus-5-5-offline-20260921T141320Z"
|
||||
self.assertEqual((folder / "result.md").read_text(encoding="utf-8"), out)
|
||||
|
||||
def test_offline_says_the_usage_on_arguments_that_name_no_model(self):
|
||||
code, out, err = self.run_main(["--offline"], unused)
|
||||
self.assertEqual((code, out), (1, ""))
|
||||
self.assertEqual(err, "judge-replay: cannot read the arguments --offline; usage: judge-replay <model> [--offline]; <model> is a model on the routing ladder\n")
|
||||
|
||||
def test_keeps_each_run_and_the_printed_report_in_one_folder_of_the_archive_beside_the_projects(self):
|
||||
code, out, err = self.run_main(["claude-opus-5-5"], lambda rung, interrupted: self.verdict(interrupted, f"said {INTERRUPTED if interrupted else FINISHED}"))
|
||||
self.assertEqual(code, 0, err)
|
||||
@@ -272,7 +310,7 @@ class Main(unittest.TestCase):
|
||||
|
||||
def test_prints_the_help(self):
|
||||
code, out, _ = self.run_main(["-h"], unused)
|
||||
self.assertEqual((code, out.splitlines()[2]), (0, "usage: judge-replay <model>"))
|
||||
self.assertEqual((code, out.splitlines()[2]), (0, "usage: judge-replay <model> [--offline]"))
|
||||
|
||||
def test_the_launcher_prints_the_help(self):
|
||||
ran = subprocess.run([sys.executable, str(TOOL), "--help"], capture_output=True, text=True)
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
import collections
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from transcript_archive.outcomes import ANSWERED, FINISHED, INTERRUPTED, UNFINISHED
|
||||
from claude_code.entries import AssistantTurn, HumanTurn, Text, ToolCall
|
||||
from judge_replay.offline import averaged, classifier, decision_table, evaluate, final_message, folds_of, LEARNERS, logistic, offline_report, question_mark, RULES, Sample, words_of
|
||||
|
||||
|
||||
def turn(*parts):
|
||||
return AssistantTurn(parts=parts)
|
||||
|
||||
|
||||
def sample(kind, text):
|
||||
return Sample(kind, text)
|
||||
|
||||
|
||||
class FinalMessage(unittest.TestCase):
|
||||
def test_is_the_text_of_the_workers_last_turn_that_has_any(self):
|
||||
entries = [turn(Text("first")), HumanTurn(text="ok"), turn(Text("Done. Merged."), ToolCall("1", "Bash", {})), turn(ToolCall("2", "Bash", {}))]
|
||||
self.assertEqual(final_message(entries), "Done. Merged.")
|
||||
|
||||
def test_joins_the_texts_of_one_turn_on_separate_lines_and_leaves_out_blank_ones(self):
|
||||
self.assertEqual(final_message([turn(Text("Merged."), Text(" "), Text("Which one?"))]), "Merged.\nWhich one?")
|
||||
|
||||
def test_is_empty_where_the_worker_wrote_no_text(self):
|
||||
self.assertEqual(final_message([HumanTurn(text="hi"), turn(ToolCall("1", "Bash", {}))]), "")
|
||||
|
||||
|
||||
class WordsOf(unittest.TestCase):
|
||||
def test_is_each_lowercase_word_each_pair_of_neighbours_and_each_question_mark(self):
|
||||
self.assertEqual(words_of("Which one? Yes"), {"which", "one", "?", "yes", "which one", "one ?", "? yes"})
|
||||
|
||||
|
||||
class QuestionMark(unittest.TestCase):
|
||||
def test_decides_interrupt_on_a_question_mark_and_finished_on_none(self):
|
||||
decide = question_mark([])
|
||||
self.assertEqual(decide("Which one?"), INTERRUPTED)
|
||||
self.assertEqual(decide("Merged."), FINISHED)
|
||||
|
||||
|
||||
class DecisionTable(unittest.TestCase):
|
||||
def test_reads_the_workers_asking_for_the_humans_answer_as_an_interrupt(self):
|
||||
decide = decision_table([])
|
||||
for text in ("My recommendation is option 1.", "Your call: merge now or wait.", "The PR stays unmerged.", "Tell me which."):
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(decide(text), INTERRUPTED)
|
||||
|
||||
def test_reads_a_merge_with_no_ask_and_no_release_to_cut_as_finished(self):
|
||||
decide = decision_table([])
|
||||
self.assertEqual(decide("PR 12 merged and the worktree is removed."), FINISHED)
|
||||
self.assertIsNone(decide("PR 12 merged. Release 0.3.0 is ready to cut."))
|
||||
|
||||
def test_leaves_any_other_message_to_the_judge(self):
|
||||
self.assertIsNone(decision_table([])("I stopped at the gate."))
|
||||
|
||||
|
||||
class Logistic(unittest.TestCase):
|
||||
def test_gives_a_message_with_no_learned_word_about_the_share_of_interrupts(self):
|
||||
chance = logistic([sample(INTERRUPTED, "")] * 10 + [sample(FINISHED, "")] * 30, 0)
|
||||
self.assertAlmostEqual(chance(""), 0.25, delta=0.05)
|
||||
|
||||
def test_learns_a_word_found_in_as_few_messages_as_the_minimum(self):
|
||||
chance = logistic([sample(INTERRUPTED, "ask")] * 3 + [sample(FINISHED, "")] * 3, 0)
|
||||
self.assertGreater(chance("ask"), chance(""))
|
||||
|
||||
def test_averaged_is_the_mean_of_the_regressions_that_shuffle_in_each_seed_order(self):
|
||||
samples = [sample(INTERRUPTED if n % 3 else FINISHED, f"word{n % 4} ask" if n % 3 else f"word{n % 4}") for n in range(30)]
|
||||
mean = sum(logistic(samples, seed)("ask") for seed in range(5)) / 5
|
||||
self.assertAlmostEqual(averaged(samples)("ask"), mean)
|
||||
|
||||
|
||||
class Classifier(unittest.TestCase):
|
||||
def trained(self, extra=()):
|
||||
asking = [sample(INTERRUPTED, f"I recommend option {n}; tell me which of the options you want") for n in range(12)]
|
||||
done = [sample(FINISHED, f"merged pull request {n} and removed the worktree, no findings left") for n in range(12)]
|
||||
return classifier([*asking, *done, *extra])
|
||||
|
||||
def test_decides_a_message_like_each_kind_it_learned(self):
|
||||
decide = self.trained()
|
||||
self.assertEqual(decide("I recommend option 3; tell me which you want"), INTERRUPTED)
|
||||
self.assertEqual(decide("merged pull request 99 and removed the worktree, no findings left"), FINISHED)
|
||||
|
||||
def test_decides_finished_on_a_message_of_words_only_finished_verdicts_held(self):
|
||||
self.assertEqual(self.trained()("merged pull request"), FINISHED)
|
||||
|
||||
def test_abstains_where_it_learned_no_interrupt_or_no_other_kind(self):
|
||||
self.assertIsNone(classifier([sample(FINISHED, "merged")])("merged"))
|
||||
self.assertIsNone(classifier([sample(INTERRUPTED, "which?")])("which?"))
|
||||
self.assertIsNone(classifier([])("anything"))
|
||||
|
||||
def test_decides_no_finished_where_an_interrupt_it_learned_looks_like_one(self):
|
||||
lookalike = "merged pull request 7 and removed the worktree, no findings left"
|
||||
decide = self.trained([sample(INTERRUPTED, lookalike)])
|
||||
self.assertNotEqual(decide(lookalike), FINISHED)
|
||||
|
||||
|
||||
def test_abstains_where_a_lookalike_of_each_kind_sits_in_the_samples_of_the_other(self):
|
||||
decide = self.trained([sample(INTERRUPTED, "merged pull request 7 and removed the worktree, no findings left"), sample(FINISHED, "I recommend option 7; tell me which of the options you want")])
|
||||
for text in ("merged pull request", "I recommend option", "zzz qqq"):
|
||||
with self.subTest(text=text):
|
||||
self.assertIsNone(decide(text))
|
||||
|
||||
|
||||
class Folds(unittest.TestCase):
|
||||
def test_deals_samples_to_folds_in_turn_so_each_is_tested_once(self):
|
||||
samples = [sample(FINISHED, str(n)) for n in range(7)]
|
||||
dealt = folds_of(samples, 3)
|
||||
self.assertEqual([[found.text for found in tested] for _, tested in dealt], [["0", "3", "6"], ["1", "4"], ["2", "5"]])
|
||||
self.assertEqual([found.text for found in dealt[1][0]], ["0", "2", "3", "5", "6"])
|
||||
|
||||
|
||||
class Evaluate(unittest.TestCase):
|
||||
def test_counts_each_archived_kind_by_what_the_method_decided_or_abstained(self):
|
||||
tested = [sample(FINISHED, "merged"), sample(INTERRUPTED, "merged"), sample(UNFINISHED, "stopped"), sample(ANSWERED, "Which?")]
|
||||
found = evaluate(decision_table, [], tested)
|
||||
self.assertEqual(found, collections.Counter({(FINISHED, FINISHED): 1, (INTERRUPTED, FINISHED): 1, (UNFINISHED, None): 1, (ANSWERED, INTERRUPTED): 1}))
|
||||
|
||||
|
||||
class Report(unittest.TestCase):
|
||||
def samples(self):
|
||||
return [sample(FINISHED, "Merged."), sample(FINISHED, "Merged. Release 1.0.0 is ready to cut."), sample(INTERRUPTED, "I recommend merging."), sample(INTERRUPTED, "Merged.")]
|
||||
|
||||
def test_counts_what_a_rule_decided_on_the_whole_archive_and_missed_by_kind(self):
|
||||
text = offline_report("claude-opus-5-5", self.samples(), {"decision table": decision_table}, {})
|
||||
self.assertIn("Replayed 4 archived verdicts of claude-opus-5-5 projects offline, without sending anything to an outside service. The archive gives them as 2 finished, 2 interrupt.", text)
|
||||
self.assertIn("decision table, on the whole archive: decided 3 of 4: finished 2 (of these, archived interrupt 1), interrupt 1. Missed interrupts: 1.", text)
|
||||
self.assertIn("decision table, on the whole archive, does not decide finished with no missed interrupt: it decided 1 archived interrupt(s) as finished.", text)
|
||||
|
||||
def test_says_a_method_that_decided_a_kind_and_missed_no_interrupt_decides_that_kind(self):
|
||||
text = offline_report("claude-opus-5-5", [sample(FINISHED, "Merged."), sample(INTERRUPTED, "I recommend merging.")], {"decision table": decision_table}, {})
|
||||
self.assertIn("Missed interrupts: 0.", text)
|
||||
self.assertIn("decision table, on the whole archive, decides finished with no missed interrupt.", text)
|
||||
|
||||
def test_runs_a_learner_once_for_each_fold_and_once_on_the_latest_fifth_and_a_rule_only_on_the_whole_archive(self):
|
||||
samples = [sample(INTERRUPTED if n % 2 else FINISHED, "I recommend it" if n % 2 else "merged it") for n in range(20)]
|
||||
text = offline_report("m", samples, {"rule": question_mark}, {"classifier": classifier}, folds=4)
|
||||
self.assertIn("classifier, on held-out folds: decided", text)
|
||||
self.assertIn("classifier, on the latest fifth after learning from the rest: decided", text)
|
||||
self.assertIn("rule, on the whole archive: decided 20 of 20", text)
|
||||
self.assertNotIn("rule, on held-out", text)
|
||||
self.assertNotIn("rule, on the latest fifth", text)
|
||||
|
||||
def test_words_the_whole_report_line_by_line_with_the_wrong_kinds_by_name(self):
|
||||
answers = {"f": FINISHED, "i1": FINISHED, "i2": FINISHED, "u": FINISHED, "q": None}
|
||||
samples = [sample(FINISHED, "f"), sample(INTERRUPTED, "i1"), sample(INTERRUPTED, "i2"), sample(UNFINISHED, "u"), sample(ANSWERED, "q")]
|
||||
text = offline_report("m", samples, {"rule": lambda trained: lambda said: answers[said]}, {})
|
||||
self.assertEqual(text.splitlines(), [
|
||||
"Replayed 5 archived verdicts of m projects offline, without sending anything to an outside service. The archive gives them as 1 finished, 2 interrupt, 1 quoted answer, 1 unfinished.",
|
||||
"rule, on the whole archive: decided 4 of 5: finished 4 (of these, archived interrupt 2, unfinished 1). Missed interrupts: 2.",
|
||||
"rule, on the whole archive, does not decide finished with no missed interrupt: it decided 2 archived interrupt(s) as finished.",
|
||||
])
|
||||
|
||||
def test_says_a_method_that_decided_nothing_decided_0_and_names_no_kind(self):
|
||||
text = offline_report("m", self.samples(), {"rule": lambda trained: lambda said: None}, {})
|
||||
self.assertEqual(text.splitlines()[1:], ["rule, on the whole archive: decided 0 of 4. Missed interrupts: 0."])
|
||||
|
||||
def test_tests_the_latest_fifth_of_the_samples_after_learning_from_the_earlier_ones(self):
|
||||
trained = []
|
||||
|
||||
def learner(samples):
|
||||
trained.append(len(samples))
|
||||
return lambda said: FINISHED
|
||||
|
||||
text = offline_report("m", [sample(FINISHED, str(n)) for n in range(7)], {}, {"learner": learner}, folds=2)
|
||||
self.assertEqual(trained, [3, 4, 6])
|
||||
self.assertEqual(text.splitlines()[-2:], ["learner, on the latest fifth after learning from the rest: decided 1 of 1: finished 1. Missed interrupts: 0.", "learner, on both tests, decides finished with no missed interrupt."])
|
||||
|
||||
def test_says_a_learner_that_missed_an_interrupt_on_either_test_does_not_decide_the_kind(self):
|
||||
def learner(samples):
|
||||
return lambda said: FINISHED if said == "miss" else None
|
||||
|
||||
samples = [sample(FINISHED, "ok")] * 4 + [sample(INTERRUPTED, "miss")]
|
||||
text = offline_report("m", samples, {}, {"learner": learner}, folds=2)
|
||||
self.assertEqual(text.splitlines()[-1], "learner, on both tests, does not decide finished with no missed interrupt: it decided 2 archived interrupt(s) as finished.")
|
||||
|
||||
def test_counts_the_verdicts_it_skipped_only_where_it_skipped_some(self):
|
||||
self.assertNotIn("Skipped", offline_report("m", self.samples(), {}, {}))
|
||||
self.assertIn("Skipped 2 verdicts whose transcript could not be read.", offline_report("m", self.samples(), {}, {}, skipped=2))
|
||||
|
||||
def test_the_shipped_methods_are_the_question_mark_rule_the_table_and_the_classifier(self):
|
||||
self.assertEqual(list(RULES), ['"?" rule', "decision table"])
|
||||
self.assertEqual(list(LEARNERS), ["classifier"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Executable
+9
@@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
|
||||
|
||||
from markdown_section import main
|
||||
|
||||
sys.exit(main.main(sys.argv[1:]))
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Print one section of a markdown file, picked by a fragment of its heading.
|
||||
|
||||
usage: markdown-section <file> <fragment>
|
||||
|
||||
The section starts at the heading whose text holds the fragment, in any case, and ends before the
|
||||
next heading of the same or a higher level, so it holds its sub-sections. A heading's text is what
|
||||
follows its `#`s. A line inside a fenced code block is never a heading.
|
||||
|
||||
Where no heading holds the fragment, it fails saying so. Where several headings do, it fails
|
||||
listing each with its line number and prints no section: pass more of the heading you want.
|
||||
A heading whose text is the fragment, in any case, is the one picked even where longer headings
|
||||
hold it too.
|
||||
|
||||
To read a record kept outside the checkout, pass the file `project-record open <record>` prints.
|
||||
|
||||
Exit 2 on bad arguments, and 1, saying why, on any other failure.
|
||||
"""
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
from failure import Failure
|
||||
from text_files.markdown_text import fenced_lines, HEADING
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
if argv[:1] in (["-h"], ["--help"]):
|
||||
print(__doc__.strip())
|
||||
return 0
|
||||
if len(argv) != 2 or not argv[1].strip():
|
||||
print("usage: markdown-section <file> <fragment>", file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
text = pathlib.Path(argv[0]).read_text()
|
||||
except (OSError, UnicodeDecodeError) as error:
|
||||
print(f"markdown-section: {error}", file=sys.stderr)
|
||||
return 1
|
||||
found = section(text, argv[1])
|
||||
if isinstance(found, Failure):
|
||||
print(f"markdown-section: {found.reason}", file=sys.stderr)
|
||||
return 1
|
||||
print(found, end="")
|
||||
return 0
|
||||
|
||||
|
||||
def headings(text: str) -> list[tuple[int, int, str]]:
|
||||
"""Each heading of text outside a fenced code block, as its line's index, its level and its text."""
|
||||
found = []
|
||||
for index, (line, fenced) in enumerate(fenced_lines(text)):
|
||||
if not fenced and (heading := HEADING.match(line)):
|
||||
found.append((index, len(heading.group(1)), heading.group(2)))
|
||||
return found
|
||||
|
||||
|
||||
def section(text: str, fragment: str) -> str | Failure:
|
||||
"""The section of text under the heading that fragment picks, or a Failure where no heading holds it or several do."""
|
||||
found = headings(text)
|
||||
wanted = fragment.strip().lower()
|
||||
matching = [heading for heading in found if wanted in heading[2].lower()]
|
||||
matching = [heading for heading in matching if heading[2].lower() == wanted] or matching
|
||||
if not matching:
|
||||
return Failure(f"no heading holds '{fragment}'")
|
||||
lines = text.splitlines()
|
||||
if len(matching) > 1:
|
||||
listed = "\n".join(f"{index + 1}: {lines[index]}" for index, _, _ in matching)
|
||||
return Failure(f"{len(matching)} headings hold '{fragment}', so use more of one:\n{listed}")
|
||||
start, level, _ = matching[0]
|
||||
end = next((index for index, deeper, _ in found if index > start and deeper <= level), None)
|
||||
return "\n".join(lines[start:end]).rstrip("\n") + "\n"
|
||||
@@ -0,0 +1,138 @@
|
||||
import contextlib
|
||||
import io
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from failure import Failure
|
||||
from markdown_section import main
|
||||
|
||||
TOOL = pathlib.Path(__file__).resolve().parents[1] / "markdown-section"
|
||||
|
||||
DOCUMENT = """# Title
|
||||
|
||||
Intro.
|
||||
|
||||
## Alpha
|
||||
|
||||
Alpha text.
|
||||
|
||||
### Alpha detail
|
||||
|
||||
Detail text.
|
||||
|
||||
## Beta Gamma
|
||||
|
||||
Beta text.
|
||||
|
||||
```
|
||||
## Not a heading
|
||||
```
|
||||
|
||||
## Delta
|
||||
|
||||
Delta text.
|
||||
"""
|
||||
|
||||
|
||||
class Section(unittest.TestCase):
|
||||
def test_a_section_runs_to_the_next_heading_of_its_level_or_a_higher_one(self):
|
||||
cases = {
|
||||
"a sub-section stops at a higher level heading": ("alpha detail", "### Alpha detail\n\nDetail text.\n"),
|
||||
"the text under a heading isn't a heading": ("alpha text", Failure("no heading holds 'alpha text'")),
|
||||
"the last section runs to the end of the file": ("delta", "## Delta\n\nDelta text.\n"),
|
||||
"a level one heading holds the rest of the document": ("title", DOCUMENT),
|
||||
"a fenced line that looks like a heading doesn't end it": ("beta", "## Beta Gamma\n\nBeta text.\n\n```\n## Not a heading\n```\n"),
|
||||
}
|
||||
for name, (fragment, expected) in cases.items():
|
||||
with self.subTest(name):
|
||||
self.assertEqual(main.section(DOCUMENT, fragment), expected)
|
||||
|
||||
def test_the_fragment_matches_any_part_of_the_heading_text_in_any_case(self):
|
||||
for fragment in ("gamma", "BETA g", " beta gamma "):
|
||||
with self.subTest(fragment):
|
||||
self.assertEqual(str(main.section(DOCUMENT, fragment)).splitlines()[0], "## Beta Gamma")
|
||||
self.assertEqual(main.section(DOCUMENT, "## beta"), Failure("no heading holds '## beta'"))
|
||||
|
||||
def test_a_heading_whose_text_is_the_fragment_wins_over_longer_ones(self):
|
||||
self.assertEqual(main.section(DOCUMENT, "Alpha"), "## Alpha\n\nAlpha text.\n\n### Alpha detail\n\nDetail text.\n")
|
||||
self.assertEqual(main.section("## Plan\n\na\n\n## Plan b\n", "PLAN"), "## Plan\n\na\n")
|
||||
|
||||
def test_several_matching_headings_fail_listing_each_with_its_line(self):
|
||||
listed = main.section("## Plan one\n\ntext\n\n## Plan two\n", "plan")
|
||||
self.assertEqual(listed, Failure("2 headings hold 'plan', so use more of one:\n1: ## Plan one\n5: ## Plan two"))
|
||||
|
||||
def test_what_counts_as_a_heading_or_a_fence(self):
|
||||
cases = {
|
||||
"a hash with no space after it is no heading": ("#hashtag\n\n## Real\n", "hashtag", Failure("no heading holds 'hashtag'")),
|
||||
"a fence with an info string closes on a bare fence": ("```python\n## In\n```\n## Out\n\ntext\n", "out", "## Out\n\ntext\n"),
|
||||
"a fence of four backticks holds a line of three": ("````\n```\n## In\n````\n## Out\n", "out", "## Out\n"),
|
||||
"a tilde fence holds a heading": ("~~~\n## In\n~~~\n## Out\n", "in", Failure("no heading holds 'in'")),
|
||||
"an indented fence holds a heading": (" ```\n## In\n ```\n", "in", Failure("no heading holds 'in'")),
|
||||
"an unclosed fence holds the rest": ("## A\n\n```\n## In\n", "a", "## A\n\n```\n## In\n"),
|
||||
}
|
||||
for name, (text, fragment, expected) in cases.items():
|
||||
with self.subTest(name):
|
||||
self.assertEqual(main.section(text, fragment), expected)
|
||||
|
||||
def test_only_blank_lines_are_cut_from_the_end_of_a_section(self):
|
||||
self.assertEqual(main.section("## A\n\nends in X\n\n\n## B\n", "a"), "## A\n\nends in X\n")
|
||||
self.assertEqual(main.section("## A\n\ncode \n\n", "a"), "## A\n\ncode \n")
|
||||
|
||||
def test_no_heading_is_a_failure(self):
|
||||
self.assertEqual(main.section(DOCUMENT, "zeta"), Failure("no heading holds 'zeta'"))
|
||||
|
||||
|
||||
class Main(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.file = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory())) / "notes.md"
|
||||
self.file.write_text(DOCUMENT)
|
||||
|
||||
def run_main(self, *args: str) -> tuple[int, str, str]:
|
||||
out, err = io.StringIO(), io.StringIO()
|
||||
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
|
||||
code = main.main(list(args))
|
||||
return code, out.getvalue(), err.getvalue()
|
||||
|
||||
def test_prints_the_section(self):
|
||||
self.assertEqual(self.run_main(str(self.file), "delta"), (0, "## Delta\n\nDelta text.\n", ""))
|
||||
|
||||
def test_a_failure_is_printed_to_stderr_with_exit_1(self):
|
||||
self.assertEqual(self.run_main(str(self.file), "zeta"), (1, "", "markdown-section: no heading holds 'zeta'\n"))
|
||||
|
||||
def test_a_file_that_cant_be_read_exits_1_naming_it(self):
|
||||
self.file.write_bytes(b"\xff\xfe## A\n")
|
||||
missing = self.file.with_name("missing.md")
|
||||
for unreadable, reason in ((missing, "missing.md"), (self.file, "decode")):
|
||||
with self.subTest(reason):
|
||||
code, out, err = self.run_main(str(unreadable), "a")
|
||||
self.assertEqual((code, out), (1, ""))
|
||||
self.assertRegex(err, rf"^markdown-section: .*{reason}")
|
||||
|
||||
def test_wrong_arguments_exit_2_with_the_usage(self):
|
||||
for args in ((), (str(self.file),), (str(self.file), "a", "b"), (str(self.file), " ")):
|
||||
with self.subTest(args):
|
||||
self.assertEqual(self.run_main(*args), (2, "", "usage: markdown-section <file> <fragment>\n"))
|
||||
|
||||
def test_help_prints_the_usage_and_exits_0_whatever_follows_it(self):
|
||||
for args in (("-h",), ("--help",), ("--help", "x")):
|
||||
with self.subTest(args):
|
||||
code, out, err = self.run_main(*args)
|
||||
self.assertEqual((code, err), (0, ""))
|
||||
self.assertTrue(out.startswith("Print one section of a markdown file"))
|
||||
|
||||
|
||||
class Launcher(unittest.TestCase):
|
||||
def test_the_launcher_runs_the_tool(self):
|
||||
file = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory())) / "notes.md"
|
||||
file.write_text(DOCUMENT)
|
||||
ran = subprocess.run([sys.executable, str(TOOL), str(file), "delta"], capture_output=True, text=True)
|
||||
self.assertEqual((ran.returncode, ran.stdout, ran.stderr), (0, "## Delta\n\nDelta text.\n", ""))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -448,15 +448,15 @@ def children(epic: Epic, authorization: str, http: Http) -> list[object] | Failu
|
||||
seen = http("GET", f"{epic.site}/rest/api/3/issue/{epic.key}?fields=summary", authorization, None)
|
||||
if isinstance(seen, Failure):
|
||||
return Failure(f"cannot read the epic {epic.key}: {seen.reason}")
|
||||
body: dict[str, object] = {
|
||||
"jql": f'parent = "{epic.key}" AND statusCategory != Done ORDER BY Rank ASC',
|
||||
"fields": ["summary", "description", "fixVersions", "issuelinks"],
|
||||
"properties": [PROPERTY],
|
||||
"maxResults": PAGE,
|
||||
}
|
||||
return searched(epic.site, f"{epic.key}'s children", f'parent = "{epic.key}" AND statusCategory != Done ORDER BY Rank ASC', {"fields": ["summary", "description", "fixVersions", "issuelinks"], "properties": [PROPERTY]}, authorization, http)
|
||||
|
||||
|
||||
def searched(site: str, what: str, jql: str, asked: dict[str, object], authorization: str, http: Http) -> list[object] | Failure:
|
||||
"""Every issue a JQL search on site finds, page by page. The dict asked holds the request's `fields` and `properties` lists. The text what names the issues sought, in the Failure of a search that doesn't end."""
|
||||
body: dict[str, object] = {"jql": jql, "maxResults": PAGE} | asked
|
||||
found: list[object] = []
|
||||
for _ in range(MAX_PAGES):
|
||||
page = http("POST", f"{epic.site}/rest/api/3/search/jql", authorization, body)
|
||||
page = http("POST", f"{site}/rest/api/3/search/jql", authorization, body)
|
||||
if isinstance(page, Failure):
|
||||
return page
|
||||
if not isinstance(page, dict) or not isinstance(page.get("issues"), list):
|
||||
@@ -465,7 +465,7 @@ def children(epic: Epic, authorization: str, http: Http) -> list[object] | Failu
|
||||
if not page.get("nextPageToken"):
|
||||
return found
|
||||
body = body | {"nextPageToken": page["nextPageToken"]}
|
||||
return Failure(f"the search for {epic.key}'s children did not end within {MAX_PAGES} pages")
|
||||
return Failure(f"the search for {what} did not end within {MAX_PAGES} pages")
|
||||
|
||||
|
||||
def issue_of(raw: object) -> Issue | Failure:
|
||||
|
||||
@@ -6,6 +6,9 @@ Run inside a checkout, or a worktree of one. It finds the config that names the
|
||||
~/.claude-projects/<key>/config.toml, and prints each setting and its absolute path, as in
|
||||
`plan: /home/me/code/acme/todo.md`. Given a setting, it prints only that setting's path.
|
||||
|
||||
`~/.claude-projects/do-epic/` holds do-epic's machine-wide settings, not a project's. It needs no `checkout`,
|
||||
and project-config doesn't read anything from it, so no project may use `do-epic` as its key.
|
||||
|
||||
The config is TOML. `checkout` names the project's main checkout, and every setting in the example below is a path:
|
||||
"checkout:<path>" inside the checkout, "folder:<path>" inside the config's own folder, or one that
|
||||
starts with / or ~/. In a worktree, a "checkout:" path is the worktree's own file.
|
||||
@@ -51,7 +54,9 @@ Where the config leaves them out, `todo-item finish` fails in a Jira plan. Each
|
||||
`goals-and-audience` may name a Jira Cloud epic the same way, where `plan` names one too. The
|
||||
epic's description is then the goals file, with ## Goals and ## Audience as headings, and
|
||||
`project-config goals-and-audience` prints the setting as the config spells it. The description is
|
||||
only read: edit it in Jira.
|
||||
written by `project-record save`, except a change to ## Goals or ## Audience, which is refused:
|
||||
edit those two in Jira. A save rewrites the whole description from markdown, which keeps no
|
||||
mention, macro or block id that markdown can't hold.
|
||||
|
||||
Each record, `decisions`, `score-anchors`, `comprehension-history`, `reading-cost` and `changelog`,
|
||||
may name a Confluence Cloud page in place of a path, as "confluence:" and the page's address, with or
|
||||
|
||||
@@ -17,6 +17,10 @@ from plan.ranking import variables
|
||||
|
||||
PROJECTS = ".claude-projects"
|
||||
CONFIG = "config.toml"
|
||||
# The key of do-epic's machine-wide settings: the folder's config.toml has no `checkout`, so no project may use the key.
|
||||
DO_EPIC = "do-epic"
|
||||
# A Jira Cloud or Confluence Cloud site's address: only https, so the credentials are never sent in the clear.
|
||||
SITE = r"https://[A-Za-z0-9.-]+(?::[0-9]+)?"
|
||||
# Every path a project sets, by its role. Three readers, seated as the human who writes the config, picked the spelling from four candidates. They picked role names over file names, which a client's project doesn't share.
|
||||
SETTINGS = ("plan", "working-rules", "decisions", "score-anchors", "comprehension-history", "reading-cost", "changelog", "goals-and-audience", "principles", "standards")
|
||||
# They also picked a prefix over bare relative paths: each line says its own place, and a relative path with no prefix fails instead of landing somewhere by default.
|
||||
@@ -177,7 +181,7 @@ def load(top: pathlib.Path, main: pathlib.Path, home: pathlib.Path) -> ProjectCo
|
||||
top is the worktree that asks, and main itself outside a worktree. A `checkout:` path lands in top.
|
||||
A config that cannot be read fails every project, since it may be the one that names main."""
|
||||
named: list[tuple[pathlib.Path, dict[str, object]]] = []
|
||||
for file in sorted((home / PROJECTS).glob(f"*/{CONFIG}")):
|
||||
for file in configs(home):
|
||||
config = read(file, home)
|
||||
if isinstance(config, Failure):
|
||||
return config
|
||||
@@ -290,14 +294,14 @@ def name_of(value: object) -> str | None:
|
||||
|
||||
|
||||
def epic_of(value: object) -> Epic | None:
|
||||
"""The epic a setting's value spells; None where it spells none. Only https, so the credentials are never sent in the clear."""
|
||||
spelt = isinstance(value, str) and re.fullmatch(rf"{JIRA}(https://[A-Za-z0-9.-]+(?::[0-9]+)?)/browse/([A-Z][A-Z0-9_]*-[0-9]+)", value)
|
||||
"""The epic a setting's value spells; None where it spells none."""
|
||||
spelt = isinstance(value, str) and re.fullmatch(rf"{JIRA}({SITE})/browse/([A-Z][A-Z0-9_]*-[0-9]+)", value)
|
||||
return Epic(spelt.group(1), spelt.group(2)) if spelt else None
|
||||
|
||||
|
||||
def page_of(value: object) -> Page | None:
|
||||
"""The page a setting's value spells; None where it spells none. Only https, as for an epic."""
|
||||
spelt = isinstance(value, str) and re.fullmatch(rf"{CONFLUENCE}(https://[A-Za-z0-9.-]+(?::[0-9]+)?)/wiki/spaces/[^/?#]+/pages/([0-9]+)(?:/[^/?#]*)?", value)
|
||||
"""The page a setting's value spells; None where it spells none."""
|
||||
spelt = isinstance(value, str) and re.fullmatch(rf"{CONFLUENCE}({SITE})/wiki/spaces/[^/?#]+/pages/([0-9]+)(?:/[^/?#]*)?", value)
|
||||
return Page(spelt.group(1), spelt.group(2), spelt.string) if spelt else None
|
||||
|
||||
|
||||
@@ -339,25 +343,51 @@ def lies_in(path: pathlib.Path, *places: pathlib.Path) -> bool:
|
||||
def held(folder: pathlib.Path, home: pathlib.Path) -> ProjectConfig | Failure | None:
|
||||
"""The config in folder, read as its project's main checkout reads it; None where folder is not a config's folder under home."""
|
||||
# Matched on resolved paths, as git names a folder, so a config's folder reached through a symlink is still found.
|
||||
file = next((file for file in (home / PROJECTS).glob(f"*/{CONFIG}") if file.parent.resolve() == folder.resolve()), None)
|
||||
file = next((file for file in configs(home) if file.parent.resolve() == folder.resolve()), None)
|
||||
if file is None:
|
||||
return None
|
||||
config = read(file, home)
|
||||
return config if isinstance(config, Failure) else load(config[0], config[0], home)
|
||||
|
||||
|
||||
def by_name(name: str, home: pathlib.Path) -> ProjectConfig | Failure | None:
|
||||
"""The config of the project whose main checkout is a folder called name, though that folder may not exist yet, loaded as `held` does; None where no config names one. A config that cannot be read fails every name, since it may be the one asked for."""
|
||||
files = []
|
||||
for file in configs(home):
|
||||
config = read(file, home)
|
||||
if isinstance(config, Failure):
|
||||
return config
|
||||
if config[0].name == name:
|
||||
files.append(file)
|
||||
if len(files) > 1:
|
||||
return Failure(f"{' and '.join(str(file) for file in files)} each name a checkout called {name}; only one config may")
|
||||
return held(files[0].parent, home) if files else None
|
||||
|
||||
|
||||
def configs(home: pathlib.Path) -> list[pathlib.Path]:
|
||||
"""Every project's config file under home, in key order. do-epic's machine-wide config is not a project's."""
|
||||
return sorted(file for file in (home / PROJECTS).glob(f"*/{CONFIG}") if file.parent.name != DO_EPIC)
|
||||
|
||||
|
||||
def env_files(home: pathlib.Path) -> list[pathlib.Path]:
|
||||
"""Every config's env-file, read without the checks load makes, so a config that fails them still keeps its credentials guarded; a config that cannot be read names none."""
|
||||
configs = [read(file, home) for file in sorted((home / PROJECTS).glob(f"*/{CONFIG}"))]
|
||||
return [path for config in configs if not isinstance(config, Failure) and (path := anywhere(config[1].get(ENV_FILE), home))]
|
||||
"""Every config's env-file, do-epic's included. Each is read without the checks load makes, so a config that fails them still keeps its credentials guarded. A config that can't be read doesn't name one."""
|
||||
values = [parsed(file) for file in (*configs(home), home / PROJECTS / DO_EPIC / CONFIG)]
|
||||
return [path for found in values if not isinstance(found, Failure) and (path := anywhere(found.get(ENV_FILE), home))]
|
||||
|
||||
|
||||
def parsed(file: pathlib.Path) -> dict[str, object] | Failure:
|
||||
"""The TOML values in file; a Failure where file can't be read or parsed."""
|
||||
try:
|
||||
return tomllib.loads(file.read_text(encoding="utf-8"))
|
||||
except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError) as failure:
|
||||
return Failure(f"cannot read {file}: {failure}")
|
||||
|
||||
|
||||
def read(file: pathlib.Path, home: pathlib.Path) -> tuple[pathlib.Path, dict[str, object]] | Failure:
|
||||
"""The checkout a config names, as an absolute path, and its other values."""
|
||||
try:
|
||||
values = tomllib.loads(file.read_text(encoding="utf-8"))
|
||||
except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError) as failure:
|
||||
return Failure(f"cannot read {file}: {failure}")
|
||||
values = parsed(file)
|
||||
if isinstance(values, Failure):
|
||||
return values
|
||||
checkout = anywhere(values.get("checkout"), home)
|
||||
if checkout is None:
|
||||
found = f"not {values['checkout']!r}" if "checkout" in values else "and is missing"
|
||||
|
||||
@@ -9,7 +9,7 @@ import unittest.mock
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from project_config.settings import env_files, Epic, Finish, held, JiraScoring, load, NOTIFIERS, OUTSIDE, Page, ProjectConfig, SETTINGS, Unnamed
|
||||
from project_config.settings import by_name, env_files, Epic, Finish, held, JiraScoring, load, NOTIFIERS, OUTSIDE, Page, ProjectConfig, SETTINGS, Unnamed
|
||||
from failure import Failure
|
||||
from test_env import isolated_env
|
||||
from project_config.main import located, title_templates
|
||||
@@ -91,6 +91,13 @@ class LoadTest(ConfigCase):
|
||||
self.assertEqual(found.paths["standards"], self.home / ".claude" / "skills" / "larv-review" / "standards")
|
||||
self.assertEqual(tuple(found.paths), SETTINGS)
|
||||
|
||||
def test_the_do_epic_folder_is_no_projects_config_so_it_needs_no_checkout(self) -> None:
|
||||
self.write("do-epic", 'jira-url = "https://acme.atlassian.net"\n')
|
||||
self.write("acme", CLIENT)
|
||||
found = load(self.checkout, self.checkout, self.home)
|
||||
assert isinstance(found, ProjectConfig), found
|
||||
self.assertEqual(found.key, "acme")
|
||||
|
||||
def test_a_plan_spelt_as_a_jira_epic_is_no_path(self) -> None:
|
||||
self.write("acme", JIRA_CLIENT)
|
||||
found = load(self.checkout, self.checkout, self.home)
|
||||
@@ -578,6 +585,10 @@ class HeldTest(ConfigCase):
|
||||
(elsewhere / "config.toml").write_text(CLIENT, encoding="utf-8")
|
||||
self.assertIsNone(held(elsewhere, self.home), "a config.toml outside ~/.claude-projects/")
|
||||
|
||||
def test_the_do_epic_folder_holds_no_projects_config(self) -> None:
|
||||
folder = self.write("do-epic", CLIENT)
|
||||
self.assertIsNone(held(folder, self.home))
|
||||
|
||||
def test_a_configs_folder_reached_through_a_symlink_holds_its_config(self) -> None:
|
||||
target = self.home / "kept" / "acme"
|
||||
target.mkdir(parents=True)
|
||||
@@ -587,6 +598,32 @@ class HeldTest(ConfigCase):
|
||||
self.assertIsInstance(held(target, self.home), ProjectConfig, "git names the folder by its resolved path")
|
||||
|
||||
|
||||
class ByNameTest(ConfigCase):
|
||||
def test_finds_the_config_of_the_project_whose_main_checkout_has_the_name_though_the_checkout_is_missing(self) -> None:
|
||||
folder = self.write("acme", CLIENT)
|
||||
found = by_name("acme-billing", self.home)
|
||||
assert isinstance(found, ProjectConfig), found
|
||||
self.assertEqual((found.folder, found.top), (folder, self.checkout))
|
||||
|
||||
def test_finds_none_where_no_config_names_a_checkout_of_that_name(self) -> None:
|
||||
self.write("acme", CLIENT)
|
||||
self.assertIsNone(by_name("billing", self.home), "a part of a name is not the name")
|
||||
self.assertIsNone(by_name("acme", self.home), "a config's key is not a repo's name")
|
||||
self.assertIsNone(by_name("do-epic", self.home))
|
||||
|
||||
def test_fails_where_two_configs_name_a_checkout_of_that_name(self) -> None:
|
||||
self.write("acme", CLIENT)
|
||||
self.write("acme-two", CLIENT.replace("~/code/acme-billing", "~/other/acme-billing"))
|
||||
found = by_name("acme-billing", self.home)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("acme/config.toml and ", found.reason)
|
||||
self.assertIn("acme-two/config.toml each name a checkout called acme-billing", found.reason)
|
||||
|
||||
def test_fails_where_a_config_cannot_be_read_since_it_may_be_the_one_asked_for(self) -> None:
|
||||
self.write("acme", "this is not toml")
|
||||
self.assertIsInstance(by_name("acme-billing", self.home), Failure)
|
||||
|
||||
|
||||
class EnvFilesTest(ConfigCase):
|
||||
def test_lists_each_readable_configs_env_file_even_one_load_refuses(self) -> None:
|
||||
self.write("acme", CLIENT + 'env-file = "~/acme.env"\n')
|
||||
@@ -595,6 +632,10 @@ class EnvFilesTest(ConfigCase):
|
||||
self.write("none", CLIENT)
|
||||
self.assertEqual(env_files(self.home), [self.home / "acme.env", pathlib.Path("/srv/other.env")])
|
||||
|
||||
def test_lists_do_epics_env_file_though_its_config_names_no_checkout(self) -> None:
|
||||
self.write("do-epic", 'env-file = "~/jira.env"\n')
|
||||
self.assertEqual(env_files(self.home), [self.home / "jira.env"])
|
||||
|
||||
|
||||
class ToolTest(ConfigCase):
|
||||
def test_prints_a_jira_plan_as_the_config_spells_it(self) -> None:
|
||||
|
||||
@@ -5,8 +5,8 @@ usage: project-record open <record>
|
||||
|
||||
Run inside a checkout, or a worktree of one. <record> is one of decisions, score-anchors,
|
||||
comprehension-history, reading-cost and changelog, each kept where the project's config names it,
|
||||
or goals-and-audience, the goals file. Goals kept in a Jira epic's description are only read, so
|
||||
save refuses them: edit the epic in Jira.
|
||||
or goals-and-audience, the goals file. Save refuses a change to the ## Goals or ## Audience of goals
|
||||
kept in a Jira epic's description: edit those two in Jira.
|
||||
|
||||
open prints the path of a file holding the record's text, and nothing else. Read and edit that
|
||||
file. Where the record is kept as a file, the path is that file. Where it is kept elsewhere, the
|
||||
@@ -20,8 +20,8 @@ save stores the file's text in the record, and prints one line saying so. Where
|
||||
as a file, there is nothing left to store, and save says so. Run it after every edit, so the same
|
||||
steps work wherever a project keeps the record.
|
||||
|
||||
For a record kept on a Confluence page, save refuses where someone saved the page after open read
|
||||
it or after save last wrote it, and the human decides what follows. The copy keeps the unsaved edit
|
||||
For a record kept on a Confluence page or in a Jira epic's description, save refuses where someone
|
||||
saved it after open read it or after save last wrote it, and the human decides what follows. The copy keeps the unsaved edit
|
||||
until open runs again and writes the copy afresh.
|
||||
|
||||
Exit 2 on bad arguments, and 1, saying why, on any other failure.
|
||||
|
||||
@@ -2,16 +2,15 @@
|
||||
|
||||
A write never overwrites a teammate's edit: it sends the version after the one held() read, and Confluence refuses it where someone saved the page since. The human then decides (lilleman, 2026-11-12)."""
|
||||
import dataclasses
|
||||
import hashlib
|
||||
import json
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from adf_codec.convert import adfs_to_markdown, markdown_to_adf, Refused
|
||||
from atlassian.client import basic_auth, confluence_json, Http, ToAdf, ToMarkdown
|
||||
from record_store.held import digest_of, Held
|
||||
from rest_client.json_request import ErrorAnswer
|
||||
from failure import Failure
|
||||
from record_store.held import Held
|
||||
from project_config.settings import Page
|
||||
|
||||
# What Confluence answers to a write whose version isn't the one after the version it holds.
|
||||
@@ -141,7 +140,3 @@ class ConfluencePage:
|
||||
return Seen(int(held["version"]), str(held["title"]), held.get("digest"))
|
||||
except (ValueError, KeyError, TypeError) as failure:
|
||||
return Failure(f"cannot read the version of {self.label} to write over, {over!r}: {failure}")
|
||||
|
||||
|
||||
def digest_of(text: str) -> str:
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Record stores for tests, held to record_store.contract like every backend."""
|
||||
"""Record stores for tests, held to record_store.contract like every backend, and a fake Jira."""
|
||||
import dataclasses
|
||||
|
||||
from failure import Failure
|
||||
@@ -29,3 +29,33 @@ class MemoryStore:
|
||||
return Failure(f"someone saved {self.label} since version {over}")
|
||||
self.text, self.version = text, self.version + 1
|
||||
return Held(text, str(self.version))
|
||||
|
||||
|
||||
class FakeJira:
|
||||
"""One epic as Jira Cloud's REST API version 3 serves it: a description that is null or a document, and no version a write could check. Its documents hold the text in one paragraph, which fake_adf and fake_markdown convert."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.description: dict[str, object] | None = None
|
||||
self.calls: list[tuple[str, str, str, object]] = []
|
||||
|
||||
def http(self, method: str, url: str, authorization: str, body: object) -> object:
|
||||
self.calls.append((method, url, authorization, body))
|
||||
if method == "GET":
|
||||
return {"key": "ABC-12", "fields": {"description": self.description}}
|
||||
assert isinstance(body, dict)
|
||||
self.description = body["fields"]["description"]
|
||||
return None
|
||||
|
||||
def teammate_saves(self, text: str) -> None:
|
||||
self.description = fake_adf(text)
|
||||
|
||||
def puts(self) -> list[object]:
|
||||
return [call[3] for call in self.calls if call[0] == "PUT"]
|
||||
|
||||
|
||||
def fake_adf(text: str) -> dict[str, object]:
|
||||
return {"type": "doc", "version": 1, "content": [{"type": "paragraph", "content": [{"type": "text", "text": text}]}]}
|
||||
|
||||
|
||||
def fake_markdown(documents: list[dict[str, object]]) -> list[str]:
|
||||
return [document["content"][0]["content"][0]["text"] for document in documents] # type: ignore[index]
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""A record as a read found it or a write left it, which every record backend gives."""
|
||||
import dataclasses
|
||||
import hashlib
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -7,3 +8,8 @@ class Held:
|
||||
"""A record's text as one read found it or one write left it, with its version: what a write over it names, so the write refuses where someone saved the record since. None for a backend whose writes check no version, as a file's."""
|
||||
text: str
|
||||
version: str | None = None
|
||||
|
||||
|
||||
def digest_of(text: str) -> str:
|
||||
"""A fingerprint of a record's text or document, for a version a write names."""
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
@@ -1,25 +1,31 @@
|
||||
"""A project's goals and audience kept as the description of one Jira Cloud epic, read over Jira's REST API with the credentials of the config's env-file, and converted by the adf-codec shim.
|
||||
"""A project's goals and audience kept as the description of one Jira Cloud epic, read and written over Jira's REST API with the credentials of the config's env-file, and converted by the adf-codec shim.
|
||||
|
||||
The store only reads: a write, such as `project-record save goals-and-audience`, fails, saying to edit the epic in Jira."""
|
||||
Jira keeps no version of a description, so the version a read gives is a digest of the document, and a write refuses where the description no longer has that digest. A write changes ## Goals or ## Audience only where the store was told the human approved that section in an interrupt (`approved`)."""
|
||||
import dataclasses
|
||||
import json
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from adf_codec.convert import adfs_to_markdown, Refused
|
||||
from atlassian.client import basic_auth, Http, jira_json, ToMarkdown
|
||||
from adf_codec.convert import adfs_to_markdown, markdown_to_adf, Refused
|
||||
from atlassian.client import basic_auth, Http, jira_json, ToAdf, ToMarkdown
|
||||
from record_store.held import digest_of, Held
|
||||
from project_config.settings import Epic
|
||||
from failure import Failure
|
||||
from record_store.held import Held
|
||||
from text_files.markdown_text import sections
|
||||
|
||||
READ_ONLY = "goals and audience kept in Jira are only read: edit the epic's description in Jira"
|
||||
# The sections of the description a write changes only after the human approves.
|
||||
APPROVED_SECTIONS = ("Goals", "Audience")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class EpicDescription:
|
||||
"""The description of epic as markdown, read with the credentials env_file holds."""
|
||||
"""The description of epic as markdown, read and written with the credentials env_file holds. approved names the APPROVED_SECTIONS whose change the human approved."""
|
||||
epic: Epic
|
||||
env_file: pathlib.Path
|
||||
http: Http = jira_json
|
||||
to_markdown: ToMarkdown = adfs_to_markdown
|
||||
to_adf: ToAdf = markdown_to_adf
|
||||
approved: frozenset[str] = frozenset()
|
||||
|
||||
@property
|
||||
def label(self) -> str:
|
||||
@@ -31,28 +37,70 @@ class EpicDescription:
|
||||
|
||||
def read(self) -> str | Failure:
|
||||
"""The description as markdown; empty where the epic has none."""
|
||||
held = self.held()
|
||||
return held if isinstance(held, Failure) else held.text
|
||||
|
||||
def held(self) -> Held | Failure:
|
||||
"""The description as markdown, with a digest of the document Jira holds as the version a write over it names."""
|
||||
description = self.fetched()
|
||||
if isinstance(description, Failure):
|
||||
return description
|
||||
text = "" if description is None else self.markdown_of(description)
|
||||
return text if isinstance(text, Failure) else Held(text, digest_of(json.dumps(description, sort_keys=True)))
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
return f"{self.epic.site}/rest/api/3/issue/{self.epic.key}"
|
||||
|
||||
def fetched(self) -> dict[str, typing.Any] | None | Failure:
|
||||
"""The description as Jira holds it, an ADF document; None where the epic has none."""
|
||||
authorization = basic_auth(self.env_file)
|
||||
if isinstance(authorization, Failure):
|
||||
return authorization
|
||||
key = self.epic.key
|
||||
answer = self.http("GET", f"{self.epic.site}/rest/api/3/issue/{key}?fields=description", authorization, None)
|
||||
answer = self.http("GET", self.url + "?fields=description", authorization, None)
|
||||
if isinstance(answer, Failure):
|
||||
return Failure(f"cannot read the epic {key}: {answer.reason}")
|
||||
fields = answer.get("fields") if isinstance(answer, dict) else None
|
||||
if not isinstance(fields, dict) or "description" not in fields or not isinstance(fields["description"], dict | None):
|
||||
return Failure(f"the epic {key} is not in the shape Jira Cloud's REST API version 3 documents: its description field is missing, or neither a document nor null")
|
||||
if fields["description"] is None:
|
||||
return ""
|
||||
texts = self.to_markdown([fields["description"]])
|
||||
return fields["description"]
|
||||
|
||||
def markdown_of(self, document: dict[str, typing.Any]) -> str | Failure:
|
||||
key = self.epic.key
|
||||
texts = self.to_markdown([document])
|
||||
if isinstance(texts, Refused):
|
||||
return Failure(f"{key}'s description: {texts.failure.reason}")
|
||||
if isinstance(texts, Failure):
|
||||
return Failure(f"cannot convert {key}'s description to markdown: {texts.reason}")
|
||||
return texts[0]
|
||||
|
||||
def held(self) -> Held | Failure:
|
||||
text = self.read()
|
||||
return text if isinstance(text, Failure) else Held(text)
|
||||
|
||||
def write(self, text: str, over: str | None) -> Held | Failure:
|
||||
return Failure(READ_ONLY)
|
||||
"""Replaces the description with text, over the version held() gave. A text that is the description as read leaves it as Jira holds it, since converting the markdown back would drop each block's localId and Jira's spacing paragraphs; any other text rewrites the whole description, which keeps only what markdown can hold. Jira checks no version, so the check of over is a read just before the request, and an edit a teammate saves between the two is overwritten."""
|
||||
key = self.epic.key
|
||||
current = self.held()
|
||||
if isinstance(current, Failure):
|
||||
return current
|
||||
if over is not None and current.version != over:
|
||||
return Failure(f"someone saved the description of {key} since the loop read it, so the loop didn't write it: never overwrite their edit, but ask the human what to do")
|
||||
if text == current.text:
|
||||
return current
|
||||
document = self.to_adf(text) if text else None
|
||||
if isinstance(document, Failure):
|
||||
return Failure(f"cannot convert the text for {key}'s description to ADF: {document.reason}")
|
||||
# The sections of the description Jira will hold, as the codec reads them: a heading spelt another way than `## Goals` still counts.
|
||||
sent = "" if document is None else self.markdown_of(document)
|
||||
if isinstance(sent, Failure):
|
||||
return sent
|
||||
before, after = sections(current.text), sections(sent)
|
||||
if repeated := [name for name, bodies in after.items() if len(bodies) > max(1, len(before.get(name, [])))]:
|
||||
return Failure(f"the write would repeat ## {repeated[0]} in {key}'s description, which locks that section for reads and writes: leave a heading out of the text, or edit the description in Jira")
|
||||
if unapproved := [name for name in APPROVED_SECTIONS if name not in self.approved and after.get(name) != before.get(name)]:
|
||||
return Failure(f"the write changes ## {unapproved[0]} of {key}'s description without the human's approval: have the human approve it in an interrupt, or edit the section in Jira")
|
||||
authorization = basic_auth(self.env_file)
|
||||
if isinstance(authorization, Failure):
|
||||
return authorization
|
||||
answer = self.http("PUT", self.url, authorization, {"fields": {"description": document}})
|
||||
if isinstance(answer, Failure):
|
||||
return Failure(f"cannot write the description of {key}: {answer.reason}")
|
||||
return self.held()
|
||||
|
||||
@@ -87,7 +87,7 @@ def touch_predictions_store(config: ProjectConfig) -> RecordStore | None:
|
||||
|
||||
|
||||
def goals_store(config: ProjectConfig) -> RecordStore:
|
||||
"""The store config names for the goals file, which holds ## Goals and ## Audience: a markdown file, or a Jira epic's description, which the store only reads."""
|
||||
"""The store config names for the goals file, which holds ## Goals and ## Audience: a markdown file, or a Jira epic's description. This store is built without an approval, so it refuses a change to the epic's ## Goals or ## Audience."""
|
||||
if config.goals_epic is None:
|
||||
return FileStore(config.paths["goals-and-audience"], config.tracked("goals-and-audience"))
|
||||
# load refuses a config that names an epic and no env-file.
|
||||
|
||||
@@ -9,10 +9,14 @@ sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
from project_config.settings import Epic
|
||||
from record_store.jira_description import EpicDescription
|
||||
from failure import Failure
|
||||
from record_store.fakes import fake_adf, fake_markdown, FakeJira
|
||||
from record_store.held import Held
|
||||
from record_store.contract import RecordStoreContract
|
||||
from adf_codec.convert import Refused
|
||||
|
||||
EPIC = Epic("https://acme.atlassian.net", "ABC-12")
|
||||
AUTHORIZATION = "Basic " + base64.b64encode(b"me@example.com:s3cret").decode()
|
||||
URL = "https://acme.atlassian.net/rest/api/3/issue/ABC-12"
|
||||
GOALS = {"type": "doc", "version": 1, "content": [{"type": "heading", "attrs": {"level": 2}, "content": [{"type": "text", "text": "Goals"}]}]}
|
||||
|
||||
|
||||
@@ -69,20 +73,176 @@ class EpicDescriptionTest(unittest.TestCase):
|
||||
for markdown in (Refused(0, Failure("an unsupported node")), Failure("docker is not running")):
|
||||
with self.subTest(markdown):
|
||||
self.markdown = markdown
|
||||
found = self.store().read()
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("ABC-12", found.reason)
|
||||
for found in (self.store().read(), self.store().held()):
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("ABC-12", found.reason)
|
||||
|
||||
def test_missing_credentials_fail(self) -> None:
|
||||
self.env.unlink()
|
||||
self.assertIsInstance(self.store().read(), Failure)
|
||||
self.assertEqual(self.calls, [])
|
||||
|
||||
def test_writing_fails_since_the_store_only_reads(self) -> None:
|
||||
found = self.store().write("## Goals\n", None)
|
||||
|
||||
class WriteCase(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.env = pathlib.Path(self.enterContext(tempfile.TemporaryDirectory())) / "acme.env"
|
||||
self.env.write_text("ATLASSIAN_EMAIL=me@example.com\nATLASSIAN_API_TOKEN=s3cret\n")
|
||||
self.env.chmod(0o600)
|
||||
self.jira = FakeJira()
|
||||
|
||||
def new_store(self, approved: frozenset[str] = frozenset()) -> EpicDescription:
|
||||
return EpicDescription(EPIC, self.env, self.jira.http, fake_markdown, fake_adf, approved)
|
||||
|
||||
|
||||
class EpicDescriptionContract(WriteCase, RecordStoreContract):
|
||||
pass
|
||||
|
||||
|
||||
GOALS_AND_SCORING = "## Goals\n1. **Achieve:** Ship.\n\n## Audience\nMe.\n\n## Scoring\nValue.\n"
|
||||
|
||||
|
||||
class EpicDescriptionWriteTest(WriteCase):
|
||||
def held(self, text: str) -> Held:
|
||||
self.jira.teammate_saves(text)
|
||||
held = self.new_store().held()
|
||||
assert isinstance(held, Held), held
|
||||
return held
|
||||
|
||||
def test_writes_the_text_as_the_epics_description(self) -> None:
|
||||
held = self.held(GOALS_AND_SCORING)
|
||||
text = GOALS_AND_SCORING.replace("Value.", "Value, then risk.")
|
||||
written = self.new_store().write(text, held.version)
|
||||
assert isinstance(written, Held), written
|
||||
self.assertEqual(self.jira.puts(), [{"fields": {"description": fake_adf(text)}}])
|
||||
put = next(call for call in self.jira.calls if call[0] == "PUT")
|
||||
self.assertEqual(put[:3], ("PUT", URL, AUTHORIZATION))
|
||||
self.assertEqual(written, self.new_store().held())
|
||||
|
||||
def test_writing_empty_text_clears_the_description(self) -> None:
|
||||
held = self.held("## Decisions\n- one\n")
|
||||
self.assertIsInstance(self.new_store().write("", held.version), Held)
|
||||
self.assertEqual(self.jira.puts(), [{"fields": {"description": None}}])
|
||||
|
||||
def test_saving_the_text_a_read_found_leaves_the_description_as_jira_holds_it(self) -> None:
|
||||
held = self.held(GOALS_AND_SCORING)
|
||||
before = self.jira.description
|
||||
self.assertEqual(self.new_store().write(held.text, held.version), held)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
self.assertIs(self.jira.description, before)
|
||||
|
||||
def test_refuses_to_write_where_a_teammate_saved_since_the_read(self) -> None:
|
||||
held = self.held(GOALS_AND_SCORING)
|
||||
self.jira.teammate_saves("## Goals\n1. **Achieve:** Their goal.\n")
|
||||
for text in (held.text, GOALS_AND_SCORING.replace("Value.", "Risk.")):
|
||||
with self.subTest(text):
|
||||
found = self.new_store().write(text, held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("someone saved", found.reason)
|
||||
self.assertIn("ABC-12", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
self.assertEqual(self.new_store().read(), "## Goals\n1. **Achieve:** Their goal.\n")
|
||||
|
||||
def test_refuses_to_write_over_the_version_of_an_epic_that_had_no_description(self) -> None:
|
||||
held = self.new_store().held()
|
||||
assert isinstance(held, Held), held
|
||||
self.jira.teammate_saves("## Goals\nTheirs.\n")
|
||||
found = self.new_store(frozenset({"Goals"})).write("## Goals\nMine.\n", held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("Jira", found.reason)
|
||||
self.assertEqual(self.calls, [])
|
||||
self.assertIn("someone saved", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_refuses_to_change_goals_or_audience_before_the_human_approves(self) -> None:
|
||||
held = self.held(GOALS_AND_SCORING)
|
||||
changes = [
|
||||
("Goals", GOALS_AND_SCORING.replace("Ship.", "Ship less.")),
|
||||
("Audience", GOALS_AND_SCORING.replace("Me.", "Everyone.")),
|
||||
("Goals", GOALS_AND_SCORING.replace("## Goals\n1. **Achieve:** Ship.\n\n", "")),
|
||||
]
|
||||
for name, text in changes:
|
||||
with self.subTest(name, text=text):
|
||||
found = self.new_store().write(text, held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn(name, found.reason)
|
||||
self.assertIn("approval", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_counts_a_heading_the_codec_reads_as_goals_even_where_the_markdown_spells_it_another_way(self) -> None:
|
||||
held = self.held("## Scoring\nValue.\n")
|
||||
as_atx = lambda documents: [fake_markdown(documents)[0].replace("Goals\n=====", "## Goals")]
|
||||
store = EpicDescription(EPIC, self.env, self.jira.http, as_atx, fake_adf)
|
||||
found = store.write("Goals\n=====\nShip.\n\n## Scoring\nValue.\n", held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("## Goals", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_refuses_a_write_that_repeats_a_section_the_codec_reads_from_another_spelling_of_its_heading(self) -> None:
|
||||
held = self.held("## Decisions\n- one\n")
|
||||
as_atx = lambda documents: [fake_markdown(documents)[0].replace("Decisions\n=====", "## Decisions")]
|
||||
store = EpicDescription(EPIC, self.env, self.jira.http, as_atx, fake_adf)
|
||||
found = store.write("## Decisions\n- one\n\n## Scoring\nValue.\n\nDecisions\n=====\n- copied\n", held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("repeat ## Decisions", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_keeps_a_section_that_was_already_repeated_in_the_description(self) -> None:
|
||||
held = self.held("## Decisions\n- one\n\n## Decisions\n- two\n")
|
||||
written = self.new_store().write("## Decisions\n- one\n\n## Decisions\n- two\n\n## Scoring\nValue.\n", held.version)
|
||||
self.assertIsInstance(written, Held)
|
||||
|
||||
def test_the_version_does_not_depend_on_the_order_of_the_documents_keys(self) -> None:
|
||||
held = self.held("## Scoring\nValue.\n")
|
||||
self.jira.description = dict(reversed(list(fake_adf("## Scoring\nValue.\n").items())))
|
||||
self.assertEqual(self.new_store().held(), held)
|
||||
|
||||
def test_refuses_to_write_where_a_teammate_saved_a_change_the_markdown_does_not_show(self) -> None:
|
||||
held = self.held("## Scoring\nValue.\n")
|
||||
self.jira.description = {**fake_adf("## Scoring\nValue.\n"), "localId": "teammate"}
|
||||
found = self.new_store().write("## Scoring\nRisk.\n", held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("someone saved", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_refuses_to_add_a_goals_section_before_the_human_approves(self) -> None:
|
||||
held = self.held("## Scoring\nValue.\n")
|
||||
found = self.new_store().write("## Goals\n1. **Achieve:** Ship.\n\n## Scoring\nValue.\n", held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("Goals", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_changes_goals_once_the_human_approved_that_section_only(self) -> None:
|
||||
held = self.held(GOALS_AND_SCORING)
|
||||
both = GOALS_AND_SCORING.replace("Ship.", "Ship less.").replace("Me.", "Everyone.")
|
||||
found = self.new_store(frozenset({"Goals"})).write(both, held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("Audience", found.reason)
|
||||
self.assertNotIn("Goals", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
written = self.new_store(frozenset({"Goals", "Audience"})).write(both, held.version)
|
||||
self.assertIsInstance(written, Held)
|
||||
self.assertEqual(self.jira.puts(), [{"fields": {"description": fake_adf(both)}}])
|
||||
|
||||
def test_changes_scoring_and_decisions_without_approval(self) -> None:
|
||||
held = self.held(GOALS_AND_SCORING)
|
||||
text = GOALS_AND_SCORING.replace("Value.", "Risk.") + "\n## Decisions\n- A choice.\n"
|
||||
self.assertIsInstance(self.new_store().write(text, held.version), Held)
|
||||
self.assertEqual(self.jira.puts(), [{"fields": {"description": fake_adf(text)}}])
|
||||
|
||||
def test_a_text_the_codec_cannot_convert_is_not_sent(self) -> None:
|
||||
held = self.held("## Decisions\n- one\n")
|
||||
store = EpicDescription(EPIC, self.env, self.jira.http, fake_markdown, lambda text: Failure("unsupported markdown"))
|
||||
found = store.write("## Decisions\n- two\n", held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("ABC-12", found.reason)
|
||||
self.assertIn("unsupported markdown", found.reason)
|
||||
self.assertEqual(self.jira.puts(), [])
|
||||
|
||||
def test_a_refused_request_fails_naming_the_epic(self) -> None:
|
||||
held = self.held("## Decisions\n- one\n")
|
||||
refusing = lambda method, url, authorization, body: Failure("HTTP 403: no permission") if method == "PUT" else self.jira.http(method, url, authorization, body)
|
||||
found = EpicDescription(EPIC, self.env, refusing, fake_markdown, fake_adf).write("## Decisions\n- two\n", held.version)
|
||||
assert isinstance(found, Failure), found
|
||||
self.assertIn("ABC-12", found.reason)
|
||||
self.assertIn("HTTP 403", found.reason)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Whether a Read or a Grep reads a credentials file, or a Bash command names it or searches a folder holding it, for the `env-file-read` rule.
|
||||
|
||||
The files are `~/.claude/.env` and every config's `env-file`, not only the calling project's: a worker in one client's project must not read another client's credentials either, and listing every config's file needs no git call, as finding the calling project's would.
|
||||
The files are `~/.claude/.env` and every config's `env-file`, do-epic's included, not only the calling project's: a worker in one client's project must not read another client's credentials either, and listing every config's file needs no git call, as finding the calling project's would.
|
||||
A Bash command names one where its text spells the file's path, or one of its words, a redirect's target included, leads to the file from the directory `cd` left the command in. Whether the command reads the file is not judged, so one that only mentions the path is refused too.
|
||||
It searches a folder holding one where `folder_search.searches` finds that an `ag`, `grep -r` or `rg` reaches the file.
|
||||
This guards against an agent reading the file in passing, not against one that hides the path, such as behind a glob, a variable of its own, `xargs` or `find -exec`."""
|
||||
|
||||
+41
-1
@@ -19,6 +19,7 @@ Each project principle states one rule that the decisions below apply; their ent
|
||||
- bin/: the tools support only the latest stable Python, and the tests, written in plain unittest, run on it in four pytest-xdist workers
|
||||
- chunk-pr: the chunks of a project merge one at a time on a machine, each finishing its plan items in its turn, and a pull request that waits for an approval leaves the queue while it waits
|
||||
- deploy: this repo keeps no `AGENTS.md`, since `~/.claude` is its checkout; its principles and decision titles head `docs/decisions.md`, its working rules follow them there, and its `.claude/settings.json` excludes every `.claude/CLAUDE.md`, so a session in the clone loads only the clone's `CLAUDE.md`
|
||||
- do-epic: drives one Jira epic whose items span repos, or have none, ranked by the epic's goals, then code health, then each repo's goals, and closes the epic only once the human confirms every goal
|
||||
- loop-claude: each verdict records the claude-config release and commit that ran it, the raw tokens each thread spent by model and kind since the previous outcome that recorded a spend, and the human's typed answer the judge quotes as reversing a choice the worker took on its own recommendation
|
||||
- loop-claude: each chunk runs in its own tmux window, of one tmux session per project, up to `parallel-chunks` at once, and needs tmux 3.2; its claude session is named after its window, its id derived from that name; a chunk's commit in the config's folder takes only the file it changed
|
||||
- machine load: a heavy command waits for one of the machine's heavy slots, one for every 4 CPUs, which every session shares, and loop-claude starts a worker only while at least a quarter of the machine's memory is available
|
||||
@@ -139,6 +140,25 @@ Valid while `~/.claude` is a checkout of this repo.
|
||||
|
||||
2026-11-08, lilleman, todo item 271. Serves goal 1.5 and goal 2. Nobody can push to `main` on the remote, and it merges only by fast-forward, so a cut that commits on `main` and pushes no longer works. The tool cuts only the release `chunk-preflight` would call ready to cut, through `bin/release_plan/release_standing.py`, so the two never disagree on which release is next or whether it is ready; the version is that release's, never the tool's choice. It builds the cut commit in a worktree of its own, so the clone's checkout stays as it is. It finds the merged commit on `main` by its title, `Cut release <version>`, since GitHub's nearest merge, a rebase merge, may give the commit a new hash. It takes only a pull request from origin's own branch whose head is the commit that branch points at, and has the forge merge it only while its head is still that commit, so a fork's pull request from a branch of the same name, or a push after the checks passed, is never merged and deployed. Each step first checks whether an earlier run did it, so a rerun continues. Releases stay the human's, so the human runs it, and the forge credentials stay theirs. Valid while `main` takes changes only through a pull request.
|
||||
|
||||
## do-epic: drives one Jira epic whose items span repos, or have none, ranked by the epic's goals, then code health, then each repo's goals, and closes the epic only once the human confirms every goal
|
||||
|
||||
2026-11-18, lilleman's answers to claude's questions while planning release 0.12.0. Serves goal 1, so development scales past one repo, and goal 1.6, since the epic stays in the client's Jira. Valid while Jira is the only backend; another backend goes behind the same interface.
|
||||
|
||||
- **What it is.** `do-epic <EPIC-KEY>` is a sibling of loop-claude, built on the same cycle, judge, interrupt, tmux and routing packages, not a copy (goal 1.3). It runs one tmux session per epic.
|
||||
- **Machine settings.** `~/.claude-projects/do-epic/` holds `config.toml` (the Jira site, env-file, the folder repos are cloned into, the default language, and the model for a no-repo item) and `repo-rule.md`. `repo-rule.md` is a prose instruction that says how to tell an item's repo. It is the same for every epic on the machine. On the first machine, an item's label that equals a repo's name names that repo.
|
||||
- **The epic's description is its record.** It holds `## Goals`, `## Audience`, `## Scoring` (formula and bar), `## Decisions` and a language line. do-epic writes all of it, but it changes Goals or Audience only after the human approves in an interrupt.
|
||||
- **Items live only in the epic.** A repo's own plan never mirrors them. Where `repo-rule.md` leaves an item's repo unclear, do-epic asks. An item without a repo is either Claude's work, done without a PR, or a human task; where the item doesn't say which, do-epic asks.
|
||||
- **One item, several repos.** do-epic splits such an item when it plans it, into one Jira sub-task per repo, each labelled for its repo. The parent tracks the whole change. A person's issue is split too, with a comment for the team.
|
||||
- **Ranking.** Score, then Blocked by, in one weight chain: the epic's achieve goals, then code health, then the repo's goals. Where the epic and a repo disagree, the epic wins. do-epic re-ranks Jira's backlog to match, and people's issues keep their rank.
|
||||
- **Releases.** An epic item has no release. It ships into its repo's unreleased work: a bullet under `CHANGELOG.md` → `## Unreleased`, or nothing where the repo's plan is in Jira.
|
||||
- **Repos.** A repo's name is its main checkout's folder name, and a project config names that checkout. A missing checkout is cloned from the host of the epic's other repos; where unsure, do-epic asks. Unsure means the other repos don't all sit under one host and owner (the same host spelt as https and as ssh counts as two), an origin can't be read or isn't an https, ssh or scp-style address, the folder to clone into holds something else, or the clone fails. A repo no config names is cloned into the clone folder, and the human is asked to write its config. Each chunk uses its repo's model, routing, and language for commits and PRs. Jira text is in the epic's language, or else the machine's default.
|
||||
- **Running.** The epic caps the chunks running at once, and each repo's `parallel-chunks` still applies. do-epic and loop-claude never run chunks in the same repo at once.
|
||||
- **Long gates.** A chunk whose PR waits on a gate, such as a human approval, parks. It keeps its worktree, PR and session, and its state is in a file that survives a reboot. A parked chunk takes no running slot, and the items it blocks wait. do-epic polls the forge and resumes the chunk's session when the gate passes or a reviewer comments. It fixes reasonable feedback, replies to the rest, and changes the setup so review would have caught it. `do-epic <key> --status` and the tmux window names show what each chunk waits on.
|
||||
- **Human tasks.** do-epic DMs the human when one blocks the next item, and polls until its issue closes.
|
||||
- **Findings.** A finding about the epic's goals is filed in the epic. A finding about a repo's goals follows that repo's rules. Where unsure which, do-epic asks. A set of new items toward an epic goal needs the human's approval before work starts; a finding filed during a chunk doesn't.
|
||||
- **Finishing.** When the epic holds no open item: release-review passes on each repo the epic changed, then the items they file, then the human confirms each achieve goal and that no maintain goal has a defect. For a goal that isn't met, do-epic proposes items for approval. Then a summary comment on the epic, and closing the epic in Jira, as the very last step.
|
||||
- **Tests.** Black-box tests run against a fake Jira backend. An opt-in e2e test runs as its own command against a sandbox epic on a real Jira site.
|
||||
|
||||
## larv-review: within a chunk lifting a project towards the comprehension floor, work on the project's lowest-scoring dimension comes first, and the floor doesn't block anything
|
||||
|
||||
2026-09-20, lilleman. Serves goal 1. Across six projects scored with the four seats, Locality or Self-sufficiency was the lowest dimension at every seat and Navigation at none, so a layout fix would not lift a project under the floor. Valid while the scoring run keeps those four seats.
|
||||
@@ -308,7 +328,7 @@ Valid while no published standard sets either number, the reading-cost window st
|
||||
|
||||
2026-10-08, lilleman. Serves goal 1.4: run beside the full scan, `/security-review` started seconds apart from it each round and returned a near-identical report. Valid while the full scan covers what `/security-review` reports.
|
||||
|
||||
## review: a fix for a 🔴 reruns the pass that found it, unless that pass is at its cap, and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟡, a 🟢, a ❓ or a prose finding reruns none; the stability pass doesn't rerun on a round's fixes where every 🔴 and 🟡 of that round asked only that a named mutant be killed or a named failing test pass, and the tests and the mutation command run clean on the fixes
|
||||
## review: a fix for a 🔴 reruns the pass that found it, unless that pass is at its cap, and each pass whose row in `larv-review` §3's re-entry table the fix meets; from round 2 a pass reviews only the answers to its last round's findings and any other pass's fix that re-entered it; a fix for a 🟢, a ❓ or a prose finding doesn't rerun any pass, and a fix for a 🟡 reruns the pass that raised it only when the fix weakens, skips or guards a test; the stability pass doesn't rerun on a round's fixes where every 🔴 and 🟡 of that round asked only that a named mutant be killed or a named failing test pass, and the tests and the mutation command run clean on the fixes
|
||||
|
||||
2026-10-08, lilleman. Serves goal 1.4. An audit of the four costliest archived chunks found nearly every finding that mattered in a pass's first round; when every fix, a nit's included, re-entered review, later rounds mostly re-checked the last fix, reworded what an earlier round wrote, or contradicted another round on the same line. A round 2 caught real bugs only where a fix changed behaviour, so stability still reruns on every fix that does, and security on every fix that opens an untrusted-input boundary or touches authn/authz, secrets or crypto. Valid while round-2 reviews of nit and prose fixes change only what a user wouldn't notice.
|
||||
|
||||
@@ -318,6 +338,8 @@ Valid while no published standard sets either number, the reading-cost window st
|
||||
|
||||
2026-11-12, lilleman through todo item 308. Serves goal 1.4. Since the worker runs the mutation command before each stability round, a finding that asks only that a named mutant be killed or a named failing test pass has an answer a run can check. The archive from 2026-09-30 to 2026-11-12 holds 326 stability reports of round 2 or later, from 13 projects, and 81 of those reports open with BLOCK (`stability-skip-count/`). For each of the 326, the 🔴 and 🟡 lines of the report before it were matched on mutant and test-failure wording. Every such line matched in 5 of those earlier reports. Read by hand, 1 of the 5 asked only that a failing test pass, and the round 2 after each of the 5 approved. So none of the 81 BLOCKs fell in a round this skips. The skip needs every 🟡 to be of that kind too: in one chunk on 2026-11-12, round 1's only 🔴 was a red suite, and round 2 blocked on a 🟡 whose answer was wrong, which no test or mutant showed. A fix may not edit the named failing test, since a test made to pass that way doesn't show anything. Fixes from such a round still rerun every other pass whose row they meet. Valid while code that such a skipped round's fixes wrote is rarely what a later chunk's 🔴 findings fix.
|
||||
|
||||
2026-11-18, lilleman through todo item 399. Serves goals 1.1 and 1.4. The 2026-10-29 entry's validity condition broke once. In a client project's pull request, stability round 1 raised a 🟡: an end-to-end payroll test passes on a used database. It proposed asserting the created row counts. The answer commit added that assertion behind a `test.skip` that runs when every count is zero. A broken payroll also creates zero rows, so a regression became a skip. The gate and the mutation run don't catch it, since a skipped test counts as passing; only the forge's auto-reviewer did. So a 🟡 fix that removes or loosens an assertion, changes an expected value, or skips, deletes or guards a test or assertion reruns the pass that raised it, as a 🔴 fix does. A 🟡 fix that only kills a named mutant or makes a named failing test pass doesn't, since the tests and the mutation command check it, and a skipped test kills no mutant. Every other 🟡 fix keeps the 2026-10-29 rule: the 2026-10-08 audit found that reruns of fixes to minor findings mostly re-checked wording. The rerun reviews only the answers, and the cap of two rounds still ends the pass. Valid while fixes of this kind are a small share of 🟡 fixes, so the extra round stays cheap.
|
||||
|
||||
## review: a comment is prose in any file; a branch changing only Markdown no agent follows runs only the prose pass and, for README or docs an outside reader lands on, the product-owner pass; plan rows a chunk files from a finding, re-scores or moves get no review
|
||||
|
||||
2026-10-29, lilleman. Serves goals 1.4 and 1.5. One project's chunk on 2026-10-29 changed a README paragraph, `todo.md` and three lines of `gate.yml`, and the principles, security and stability passes all ran on it, each twice: 6 of the chunk's 10 rounds. A change to config is one that changes what runs, so a comment in it fires only the prose pass, as a comment in code already did. A branch that changes only Markdown no agent follows changes no code, config or tests, so no pass that reads those runs on it. A plan row a chunk files from a finding, re-scores or moves is read by the chunk that ships it, so reviewing it at filing reviewed it twice: that chunk spent a prose round of 4 minutes on its plan change. Markdown an agent follows, such as a skill, is config, since a change there changes what agents do. Valid while defects in prose-only chunks stay rare in later chunks' fixes.
|
||||
@@ -399,6 +421,8 @@ Messages keep each file's usual name, such as `todo.md`, as its label, since the
|
||||
|
||||
2026-11-17, claude, through item 391. Serves goal 1.6: a team's Jira and Confluence keep each block's localId and the empty paragraphs Jira uses for spacing, and agents, the judge and people still read the text easily, which keeps goal 1.4's reading cost down. adf-codec converts markdown and ADF by two pairs of calls, as `bin/adf_codec/convert.py` says. The lossless pair keeps the localIds and spacing paragraphs, but wraps every block in two `!adf:` marker lines: a five-line detail became 17 lines in a trial (larvit/adf-codec#185). The portable pair drops both. So the loop reads by the portable pair, and leaves a document alone where a save's text is the text a read showed. `JiraPlan.shows` compares a new detail with the description converted again. `ConfluencePage` keeps the digest of the text it read in the version a write names, and checks that the page still holds that version. Valid while the lossless markdown wraps every block in marker lines. A save that changes one line still rewrites the whole document by the portable pair, and so drops every localId and spacing paragraph, until `todo.md` item 395 keeps each block the save didn't change.
|
||||
|
||||
2026-11-18, claude, item 402. Serves goal 1.6. A Jira epic's description is written by the portable pair too, though the item named the lossless pair: that pair shows the loop marker lines around its blocks, which this decision rejects for the reading cost. `EpicDescription.write` sends nothing for the text a read showed, refuses a write over a description whose document differs from the one read, and refuses a change to `## Goals` or `## Audience` unless the human approved that section. Until item 395, a save that changes one section also reduces a mention or macro in Goals or Audience to its text, which `project-config`'s help and the README tell the reader.
|
||||
|
||||
## project config: `tooling-on-host = true` lets a project with no container setup run its language runtime, package manager, tests and task runner on the host
|
||||
|
||||
2026-11-12, lilleman, through todo item 358. Serves goal 1.6: a legacy product may have no container setup, and the loop works in it without its owners adding one. `CLAUDE.md`'s container rule and the stability reviewer follow the setting. rule-hook checks only how a container starts, its memory limits and its image's pin, never where a runtime runs, so it needs no change. A decision may still put one tool on the host where the project has containers, as this repo's `bin/` decision does. Readers seated as the person who writes the config picked the key, 2 of 3. Valid while the loop runs on the machine that holds the checkout.
|
||||
@@ -664,6 +688,22 @@ A project's rung is its model's cheapest rung, which for Opus 5.5 is its default
|
||||
|
||||
Valid while the index ranks these models, and while no newer model joins the catalog.
|
||||
|
||||
## routing: the offline classifier may decide finished, since lilleman ruled a miss rate of about 1 in 100 decided close enough to none; the judge's model keeps every other verdict kind
|
||||
|
||||
2026-11-18, claude. Serves goals 1.2 and 1.4, and applies project principle 1: a cost cut waits for a measurement that shows quality holds.
|
||||
|
||||
`bin/judge-replay claude-opus-5-5 --offline` replays all archived verdicts of the Opus 5.5 projects through three deciders. Each reads only the worker's final message, doesn't send anything to an outside service, and decides a kind or leaves the verdict to the judge's model. A missed interrupt is an archived interrupt decided as another kind, which lets the worker settle a decision that is lilleman's. Deciding interrupt can't miss one: a wrong interrupt only asks lilleman more. On 2026-11-17 the archive held 1,104 verdicts (660 finished, 333 interrupt, 33 quoted answer, 78 unfinished); on 2026-11-18 it held 1,116 (668, 336, 34, 78).
|
||||
|
||||
- **A "?" rule** (interrupt where the message holds a "?", finished where it holds none) decided every one of the 1,104 and missed 197 interrupts.
|
||||
- **A decision table**, the rows of `DECISION_TABLE` in `bin/judge_replay/offline.py`. Its rows were read from this archive's interrupts, so its run isn't a held-out test. It decided 974 of the 1,104 and missed 55. Most of its misses are a worker reporting a merged chunk while a release is ready to cut, which depends on the plan; the rest ask for an answer in words no row names.
|
||||
- **A classifier**, a logistic regression over the message's words, trained on the archive. It decides finished only where its chance of an interrupt is below that of every interrupt among its training folds when held out, and interrupt only where it is above that of every other kind. It averages five regressions that differ only in the order they shuffle the samples, since a single shuffle moved its missed interrupts between 0 and 2 on the same archive. Its bar is no missed interrupt on both tests: held-out folds, each tested by a model trained on the other folds, and the latest fifth of the verdicts, learning from the rest.
|
||||
- On the 1,104 verdicts, held-out folds: 103 decided, none missed. Latest fifth: 29 decided, all as finished, and 1 interrupt missed. That fails the bar, and shuffles 5 to 9 and 10 to 14 failed it alike.
|
||||
- On the 1,116 verdicts, held-out folds: 95 decided (84 finished, 11 interrupt), none missed. Latest fifth: 20 decided, all as finished, none missed. That meets the bar for finished.
|
||||
|
||||
The classifier decides about 1 in 12 verdicts, and about 1 in 100 of what it decides were an interrupt on the 1,104. A latest fifth that starts a few verdicts later no longer holds the one it missed, so the bar flips with the archive's size and doesn't separate a classifier that never misses from one that rarely does. On 2026-11-18 lilleman ruled, at loop-claude's terminal, that a miss rate of about 1 in 100 decided is close enough to none. So the classifier may decide finished and skip the judge's call; the plan holds that change as an item. The judge's model keeps every verdict the classifier leaves, the escalation it quotes and whether a release is ready to cut, since the replay didn't test the classifier on those.
|
||||
|
||||
Valid while the archived verdicts hold interrupts that read like a finished chunk. Replay with `judge-replay <model> --offline` when the archive has doubled, or when a decider reads more than the final message.
|
||||
|
||||
## routing: the worker changes rung only as a chunk starts, after every second unfinished verdict, when it names a chunk item that picks a higher rung, or when it cites an escalation trigger the judge finds
|
||||
|
||||
2026-10-14, lilleman. Serves goal 1.4.
|
||||
|
||||
@@ -2541,3 +2541,102 @@ Read most (opens and searches):
|
||||
- `bin/todo-item` 1
|
||||
- `rules/decisions.md` 1
|
||||
- `todo.md` 1
|
||||
|
||||
## Reading cost, chunk 8174e3b60f07, finished 2026-11-17T08:26:23Z
|
||||
|
||||
92410 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `bin/loop_claude/cycle/prompts.py` 3
|
||||
- `rules/todo-items.md` 3
|
||||
- `workflow-bench/bench.py` 2
|
||||
- `bin/loop_claude/cycle/run.py` 1
|
||||
- `bin/loop_claude/cycle/worker.py` 1
|
||||
|
||||
## Reading cost, chunk 91d4ddb5d55e, finished 2026-11-17T11:39:35Z
|
||||
|
||||
68667 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `.gitignore` 4
|
||||
- `README.md` 4
|
||||
- `CLAUDE.md` 3
|
||||
- `bin/project-record` 2
|
||||
- `bin/project_record/main.py` 2
|
||||
- `bin/project_record/test_project_record.py` 2
|
||||
- `bin/todo_item/main.py` 2
|
||||
- `bin/failure.py` 1
|
||||
- `bin/markdown_section/test_markdown_section.py` 1
|
||||
- `bin/run-tests` 1
|
||||
|
||||
## Reading cost, chunk 4655673a30a9, finished 2026-11-17T11:46:22Z
|
||||
|
||||
111449 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `bin/review_replay/test_review_replay.py` 6
|
||||
- `README.md` 2
|
||||
- `bin/review_replay/main.py` 2
|
||||
- `bin/review_replay/replay.py` 2
|
||||
- `docs/decisions.md` 2
|
||||
- `CHANGELOG.md` 1
|
||||
- `bin/review_replay/omit_comparison.py` 1
|
||||
- `bin/rung_replay/archive.py` 1
|
||||
- `bin/rung_replay/search.py` 1
|
||||
- `bin/rung_replay/test_rung_replay.py` 1
|
||||
|
||||
## Reading cost, chunk 59dcab68a4fb, finished 2026-11-18T09:15:05Z
|
||||
|
||||
52225 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `docs/decisions.md` 6
|
||||
- `skills/larv-review/SKILL.md` 5
|
||||
|
||||
## Reading cost, chunk 3cc43a838262, finished 2026-11-18T09:22:31Z
|
||||
|
||||
57869 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `CLAUDE.md` 5
|
||||
- `skills/larv-review/SKILL.md` 4
|
||||
- `docs/decisions.md` 1
|
||||
|
||||
## Reading cost, chunk e880525b42ae, finished 2026-11-18T09:56:42Z
|
||||
|
||||
89443 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `README.md` 6
|
||||
- `bin/plan_store/jira_epic.py` 6
|
||||
- `bin/plan_store/test_jira_epic.py` 5
|
||||
- `.gitignore` 4
|
||||
- `bin/atlassian/client.py` 3
|
||||
- `bin/plan_store/contract.py` 2
|
||||
- `docs/decisions.md` 2
|
||||
- `bin/adf_codec/convert.py` 1
|
||||
- `bin/do_epic/backend/contract.py` 1
|
||||
- `bin/do_epic/backend/protocol.py` 1
|
||||
|
||||
## Reading cost, chunk 3312afb8e236, finished 2026-11-18T11:56:17Z
|
||||
|
||||
502050 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `README.md` 14
|
||||
- `bin/project_config/settings.py` 13
|
||||
- `docs/decisions.md` 10
|
||||
- `bin/project_config/main.py` 9
|
||||
- `.gitignore` 7
|
||||
- `skills/larv-review/SKILL.md` 7
|
||||
- `bin/project_config/test_project_config.py` 6
|
||||
- `bin/model_routing/ladder.py` 3
|
||||
- `ci.sh` 3
|
||||
- `docs/reading-cost.md` 3
|
||||
|
||||
@@ -87,18 +87,19 @@ Give a reviewer without Bash `git ls-files`'s output in its prompt, since some h
|
||||
- **Done.** A pass is done when its latest round leaves no 🔴 or 🟡 standing: none was reported, or each was shown factually wrong, declined, filed, or named under the bar per "Findings", or settled in the PR summary per "Comprehension check" → "Places", or, for the prose pass and a pass at its cap, fixed. A stability pass whose round's fixes skip the next round per "Re-entry" → "Mechanical answers" is done once that rule's run is clean. Resolve that round's 🟢s and ❓s per "Findings" too.
|
||||
- **Cap.** A pass takes at most two rounds per chunk, and the release review's architecture and principles passes five; don't ask the user for more. Answer its last round's findings per "Findings": those fixes merge once the gate passes, unreviewed by that pass. rule-hook's `round-cap` refuses a round commit that would take a pass past its cap, and no override waives it. A review the auto-reviewer posts after its second isn't a round: fix none of it and rerun no pass for it, but file a security finding it raises per `~/.claude/CLAUDE.md` → "Working loop", and any other 🔴 in `todo.md`.
|
||||
|
||||
**Re-entry.** A fix is a change no pass has seen. A fix for a 🟡, a 🟢, a ❓ or a prose finding reruns no pass, whatever it changes. A round's fixes for 🔴 findings re-enter together: run every pass that a row below names for any of them, a done pass included, in the waves of "Waves", then §4; "Mechanical answers" below names the one exception. A pass at its cap doesn't run again ("Round cap" → "Cap").
|
||||
**Re-entry.** A fix is a change no pass has seen. A fix for a 🟢, a ❓ or a prose finding doesn't rerun any pass, whatever it changes. A fix for a 🟡 meets only the 🟡 row below. A round's fixes for 🔴 findings, and its 🟡 fixes that meet that row, re-enter together: run every pass that a row below names for any of them, a done pass included, in the waves of "Waves", then §4; "Mechanical answers" below names the one exception. A pass at its cap doesn't run again ("Round cap" → "Cap").
|
||||
|
||||
| The fix | Reruns |
|
||||
|---------|--------|
|
||||
| answers a 🔴 of a pass other than prose | that pass |
|
||||
| answers a 🟡 of a pass other than prose, and removes or loosens an assertion, changes an expected value, or skips, deletes or guards a test or assertion with a condition or a catch, unless it only kills a named mutant or makes a named failing test pass | that pass |
|
||||
| carries an architectural decision, in the release review | Architecture |
|
||||
| changes code, config or tests, in the release review | Principles |
|
||||
| opens an untrusted-input boundary or touches authn/authz, secrets or crypto | Security, unless the full scan reruns on it |
|
||||
| made rule-hook log an override | Overrides |
|
||||
| changes behaviour: what the code or its tests do | Stability |
|
||||
|
||||
A fix meets every row it fits. Prose has no row, so a comment or doc edit inside a fix doesn't rerun any pass. Example: a fix for a stability 🔴 that changes code and a comment reruns Principles and Stability.
|
||||
A fix meets every row it fits, except that a fix for a 🟡 meets only the 🟡 row. Such a fix meets no row once §2's tests and mutation command run clean on it and the named failing test is unedited ("Mechanical answers"); a skipped test kills no mutant. Prose has no row, so a comment or doc edit inside a fix doesn't rerun any pass. Example: a fix for a stability 🔴 that changes code and a comment reruns Principles and Stability.
|
||||
|
||||
**Mechanical answers.** A stability round's fixes may skip the next stability round. That holds where the round reported a 🔴, and each of its 🔴 and 🟡 findings asked only that a named mutant be killed or a named failing test pass. Run §2's tests and mutation command on the fixes, and fix again per §2 until they leave no failing test and no surviving mutant that changes behaviour. Each named failing test must pass as the round found it, not edited or deleted. Then the stability pass doesn't run again, and the round's answer commit title ends `; tests and mutants clean at <commit>`. The fixes still rerun each other pass whose row they meet. Where the run can't be made clean, or a fix edits a named failing test, the stability pass runs again as the table says.
|
||||
|
||||
@@ -195,7 +196,7 @@ Launch prose-reviewer with what the diff touched and every section this review c
|
||||
3. The forge's checks: `chunk-pr merge` waits for them to pass (§5 step 4). A red check is a finding.
|
||||
4. No CI configured → the gate is local tests only; note it in the summary.
|
||||
5. The local tests green → mark the PR ready with `~/.claude/bin/chunk-pr ready '<final title>'`, which sets the title as it marks the PR ready.
|
||||
6. **External auto-reviewer**, when the repo has one — its config file (`.coderabbit.yaml` and the like), or its comments on earlier PRs; none → skip it, say so. Marking the PR ready is what summons it. Wait up to **10 minutes** for its first review, checking about once a minute; nothing by then → note it in the summary and carry on, never waiting twice. Its rounds, §3 → "Round cap". Once it has reviewed, each later return here before its cap, after a fix re-entered per §3, waits up to **5 minutes** for its re-review of the new commit.
|
||||
6. **External auto-reviewer**, when the repo has one — its config file (`.coderabbit.yaml` and the like), or its comments on earlier PRs; none → skip it, say so. Marking the PR ready is what summons it. Wait up to **10 minutes** for its first review, checking about once a minute; nothing by then → note it in the summary and carry on, never waiting twice. Its rounds, §3 → "Round cap". Once it has reviewed, each later return here before its cap, after a fix re-entered per §3, waits up to **5 minutes** for its re-review of the new commit. A fix of its finding ends with the setup change that `~/.claude/CLAUDE.md` → "Working loop", "Change the setup for each reasonable fix an outside reviewer asked for on a PR" asks for.
|
||||
|
||||
## 5. Goal alignment, then merge
|
||||
|
||||
@@ -206,7 +207,7 @@ Launch prose-reviewer with what the diff touched and every section this review c
|
||||
- **A merge conflict.** Resolve it, commit and push. Rerun only the passes whose triggers the resolution's own lines meet (§3), each in a round of its own.
|
||||
- **A paired ruling to run again.** `chunk-pr merge` has merged the default branch in and pushed. Run the ruling at that head, record it, commit and push. Don't return to §4 for it: the next turn waits for the checks at the new head.
|
||||
- **An owed scoring run, or a queue another pull request holds.** Record the run, or wait, as the message says.
|
||||
- **A reviewer's comment**, where the project's config sets `wait-for-approval` (`chunk-pr --help`) and the pull request waits for a human's approval. Answer it in its thread on the forge, and fix what it asks as a finding of the chunk. The wait doesn't hold the queue, so other chunks merge meanwhile.
|
||||
- **A reviewer's comment**, where the project's config sets `wait-for-approval` (`chunk-pr --help`) and the pull request waits for a human's approval. Answer it in its thread on the forge, and fix what it asks as a finding of the chunk. A fix of it ends with the same setup change (§4 step 6). The wait doesn't hold the queue, so other chunks merge meanwhile.
|
||||
|
||||
A repo whose `AGENTS.md` names another method merges that way with the forge's CLI instead, after it finishes the chunk's items with `todo-item finish` in the branch, and once the pull request's checks pass, `~/.claude/bin/larv-review scoring-run-owed` exits 0 and, where `~/.claude/bin/project-config wait-for-approval` prints `true`, GitHub reports the pull request approved. A repo withholds this in its `AGENTS.md` → leave the PR ready and unmerged, and say which line withheld it.
|
||||
5. Merged → run `~/.claude/bin/chunk-pr clean` in the chunk's worktree, and put a failure it prints in the summary.
|
||||
@@ -214,4 +215,4 @@ Launch prose-reviewer with what the diff touched and every section this review c
|
||||
|
||||
## 6. Wrap up
|
||||
|
||||
Summarize: PR URL and final state, every reviewer's verdict, rounds per pass, passes skipped, principles files left out with why, findings fixed vs answered vs declined vs open — each answer with what settles it, each decline with its reason, a 🟡's with the goal or technical principle it rests on — CI status, rule overrides with their rulings, decisions recorded (file paths), `todo.md`/`CHANGELOG.md` updated, and any 🔀 escalation left for the user.
|
||||
Summarize: PR URL and final state, every reviewer's verdict, rounds per pass, passes skipped, principles files left out with why, findings fixed vs answered vs declined vs open — each answer with what settles it, each decline with its reason, a 🟡's with the goal or technical principle it rests on — CI status, rule overrides with their rulings, decisions recorded (file paths), for each fixed outside-reviewer comment, the setup change it led to, or that no setup gap caused it, `todo.md`/`CHANGELOG.md` updated, and any 🔀 escalation left for the user.
|
||||
|
||||
@@ -6,26 +6,43 @@
|
||||
|
||||
`Bar = 0.1`
|
||||
|
||||
`Next ID = 398`
|
||||
`Next ID = 424`
|
||||
|
||||
## Items
|
||||
|
||||
| ID | Release | Blocked by | Exempt | Item | L | E | I | R | S | Moves | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 396 | 0.12.0 | | | **Print one section of a markdown file, picked by a fragment of its heading, with `markdown-section <file> <fragment>`.** | 2 | 1 | 3 | 4 | 4 | 1.4 by 3, 1.5 by 2 | 1.4 |
|
||||
| 299 | 0.12.0 | | | **Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 401 | 0.12.0 | | | **Move loop-claude's cycle, judge, interrupt, tmux and routing into a package a second entry point drives, with loop-claude's behaviour unchanged.** | 3 | 2 | 3 | 6 | 6 | 1 by 3, code health by 2 | 1.8 |
|
||||
| 406 | 0.12.0 | | | **Pick each epic item's repo, or for an item without one whether it is Claude's work or a human task, by `repo-rule.md`, record the pick on the issue, and ask the human where the rule leaves it unclear.** | 2 | 2 | 2 | 4 | 4 | 1 by 5 | 1.6 |
|
||||
| 408 | 0.12.0 | | | **Rank an epic's items by one weight chain, the epic's achieve goals, then code health, then the item's repo goals, then by Blocked by, and re-rank Jira's backlog to match, with people's issues keeping their rank.** | 3 | 2 | 2 | 5 | 4 | 1 by 5 | 1.6 |
|
||||
| 410 | 0.12.0 | 401, 406, 408 | | **Run `do-epic <EPIC-KEY>`: one tmux session per epic, a window per chunk, up to the epic's cap of running chunks and each repo's `parallel-chunks`, and never in a repo a loop-claude is running chunks in.** | 3 | 2 | 3 | 6 | 6 | 1 by 5 | 0.8 |
|
||||
| 416 | 0.12.0 | 410 | | **Finish an epic's repo item with a bullet under its repo's `CHANGELOG.md` → `## Unreleased`, or with none where the repo's plan is in Jira.** | 1 | 1 | 1 | 1 | 2 | 1 by 5 | 2.1 |
|
||||
| 413 | 0.12.0 | 410 | | **File a finding about an epic goal in the epic and one about a repo goal by that repo's rules, ask the human where unsure which, and give the judge the epic's records beside the repo's.** | 3 | 2 | 3 | 6 | 4 | 1 by 5 | 1.6 |
|
||||
| 415 | 0.12.0 | 410, 413 | | **Finish an epic: run release-review passes on each repo it changed, work what they file, have the human confirm each goal, post a summary comment, and close the epic in Jira last.** | 3 | 2 | 3 | 6 | 5 | 1 by 5 | 1.2 |
|
||||
| 418 | 0.12.0 | 415 | | **Describe do-epic in `README.md`'s tools and setup, with its machine settings and `repo-rule.md`.** | 1 | 2 | 1 | 2 | 2 | 1 by 5 | 2.1 |
|
||||
| 400 | 0.12.0 | | | **Forbid a test that decides at run time whether to check anything in `principles/technical-principles.md`, and point stability-reviewer's Tests checks at it.** | 1 | 2 | 3 | 4 | 2 | 1.1 by 5 | 2.0 |
|
||||
| 420 | 0.12.0 | | | **End a worker that printed its final result and lingers, by killing the background shells it left, after a few minutes, and print each kill with its evidence on loop-claude's terminal.** | 2 | 2 | 2 | 4 | 3 | 1.5 by 6 | 2.0 |
|
||||
| 409 | 0.12.0 | 406 | | **Split an epic item that changes several repos into one Jira sub-task per repo, each labelled for its repo, with the parent tracking the whole change, when do-epic plans the item.** | 2 | 2 | 3 | 5 | 3 | 1 by 5 | 1.9 |
|
||||
| 411 | 0.12.0 | 410 | | **Park a chunk whose PR waits on a long gate, keeping its worktree, PR and session in a state file that survives a reboot, and resume its session when the gate passes or a reviewer comments.** | 3 | 2 | 3 | 6 | 5 | 1 by 5 | 1.2 |
|
||||
| 412 | 0.12.0 | 410, 411 | | **Print each chunk of an epic, running, parked with its gate and since when, or stopped, each human task and each open interrupt with `do-epic <key> --status`, and name a parked chunk's gate in its tmux window's name.** | 2 | 1 | 1 | 2 | 3 | 1 by 5 | 1.9 |
|
||||
| 414 | 0.12.0 | 410 | | **DM the human through the notifier when a human-task issue blocks the epic's next item, and poll until it closes.** | 2 | 2 | 2 | 4 | 3 | 1 by 5 | 1.9 |
|
||||
| 417 | 0.12.0 | 410 | | **Run do-epic end to end against a sandbox epic on a real Jira site with a command of its own, outside `run-tests`.** | 2 | 1 | 1 | 2 | 3 | 1 by 5 | 1.9 |
|
||||
| 421 | 0.12.0 | | | **Refuse in rule-hook a Bash or Monitor wait that has no end: a loop without a time or count cap, and a `pgrep -f` or `pkill -f` whose pattern the same command line contains.** | 2 | 2 | 3 | 5 | 3 | 1.5 by 4 | 1.3 |
|
||||
| 311 | 0.12.0 | | | **Hand each worker's claude a Bash output ceiling that saves longer output to a file, and refuse a Read of a session transcript under `~/.claude/projects`.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 316 | 0.12.0 | | | **Launch comprehension seats as an agent type of their own under `agents/`, without `CLAUDE.md` and the skill and agent listings a general-purpose subagent carries.** | 2 | 2 | 2 | 4 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 339 | 0.12.0 | | | **Lower the reading cost of the files a chunk reads most before its first commit.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 341 | 0.12.0 | | | **Hand each worker's claude a lower Read limit through `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS`, kept only where replays of the archived review rounds find what two unchanged replays find.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 310 | 0.12.0 | | | **Compact a waiting worker's session shortly before its prompt cache expires, only when the human has not answered an interrupt by then.** | 3 | 2 | 3 | 6 | 4 | 1.4 by 3 | 0.9 |
|
||||
| 298 | 0.12.0 | | | **Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.** | 3 | 3 | 3 | 7 | 4 | 1.4 by 2, 1.5 by 1 | 0.9 |
|
||||
| 422 | 0.12.0 | 401, 420 | | **Check every 5 minutes whether a tmux window of the loop looks stuck by other signs than a lingering worker, investigate it with a headless claude, and kill only the process the investigation names.** | 3 | 2 | 3 | 6 | 5 | 1.5 by 4 | 0.8 |
|
||||
| 365 | 0.13.0 | | | **Have `score-outcomes` read the finished and dropped items of a plan outside the checkout, from the project repo's history or from Jira's issue changelogs.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
|
||||
| 368 | 0.13.0 | | | **Call a release review due, call a release ready to cut, and tell whether a chunk owes a scoring run, for a plan outside the checkout, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.1 by 2 | 1.2 |
|
||||
| 309 | 0.13.0 | | | **Run a scoring run's second pass on one head with two seats in place of four, once `larv-review agreement` shows the seats agree with each other more than a run agrees with its rerun.** | 2 | 2 | 2 | 4 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 419 | 0.13.0 | | | **Run the external auto-reviewer until its review of the PR's head has no findings, and answer each of its threads on the PR.** | 3 | 2 | 3 | 6 | 5 | 1.1 by 5 | 1.1 |
|
||||
| 366 | 0.13.0 | | | **Count the chunks of a project whose plan sits outside the checkout in the routing figures, from the project repo's history or from Jira.** | 3 | 2 | 2 | 5 | 4 | 1.6 by 2, 1.4 by 2 | 1.1 |
|
||||
| 339 | 0.13.0 | | | **Lower the reading cost of the files a chunk reads most before its first commit.** | 2 | 2 | 1 | 3 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 341 | 0.13.0 | 311 | | **Hand each worker's claude a lower Read limit through `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS`, kept only where replays of the archived review rounds find what two unchanged replays find.** | 2 | 2 | 3 | 5 | 3 | 1.4 by 3 | 1.0 |
|
||||
| 298 | 0.13.0 | | | **Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.** | 3 | 3 | 3 | 7 | 4 | 1.4 by 2, 1.5 by 1 | 0.9 |
|
||||
| 395 | 0.13.0 | | decision | **Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.** | 3 | 2 | 2 | 5 | 5 | 1.6 by 4 | 0.8 |
|
||||
| 367 | 0.13.0 | | | **Check that every write to a Jira plan leaves it ranked, as rule-hook's `todo-rank-check` checks `todo.md` today.** | 2 | 2 | 3 | 5 | 3 | 1.6 by 2 | 0.7 |
|
||||
| 423 | 0.13.0 | | | **Skip the judge's model call on a worker's final message that the offline classifier decides is finished.** | 3 | 3 | 3 | 7 | 5 | 1.4 by 2 | 0.4 |
|
||||
| 77 | 0.14.0 | | | **Teach `bin/metric-bench` Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.** | 4 | 2 | 1 | 5 | 6 | 1.1 by 7, 1.4 by 7 | 2.1 |
|
||||
| 196 | 0.14.0 | | decision | **Replay the architecture, principles, overrides and product-owner passes with `review-replay` on Opus 5.5 projects, record each run in the review-replay routing decision, and, where a run qualifies a rung, add back the code that lowers a pass to it.** | 2 | 2 | 3 | 5 | 2 | 1.4 by 3 | 1.2 |
|
||||
| 321 | 0.14.0 | 196 | decision | **Evaluate the review passes that run on a rung `review-replay` qualified, against the chunks before them, and revert any pass whose findings or later fixes worsened.** | 1 | 2 | 1 | 2 | 2 | 1.1 by 3, 1.4 by 2 | 2.0 |
|
||||
@@ -66,33 +83,200 @@
|
||||
|
||||
## Details
|
||||
|
||||
### 396. Print one section of a markdown file, picked by a fragment of its heading, with `markdown-section <file> <fragment>`.
|
||||
### 401. Move loop-claude's cycle, judge, interrupt, tmux and routing into a package a second entry point drives, with loop-claude's behaviour unchanged.
|
||||
|
||||
Found by item 338's audit of release 0.9.0. All 11 of its chunks read a section of a markdown file by hand: `grep -n` for the heading, then `sed -n <from>,<to>p` with a guessed range. 115 `sed -n` ranges and 42 `grep -n` calls on README.md, SKILL.md, CHANGELOG.md, `docs/` and `~/.claude/rules/*.md` cost about 10k command tokens and 112k result tokens (characters / 4), 447k characters. `docs/decisions.md` alone took 74 such calls in 10 chunks and 310k result characters; each of its 80 decisions is a `## ` section.
|
||||
|
||||
The tool prints from the first heading whose text holds the fragment, case-insensitive, to the next heading of the same or a higher level. Where nothing matches it fails, saying so; where several headings match it lists them. A record kept outside the checkout is the file `project-record open <record>` prints, so the tool takes that path too. It saves the locating call and the misaimed ranges, not the reading of the section; how many of the 157 calls only located a section is not measured. Once it ships, `CLAUDE.md` → "Working loop" → "Read the plan and the decisions only as far as the chunk needs" names it for reading one entry of `docs/decisions.md`, and README → Tools lists it. That list, `README.md:197`, is a place the comprehension seats found hard to navigate; make it easier to read where the entry joins it.
|
||||
|
||||
Left with the model: reading the section and judging what it says; and reading code, which item 342 researches.
|
||||
|
||||
Scored L 2: a new tool, but a small one in the pattern `todo-item show` and `project-record` already set, a package beside its launcher. I 3, since `CLAUDE.md` puts it in every session.
|
||||
do-epic runs on these, not on a copy (`docs/decisions.md` → `do-epic: drives one Jira epic …`). Move the reusable modules into `bin/agent_cycle/`: `tmux`, `judge`, `interrupt`, `verdicts`, `retry`, `spend`, `worker`, `states`, `stop`, and the parts of `routing` that take a `ProjectRung` and a `Route`. Keep `routed_item`, `loop/`, `project/`, `escalation_mark` and `chunk_items` in `loop_claude`. Decouple `cycle/run.py`'s archive and `Chunk items:` handling, and `cycle/prompts.py`'s goals-file and `todo.md` names, behind parameters. The places the comprehension seats found hard here, `routing.py` `unfinished_route` and `cycle/judge.py` `quotes_checked`, must read no harder after the move.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/markdown-section`
|
||||
- `bin/markdown_section/__init__.py`
|
||||
- `bin/markdown_section/main.py`
|
||||
- `bin/markdown_section/test_markdown_section.py`
|
||||
- `bin/loop_claude/tmux.py`
|
||||
- `bin/loop_claude/cycle/judge.py`
|
||||
- `bin/loop_claude/cycle/interrupt.py`
|
||||
- `bin/loop_claude/cycle/retry.py`
|
||||
- `bin/loop_claude/cycle/spend.py`
|
||||
- `bin/loop_claude/cycle/worker.py`
|
||||
- `bin/loop_claude/cycle/states.py`
|
||||
- `bin/loop_claude/verdicts.py`
|
||||
- `bin/loop_claude/stop.py`
|
||||
- `bin/loop_claude/routing.py`
|
||||
- `bin/loop_claude/cycle/run.py`: `Cycle`, `run_cycle`, `step`, `rule`
|
||||
- `bin/loop_claude/cycle/prompts.py`: `QUOTE_SOURCES`, `ANSWER_DESTINATIONS`
|
||||
- `bin/loop_claude/test_loop_claude.py`
|
||||
- `bin/loop_claude/test_black_box.py`
|
||||
- `bin/judge_replay/main.py`
|
||||
- `bin/judge_replay/replay.py`
|
||||
- `bin/judge_replay/archived_prompt.py`
|
||||
- `bin/rung_replay/command.py`
|
||||
- `bin/rung_replay/search.py`
|
||||
- `README.md`: `` ### Packages under `bin/` `` (small)
|
||||
|
||||
### 406. Pick each epic item's repo, or for an item without one whether it is Claude's work or a human task, by `repo-rule.md`, record the pick on the issue, and ask the human where the rule leaves it unclear.
|
||||
|
||||
The pick is a headless claude call with a JSON schema, as the judge makes (`docs/decisions.md` → `do-epic: drives one Jira epic …`). A recorded pick is read back without a new call.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/claude_code/reply.py`: `claude_reply` (small)
|
||||
- `bin/do_epic/repo_pick.py`
|
||||
|
||||
### 408. Rank an epic's items by one weight chain, the epic's achieve goals, then code health, then the item's repo goals, then by Blocked by, and re-rank Jira's backlog to match, with people's issues keeping their rank.
|
||||
|
||||
Where the epic and a repo disagree, the epic wins (`docs/decisions.md` → `do-epic: drives one Jira epic …`). An item without a repo weighs only the epic's goals. Reuse `goal_weights` and `weight_of`, and `JiraPlan.rank_refusal` for the rank call. `plan/ranking.py` `score` is a place the seats found hard, so the combiner must not add a branch to it.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/plan/ranking.py`: `weigh`, `weight_of` (small)
|
||||
- `bin/plan/goals_file.py`: `goal_weights` (small)
|
||||
- `bin/plan_store/jira_epic.py`: `JiraPlan.rank_refusal` (small)
|
||||
- `bin/do_epic/ranking.py`
|
||||
|
||||
### 410. Run `do-epic <EPIC-KEY>`: one tmux session per epic, a window per chunk, up to the epic's cap of running chunks and each repo's `parallel-chunks`, and never in a repo a loop-claude is running chunks in.
|
||||
|
||||
The lock that keeps do-epic and loop-claude apart is the per-project `starting` flock, shared by both (`docs/decisions.md` → `do-epic: drives one Jira epic …`). Each chunk routes on its repo's model and ladder, a no-repo chunk on the machine setting's model. `loop/chunk_windows.py` `start_chunks` is a place the seats found hard: make it easier to read where it changes.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/loop_claude/loop/chunk_windows.py`: `starting`, `start_chunks` (small)
|
||||
- `bin/loop_claude/tmux.py`: `session_name` (small)
|
||||
- `bin/do-epic`
|
||||
- `bin/do_epic/main.py`
|
||||
- `bin/do_epic/scheduler.py`
|
||||
|
||||
### 416. Finish an epic's repo item with a bullet under its repo's `CHANGELOG.md` → `## Unreleased`, or with none where the repo's plan is in Jira.
|
||||
|
||||
An epic item has no release (`docs/decisions.md` → `do-epic: drives one Jira epic …`). do-epic runs `chunk-pr merge --finish` in the repo's checkout, and picks the flag by the repo's plan type.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/do_epic/main.py`
|
||||
|
||||
### 413. File a finding about an epic goal in the epic and one about a repo goal by that repo's rules, ask the human where unsure which, and give the judge the epic's records beside the repo's.
|
||||
|
||||
The epic's records are its description's sections (`docs/decisions.md` → `do-epic: drives one Jira epic …`). `read_records` and `read_premises` take the epic's record as a parameter, so loop-claude's judge is unchanged.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/loop_claude/project/records.py`: `read_records` (small)
|
||||
- `bin/loop_claude/project/premises.py`: `read_premises` (small)
|
||||
- `bin/loop_claude/cycle/judge.py`: `judge_prompt` (small)
|
||||
- `bin/loop_claude/cycle/prompts.py`: `ANSWER_DESTINATIONS` (small)
|
||||
- `skills/larv-review/SKILL.md`: `## 3. Passes` (small)
|
||||
- `bin/do_epic/findings.py`
|
||||
|
||||
### 415. Finish an epic: run release-review passes on each repo it changed, work what they file, have the human confirm each goal, post a summary comment, and close the epic in Jira last.
|
||||
|
||||
A goal is met only where the human confirms it: each achieve goal achieved, and no defect against a maintain goal. For a goal that isn't met, do-epic proposes a set of items, and the human approves the set before work starts (`docs/decisions.md` → `do-epic: drives one Jira epic …`). The summary names what shipped per repo and each goal's evidence, in the epic's language. The epic's transition is configured as `todo-item finish`'s is.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/plan_store/jira_epic.py`: `JiraPlan.finish`, `transition_of` (small)
|
||||
- `bin/release_plan/release_standing.py`: `release_reviewed` (small)
|
||||
- `bin/do_epic/finish.py`
|
||||
|
||||
### 418. Describe do-epic in `README.md`'s tools and setup, with its machine settings and `repo-rule.md`.
|
||||
|
||||
Each chunk keeps the README true for what it changes; this item writes the overview a new reader needs (`docs/decisions.md` → `do-epic: drives one Jira epic …`).
|
||||
|
||||
Touches:
|
||||
|
||||
- `README.md`: `## Tools`, `### Set up a project` (small)
|
||||
|
||||
### 400. Forbid a test that decides at run time whether to check anything in `principles/technical-principles.md`, and point stability-reviewer's Tests checks at it.
|
||||
|
||||
Today the rule is only implied, by "fails when that behaviour breaks" and "states what it needs and gets it". Proposed bullet under the tests principle: "A test doesn't decide at run time whether to check anything: no conditional skip, early return, or catch around its assertions. A test that needs a state creates it, or fails and names what is missing. BANNED: `test.skip(created.every((count) => count === 0))`, since a broken payroll also creates zero rows. GOOD: check the precondition before the action, failing with its cause, then `expect(created).toEqual([1, 2, 1])`."
|
||||
|
||||
Add one line to section 11 of `agents/stability-reviewer.md` that points at the bullet.
|
||||
|
||||
Source: a client project's pull request, where a fix hid a regression behind a conditional skip.
|
||||
|
||||
Touches:
|
||||
|
||||
- `principles/technical-principles.md`
|
||||
- `agents/stability-reviewer.md`: `### 11. Tests` (small)
|
||||
|
||||
### 420. End a worker that printed its final result and lingers, by killing the background shells it left, after a few minutes, and print each kill with its evidence on loop-claude's terminal.
|
||||
|
||||
On 2026-11-18 the worker of the item 402 chunk merged its pull request at about 11:47, printed its summary and then waited 95 minutes: it had started `until [ -s /tmp/merge-402.log ] && ! pgrep -f "chunk-pr merg"; do sleep 15; done` in the background, and `pgrep -f` matched the loop's own command line, so the condition never held. Its `claude -p` stayed alive ("Waiting for background work to finish"), `loop-claude` waited for it to exit, and the window's slot started no new chunk until the human found it and killed the shell.
|
||||
|
||||
Every 5 minutes loop-claude checks each worker it runs: when the worker's last transcript entry is its final result and its `claude -p` is still alive more than 3 minutes later, it kills the background shells that are children of that process, and the worker then exits. Nothing else is killed: no worker that is still working, no process of another session, and no window with a sign short of this one. A worker whose shell is still writing output or holds a `run-heavy` slot is left alone. Each kill is printed on loop-claude's terminal with the process, its age and its command, and appended to the judge's records so the pattern can become a rule-hook ban (item 421). Item 422 handles the vaguer signs of a stuck window.
|
||||
|
||||
The check lives in the main run beside the other per-cycle work; item 401 moves that code into a package, and a conflict with it is cheaper than waiting for it.
|
||||
|
||||
Before building the check, try the platform's own bound: Claude Code documents `BASH_MAX_TIMEOUT_MS` and `BASH_DEFAULT_TIMEOUT_MS` as the time limit of a background command, 10 minutes by default in `claude -p` (docs: code.claude.com/docs/en/headless.md and env-vars). Nothing in this repo sets them, yet the lingering shell ran 95 minutes on Claude Code 2.1.327, so first find out why the limit did not apply (an explicit timeout in the worker's call, or a gap in the docs). Hand the workers a cap through those variables where that bounds the wait, and build the lingering-worker check only for what the cap leaves.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/loop_claude/loop/main.py`: `run_loop` (small)
|
||||
- `bin/loop_claude/loop/lingering_workers.py`
|
||||
- `README.md`: `## Tools` (small)
|
||||
- `CLAUDE.md`: `## Working loop` (small)
|
||||
|
||||
### 299. Replay the archived judge verdicts through code rules and a classifier trained locally on the archive, and file an item for each verdict kind one of them decides with no missed interrupt.
|
||||
### 409. Split an epic item that changes several repos into one Jira sub-task per repo, each labelled for its repo, with the parent tracking the whole change, when do-epic plans the item.
|
||||
|
||||
The judge's LLM still decides three things: the verdict kind, the escalation it quotes, and whether a release is ready to cut (`docs/decisions.md` → `routing: loop-claude reads the items a chunk takes …`). This item replays the verdict kind. No single rule found so far separates interrupts: a "?" in the worker's final message appears in 48 interrupts and misses 66. A smaller LLM already failed the replay (`docs/decisions.md` → `routing: on a model judge-replay covered, …`: Sonnet returned another kind on 3 interrupts, Haiku missed 7). Untested are combined code rules, a decision table, and a small classifier trained and run on this machine on the archived verdicts (372 finished, 234 interrupt, 42 unfinished, 20 quoted). Run each offline, nothing sent to an outside service (goal 4), and count disagreements by kind; the bar is zero missed interrupts. A hosted decision model is out of scope, since it would send transcripts to an outside service.
|
||||
do-epic predicts the repos from the code, as `Touches:` predictions are made. A person's issue is split too, with a comment for the team in the epic's language (`docs/decisions.md` → `do-epic: drives one Jira epic …`). The parent is finished once every sub-task is.
|
||||
|
||||
Touches:
|
||||
- `bin/judge_replay/main.py`: `main`, `archived_cases` (small)
|
||||
- `bin/judge_replay/replay.py`: `Case`, `verdict_kind`, `report`
|
||||
- `bin/judge_replay/test_judge_replay.py`
|
||||
|
||||
- `bin/plan_store/jira_epic.py`: `Filing`, `filing_of` (small)
|
||||
- `bin/do_epic/split.py`
|
||||
|
||||
### 411. Park a chunk whose PR waits on a long gate, keeping its worktree, PR and session in a state file that survives a reboot, and resume its session when the gate passes or a reviewer comments.
|
||||
|
||||
A parked chunk takes no running slot, and the items it blocks wait (`docs/decisions.md` → `do-epic: drives one Jira epic …`). On reviewer feedback the resumed chunk fixes what is reasonable, replies to the rest, and changes the setup so review would have caught it. The forge checks need forms that return instead of blocking. `chunk_pr/main.py` `merged` is a place the seats found hard: make it easier to read where it changes.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/chunk_pr/main.py`: `merged`, `merge_turn` (small)
|
||||
- `bin/chunk_pr/approval.py`: `approval`, `unanswered` (small)
|
||||
- `bin/forge/pulls.py`: `checks_passed` (small)
|
||||
- `bin/do_epic/gates.py`
|
||||
- `bin/do_epic/parked.py`
|
||||
|
||||
### 412. Print each chunk of an epic, running, parked with its gate and since when, or stopped, each human task and each open interrupt with `do-epic <key> --status`, and name a parked chunk's gate in its tmux window's name.
|
||||
|
||||
A plain `do-epic <key>` resumes all of them, as after a reboot (`docs/decisions.md` → `do-epic: drives one Jira epic …`). The window suffix generalises the interrupt's " ?".
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/loop_claude/tmux.py`: `WAITS`, `mark` (small)
|
||||
- `bin/do_epic/status.py`
|
||||
|
||||
### 414. DM the human through the notifier when a human-task issue blocks the epic's next item, and poll until it closes.
|
||||
|
||||
do-epic leaves the issue as the team filed it (`docs/decisions.md` → `do-epic: drives one Jira epic …`).
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/loop_claude/direct_message.py`: `tell` (small)
|
||||
- `bin/do_epic/human_tasks.py`
|
||||
|
||||
### 417. Run do-epic end to end against a sandbox epic on a real Jira site with a command of its own, outside `run-tests`.
|
||||
|
||||
The sandbox is the Jira site `larvit.atlassian.net`, project `SCRUM`: a team-managed test-data space with Epic, Task, Story and Subtask issue types. The test creates its own epic there, with children and sub-tasks, and deletes them when it ends, so it leaves the space's other issues alone. Its account may create, edit, transition, comment on and delete issues in `SCRUM`. The command reads the env-file's path, holding `ATLASSIAN_EMAIL` and `ATLASSIAN_API_TOKEN`, from a setting of its own, never from the repo.
|
||||
|
||||
The test never decides at run time whether to check anything: the command fails, naming what is missing, where the site or its env-file isn't set (`docs/decisions.md` → `do-epic: drives one Jira epic …`).
|
||||
|
||||
Touches:
|
||||
|
||||
- `README.md`: `## Test` (small)
|
||||
- `bin/do-epic-e2e`
|
||||
- `bin/do_epic/e2e/`
|
||||
|
||||
### 421. Refuse in rule-hook a Bash or Monitor wait that has no end: a loop without a time or count cap, and a `pgrep -f` or `pkill -f` whose pattern the same command line contains.
|
||||
|
||||
On 2026-11-18 a worker in the item 402 chunk started `until [ -s /tmp/merge-402.log ] && ! pgrep -f "chunk-pr merg"; do sleep 15; done` in the background. The `pgrep -f` pattern is part of the loop's own command line, so it matched itself and the loop never ended; the worker's `claude -p` waited on it for 95 minutes and its slot started no new chunk (item 420 ends such a worker; item 422 handles other stuck windows). A wait that cannot end is cheaper to refuse before it starts.
|
||||
|
||||
rule-hook refuses a Bash command, or a Monitor call, that starts a wait of either kind. A loop (`until`, `while`, `for`) that sleeps and has neither a time limit (`timeout`, a deadline compared with `date`) nor a bounded count (`for i in $(seq …)`); `~/.claude/bin/wait-for` always passes, since it takes `--timeout`. A `pgrep -f` or `pkill -f` whose pattern text also appears in the command line that holds it, quoted or not, so it can match its own shell. The refusal names which kind it found and says to use `wait-for <file> [<pattern>] --timeout <seconds>`, or a `pgrep -x` on the process name.
|
||||
|
||||
`settings.json` hooks only `Agent|Bash|Edit|Grep|MultiEdit|Read|Skill|Write`, so rule-hook never sees a Monitor call: add `Monitor` to that matcher, and have rule-hook read a Monitor call's command as it reads a Bash one. Claude Code 2.1.314 and later send Monitor-started commands to a PreToolUse hook; a hook's allow and updatedInput are ignored for them, so rule-hook can refuse but not rewrite (docs: code.claude.com/docs/en/hooks.md).
|
||||
|
||||
The test covers each kind refused, a bounded loop that passes, `wait-for` passing, and a `pgrep -f` on a pattern absent from the command passing. The rule's reason goes in rule-hook's rule table beside the other rules.
|
||||
|
||||
Touches:
|
||||
|
||||
- `bin/rule_hook/rules.py`: `RULES` (small)
|
||||
- `bin/rule_hook/endless_wait.py`
|
||||
- `bin/rule_hook/test_rule_hook.py`: `CommandBreaches` (small)
|
||||
- `settings.json`
|
||||
|
||||
### 311. Hand each worker's claude a Bash output ceiling that saves longer output to a file, and refuse a Read of a session transcript under `~/.claude/projects`.
|
||||
|
||||
@@ -125,32 +309,6 @@ Touches:
|
||||
- `bin/rule_hook/test_rule_hook.py`: `ReviewerRung`
|
||||
- `bin/larv_review/test_scoring_setup.py`
|
||||
|
||||
### 339. Lower the reading cost of the files a chunk reads most before its first commit.
|
||||
|
||||
`reading-cost` printed `Rising:` on 2026-11-12: the last 5 chunks' median is 128,199 new input tokens before their first commit, the 5 before them 101,795. Read most over the last 5, as opens and searches: `todo.md` 34, `docs/decisions.md` 26, `README.md` 19, `skills/larv-review/SKILL.md` 13, `bin/loop_claude/routing.py` 8, `CHANGELOG.md` 6. The chunk that ships this runs `reading-cost --reset`.
|
||||
|
||||
Touches:
|
||||
- `README.md`
|
||||
- `skills/larv-review/SKILL.md`
|
||||
- `bin/loop_claude/routing.py`
|
||||
|
||||
### 341. Hand each worker's claude a lower Read limit through `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS`, kept only where replays of the archived review rounds find what two unchanged replays find.
|
||||
|
||||
Raised by item 312's research round, 2026-11-12. Over the 53 chunks archived since 2026-09-30, Read results held about 3.17M tokens (characters / 4) in 1,617 reads. Results over 20,000 characters, about 5,000 tokens, held 479k tokens beyond that size in 115 reads: prose reviewers 144k, stability reviewers 119k, product-owner reviewers 102k, general-purpose subagents 65k, principles auditors 28k, architects 21k. Workers had none, under the reading rule in their prompt. Measured with a script over loop-claude's archive that sums each Read tool_result's characters by the subagent's `agentType`. The decision of 2026-10-27 found Read results out of a PostToolUse hook's reach; this variable reaches them.
|
||||
|
||||
`CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS` overrides the Read tool's default limit of 25,000 tokens (https://code.claude.com/docs/en/env-vars, read 2026-11-12: "Override the default token limit for file reads, which is 25,000 tokens."). Claude Code 2.1.322's binary holds the name. A read over the limit fails and asks for an offset and a limit, so an agent may page through the same file and save nothing; the replay measures that.
|
||||
|
||||
loop-claude can hand it as it hands `CLAUDE_CODE_AUTO_COMPACT_WINDOW`, and subagents inherit it, the principles auditors included. The choice of 2026-10-27 that the principles audit reads the project whole holds as long as it pages through what it reads.
|
||||
|
||||
The replay: give `review-replay` a way to set the variable, then replay the archived prose, stability, product-owner and principles rounds at 5,000 tokens, beside two unchanged replays of the same rounds. Keeps quality where each pass finds every finding the two unchanged replays both find, or misses no more than the most an unchanged replay misses, as `review-replay --omit-claude-md` counts it. Keep it only where the replays' Read characters also fall; item 311's stability replay can share the baseline runs.
|
||||
|
||||
Item 311's chunk built the way to set the variable: `review-replay --read-limit` replays a pass's rounds three times as archived and once at 5,000 tokens (`bin/review_replay/changes.py`), and `bin/claude_code/output_limits.py` holds `READ_LIMIT` and `READ_LIMIT_TOKENS`. What remains is to run that replay for the four passes, and to hand the variable over where it passes.
|
||||
|
||||
Touches:
|
||||
- `bin/loop_claude/cycle/run.py`: `worker_environment` (small)
|
||||
- `bin/loop_claude/loop/usage.py`: `HELP` (small)
|
||||
- `bin/loop_claude/test_loop_claude.py`: `Cycles` (small), `HelpLimits` (small)
|
||||
|
||||
### 310. Compact a waiting worker's session shortly before its prompt cache expires, only when the human has not answered an interrupt by then.
|
||||
|
||||
281 worker resumes after the 1-hour cache expired rewrote a whole context, $379 of the $9,154 measured from 2026-09-30 to 2026-11-11 (4.1%). A compaction while the cache is warm reads the context at a twentieth of the input price and leaves about 20k tokens to rewrite later.
|
||||
@@ -168,17 +326,19 @@ Touches:
|
||||
- `bin/loop_claude/test_loop_claude.py`: `FakeEffects`, `Interrupts`
|
||||
- `bin/loop_effects/test_effects.py`
|
||||
|
||||
### 298. Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.
|
||||
### 422. Check every 5 minutes whether a tmux window of the loop looks stuck by other signs than a lingering worker, investigate it with a headless claude, and kill only the process the investigation names.
|
||||
|
||||
153 `Judge cost:` entries total $109.76, a median of about $0.76 a verdict at every rung, and the cost follows the size of the records, not the transcript: about 76k characters in log and 188k in claude-config. In 224 of 251 recent verdicts (89%) the evidence sits in the worker's final message. A missed interrupt is the costly error, so the bar is zero changed interrupts and zero new rejections across the replay. The judge is about 4.5% of dollars, so the saving is small; each cycle also gets faster.
|
||||
Item 420 ends a worker that printed its final result and lingers. A window can also stall earlier: a test run that hangs on a shell ignoring SIGINT (item 402's first full run), a pane with no output while its process tree does no work, or a background shell older than the chunk's usual length.
|
||||
|
||||
Every 5 minutes loop-claude lists its own session's windows and, for one with such a sign, launches a headless claude on the project's rung that reads the pane, the process tree and the chunk's log files and returns `working`, `waiting on a person` or `stuck`, with the process that holds the window and why. Only `stuck` kills, and only the named process, never the worker or the loop. A window waiting on a gate (`run-heavy`, a test run that still writes output) or on the human's answer is not `stuck`. Each check and kill is printed on loop-claude's terminal with the evidence.
|
||||
|
||||
Run it first as a report that kills nothing, and turn the kill on once its verdicts over a few days match what the human would have done; record that in the item's decision. Blocked by 401, which moves loop-claude's cycle into a package; this item lands in the package.
|
||||
|
||||
Touches:
|
||||
- `bin/loop_claude/cycle/judge.py`: `Brief`, `judge_prompt`, `JUDGE_PROMPT_CHARS`
|
||||
- `bin/loop_claude/project/records.py`: `read_records`
|
||||
- `bin/judge_replay/main.py`: `rule` (small)
|
||||
- `bin/judge_replay/replay.py`: `case_from` (small)
|
||||
- `bin/loop_claude/test_loop_claude.py`: `JudgePrompt`, `Records`
|
||||
- `bin/judge_replay/test_judge_replay.py`
|
||||
|
||||
- `bin/loop_claude/loop/stuck_windows.py`
|
||||
- `bin/loop_claude/tmux.py`: `Listed`, `listed`
|
||||
- `README.md`: `## Tools` (small)
|
||||
|
||||
### 365. Have `score-outcomes` read the finished and dropped items of a plan outside the checkout, from the project repo's history or from Jira's issue changelogs.
|
||||
|
||||
@@ -217,10 +377,63 @@ Touches:
|
||||
- `skills/larv-review/SKILL.md`: `### Comprehension check`
|
||||
- `skills/comprehension-panel/SKILL.md`: `## The scoring run` (small)
|
||||
|
||||
### 419. Run the external auto-reviewer until its review of the PR's head has no findings, and answer each of its threads on the PR.
|
||||
|
||||
Today `larv-review` §3 → "Round cap" stops the auto-reviewer after two reviews. On a client project's pull request, CodeRabbit spent both reviews on the draft, before the internal passes were done, so its last finding got no answer on the PR. Change:
|
||||
|
||||
- §3 → "Round cap": drop the auto-reviewer's cap. Every finding it makes, whatever its severity, is fixed or shown wrong, and each push waits for its review of the new head. The pass is done when that review adds no finding.
|
||||
- §4 step 6: detect a clean review. CodeRabbit posts no review when it has no findings. Instead it updates its summary comment with "No actionable comments were generated in the recent review" and a `coveredCommitId` that matches the head. Wait for that, not for a new review.
|
||||
- Answer every thread in place, with the fix's commit or the reason for not making it.
|
||||
- `chunk-pr ready` and `chunk-pr merge` refuse while an auto-reviewer thread has neither our reply nor resolved status, read from the thread state GitHub's API reports.
|
||||
- `round-count`, rule-hook's round-cap and the "Answer … round" titles need no change, since the auto-reviewer's commits already take ordinary titles.
|
||||
- Once this ships, delete the first two bullets of the machine-local CodeRabbit rule file, which say the same thing for one machine.
|
||||
|
||||
The change touches `skills/larv-review/SKILL.md` "Re-entry", which the comprehension seats found hard to follow (Locality): make it easier to read while editing it.
|
||||
|
||||
### 366. Count the chunks of a project whose plan sits outside the checkout in the routing figures, from the project repo's history or from Jira.
|
||||
|
||||
One of 0.13.0's history ports, per lilleman's call of 2026-11-12.
|
||||
|
||||
### 339. Lower the reading cost of the files a chunk reads most before its first commit.
|
||||
|
||||
`reading-cost` printed `Rising:` on 2026-11-12: the last 5 chunks' median is 128,199 new input tokens before their first commit, the 5 before them 101,795. Read most over the last 5, as opens and searches: `todo.md` 34, `docs/decisions.md` 26, `README.md` 19, `skills/larv-review/SKILL.md` 13, `bin/loop_claude/routing.py` 8, `CHANGELOG.md` 6. The chunk that ships this runs `reading-cost --reset`.
|
||||
|
||||
Touches:
|
||||
- `README.md`
|
||||
- `skills/larv-review/SKILL.md`
|
||||
- `bin/loop_claude/routing.py`
|
||||
|
||||
### 341. Hand each worker's claude a lower Read limit through `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS`, kept only where replays of the archived review rounds find what two unchanged replays find.
|
||||
|
||||
Raised by item 312's research round, 2026-11-12. Over the 53 chunks archived since 2026-09-30, Read results held about 3.17M tokens (characters / 4) in 1,617 reads. Results over 20,000 characters, about 5,000 tokens, held 479k tokens beyond that size in 115 reads: prose reviewers 144k, stability reviewers 119k, product-owner reviewers 102k, general-purpose subagents 65k, principles auditors 28k, architects 21k. Workers had none, under the reading rule in their prompt. Measured with a script over loop-claude's archive that sums each Read tool_result's characters by the subagent's `agentType`. The decision of 2026-10-27 found Read results out of a PostToolUse hook's reach; this variable reaches them.
|
||||
|
||||
`CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS` overrides the Read tool's default limit of 25,000 tokens (https://code.claude.com/docs/en/env-vars, read 2026-11-12: "Override the default token limit for file reads, which is 25,000 tokens."). Claude Code 2.1.322's binary holds the name. A read over the limit fails and asks for an offset and a limit, so an agent may page through the same file and save nothing; the replay measures that.
|
||||
|
||||
loop-claude can hand it as it hands `CLAUDE_CODE_AUTO_COMPACT_WINDOW`, and subagents inherit it, the principles auditors included. The choice of 2026-10-27 that the principles audit reads the project whole holds as long as it pages through what it reads.
|
||||
|
||||
The replay: give `review-replay` a way to set the variable, then replay the archived prose, stability, product-owner and principles rounds at 5,000 tokens, beside two unchanged replays of the same rounds. Keeps quality where each pass finds every finding the two unchanged replays both find, or misses no more than the most an unchanged replay misses, as `review-replay --omit-claude-md` counts it. Keep it only where the replays' Read characters also fall; item 311's stability replay can share the baseline runs.
|
||||
|
||||
Blocked by item 311, 2026-11-17: item 311's chunk already generalizes `review-replay`'s `--omit-claude-md` comparison into a `Change` of the replayed reviewer's environment (`bin/review_replay/changes.py`, `change_comparison.py`) and edits the same lines of `worker_environment`, `HELP` and `launch`; this item adds its own `Change` for `CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS` on top of it once it merges.
|
||||
|
||||
Item 311's chunk built the way to set the variable: `review-replay --read-limit` replays a pass's rounds three times as archived and once at 5,000 tokens (`bin/review_replay/changes.py`), and `bin/claude_code/output_limits.py` holds `READ_LIMIT` and `READ_LIMIT_TOKENS`. What remains is to run that replay for the four passes, and to hand the variable over where it passes.
|
||||
|
||||
Touches:
|
||||
- `bin/loop_claude/cycle/run.py`: `worker_environment` (small)
|
||||
- `bin/loop_claude/loop/usage.py`: `HELP` (small)
|
||||
- `bin/loop_claude/test_loop_claude.py`: `Cycles` (small), `HelpLimits` (small)
|
||||
|
||||
### 298. Send the judge only the decision entries and the transcript tail a verdict needs, and keep the cut only where `judge-replay` changes no verdict kind.
|
||||
|
||||
153 `Judge cost:` entries total $109.76, a median of about $0.76 a verdict at every rung, and the cost follows the size of the records, not the transcript: about 76k characters in log and 188k in claude-config. In 224 of 251 recent verdicts (89%) the evidence sits in the worker's final message. A missed interrupt is the costly error, so the bar is zero changed interrupts and zero new rejections across the replay. The judge is about 4.5% of dollars, so the saving is small; each cycle also gets faster.
|
||||
|
||||
Touches:
|
||||
- `bin/loop_claude/cycle/judge.py`: `Brief`, `judge_prompt`, `JUDGE_PROMPT_CHARS`
|
||||
- `bin/loop_claude/project/records.py`: `read_records`
|
||||
- `bin/judge_replay/main.py`: `rule` (small)
|
||||
- `bin/judge_replay/replay.py`: `case_from` (small)
|
||||
- `bin/loop_claude/test_loop_claude.py`: `JudgePrompt`, `Records`
|
||||
- `bin/judge_replay/test_judge_replay.py`
|
||||
|
||||
### 395. Keep each block a save didn't change, with its localId, and the spacing paragraphs beside it as Jira or Confluence holds them.
|
||||
|
||||
Today a Jira detail edited in one line, or a Confluence record with one entry appended, is written back whole by adf-codec's portable pair, which drops each block's localId and the empty paragraphs Jira uses for spacing. A save whose text is the text a read showed already sends nothing (`docs/decisions.md` → `project config: the loop shows a Jira description and a Confluence page as adf-codec's portable markdown, …`). Match the saved markdown's top-level blocks against the held document's, each as the portable pair reads it, and send the held block where they match. The decision's limit on partial saves ends with this item.
|
||||
@@ -229,6 +442,21 @@ Today a Jira detail edited in one line, or a Confluence record with one entry ap
|
||||
|
||||
One of 0.13.0's history ports, per lilleman's call of 2026-11-12.
|
||||
|
||||
### 423. Skip the judge's model call on a worker's final message that the offline classifier decides is finished.
|
||||
|
||||
On 2026-11-18 lilleman ruled that the classifier's miss rate of about 1 in 100 decided is close enough to no missed interrupt (`docs/decisions.md` → `routing: the offline classifier may decide finished, since lilleman ruled a miss rate of about 1 in 100 decided close enough to none; the judge's model keeps every other verdict kind`).
|
||||
|
||||
Taking the item builds this: before the judge call, loop-claude runs the classifier trained on its archive over the worker's final message. Where the classifier decides finished, loop-claude skips the call. The classifier is re-trained as the archive grows. The judge call sits in `bin/loop_claude/cycle/run.py` → `rule`, a place the comprehension seats found hard, since the order of stamping, archiving and stepping matters across three functions: make it easier to read.
|
||||
|
||||
The judge's model still decides every verdict the classifier leaves, the escalation a worker quotes, and whether a release is ready to cut, since the replay didn't test the classifier on those.
|
||||
|
||||
Evidence, from `judge-replay claude-opus-5-5 --offline`: the classifier reads only the final message, averages five seeded logistic regressions, and decides finished only where its chance of an interrupt is below that of every interrupt among its held-out training folds.
|
||||
- On the 1,116 archived verdicts it decided finished with no missed interrupt on both tests: held-out folds, 95 decided and none missed; latest fifth, 20 decided and none missed.
|
||||
- On the first 1,104 it missed one interrupt on the latest fifth, while the held-out folds decided 103 and missed none.
|
||||
- It decides about 1 in 12 verdicts. About 1 in 100 of those it decided were an interrupt, on the 1,104.
|
||||
|
||||
Savings: one judge call per about 12 verdicts. That is small, and nobody has measured it.
|
||||
|
||||
### 77. Teach `bin/metric-bench` Go and TypeScript, rerun it once two projects' comprehension histories name places, and bring into review only a metric that tracks the panel.
|
||||
|
||||
2026-10-08, lilleman: no metric enters review until a rerun on more data shows it tracking the panel.
|
||||
|
||||
Reference in New Issue
Block a user