Merge pull request 'Route each chunk's worker by its item's R + C on one ladder of model and effort rungs' (#262) from route-worker into main
This commit was merged in pull request #262.
This commit is contained in:
@@ -35,6 +35,8 @@
|
||||
!bin/metric-bench
|
||||
!bin/metric_bench/
|
||||
!bin/metric_bench/*.py
|
||||
!bin/model_routing/
|
||||
!bin/model_routing/*.py
|
||||
!bin/project_key.py
|
||||
!bin/rank_todo.py
|
||||
!bin/reading-cost
|
||||
|
||||
@@ -33,6 +33,7 @@ Decisions live in `docs/decisions.md`:
|
||||
- worker: a chunk's worker writes the simplest change that solves its item, test first wherever a test can check the change, and reads a technical principle only where the item, a finding or its own record names it; larv-review's reviewers hold the change to the principles and to the standards in `skills/larv-review/standards/`
|
||||
- review: stability and product-owner report only a defect, a broken requirement of the plan or the goals, a breach of a technical principle, or for stability a hard place the change touches; each architect finding names a later change the PR makes harder; prose runs only on a chunk that changes prose
|
||||
- review: the stability pass mutation-tests the lines a chunk changes, in place of measuring test coverage
|
||||
- routing: one ladder of (model, effort) rungs, ordered by the Artificial Analysis Intelligence Index with Fable 5.1 at max on top; a task's R + C picks a rung by fixed bands, counted from the cheapest rung of the project's model, and a chunk's worker never runs below that rung
|
||||
|
||||
## Working rules
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- `loop-claude` routes each chunk's worker on a ladder of model and effort rungs, from Haiku 5.5 at low effort to Fable 5.1 at max, when the project's model is on it. As a chunk starts, the R + C of `todo.md`'s top item in the next release picks the rung: 11–12 one rung above the project's model, which for Opus 5.5 is Opus 5.5 at high effort, and so on up to four rungs. The worker never runs below the project's model's cheapest rung, which for Opus 5.5 is medium effort, below the xhigh `settings.json` sets. It steps up one rung after every second unfinished verdict on a chunk, and to a higher rung when it takes in an item of higher R + C. Each cycle's header, and each finished or unfinished verdict, stop and step-up in the archive, name the worker's rung; a rerun resumes an open chunk on the latest rung archived for it. `docs/decisions.md` → `routing:` gives the ladder, its source and the bands. Expect a chunk on an easy item to cost less than under xhigh, and one on an item of R + C 15 or more to cost more.
|
||||
- A session reads `todo.md` up to `## Details`, and the details of only the items its chunk takes. It reads a `docs/decisions.md` entry only where the entry's title leaves open what it needs. Both rules are in `CLAUDE.md`, so they hold from a session's start. The decisions rule replaces one in `rules/decisions.md`, which loads only once a session has opened `docs/decisions.md` or `AGENTS.md`. A chunk's opening line now counts the unreleased changes instead of listing them. Expect fewer tokens read before a chunk's first commit.
|
||||
- larv-review's stability and product-owner reviewers report only a defect, a requirement of the plan or the goals that the change breaks, a breach of a technical principle, or for stability a hard place the change touches; the stability reviewer leaves the style of comments to the prose reviewer, which now also reads help and error text. Each architect finding names a concrete later change the PR makes harder, and one that names none is dropped. The prose pass runs only on a chunk that changes prose a reader reads, such as docs, comments, help text or `todo.md`, and skips a code-only chunk. Expect fewer findings and fewer review rounds.
|
||||
- Every `todo.md` item carries a C score, how hard the change is to get right, from 1 to 10, in a column after S. C stays out of the score. `rank_todo.py` fails on an items table without the column, and the rule hook then blocks committing that `todo.md`. Before the next commit that changes a project's `todo.md`, add the column and score every item's C, as `rank_todo.py --help` shows.
|
||||
|
||||
@@ -91,7 +91,7 @@ pytest_add_cli_args=-k
|
||||
|
||||
Then, in the copy: `uvx --with pytest 'mutmut>=3,<4' run && uvx --with pytest 'mutmut>=3,<4' results`, and `uvx --with pytest 'mutmut>=3,<4' show <mutant>` to see a mutant's change. The config keys are mutmut 3's.
|
||||
|
||||
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them, `rule_hook` imports `rank_todo.py`'s check, and `score_outcomes` imports `reading_cost`'s record path. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
|
||||
`bin/` is the import root: each launcher puts it on `sys.path`, so a module imports another by its path under `bin/`, such as `from claude_code.entries import parse_entry`. Each tool's package holds its tests; `rank_todo.py` and `statusline.py` are tools of one file each, tested beside them, `rule_hook` imports `rank_todo.py`'s check, and `score_outcomes` imports `reading_cost`'s record path. The other non-test modules at `bin/`'s root, `claude_code/` and `transcript_archive/` serve more than one tool; `model_routing/` holds the routing ladder, which loop-claude reads. `test_env.py` serves the tests, and `repo_checks/` tests the repo itself.
|
||||
|
||||
## Tracking a new file
|
||||
|
||||
|
||||
@@ -6,13 +6,14 @@ from failure import Failure
|
||||
from claude_code.transcript import human_turns, read_transcript
|
||||
from loop_claude.stop import needs_human, needs_terminal, Stop
|
||||
from loop_claude.paths import Paths
|
||||
from model_routing.ladder import Rung
|
||||
from claude_code.trust import trust
|
||||
from loop_claude.project.checkout import worktrees
|
||||
|
||||
ANSWER_QUESTION = "answer the question above"
|
||||
|
||||
|
||||
def interrupt(root: str, session_id: uuid.UUID, model: str, path: str, question: str, paths: Paths, effects: Effects) -> Stop | None:
|
||||
def interrupt(root: str, session_id: uuid.UUID, worker: Rung, path: str, question: str, paths: Paths, effects: Effects) -> Stop | None:
|
||||
print(f"\n{question}", file=sys.stderr)
|
||||
if stop := needs_terminal(ANSWER_QUESTION, effects.on_terminal()):
|
||||
return stop
|
||||
@@ -22,7 +23,7 @@ def interrupt(root: str, session_id: uuid.UUID, model: str, path: str, question:
|
||||
return Stop(1, entries.reason)
|
||||
before = human_turns(entries)
|
||||
try:
|
||||
effects.claude_on_terminal(["--model", model, "--permission-mode", "bypassPermissions", "--resume", str(session_id)], root)
|
||||
effects.claude_on_terminal([*worker.flags(), "--permission-mode", "bypassPermissions", "--resume", str(session_id)], root)
|
||||
except OSError as failure:
|
||||
return Stop(1, f"could not start claude for the interrupt: {failure.strerror}")
|
||||
entries = read_transcript(path)
|
||||
|
||||
@@ -10,6 +10,7 @@ from claude_code.entries import hit_limit, limit_said, parse_entry, structured_o
|
||||
from loop_claude.project.records import HUMAN_RULE_NAMES
|
||||
from loop_claude.cycle.retry import Limited, retried
|
||||
from loop_claude.stop import Stop
|
||||
from todo_table import TOP_SCORE
|
||||
|
||||
# At 3 to 4 characters a token, 100,000 to 133,000 tokens, which leaves a 200,000-token context room for the reply; the largest session rendered here was 364,000 characters on 2026-10-03.
|
||||
JUDGE_PROMPT_CHARS = 400000
|
||||
@@ -49,6 +50,8 @@ JUDGE_RULES = (
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Reply:
|
||||
answer: str = dataclasses.field(metadata={"description": "With interrupt false, the passage of README.md or the records that answers the agent's question, quoted verbatim, without elisions. Empty otherwise."})
|
||||
# Routes the next cycle's worker. Counting the item the chunk was routed on is harmless: loop-claude never steps the worker down.
|
||||
bundled: int = dataclasses.field(metadata={"description": "The highest R + C the agent states for a todo.md item it took into this chunk; 0 when it states none."})
|
||||
evidence: str = dataclasses.field(metadata={"description": "A verbatim quote of the one transcript passage the verdict rests on, without elisions: what triggers the interrupt; for a quoted answer, the agent's question; else what shows the chunk finished or the work remaining."})
|
||||
finished: bool = dataclasses.field(metadata={"description": "Whether the chunk is finished."})
|
||||
interrupt: bool = dataclasses.field(metadata={"description": "Whether the human must now be interrupted."})
|
||||
@@ -56,7 +59,8 @@ class Reply:
|
||||
|
||||
|
||||
REPLY_TYPES = typing.get_type_hints(Reply)
|
||||
JSON_TYPES = {bool: "boolean", str: "string"}
|
||||
JSON_TYPES = {bool: "boolean", int: "integer", str: "string"}
|
||||
HIGHEST_R_PLUS_C = 2 * TOP_SCORE
|
||||
JUDGE_SCHEMA = {
|
||||
"properties": {field.name: {**field.metadata, "type": JSON_TYPES[REPLY_TYPES[field.name]]} for field in dataclasses.fields(Reply)},
|
||||
"required": list(REPLY_TYPES),
|
||||
@@ -86,6 +90,8 @@ class Finished:
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Unfinished:
|
||||
evidence: str
|
||||
# The bundled R + C the judge saw, 0 for none.
|
||||
bundled: int = 0
|
||||
|
||||
|
||||
Verdict = Interrupt | QuotedAnswer | Finished | Unfinished
|
||||
@@ -95,7 +101,8 @@ def verdict_from(output: object) -> Verdict | Failure:
|
||||
if not isinstance(output, dict):
|
||||
return Failure("the judge returned no reply")
|
||||
for name, kind in REPLY_TYPES.items():
|
||||
if not isinstance(output.get(name), kind):
|
||||
# type(), not isinstance(): JSON true is no integer.
|
||||
if type(output.get(name)) is not kind:
|
||||
return Failure(f"the judge's reply has no {name} of type {JSON_TYPES[kind]}")
|
||||
if output["interrupt"]:
|
||||
if not output["question"].strip():
|
||||
@@ -105,7 +112,11 @@ def verdict_from(output: object) -> Verdict | Failure:
|
||||
if not output["question"].strip():
|
||||
return Failure("the judge quoted an answer without a question")
|
||||
return QuotedAnswer(output["evidence"], output["question"], output["answer"])
|
||||
return Finished(output["evidence"]) if output["finished"] else Unfinished(output["evidence"])
|
||||
if output["finished"]:
|
||||
return Finished(output["evidence"])
|
||||
if not 0 <= output["bundled"] <= HIGHEST_R_PLUS_C:
|
||||
return Failure(f"the judge's bundled R + C {output['bundled']} is not from 0 to {HIGHEST_R_PLUS_C}")
|
||||
return Unfinished(output["evidence"], output["bundled"])
|
||||
|
||||
|
||||
def judge(root: str, task: str, premises: str, records: str, rendered_transcript: str, model: str, effects: Effects) -> Verdict | Stop:
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
from loop_claude.project.records import HUMAN_RULE_NAMES, RECORD_FILES
|
||||
from markdown_text import quoted
|
||||
from loop_claude.routing import Route
|
||||
|
||||
def either(names: tuple[str, ...]) -> str:
|
||||
"""The names as `a, b or c`."""
|
||||
@@ -13,8 +14,13 @@ ANSWER_DESTINATIONS = "README.md's Goals or Audience, " + ", ".join(RECORD_FILES
|
||||
ANSWERS_RECORDED = f"every answer the human typed in it that a later chunk could need is recorded in {ANSWER_DESTINATIONS} at the one place it governs"
|
||||
|
||||
|
||||
def fresh(task: str, premises: str) -> str:
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.\n\nThe chunk's todo.md items, or else the task above, set its scope. The goals below choose between ways to deliver that scope, and never widen it.\n\nFrom README.md:\n{premises}"
|
||||
def fresh(task: str, premises: str, route: Route) -> str:
|
||||
routed = f"{routed_by(route)}\n\n" if route.item else ""
|
||||
return f"{task}\n\nContinue any open draft PR or worktree for the chunk before creating a new one.\n\n{routed}The chunk's todo.md items, or else the task above, set its scope. The goals below choose between ways to deliver that scope, and never widen it.\n\nFrom README.md:\n{premises}"
|
||||
|
||||
|
||||
def routed_by(route: Route) -> str:
|
||||
return f"loop-claude runs you on {route.rung}, routed by item {route.item}'s R + C of {route.risk_and_complexity}. Name each todo.md item this chunk takes, with its R + C, whether or not it is item {route.item}."
|
||||
|
||||
|
||||
def resumed(unfinished_evidence: str) -> str:
|
||||
|
||||
@@ -9,12 +9,12 @@ from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.cycle.judge import Finished, Interrupt, judge, QuotedAnswer, Unfinished
|
||||
from loop_claude.cycle.interrupt import interrupt
|
||||
from loop_claude.project.model_choice import Models
|
||||
from transcript_archive import outcomes
|
||||
from loop_claude.paths import Paths
|
||||
from loop_claude.cycle import prompts
|
||||
from claude_code.transcript import read_transcript, render
|
||||
from loop_claude.cycle.retry import retried
|
||||
from loop_claude.routing import Route
|
||||
from loop_claude.cycle.worker import run_worker, worker_args
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
@@ -22,7 +22,7 @@ from loop_claude.stop import Stop
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Cycle:
|
||||
effects: Effects
|
||||
models: Models
|
||||
judge_model: str
|
||||
paths: Paths
|
||||
premises: str
|
||||
project_archive: pathlib.Path
|
||||
@@ -30,6 +30,7 @@ class Cycle:
|
||||
root: str
|
||||
session: Session
|
||||
task: str
|
||||
route: Route
|
||||
# Empty when the previous cycle's verdict was not Unfinished.
|
||||
unfinished_evidence: str
|
||||
|
||||
@@ -49,7 +50,7 @@ def run_cycle(cycle: Cycle) -> Stop | Finished | Unfinished:
|
||||
|
||||
def stopped(cycle: Cycle, stop: Stop, why: str) -> Stop:
|
||||
path = transcript_file(cycle.session)
|
||||
failure = archive(cycle.project_archive, path, cycle.task, outcomes.stopped(why), cycle.effects.now()) if path else None
|
||||
failure = archive(cycle.project_archive, path, cycle.task, outcomes.stopped(str(cycle.route), why), cycle.effects.now()) if path else None
|
||||
# A failed archive exits 1 whatever the stop; when the stop was this archive failing, its reason is said once.
|
||||
return Stop(1, f"{failure.reason}\n{stop.reason}".rstrip()) if failure and failure.reason != stop.reason else stop
|
||||
|
||||
@@ -82,10 +83,10 @@ def work(cycle: Cycle, answered: QuotedAnswer | None) -> WorkerRan | Stop:
|
||||
|
||||
def worker_start(cycle: Cycle, answered: QuotedAnswer | None) -> list[str]:
|
||||
if answered is not None:
|
||||
return worker_args(cycle.session, cycle.models.worker, prompts.answered(answered.question, answered.answer), resume=True)
|
||||
return worker_args(cycle.session, cycle.route.rung, prompts.answered(answered.question, answered.answer), resume=True)
|
||||
if session_open(cycle.session):
|
||||
return worker_args(cycle.session, cycle.models.worker, prompts.resumed(cycle.unfinished_evidence), resume=True)
|
||||
return worker_args(cycle.session, cycle.models.worker, prompts.fresh(cycle.task, cycle.premises), resume=False)
|
||||
return worker_args(cycle.session, cycle.route.rung, prompts.resumed(cycle.unfinished_evidence), resume=True)
|
||||
return worker_args(cycle.session, cycle.route.rung, prompts.fresh(cycle.task, cycle.premises, cycle.route), resume=False)
|
||||
|
||||
|
||||
def keep_answer(cycle: Cycle, answered: QuotedAnswer) -> Stop | None:
|
||||
@@ -99,7 +100,7 @@ def rule(cycle: Cycle, transcript_path: str) -> Ruled | Stop:
|
||||
entries = read_transcript(transcript_path)
|
||||
if isinstance(entries, Failure):
|
||||
return Stop(1, entries.reason)
|
||||
verdict = judge(cycle.root, cycle.task, cycle.premises, cycle.records, render(entries), cycle.models.judge, cycle.effects)
|
||||
verdict = judge(cycle.root, cycle.task, cycle.premises, cycle.records, render(entries), cycle.judge_model, cycle.effects)
|
||||
return verdict if isinstance(verdict, Stop) else Ruled(verdict)
|
||||
|
||||
|
||||
@@ -108,7 +109,7 @@ def ask(cycle: Cycle, transcript_path: str, ruling: Interrupt | QuotedAnswer) ->
|
||||
# Archived now, so the transcript the question was asked on is kept with it; the human's answer then grows the transcript.
|
||||
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.interrupted(question, ruling.evidence), cycle.effects.now()):
|
||||
return Stop(1, failure.reason)
|
||||
return interrupt(cycle.root, cycle.session.id, cycle.models.worker, transcript_path, question, cycle.paths, cycle.effects) or HumanAnswered()
|
||||
return interrupt(cycle.root, cycle.session.id, cycle.route.rung, transcript_path, question, cycle.paths, cycle.effects) or HumanAnswered()
|
||||
|
||||
|
||||
def question_for(ruling: Interrupt | QuotedAnswer) -> str:
|
||||
@@ -119,7 +120,7 @@ def question_for(ruling: Interrupt | QuotedAnswer) -> str:
|
||||
|
||||
def close(cycle: Cycle, transcript_path: str, verdict: Finished | Unfinished) -> Archived | Stop:
|
||||
finished = isinstance(verdict, Finished)
|
||||
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.finished(verdict.evidence) if finished else outcomes.unfinished(verdict.evidence), cycle.effects.now()):
|
||||
if failure := archive(cycle.project_archive, transcript_path, cycle.task, outcomes.finished(str(cycle.route), verdict.evidence) if finished else outcomes.unfinished(str(cycle.route), verdict.evidence), cycle.effects.now()):
|
||||
return Stop(1, failure.reason)
|
||||
if finished and (failure := delete_session(cycle.session)):
|
||||
return Stop(1, failure.reason)
|
||||
|
||||
@@ -3,6 +3,7 @@ from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.cycle.retry import Limited
|
||||
from claude_code.transcript import part_line
|
||||
from model_routing.ladder import Rung
|
||||
from claude_code.session import Session
|
||||
|
||||
PRINTED_TOOL_INPUT = 160
|
||||
@@ -37,7 +38,7 @@ def worker_outcome(code: int, result: dict | None, resets: list[int]) -> Failure
|
||||
return None if result is not None else Failure("claude exited 0 without a result event")
|
||||
|
||||
|
||||
def worker_args(session: Session, model: str, prompt: str, resume: bool) -> list[str]:
|
||||
def worker_args(session: Session, worker: Rung, prompt: str, resume: bool) -> list[str]:
|
||||
# --session-id creates the session under the id loop-claude keeps; --resume continues that session.
|
||||
start = ["--resume", str(session.id)] if resume else ["--name", session.name, "--session-id", str(session.id)]
|
||||
# -p fails without --verbose under stream-json.
|
||||
@@ -47,5 +48,5 @@ def worker_args(session: Session, model: str, prompt: str, resume: bool) -> list
|
||||
"--output-format", "stream-json",
|
||||
"--permission-mode", "bypassPermissions",
|
||||
"--verbose",
|
||||
"--model", model,
|
||||
*worker.flags(),
|
||||
]
|
||||
|
||||
@@ -40,6 +40,20 @@ Every claude loop-claude starts runs the project's model, saved per <key> in
|
||||
run, is asked for one. LOOP_CLAUDE_JUDGE_MODEL overrides it for the judge, LOOP_CLAUDE_WORKER_MODEL
|
||||
for the worker and the interrupt; with both set, nothing is asked.
|
||||
|
||||
Where that model is on the routing ladder (bin/model_routing/ladder.py), the worker and the
|
||||
interrupt run on one of the ladder's rungs: a model at an effort. As a chunk starts, the R + C of
|
||||
todo.md's top item in the next release, skipping an item marked question, picks the rung. It
|
||||
counts from the model's cheapest rung and never goes below it; without such an item, or with a C
|
||||
missing, the chunk runs on the model's cheapest rung. The worker's first prompt names the rung and
|
||||
the item, and asks the worker to name each item the chunk takes, with its R + C. The judge reports
|
||||
the highest R + C the worker states, and the next cycle steps up to the rung that R + C picks,
|
||||
never down. Every second unfinished verdict on a chunk steps it up one rung. Each finished or
|
||||
unfinished verdict, stop and step-up in the archive names the worker's rung, with the item and
|
||||
R + C that routed the chunk; a rerun resumes an open chunk on the latest rung named there, and
|
||||
counts its unfinished verdicts afresh. A rung no longer on the ladder, or
|
||||
below the model's cheapest, is dropped, and the chunk is routed afresh from todo.md. A todo.md or
|
||||
an archive that cannot be read or parsed is said on stderr, and the chunk is routed without it.
|
||||
|
||||
A change to bin/loop-claude or any bin/**/*.py takes effect at the next cycle boundary, once
|
||||
loop-claude --help succeeds: the loop reloads into the edited script.
|
||||
|
||||
@@ -60,8 +74,9 @@ Exit 1, saying why, when:
|
||||
- the worker or the judge fails three times running, retried a minute apart within the cycle. A
|
||||
worker fails when claude cannot start, exits non-zero, is killed, or ends without a result event;
|
||||
a judge when claude cannot start or exits non-zero, or its reply misses or mistypes a field,
|
||||
interrupts or quotes an answer without a question, or quotes a passage the transcript does not
|
||||
hold; it holds an elided passage when each fragment appears in order.
|
||||
interrupts or quotes an answer without a question, gives an unfinished verdict an R + C outside
|
||||
0 to 20, or quotes a passage the transcript does not hold; it holds an elided passage when each
|
||||
fragment appears in order.
|
||||
A usage limit is not a failure, and restarts the count of three: the worker or the judge is
|
||||
retried 2 minutes after the limit resets, or every 10 minutes when the reset time is unknown or
|
||||
more than 2 minutes past. The judge's reply gives its reset time only as text, so the judge is
|
||||
@@ -79,19 +94,26 @@ import pathlib
|
||||
import sys
|
||||
import typing
|
||||
|
||||
from transcript_archive.write import archive
|
||||
from loop_claude.project.checkout import checkout_root
|
||||
from loop_claude.cycle.run import Cycle, run_cycle
|
||||
from loop_claude.loop.reload import decode_handed, exec_if_edited, fingerprint, LoopRun, RELOAD_FLAG
|
||||
from loop_claude.effects import Effects
|
||||
from failure import Failure
|
||||
from loop_claude.cycle.judge import Finished, Unfinished
|
||||
from loop_claude.routing import first_route, kept, Route, route_from, routed_item, stepped
|
||||
from loop_claude.project.model_choice import get_or_ask_models
|
||||
from todo_table import Item
|
||||
from transcript_archive.read import latest_worker
|
||||
from transcript_archive import outcomes
|
||||
from loop_claude.paths import Paths
|
||||
from transcript_archive.location import project_archive
|
||||
from project_key import project_key
|
||||
from model_routing.ladder import project_rung, Rung
|
||||
from loop_claude.project.premises import read_premises
|
||||
from loop_claude.project.records import read_records
|
||||
from claude_code.session import session_named
|
||||
from claude_code.session import Session, session_named, transcript_file
|
||||
from loop_claude.stop import Stop
|
||||
from loop_claude.cycle.judge import Unfinished
|
||||
|
||||
|
||||
def main(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launcher: pathlib.Path, cwd: pathlib.Path, effects: Effects) -> int:
|
||||
@@ -110,7 +132,7 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
return loop_run
|
||||
baseline = fingerprint(launcher)
|
||||
key = project_key(loop_run.root, paths.home)
|
||||
archive = project_archive(loop_run.root, paths.home, env)
|
||||
archive_folder = project_archive(loop_run.root, paths.home, env)
|
||||
# "A " is the human's cue in the session picker that the session is the loop's.
|
||||
session = session_named(f"A {key} loop-claude", paths.claude_code_transcripts)
|
||||
while True:
|
||||
@@ -126,11 +148,66 @@ def run_loop(argv: list[str], env: typing.Mapping[str, str], paths: Paths, launc
|
||||
models = get_or_ask_models(key, paths, env, effects)
|
||||
if isinstance(models, Stop):
|
||||
return models
|
||||
print(f"\n=== cycle {loop_run.cycle_number} ===", flush=True)
|
||||
verdict = run_cycle(Cycle(effects=effects, models=models, paths=paths, premises=premises, project_archive=archive, records=records, root=loop_run.root, session=session, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence))
|
||||
anchor = project_rung(models.worker)
|
||||
root = loop_run.root
|
||||
loop_run = routed(loop_run, anchor, lambda: archived_route(archive_folder, session), lambda: plan_item(root))
|
||||
route = loop_run.route or Route(Rung(models.worker))
|
||||
print(f"\n=== cycle {loop_run.cycle_number}, worker on {route.rung} ===", flush=True)
|
||||
verdict = run_cycle(Cycle(effects=effects, judge_model=models.judge, paths=paths, premises=premises, project_archive=archive_folder, records=records, root=loop_run.root, route=route, session=session, task=loop_run.task, unfinished_evidence=loop_run.unfinished_evidence))
|
||||
if isinstance(verdict, Stop):
|
||||
return verdict
|
||||
loop_run = dataclasses.replace(loop_run, unfinished_evidence=verdict.evidence if isinstance(verdict, Unfinished) else "")
|
||||
before, loop_run = loop_run.route, after_verdict(loop_run, anchor, verdict)
|
||||
if stop := archive_step(archive_folder, session, loop_run, before, effects):
|
||||
return stop
|
||||
|
||||
|
||||
def routed(loop_run: LoopRun, anchor: Rung | None, archived: typing.Callable[[], Route | None], item: typing.Callable[[], Item | None]) -> LoopRun:
|
||||
"""The loop run with its chunk's route, none off the ladder. A chunk keeps its route, or on a rerun the one its archive last names,
|
||||
while that is at or above anchor, the project's rung; else the plan item routes it afresh."""
|
||||
if anchor is None:
|
||||
return dataclasses.replace(loop_run, route=None)
|
||||
route = loop_run.route or archived()
|
||||
return dataclasses.replace(loop_run, route=route if route and kept(route, anchor) else first_route(anchor, item()))
|
||||
|
||||
|
||||
def after_verdict(loop_run: LoopRun, anchor: Rung | None, verdict: Finished | Unfinished) -> LoopRun:
|
||||
if isinstance(verdict, Finished):
|
||||
return dataclasses.replace(loop_run, route=None, unfinished_evidence="")
|
||||
route = stepped(anchor, loop_run.route, verdict.bundled) if anchor and loop_run.route else loop_run.route
|
||||
return dataclasses.replace(loop_run, route=route, unfinished_evidence=verdict.evidence)
|
||||
|
||||
|
||||
def archive_step(archive_folder: pathlib.Path, session: Session, loop_run: LoopRun, before: Route | None, effects: Effects) -> Stop | None:
|
||||
"""Archives the rung an unfinished verdict stepped the chunk up to; the verdict itself names the rung its cycle ran on."""
|
||||
if not (before and loop_run.route) or loop_run.route.rung == before.rung:
|
||||
return None
|
||||
path = transcript_file(session)
|
||||
failure = archive(archive_folder, path, loop_run.task, outcomes.stepped_up(str(loop_run.route)), effects.now()) if path else None
|
||||
return Stop(1, failure.reason) if failure else None
|
||||
|
||||
|
||||
def archived_route(archive_folder: pathlib.Path, session: Session) -> Route | None:
|
||||
"""The route the latest verdict archived for the session's open chunk names; None for a chunk not yet judged."""
|
||||
transcript = transcript_file(session)
|
||||
worker = latest_worker(archive_folder, transcript) if transcript else None
|
||||
if isinstance(worker, Failure):
|
||||
print(f"routing the chunk afresh: {worker.reason}", file=sys.stderr)
|
||||
return None
|
||||
return route_from(worker) if worker else None
|
||||
|
||||
|
||||
def plan_item(root: str) -> Item | None:
|
||||
"""The item that routes a chunk; None without one, so the chunk runs on the project's rung."""
|
||||
try:
|
||||
item = routed_item(pathlib.Path(root, "todo.md").read_text(encoding="utf-8"))
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except (OSError, ValueError) as failure:
|
||||
item = Failure(f"could not read todo.md: {failure}")
|
||||
if isinstance(item, Failure):
|
||||
print(f"routing on the project's rung: {item.reason}", file=sys.stderr)
|
||||
return None
|
||||
return item
|
||||
|
||||
|
||||
def loop_run_from(argv: list[str], cwd: pathlib.Path, effects: Effects) -> LoopRun | Stop:
|
||||
|
||||
@@ -3,16 +3,18 @@ import hashlib
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import types
|
||||
import typing
|
||||
|
||||
from loop_claude.effects import Effects
|
||||
from loop_claude.routing import Route
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
RELOAD_FLAG = "--reloaded"
|
||||
T = typing.TypeVar("T")
|
||||
|
||||
|
||||
# A field change stops a running loop at its next reload: the new script rejects the old fields.
|
||||
# A field removed, renamed or retyped stops a running loop at its next reload: the new script rejects the old fields. A new field with a default does not.
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class LoopRun:
|
||||
"""What a reload hands the edited script, as JSON on argv."""
|
||||
@@ -20,6 +22,8 @@ class LoopRun:
|
||||
root: str
|
||||
task: str
|
||||
unfinished_evidence: str
|
||||
# None until the chunk is routed, and where the project's model is off the ladder.
|
||||
route: Route | None = None
|
||||
|
||||
|
||||
def decode_handed(handed: str, kind: type[T]) -> T | None:
|
||||
@@ -27,12 +31,26 @@ def decode_handed(handed: str, kind: type[T]) -> T | None:
|
||||
fields = json.loads(handed)
|
||||
except ValueError:
|
||||
return None
|
||||
shape = typing.get_type_hints(kind)
|
||||
if not isinstance(fields, dict) or fields.keys() != shape.keys():
|
||||
return None
|
||||
if any(type(fields[name]) is not type_ for name, type_ in shape.items()):
|
||||
return None
|
||||
return kind(**fields)
|
||||
decoded = decoded_as(fields, kind)
|
||||
return None if decoded is REJECTED else typing.cast(T, decoded)
|
||||
|
||||
|
||||
REJECTED = object()
|
||||
|
||||
|
||||
def decoded_as(value: object, kind: typing.Any) -> typing.Any:
|
||||
"""value as kind, or REJECTED: a dataclass from an object holding its fields, a union's first member that fits, else a value of exactly kind."""
|
||||
if isinstance(kind, types.UnionType):
|
||||
return next((found for option in typing.get_args(kind) if (found := decoded_as(value, option)) is not REJECTED), REJECTED)
|
||||
if isinstance(kind, type) and dataclasses.is_dataclass(kind):
|
||||
shape = typing.get_type_hints(kind)
|
||||
# A field with a default may be missing, so a field added since the running script started does not stop the loop.
|
||||
required = {field.name for field in dataclasses.fields(kind) if field.default is dataclasses.MISSING and field.default_factory is dataclasses.MISSING}
|
||||
if not isinstance(value, dict) or not required <= value.keys() <= shape.keys():
|
||||
return REJECTED
|
||||
fields = {name: decoded_as(field, shape[name]) for name, field in value.items()}
|
||||
return REJECTED if any(field is REJECTED for field in fields.values()) else kind(**fields)
|
||||
return value if type(value) is kind else REJECTED
|
||||
|
||||
|
||||
def fingerprint(launcher: pathlib.Path) -> str:
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Which ladder rung a chunk's worker runs on: docs/decisions.md → `routing:`."""
|
||||
import dataclasses
|
||||
import re
|
||||
|
||||
from failure import Failure
|
||||
from model_routing.ladder import highest, picked, Rung, stepped_up
|
||||
from todo_table import Item, items
|
||||
|
||||
UNFINISHED_PER_STEP = 2
|
||||
ROUTED = re.compile(r"(?P<model>\S+) at (?P<effort>\S+) effort(?:, routed by item (?P<item>[0-9]+)'s R \+ C of (?P<sum>[0-9]+))?")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Route:
|
||||
"""A chunk's worker rung and the plan item whose R + C picked it, 0 for none.
|
||||
|
||||
unfinished counts the unfinished verdicts towards the next one-rung step; a reload hands it over with the rung.
|
||||
"""
|
||||
rung: Rung
|
||||
item: int = 0
|
||||
risk_and_complexity: int = 0
|
||||
unfinished: int = 0
|
||||
|
||||
def __str__(self) -> str:
|
||||
"""What the archive records as each verdict's worker; ROUTED reads it back."""
|
||||
return f"{self.rung}, routed by item {self.item}'s R + C of {self.risk_and_complexity}" if self.item else str(self.rung)
|
||||
|
||||
|
||||
def route_from(worker: str) -> Route | None:
|
||||
"""The route an archived worker line names; None for a model run off the ladder."""
|
||||
found = ROUTED.fullmatch(worker)
|
||||
if not found:
|
||||
return None
|
||||
return Route(Rung(found["model"], found["effort"]), int(found["item"] or 0), int(found["sum"] or 0))
|
||||
|
||||
|
||||
def routed_item(todo: str) -> Item | Failure | None:
|
||||
"""The next release's first row that waits on no question, in the table's order, which rank_todo.py keeps ranked; None when there is none, or it has no C."""
|
||||
found = items(todo)
|
||||
if isinstance(found, Failure) or not found:
|
||||
return found if isinstance(found, Failure) else None
|
||||
next_release = min(item.release for item in found)
|
||||
takes = next((item for item in found if item.release == next_release and item.exempt != "question"), None)
|
||||
return takes if takes and takes.risk_and_complexity else None
|
||||
|
||||
|
||||
def first_route(anchor: Rung, item: Item | None) -> Route:
|
||||
"""The route a chunk starts on: the rung its item's R + C picks from anchor, the project's rung, never below it."""
|
||||
if item is None:
|
||||
return Route(anchor)
|
||||
return Route(highest(anchor, picked(anchor, item.risk_and_complexity)), item.id, item.risk_and_complexity)
|
||||
|
||||
|
||||
def kept(route: Route, anchor: Rung) -> bool:
|
||||
"""Whether a route carries on from anchor, the project's rung: its rung is on the ladder at or above it."""
|
||||
return highest(route.rung, anchor) == route.rung
|
||||
|
||||
|
||||
def stepped(anchor: Rung, route: Route, bundled: int) -> Route:
|
||||
"""The route after one more unfinished verdict; bundled is the highest R + C the judge saw the worker take into the chunk, 0 for none."""
|
||||
rung = highest(route.rung, anchor, picked(anchor, bundled) if bundled else anchor)
|
||||
if route.unfinished + 1 >= UNFINISHED_PER_STEP:
|
||||
return dataclasses.replace(route, rung=stepped_up(rung), unfinished=0)
|
||||
return dataclasses.replace(route, rung=rung, unfinished=route.unfinished + 1)
|
||||
@@ -115,14 +115,16 @@ if os.path.basename(sys.argv[0]) == "loop-claude":
|
||||
"""
|
||||
|
||||
PREMISES = "# Project\n\n## Goals\n\n1. Work.\n\n## Audience\n\nMe.\n"
|
||||
# Item 4's R + C of 11 steps one rung up from the project's.
|
||||
TODO = "## Items\n\n| ID | Release | Exempt | Item | R | S | C | A | G | Goals | Score |\n|---|---|---|---|---|---|---|---|---|---|---|\n| 4 | 0.1.0 | | **Do it.** | 5 | 2 | 6 | 8 | 9 | 1 | 25 |\n"
|
||||
|
||||
|
||||
def worker(**step):
|
||||
return {"role": "worker", **step}
|
||||
|
||||
|
||||
def judge(interrupt=False, finished=False, evidence="working", question="", **step):
|
||||
verdict = {"answer": "", "evidence": evidence, "finished": finished, "interrupt": interrupt, "question": question}
|
||||
def judge(interrupt=False, finished=False, evidence="working", question="", bundled=0, **step):
|
||||
verdict = {"answer": "", "bundled": bundled, "evidence": evidence, "finished": finished, "interrupt": interrupt, "question": question}
|
||||
return {"role": "judge", "verdict": verdict, **step}
|
||||
|
||||
|
||||
@@ -360,13 +362,51 @@ class Cycles(unittest.TestCase):
|
||||
self.assertIn("landed-it", run.calls[3]["stdin"])
|
||||
self.assertNotIn("landed-it", run.calls[5]["stdin"])
|
||||
self.assertEqual([path.name for path in checkout.transcripts()[0].parent.iterdir()], [checkout.transcripts()[0].name])
|
||||
self.assertIn("=== cycle 3 ===", run.stdout)
|
||||
self.assertIn("=== cycle 3, worker on worker-model ===", run.stdout)
|
||||
copies = {verdict.removeprefix(f"Task: {TASK}\n\n"): copy for copy, verdict in checkout.archived().items()}
|
||||
self.assertEqual(sorted(copies), ["<UTC> Stopped: Ctrl-C\n", "<UTC> Verdict: finished\n\nEvidence:\n> landed-it\n", "<UTC> Verdict: unfinished\n\nEvidence:\n> started\n"])
|
||||
self.assertIn('"text": "landed-it"', copies["<UTC> Verdict: finished\n\nEvidence:\n> landed-it\n"])
|
||||
self.assertIn('"text": "next chunk"', copies["<UTC> Stopped: Ctrl-C\n"])
|
||||
self.assertEqual(sorted(copies), ["<UTC> Stopped: Ctrl-C\n\nWorker: worker-model\n", "<UTC> Verdict: finished\n\nEvidence:\n> landed-it\n\nWorker: worker-model\n", "<UTC> Verdict: unfinished\n\nEvidence:\n> started\n\nWorker: worker-model\n"])
|
||||
self.assertIn('"text": "landed-it"', copies["<UTC> Verdict: finished\n\nEvidence:\n> landed-it\n\nWorker: worker-model\n"])
|
||||
self.assertIn('"text": "next chunk"', copies["<UTC> Stopped: Ctrl-C\n\nWorker: worker-model\n"])
|
||||
self.assertEqual(len(list(checkout.archive().glob(f"proj/*/session/{session}.orphaned-1-ab.jsonl"))), 1)
|
||||
|
||||
def test_routes_the_worker_by_its_items_r_plus_c_and_steps_it_up_within_the_chunk(self):
|
||||
checkout = Checkout(self)
|
||||
checkout.env["LOOP_CLAUDE_WORKER_MODEL"] = "claude-opus-5-5"
|
||||
(checkout.root / "todo.md").write_text(TODO)
|
||||
run = checkout.run([
|
||||
worker(say="one"), judge(evidence="one"),
|
||||
worker(say="two"), judge(evidence="two"),
|
||||
worker(say="three"), judge(evidence="three", bundled=17),
|
||||
worker(say="four"), judge(finished=True, evidence="four"),
|
||||
worker(say="next chunk"),
|
||||
])
|
||||
self.assertEqual(run.code, 130)
|
||||
workers = [call for call in run.calls if call["role"] == "worker"]
|
||||
self.assertEqual([(argument(call, "--model"), argument(call, "--effort")) for call in workers], [
|
||||
("claude-opus-5-5", "high"),
|
||||
("claude-opus-5-5", "high"),
|
||||
("claude-opus-5-5", "xhigh"),
|
||||
("claude-fable-5-1", "max"),
|
||||
("claude-opus-5-5", "high"),
|
||||
])
|
||||
self.assertEqual({(argument(call, "--model"), argument(call, "--effort")) for call in run.calls if call["role"] == "judge"}, {("judge-model", None)})
|
||||
self.assertIn("loop-claude runs you on claude-opus-5-5 at high effort, routed by item 4's R + C of 11. Name each todo.md item this chunk takes, with its R + C, whether or not it is item 4.", argument(workers[0], "-p"))
|
||||
outcomes = sorted(verdict.removeprefix(f"Task: {TASK}\n\n") for verdict in checkout.archived().values())
|
||||
self.assertIn("<UTC> Verdict: unfinished\n\nEvidence:\n> three\n\nWorker: claude-opus-5-5 at xhigh effort, routed by item 4's R + C of 11\n\n<UTC> Stepped up\n\nWorker: claude-fable-5-1 at max effort, routed by item 4's R + C of 11\n", outcomes)
|
||||
self.assertIn("<UTC> Verdict: finished\n\nEvidence:\n> four\n\nWorker: claude-fable-5-1 at max effort, routed by item 4's R + C of 11\n", outcomes)
|
||||
|
||||
def test_a_rerun_resumes_an_open_chunk_on_the_rung_it_last_ran(self):
|
||||
checkout = Checkout(self)
|
||||
checkout.env["LOOP_CLAUDE_WORKER_MODEL"] = "claude-opus-5-5"
|
||||
(checkout.root / "todo.md").write_text(TODO)
|
||||
run = checkout.run([worker(), judge(), worker(), judge()])
|
||||
self.assertEqual(run.code, 130)
|
||||
self.assertEqual(argument(run.calls[-1], "--effort"), "xhigh")
|
||||
(checkout.root / "todo.md").unlink()
|
||||
run = checkout.run([worker()])
|
||||
self.assertIsNotNone(argument(run.calls[0], "--resume"))
|
||||
self.assertEqual((argument(run.calls[0], "--model"), argument(run.calls[0], "--effort")), ("claude-opus-5-5", "xhigh"))
|
||||
|
||||
def test_exits_1_when_the_transcript_cannot_be_archived_and_resumes_on_the_next_run(self):
|
||||
checkout = Checkout(self)
|
||||
checkout.archive().parent.mkdir(parents=True)
|
||||
@@ -390,7 +430,7 @@ class Cycles(unittest.TestCase):
|
||||
self.assertIn("retrying the worker in a minute: claude exited 1", run.stderr)
|
||||
self.assertIn("claude exited 0 without a result event", run.stderr)
|
||||
self.assertEqual(run.stderr.strip().splitlines()[-1], "the worker failed three times running: claude killed by signal 9")
|
||||
self.assertNotIn("=== cycle 2 ===", run.stdout)
|
||||
self.assertNotIn("=== cycle 2,", run.stdout)
|
||||
self.assertEqual(argument(run.calls[1], "--resume"), argument(run.calls[0], "--session-id"))
|
||||
|
||||
run = checkout.run([worker(exit=1), worker(exit=1), worker(), judge(), worker(exit=1), worker(exit=1)])
|
||||
@@ -489,7 +529,7 @@ class Interrupt(unittest.TestCase):
|
||||
self.assertEqual(run.stderr.strip().splitlines()[-2:], ["A or B?", "the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"])
|
||||
[(copy, verdict)] = checkout.archived().items()
|
||||
self.assertEqual(copy, (checkout.transcripts()[0]).read_text())
|
||||
self.assertEqual(verdict, f"Task: {TASK}\n\n<UTC> Verdict: interrupt\n\nA or B?\n\nEvidence:\n> Shall I use A or B?\n\n<UTC> Stopped: the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal\n")
|
||||
self.assertEqual(verdict, f"Task: {TASK}\n\n<UTC> Verdict: interrupt\n\nA or B?\n\nEvidence:\n> Shall I use A or B?\n\n<UTC> Stopped: the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal\n\nWorker: worker-model\n")
|
||||
self.assertEqual(stat.S_IMODE((checkout.temp / "deep").stat().st_mode), 0o700)
|
||||
self.assertIn(f"transcript: {checkout.archive() / 'proj'}/", run.stderr)
|
||||
session = argument(run.calls[0], "--session-id")
|
||||
@@ -688,16 +728,16 @@ class Reload(unittest.TestCase):
|
||||
self.assertEqual(run.roles(), ["worker", "judge"] * 5)
|
||||
self.assertIn("keeping the running script", run.stderr)
|
||||
reloads = [json.loads(argv[1]) for argv in run.starts if argv[:1] == ["--reloaded"]]
|
||||
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working"} for cycle in (2, 3, 4)])
|
||||
self.assertIn("=== cycle 5 ===", run.stdout)
|
||||
self.assertEqual(reloads, [{"cycle_number": cycle, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "working", "route": None} for cycle in (2, 3, 4)])
|
||||
self.assertIn("=== cycle 5,", run.stdout)
|
||||
|
||||
outside = checkout.temp / "outside"
|
||||
outside.mkdir()
|
||||
state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK, "unfinished_evidence": ""}
|
||||
state = {"cycle_number": 3, "root": str(checkout.root), "task": TASK, "unfinished_evidence": "", "route": None}
|
||||
run = checkout.run([worker()], argv=["--reloaded", json.dumps(state)], cwd=outside)
|
||||
self.assertEqual((run.code, run.roles()), (130, ["worker", "judge"]))
|
||||
self.assertEqual(run.calls[0]["cwd"], str(checkout.root))
|
||||
self.assertIn("=== cycle 4 ===", run.stdout)
|
||||
self.assertIn("=== cycle 4,", run.stdout)
|
||||
|
||||
run = checkout.run([], argv=["--reloaded", "{}"])
|
||||
self.assertEqual((run.code, run.calls), (1, []))
|
||||
|
||||
@@ -21,9 +21,11 @@ from claude_code import entries, session
|
||||
from failure import Failure
|
||||
from loop_claude.cycle.judge import Finished, Interrupt, QuotedAnswer, Unfinished, verdict_from
|
||||
from loop_claude.cycle import interrupt, judge, prompts, retry, run, states, worker
|
||||
from model_routing.ladder import LADDER, Rung
|
||||
from loop_claude.cycle.retry import Limited
|
||||
from loop_claude.loop import main, reload
|
||||
from loop_claude import paths
|
||||
from transcript_archive import outcomes, write
|
||||
from loop_claude import paths, routing
|
||||
from transcript_archive.location import project_archive
|
||||
from loop_claude.stop import Stop
|
||||
|
||||
@@ -88,7 +90,7 @@ def ran(stdout="", code=0, stderr=""):
|
||||
|
||||
|
||||
def judged(verdict, evidence="working", answer=""):
|
||||
reply = {"answer": answer, "evidence": evidence, "finished": verdict == "finished", "interrupt": verdict == "interrupt", "question": "Which one?" if verdict in ("interrupt", "quoted") else ""}
|
||||
reply = {"answer": answer, "bundled": 0, "evidence": evidence, "finished": verdict == "finished", "interrupt": verdict == "interrupt", "question": "Which one?" if verdict in ("interrupt", "quoted") else ""}
|
||||
return ran(json.dumps({"structured_output": reply}))
|
||||
|
||||
|
||||
@@ -169,7 +171,7 @@ class Cycles(unittest.TestCase):
|
||||
self.archive = project_archive(str(self.home / "code" / "proj"), self.home, {})
|
||||
|
||||
def cycle(self, fake, records="", unfinished_evidence=""):
|
||||
return run.Cycle(effects=fake.effects(), models=model_choice.Models("judge-model", "worker-model"), paths=self.paths, premises="## Goals", project_archive=self.archive, records=records, root=str(self.home), session=self.session, task="Do it.", unfinished_evidence=unfinished_evidence)
|
||||
return run.Cycle(effects=fake.effects(), judge_model="judge-model", paths=self.paths, premises="## Goals", project_archive=self.archive, records=records, root=str(self.home), route=routing.Route(Rung("worker-model")), session=self.session, task="Do it.", unfinished_evidence=unfinished_evidence)
|
||||
|
||||
def archived_outcomes(self):
|
||||
return sorted(path.read_text() for path in self.archive.glob("*/*.outcomes.md"))
|
||||
@@ -177,7 +179,7 @@ class Cycles(unittest.TestCase):
|
||||
def test_a_finished_chunk_is_archived_and_its_session_deleted(self):
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("finished")])
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Finished("working"))
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n"])
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n\nWorker: worker-model\n"])
|
||||
self.assertIsNone(session.transcript_file(self.session))
|
||||
|
||||
def test_an_unfinished_chunks_transcript_is_archived_and_the_next_cycle_resumes_its_session_with_the_judges_evidence(self):
|
||||
@@ -189,14 +191,14 @@ class Cycles(unittest.TestCase):
|
||||
self.assertIn("--session-id", workers[0])
|
||||
self.assertIn("--resume", workers[1])
|
||||
self.assertIn("as the work remaining:\n\n> working\n\nThe chunk is finished once", workers[1][workers[1].index("-p") + 1])
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n"] * 2)
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n\nWorker: worker-model\n"] * 2)
|
||||
|
||||
def test_a_cycle_stopped_by_a_failure_or_ctrl_c_archives_what_its_session_holds(self):
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[ran(code=1)] * 3)
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)).code, 1)
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[KeyboardInterrupt()])
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Stop(130, ""))
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n", "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: the judge failed three times running: the judge exited 1\n"])
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n\nWorker: worker-model\n", "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: the judge failed three times running: the judge exited 1\n\nWorker: worker-model\n"])
|
||||
|
||||
def test_archives_each_chunks_session_folder_replacing_a_copy_only_with_what_grew_from_it(self):
|
||||
folder = self.paths.claude_code_transcripts / "-fake" / str(self.session.id)
|
||||
@@ -242,7 +244,7 @@ class Cycles(unittest.TestCase):
|
||||
return 0
|
||||
fake = FakeEffects(streams=[Worker(self.paths.claude_code_transcripts.parent)], claude=[judged("interrupt"), judged("unfinished")], on_terminal_claude=[answer], git=[ran()])
|
||||
self.assertEqual(run.run_cycle(self.cycle(fake)), Unfinished("working"))
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nEvidence:\n> working\n", "Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n"])
|
||||
self.assertEqual(self.archived_outcomes(), ["Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nEvidence:\n> working\n", "Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: unfinished\n\nEvidence:\n> working\n\nWorker: worker-model\n"])
|
||||
self.assertIn(str(self.session.id), fake.made("claude_on_terminal")[0])
|
||||
judge_prompts = [stdin for args, stdin in fake.made("claude")]
|
||||
self.assertEqual(len(judge_prompts), 2)
|
||||
@@ -260,7 +262,7 @@ class Cycles(unittest.TestCase):
|
||||
self.assertEqual(printed.getvalue().count("=== README.md's Goals and Audience, AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/ or ~/.claude/skills/larv-review/standards/ answered: Which one? ===\nUse *A*."), 3)
|
||||
self.assertIn("Which one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/ or ~/.claude/skills/larv-review/standards/ says:\nUse *A*.\n\nThose files have answered 3 questions this cycle", said.getvalue())
|
||||
quoted = "2026-09-21T14:13:20Z Verdict: quoted answer\n\nWhich one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/ or ~/.claude/skills/larv-review/standards/ says:\n> Use *A*.\n\nEvidence:\n> working\n"
|
||||
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/ or ~/.claude/skills/larv-review/standards/ says:\nUse *A*.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.\n\nEvidence:\n> working\n\n2026-09-21T14:13:20Z Stopped: the human is needed to answer the question above, but the interrupt ended with nothing typed\n", *[f"Task: Do it.\n\n{quoted}"] * 3])
|
||||
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: interrupt\n\nWhich one?\n\nREADME.md's Goals and Audience, AGENTS.md, docs/decisions.md, ~/.claude/CLAUDE.md, ~/.claude/rules/ or ~/.claude/skills/larv-review/standards/ says:\nUse *A*.\n\nThose files have answered 3 questions this cycle, the most allowed, so this one comes to you.\n\nEvidence:\n> working\n\n2026-09-21T14:13:20Z Stopped: the human is needed to answer the question above, but the interrupt ended with nothing typed\n\nWorker: worker-model\n", *[f"Task: Do it.\n\n{quoted}"] * 3])
|
||||
|
||||
def test_stops_when_a_quoted_answer_cannot_be_archived(self):
|
||||
self.archive.parent.mkdir(parents=True)
|
||||
@@ -289,7 +291,7 @@ class Cycles(unittest.TestCase):
|
||||
stop = run.run_cycle(self.cycle(fake))
|
||||
self.assertEqual(stop, Stop(1, f"could not delete {folder}: Permission denied; delete every file and folder matching {folder}*, then rerun loop-claude; a rerun before that resumes the finished chunk's session"))
|
||||
self.assertTrue(session.session_open(self.session))
|
||||
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n\n2026-09-21T14:13:20Z Stopped: {stop.reason}\n"])
|
||||
self.assertEqual(self.archived_outcomes(), [f"Task: Do it.\n\n2026-09-21T14:13:20Z Verdict: finished\n\nEvidence:\n> working\n\nWorker: worker-model\n\n2026-09-21T14:13:20Z Stopped: {stop.reason}\n\nWorker: worker-model\n"])
|
||||
|
||||
def test_stops_when_the_worker_leaves_no_transcript(self):
|
||||
fake = FakeEffects(streams=[Streamed(0)])
|
||||
@@ -349,7 +351,7 @@ class Limits(unittest.TestCase):
|
||||
class Interrupts(unittest.TestCase):
|
||||
def test_stops_off_a_terminal_before_starting_claude(self):
|
||||
fake = FakeEffects(on_terminal=False)
|
||||
stop = interrupt.interrupt("/r", uuid7(), "m", "/t", "Which one?", paths.paths_from(pathlib.Path("/h")), fake.effects())
|
||||
stop = interrupt.interrupt("/r", uuid7(), Rung("m"), "/t", "Which one?", paths.paths_from(pathlib.Path("/h")), fake.effects())
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but stdin or stdout is not a terminal; rerun loop-claude on a terminal"))
|
||||
self.assertEqual(fake.calls, [])
|
||||
|
||||
@@ -358,7 +360,8 @@ class Interrupts(unittest.TestCase):
|
||||
path = pathlib.Path(home, "t.jsonl")
|
||||
path.write_text(json.dumps(human("a")) + "\n")
|
||||
fake = FakeEffects(on_terminal_claude=[0], git=[ran()])
|
||||
stop = interrupt.interrupt(home, uuid7(), "m", str(path), "Which one?", paths.paths_from(pathlib.Path(home)), fake.effects())
|
||||
stop = interrupt.interrupt(home, uuid7(), Rung("m", "high"), str(path), "Which one?", paths.paths_from(pathlib.Path(home)), fake.effects())
|
||||
self.assertEqual(fake.made("claude_on_terminal")[0][:4], ["--model", "m", "--effort", "high"])
|
||||
self.assertEqual(stop, Stop(2, "the human is needed to answer the question above, but the interrupt ended with nothing typed"))
|
||||
|
||||
|
||||
@@ -381,13 +384,156 @@ class ModelPrompt(unittest.TestCase):
|
||||
self.assertEqual(model_choice.ask_for_model("proj", pathlib.Path("/m.json"), pathlib.Path(empty), fake.effects()), Stop(2, "the human is needed to pick proj's model, but the input ended"))
|
||||
|
||||
|
||||
TODO = """# todo
|
||||
|
||||
## Items
|
||||
|
||||
| ID | Release | Exempt | Item | R | S | C | A | G | Goals | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 7 | 0.2.0 | question | **Ask first.** | 9 | 1 | 9 | 9 | 9 | 1 | 30 |
|
||||
| 3 | 0.2.0 | | **Do the next thing.** | 5 | 3 | 6 | 8 | 9 | 1 | 25 |
|
||||
| 1 | 0.3.0 | | **Do a later thing.** | 1 | 1 | 1 | 9 | 9 | 1 | 30 |
|
||||
"""
|
||||
|
||||
|
||||
class Routing(unittest.TestCase):
|
||||
OPUS = Rung("claude-opus-5-5", "medium")
|
||||
HIGH = Rung("claude-opus-5-5", "high")
|
||||
XHIGH = Rung("claude-opus-5-5", "xhigh")
|
||||
|
||||
def test_routes_on_the_next_releases_top_item_that_waits_on_no_question(self):
|
||||
self.assertEqual(routing.routed_item(TODO).id, 3)
|
||||
self.assertIsNone(routing.routed_item(TODO.replace("| | **Do", "| question | **Do")))
|
||||
self.assertIsNone(routing.routed_item("# todo\n"))
|
||||
self.assertIsNone(routing.routed_item(TODO[:TODO.index("| 7 |")]))
|
||||
without_c = TODO.replace(" C |", "").replace("|---|---|---|---|---|---|---|---|---|---|---|", "|---|---|---|---|---|---|---|---|---|---|").replace("| 9 | 1 | 9 |", "| 9 | 1 |").replace("| 5 | 3 | 6 |", "| 5 | 3 |").replace("| 1 | 1 | 1 |", "| 1 | 1 |")
|
||||
self.assertIsNone(routing.routed_item(without_c))
|
||||
self.assertEqual(routing.routed_item(TODO.replace("| 5 | 3 |", "| x | 3 |")), Failure("item 3's R 'x' is not a whole number from 1 to 10"))
|
||||
|
||||
def test_starts_a_chunk_on_the_rung_its_items_r_plus_c_picks_never_below_the_projects(self):
|
||||
self.assertEqual(routing.first_route(self.OPUS, routing.routed_item(TODO)), routing.Route(self.HIGH, 3, 11))
|
||||
cheap = routing.routed_item(TODO.replace("| 5 | 3 | 6 |", "| 1 | 3 | 1 |"))
|
||||
self.assertEqual(routing.first_route(self.OPUS, cheap), routing.Route(self.OPUS, 3, 2))
|
||||
self.assertEqual(routing.first_route(self.OPUS, None), routing.Route(self.OPUS))
|
||||
|
||||
def test_steps_up_a_rung_after_every_second_unfinished_verdict(self):
|
||||
once = routing.stepped(self.OPUS, routing.Route(self.OPUS, 3, 9), 0)
|
||||
self.assertEqual(once, routing.Route(self.OPUS, 3, 9, 1))
|
||||
self.assertEqual(routing.stepped(self.OPUS, once, 0), routing.Route(self.HIGH, 3, 9, 0))
|
||||
self.assertEqual(routing.stepped(self.OPUS, routing.Route(LADDER[-1], unfinished=1), 0).rung, LADDER[-1])
|
||||
|
||||
def test_steps_up_to_the_rung_a_bundled_items_r_plus_c_picks_and_never_down(self):
|
||||
self.assertEqual(routing.stepped(self.OPUS, routing.Route(self.OPUS), 13).rung, self.XHIGH)
|
||||
self.assertEqual(routing.stepped(self.OPUS, routing.Route(self.XHIGH), 11).rung, self.XHIGH)
|
||||
self.assertEqual(routing.stepped(self.OPUS, routing.Route(self.OPUS, unfinished=1), 13).rung, Rung("claude-opus-5-5", "max"))
|
||||
|
||||
def test_a_rung_an_edit_took_off_the_ladder_counts_as_the_projects(self):
|
||||
self.assertEqual(routing.stepped(self.OPUS, routing.Route(Rung("claude-opus-5-5", "low")), 0).rung, self.OPUS)
|
||||
self.assertEqual(routing.stepped(self.OPUS, routing.Route(Rung("claude-opus-5-5", "low")), 2).rung, self.OPUS)
|
||||
|
||||
def test_archives_the_route_in_a_line_it_reads_back(self):
|
||||
route = routing.Route(self.HIGH, 3, 11)
|
||||
self.assertEqual(str(route), "claude-opus-5-5 at high effort, routed by item 3's R + C of 11")
|
||||
self.assertEqual(routing.route_from(str(route)), route)
|
||||
self.assertEqual(routing.route_from(str(routing.Route(self.HIGH))), routing.Route(self.HIGH))
|
||||
self.assertEqual(str(routing.Route(Rung("worker-model"))), "worker-model")
|
||||
self.assertIsNone(routing.route_from("worker-model"))
|
||||
|
||||
def test_keeps_a_chunks_route_or_its_archived_one_at_or_above_the_projects_rung_and_else_routes_afresh(self):
|
||||
def run_with(route=None):
|
||||
return main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=route)
|
||||
item = routing.routed_item(TODO)
|
||||
archived = routing.Route(self.XHIGH, 3, 11)
|
||||
def never():
|
||||
raise AssertionError("not asked")
|
||||
self.assertIsNone(main.routed(run_with(archived), None, never, never).route)
|
||||
self.assertEqual(main.routed(run_with(archived), self.OPUS, never, never).route, archived)
|
||||
self.assertEqual(main.routed(run_with(), self.OPUS, lambda: archived, never).route, archived)
|
||||
self.assertEqual(main.routed(run_with(), self.OPUS, lambda: None, lambda: item).route, routing.Route(self.HIGH, 3, 11))
|
||||
below = routing.Route(Rung("claude-sonnet-5-5", "high"))
|
||||
self.assertEqual(main.routed(run_with(below), self.OPUS, never, lambda: None).route, routing.Route(self.OPUS))
|
||||
|
||||
|
||||
class RouteSources(unittest.TestCase):
|
||||
def test_routes_on_the_plan_item_and_says_why_it_routes_on_the_projects_rung(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
self.assertIsNone(main.plan_item(root))
|
||||
todo = pathlib.Path(root, "todo.md")
|
||||
todo.write_text(TODO)
|
||||
self.assertEqual(main.plan_item(root).id, 3)
|
||||
for text, said in ((TODO.replace("| 5 | 3 |", "| x | 3 |"), "routing on the project's rung: item 3's R 'x' is not a whole number from 1 to 10\n"), (b"\xff", "routing on the project's rung: could not read todo.md: ")):
|
||||
with self.subTest(said=said):
|
||||
todo.write_bytes(text if isinstance(text, bytes) else text.encode())
|
||||
with contextlib.redirect_stderr(io.StringIO()) as stderr:
|
||||
self.assertIsNone(main.plan_item(root))
|
||||
self.assertTrue(stderr.getvalue().startswith(said), stderr.getvalue())
|
||||
|
||||
def test_resumes_on_the_archived_route_and_says_why_it_routes_afresh(self):
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
opened = session.session_named("A proj loop-claude", pathlib.Path(home, "projects"))
|
||||
archive = pathlib.Path(home, "archive")
|
||||
self.assertIsNone(main.archived_route(archive, opened))
|
||||
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
|
||||
transcript.parent.mkdir(parents=True)
|
||||
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
|
||||
self.assertIsNone(main.archived_route(archive, opened))
|
||||
route = routing.Route(Rung("claude-opus-5-5", "high"), 3, 11)
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(str(route), "working"), NOW))
|
||||
self.assertEqual(main.archived_route(archive, opened), route)
|
||||
(chunk,) = archive.iterdir()
|
||||
chunk.chmod(0)
|
||||
try:
|
||||
with contextlib.redirect_stderr(io.StringIO()) as stderr:
|
||||
self.assertIsNone(main.archived_route(archive, opened))
|
||||
finally:
|
||||
chunk.chmod(0o700)
|
||||
self.assertTrue(stderr.getvalue().startswith("routing the chunk afresh: could not read the archive"), stderr.getvalue())
|
||||
|
||||
|
||||
class Steps(unittest.TestCase):
|
||||
OPUS = Rung("claude-opus-5-5", "medium")
|
||||
|
||||
def test_a_finished_verdict_clears_the_route_and_the_evidence(self):
|
||||
run_with = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="old", route=routing.Route(self.OPUS))
|
||||
self.assertEqual(main.after_verdict(run_with, self.OPUS, Finished("x")), dataclasses.replace(run_with, route=None, unfinished_evidence=""))
|
||||
self.assertEqual(main.after_verdict(run_with, self.OPUS, Unfinished("new", 13)), dataclasses.replace(run_with, route=routing.Route(Rung("claude-opus-5-5", "xhigh"), unfinished=1), unfinished_evidence="new"))
|
||||
|
||||
def test_archives_a_step_up_as_it_happens_so_a_rerun_resumes_on_it(self):
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
opened = session.session_named("A proj loop-claude", pathlib.Path(home, "projects"))
|
||||
archive = pathlib.Path(home, "archive")
|
||||
transcript = pathlib.Path(home, "projects", "-fake", f"{opened.id}.jsonl")
|
||||
transcript.parent.mkdir(parents=True)
|
||||
transcript.write_text(json.dumps({"type": "queue-operation", "timestamp": "1"}) + "\n")
|
||||
before = routing.Route(self.OPUS, 3, 9)
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(write.archive(archive, str(transcript), "Do it.", outcomes.unfinished(str(before), "working"), NOW))
|
||||
after = main.after_verdict(main.LoopRun(cycle_number=1, root="/r", task="Do it.", unfinished_evidence="", route=before), self.OPUS, Unfinished("working", 13))
|
||||
self.assertIsNone(main.archive_step(archive, opened, after, before, FakeEffects().effects()))
|
||||
self.assertEqual(main.archived_route(archive, opened), routing.Route(Rung("claude-opus-5-5", "xhigh"), 3, 9))
|
||||
self.assertIsNone(main.archive_step(archive, opened, after, after.route, FakeEffects().effects()))
|
||||
self.assertIsNone(main.archive_step(archive, opened, after, None, FakeEffects().effects()))
|
||||
blocked = pathlib.Path(home, "blocked")
|
||||
blocked.write_text("")
|
||||
stop = main.archive_step(blocked, opened, after, before, FakeEffects().effects())
|
||||
self.assertEqual((stop.code, stop.reason.split(":")[0]), (1, "could not archive the transcript"))
|
||||
|
||||
|
||||
class WorkerArgs(unittest.TestCase):
|
||||
def test_names_the_route_in_a_fresh_prompt_only_where_an_item_routed_it(self):
|
||||
routed = prompts.fresh("Do it.", "## Goals", routing.Route(Rung("m", "high"), 3, 11))
|
||||
self.assertIn("Continue any open draft PR or worktree for the chunk before creating a new one.\n\nloop-claude runs you on m at high effort, routed by item 3's R + C of 11. Name each todo.md item this chunk takes, with its R + C, whether or not it is item 3.\n\nThe chunk's", routed)
|
||||
unrouted = prompts.fresh("Do it.", "## Goals", routing.Route(Rung("m")))
|
||||
self.assertIn("before creating a new one.\n\nThe chunk's", unrouted)
|
||||
self.assertNotIn("loop-claude runs you on", unrouted)
|
||||
|
||||
def test_opens_a_fresh_session_by_name_and_resumes_an_open_one(self):
|
||||
opened = session.session_named("A proj loop-claude", pathlib.Path("/h/.claude/projects"))
|
||||
fresh = worker.worker_args(opened, "m", prompts.fresh("Do it.", "## Goals\n1. g\n\n## Audience\nme"), resume=False)
|
||||
fresh = worker.worker_args(opened, Rung("m"), prompts.fresh("Do it.", "## Goals\n1. g\n\n## Audience\nme", routing.Route(Rung("m"))), resume=False)
|
||||
self.assertEqual(fresh[fresh.index("--session-id") + 1], str(opened.id))
|
||||
self.assertEqual(fresh[fresh.index("--name") + 1], "A proj loop-claude")
|
||||
resumed = worker.worker_args(opened, "m", prompts.resumed(""), resume=True)
|
||||
resumed = worker.worker_args(opened, Rung("m"), prompts.resumed(""), resume=True)
|
||||
self.assertEqual(resumed[resumed.index("--resume") + 1], str(opened.id))
|
||||
self.assertNotIn("--session-id", resumed)
|
||||
self.assertTrue(resumed[resumed.index("-p") + 1].startswith("Continue the task; the previous run ended before the chunk was finished. The chunk is finished once"))
|
||||
@@ -467,17 +613,25 @@ class Grounded(unittest.TestCase):
|
||||
|
||||
class Verdict(unittest.TestCase):
|
||||
def test_needs_every_field_the_schema_requires(self):
|
||||
self.assertEqual(verdict_from({"answer": "", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Finished("e"))
|
||||
self.assertEqual(verdict_from({"answer": "", "interrupt": False, "evidence": "e", "finished": False, "question": "ignored"}), Unfinished("e"))
|
||||
self.assertEqual(verdict_from({"answer": "", "interrupt": True, "evidence": "e", "finished": True, "question": "Which one?"}), Interrupt("e", "Which one?"))
|
||||
self.assertEqual(verdict_from({"answer": "A.", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?"}), QuotedAnswer("e", "Which one?", "A."))
|
||||
self.assertEqual(verdict_from({"answer": "A.", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?"}), Interrupt("e", "Which one?"))
|
||||
self.assertEqual(verdict_from({"answer": "A.", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Failure("the judge quoted an answer without a question"))
|
||||
self.assertEqual(verdict_from({"answer": "", "interrupt": "no", "evidence": "", "finished": True, "question": ""}), Failure("the judge's reply has no interrupt of type boolean"))
|
||||
self.assertEqual(verdict_from({"answer": "", "interrupt": False, "evidence": None, "finished": True, "question": ""}), Failure("the judge's reply has no evidence of type string"))
|
||||
self.assertEqual(verdict_from({"answer": "", "interrupt": True, "evidence": "x", "finished": False, "question": " "}), Failure("the judge interrupted without a question"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Finished("e"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "", "interrupt": False, "evidence": "e", "finished": False, "question": "ignored"}), Unfinished("e"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "", "interrupt": True, "evidence": "e", "finished": True, "question": "Which one?"}), Interrupt("e", "Which one?"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "A.", "interrupt": False, "evidence": "e", "finished": False, "question": "Which one?"}), QuotedAnswer("e", "Which one?", "A."))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "A.", "interrupt": True, "evidence": "e", "finished": False, "question": "Which one?"}), Interrupt("e", "Which one?"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "A.", "interrupt": False, "evidence": "e", "finished": True, "question": ""}), Failure("the judge quoted an answer without a question"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "", "interrupt": "no", "evidence": "", "finished": True, "question": ""}), Failure("the judge's reply has no interrupt of type boolean"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "", "interrupt": False, "evidence": None, "finished": True, "question": ""}), Failure("the judge's reply has no evidence of type string"))
|
||||
self.assertEqual(verdict_from({"bundled": 0, "answer": "", "interrupt": True, "evidence": "x", "finished": False, "question": " "}), Failure("the judge interrupted without a question"))
|
||||
self.assertEqual(verdict_from(None), Failure("the judge returned no reply"))
|
||||
|
||||
def test_takes_a_bundled_r_plus_c_from_0_to_20_on_an_unfinished_verdict(self):
|
||||
reply = {"answer": "", "interrupt": False, "evidence": "e", "finished": False, "question": ""}
|
||||
self.assertEqual(verdict_from({**reply, "bundled": 20}), Unfinished("e", 20))
|
||||
self.assertEqual(verdict_from({**reply, "bundled": True}), Failure("the judge's reply has no bundled of type integer"))
|
||||
for bundled in (21, -1):
|
||||
with self.subTest(bundled=bundled):
|
||||
self.assertEqual(verdict_from({**reply, "bundled": bundled}), Failure(f"the judge's bundled R + C {bundled} is not from 0 to 20"))
|
||||
|
||||
|
||||
class LoopRuns(unittest.TestCase):
|
||||
def test_stops_naming_the_reload_when_it_hands_over_anything_but_the_state_with_a_task(self):
|
||||
@@ -493,6 +647,20 @@ class LoopRuns(unittest.TestCase):
|
||||
self.assertEqual(reload.decode_handed('{"count": 1, "name": "n"}', Stringly), Stringly(1, "n"))
|
||||
self.assertIsNone(reload.decode_handed('{"count": "1", "name": "n"}', Stringly))
|
||||
|
||||
def test_decodes_a_route_inside_the_state_or_none(self):
|
||||
state = main.LoopRun(cycle_number=1, root="/r", task="do it", unfinished_evidence="", route=routing.Route(Rung("claude-opus-5-5", "high"), 3, 11, 1))
|
||||
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(state)), main.LoopRun), state)
|
||||
unrouted = dataclasses.replace(state, route=None)
|
||||
self.assertEqual(reload.decode_handed(json.dumps(dataclasses.asdict(unrouted)), main.LoopRun), unrouted)
|
||||
handed = dataclasses.asdict(state)
|
||||
handed["route"]["rung"]["effort"] = 1
|
||||
self.assertIsNone(reload.decode_handed(json.dumps(handed), main.LoopRun))
|
||||
del handed["route"]["rung"]
|
||||
self.assertIsNone(reload.decode_handed(json.dumps(handed), main.LoopRun))
|
||||
|
||||
def test_decodes_a_state_from_before_a_field_with_a_default_joined_it(self):
|
||||
self.assertEqual(reload.decode_handed(json.dumps({"cycle_number": 3, "root": "/r", "task": "do it", "unfinished_evidence": ""}), main.LoopRun), main.LoopRun(3, "/r", "do it", ""))
|
||||
|
||||
def test_stops_when_the_reload_cannot_start_python(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
launcher = pathlib.Path(root, "loop-claude")
|
||||
@@ -530,6 +698,7 @@ class ExitContract(unittest.TestCase):
|
||||
("cycle/run.py", "ask", "1", "failure.reason", read),
|
||||
("cycle/run.py", "close", "1", "failure.reason", read),
|
||||
("cycle/run.py", "close", "1", "failure.reason", "a finished chunk's session cannot be deleted"),
|
||||
("loop/main.py", "archive_step", "1", "failure.reason", read),
|
||||
("loop/main.py", "loop_run_from", "0 if task else 1", "__doc__.strip()", "no task is given"),
|
||||
("loop/main.py", "loop_run_from", "1", 'f"the edited loop-claude cannot read the state', "a reload cannot read the state the running script handed it"),
|
||||
("loop/reload.py", "exec_if_edited", "1", "f'could not reload loop-claude:", "python cannot start for a reload"),
|
||||
@@ -567,6 +736,7 @@ class JudgeFailures(unittest.TestCase):
|
||||
("verdict_from", 'f"the judge\'s reply has no {name}', field),
|
||||
("verdict_from", "'the judge interrupted without a question'", unasked),
|
||||
("verdict_from", "'the judge quoted an answer without a question'", unasked),
|
||||
("verdict_from", 'f"the judge\'s bundled R + C', "gives an unfinished verdict an R + C outside 0 to 20"),
|
||||
("run_judge", "f'could not start claude:", started),
|
||||
("run_judge", "proc.stderr.strip() or f'the judge exited", started),
|
||||
("run_judge", 'f"{verdict.reason}; it replied:', field),
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""The routing ladder, and the rung an R + C picks from the project's rung: docs/decisions.md → `routing:`."""
|
||||
import dataclasses
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Rung:
|
||||
model: str
|
||||
# Empty leaves the effort to claude's settings, as for a model off the ladder.
|
||||
effort: str = ""
|
||||
|
||||
def flags(self) -> list[str]:
|
||||
return ["--model", self.model, *(["--effort", self.effort] if self.effort else [])]
|
||||
|
||||
# The archive records this wording, and loop_claude.routing.ROUTED reads it back to resume a chunk on its rung.
|
||||
def __str__(self) -> str:
|
||||
return f"{self.model} at {self.effort} effort" if self.effort else self.model
|
||||
|
||||
|
||||
# Least competent first.
|
||||
LADDER = (
|
||||
Rung("claude-haiku-5-5", "low"),
|
||||
Rung("claude-haiku-5-5", "medium"),
|
||||
Rung("claude-haiku-5-5", "high"),
|
||||
Rung("claude-haiku-5-5", "xhigh"),
|
||||
Rung("claude-haiku-5-5", "max"),
|
||||
Rung("claude-sonnet-5-5", "high"),
|
||||
Rung("claude-opus-5-5", "medium"),
|
||||
Rung("claude-opus-5-5", "high"),
|
||||
Rung("claude-opus-5-5", "xhigh"),
|
||||
Rung("claude-opus-5-5", "max"),
|
||||
Rung("claude-fable-5-1", "max"),
|
||||
)
|
||||
# (highest R + C in the band, rungs it steps from the project's rung); a higher R + C steps ABOVE_THE_BANDS.
|
||||
BANDS = ((4, -3), (6, -2), (8, -1), (10, 0), (12, 1), (14, 2), (16, 3))
|
||||
ABOVE_THE_BANDS = 4
|
||||
|
||||
|
||||
def project_rung(model: str) -> Rung | None:
|
||||
"""The model's cheapest rung, which for Opus 5.5 is its default effort; None for a model off the ladder."""
|
||||
return next((rung for rung in LADDER if rung.model == model), None)
|
||||
|
||||
|
||||
def picked(anchor: Rung, risk_and_complexity: int) -> Rung:
|
||||
"""The rung a task's R + C picks from anchor, the project's rung, stopping at either end of the ladder."""
|
||||
return climbed(anchor, next((step for highest, step in BANDS if risk_and_complexity <= highest), ABOVE_THE_BANDS))
|
||||
|
||||
|
||||
def stepped_up(rung: Rung) -> Rung:
|
||||
"""The rung above, or the top one; a rung off the ladder steps to the bottom one."""
|
||||
return climbed(rung, 1)
|
||||
|
||||
|
||||
def highest(*rungs: Rung) -> Rung:
|
||||
"""The most competent of the rungs; one off the ladder ranks below every rung on it."""
|
||||
return max(rungs, key=position)
|
||||
|
||||
|
||||
def climbed(rung: Rung, by: int) -> Rung:
|
||||
return LADDER[max(0, min(len(LADDER) - 1, position(rung) + by))]
|
||||
|
||||
|
||||
def position(rung: Rung) -> int:
|
||||
return LADDER.index(rung) if rung in LADDER else -1
|
||||
@@ -0,0 +1,49 @@
|
||||
import pathlib
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from model_routing.ladder import highest, LADDER, picked, project_rung, Rung, stepped_up
|
||||
|
||||
OPUS = Rung("claude-opus-5-5", "medium")
|
||||
|
||||
|
||||
class Ladder(unittest.TestCase):
|
||||
def test_runs_from_haiku_at_low_effort_to_fable_at_max_with_opus_at_medium_between(self):
|
||||
self.assertEqual(LADDER[0], Rung("claude-haiku-5-5", "low"))
|
||||
self.assertEqual(LADDER[-1], Rung("claude-fable-5-1", "max"))
|
||||
self.assertEqual(len(LADDER), 11)
|
||||
self.assertEqual(len(set(LADDER)), len(LADDER))
|
||||
|
||||
def test_puts_a_project_on_its_models_cheapest_rung_and_none_off_the_ladder(self):
|
||||
self.assertEqual(project_rung("claude-opus-5-5"), OPUS)
|
||||
self.assertEqual(project_rung("claude-sonnet-5-5"), Rung("claude-sonnet-5-5", "high"))
|
||||
self.assertIsNone(project_rung("claude-opus-5"))
|
||||
self.assertIsNone(project_rung("opus"))
|
||||
|
||||
def test_steps_each_band_of_r_plus_c_from_the_projects_rung(self):
|
||||
position = LADDER.index(OPUS)
|
||||
self.assertEqual([LADDER.index(picked(OPUS, total)) - position for total in range(2, 21)], [-3, -3, -3, -2, -2, -1, -1, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 4, 4])
|
||||
|
||||
def test_a_step_past_either_end_stops_there(self):
|
||||
self.assertEqual(picked(OPUS, 4), Rung("claude-haiku-5-5", "xhigh"))
|
||||
self.assertEqual(picked(OPUS, 17), Rung("claude-fable-5-1", "max"))
|
||||
self.assertEqual(picked(LADDER[1], 2), LADDER[0])
|
||||
self.assertEqual(stepped_up(LADDER[-1]), LADDER[-1])
|
||||
self.assertEqual(stepped_up(OPUS), Rung("claude-opus-5-5", "high"))
|
||||
|
||||
def test_ranks_a_rung_off_the_ladder_below_every_rung_on_it(self):
|
||||
self.assertEqual(highest(OPUS, LADDER[-1], LADDER[0]), LADDER[-1])
|
||||
self.assertEqual(highest(Rung("claude-opus-5-5", "low"), LADDER[0]), LADDER[0])
|
||||
self.assertEqual(stepped_up(Rung("claude-opus-5-5", "low")), LADDER[0])
|
||||
|
||||
def test_passes_the_effort_only_where_the_rung_names_one(self):
|
||||
self.assertEqual(Rung("claude-opus-5-5", "high").flags(), ["--model", "claude-opus-5-5", "--effort", "high"])
|
||||
self.assertEqual(Rung("worker-model").flags(), ["--model", "worker-model"])
|
||||
self.assertEqual(str(Rung("claude-opus-5-5", "high")), "claude-opus-5-5 at high effort")
|
||||
self.assertEqual(str(Rung("worker-model")), "worker-model")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -125,7 +125,7 @@ class ReadingCostTool(unittest.TestCase):
|
||||
transcript.write_bytes(jsonl([{"timestamp": str(at)}, *worker]))
|
||||
(transcript.with_suffix("") / "subagents" / "agent-a.jsonl").write_bytes(jsonl([said("1", "s1", tokens=1)]))
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(archive(self.archive, str(transcript), "Do it.", outcome("merged"), at))
|
||||
self.assertIsNone(archive(self.archive, str(transcript), "Do it.", outcome("m", "merged"), at))
|
||||
|
||||
def run_tool(self, cwd, *args):
|
||||
return subprocess.run([sys.executable, str(TOOL), *args], capture_output=True, cwd=cwd, env=self.env, text=True)
|
||||
|
||||
+9
-2
@@ -3,6 +3,8 @@ import re
|
||||
|
||||
from failure import Failure
|
||||
|
||||
# Each of R, S, C, A and G runs from 1 to TOP_SCORE.
|
||||
TOP_SCORE = 10
|
||||
EXEMPTIONS = ("", "decision", "defect", "principle", "question", "security")
|
||||
COLUMNS = ("ID", "Release", "Exempt", "Item", "R", "S", "C", "A", "G", "Goals", "Score")
|
||||
ITEMS_HEADER = "| " + " | ".join(COLUMNS) + " |"
|
||||
@@ -19,6 +21,11 @@ class Item:
|
||||
scores: dict[str, int]
|
||||
goals: tuple[str, ...]
|
||||
|
||||
@property
|
||||
def risk_and_complexity(self) -> int:
|
||||
"""R + C, which routes the item's chunk to a rung; 0 without a C."""
|
||||
return self.scores["R"] + self.scores["C"] if "C" in self.scores else 0
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Table:
|
||||
@@ -82,8 +89,8 @@ def parse_row(row: str, header: str = ITEMS_HEADER) -> Item | Failure:
|
||||
for name in ("R", "S", "C", "A", "G"):
|
||||
if name not in named:
|
||||
continue
|
||||
if not re.fullmatch(r"[0-9]+", named[name]) or not 1 <= int(named[name]) <= 10:
|
||||
return Failure(f"item {id_}'s {name} {named[name]!r} is not a whole number from 1 to 10")
|
||||
if not re.fullmatch(r"[0-9]+", named[name]) or not 1 <= int(named[name]) <= TOP_SCORE:
|
||||
return Failure(f"item {id_}'s {name} {named[name]!r} is not a whole number from 1 to {TOP_SCORE}")
|
||||
scores[name] = int(named[name])
|
||||
cited = tuple(goal.strip() for goal in named["Goals"].split(","))
|
||||
major, minor, patch = (int(part) for part in release.split("."))
|
||||
|
||||
@@ -1,9 +1,16 @@
|
||||
import hashlib
|
||||
import pathlib
|
||||
import typing
|
||||
|
||||
from claude_code.transcript import first_timestamped_line
|
||||
from project_key import project_key
|
||||
|
||||
|
||||
def project_archive(checkout_root: str, home: pathlib.Path, env: typing.Mapping[str, str]) -> pathlib.Path:
|
||||
archive = pathlib.Path(env["LOOP_CLAUDE_TRANSCRIPTS"]).expanduser() if env.get("LOOP_CLAUDE_TRANSCRIPTS") else home / ".claude" / "loop-claude-transcripts"
|
||||
return archive / project_key(checkout_root, home)
|
||||
|
||||
|
||||
def chunk_folder(project_archive: pathlib.Path, transcript: bytes) -> pathlib.Path:
|
||||
# Each chunk's first timestamped line is its own: a finished chunk's session is deleted, so the next chunk's transcript starts afresh.
|
||||
return project_archive / hashlib.sha256(first_timestamped_line(transcript)).hexdigest()
|
||||
|
||||
@@ -5,14 +5,18 @@ from utc_stamp import STAMPED
|
||||
|
||||
FINISHED = "Verdict: finished"
|
||||
FINISHED_AT = re.compile(rf"^({STAMPED}) {re.escape(FINISHED)}$", re.MULTILINE)
|
||||
# An outcome opens with its UTC stamp; a verdict that closes a cycle, a stop, or a step up ends with the worker line.
|
||||
OUTCOME_AT = re.compile(rf"^({STAMPED}) ", re.MULTILINE)
|
||||
NAMES_A_WORKER = ("Verdict: finished\n", "Verdict: unfinished\n", "Stopped: ", "Stepped up\n")
|
||||
WORKER = "Worker: "
|
||||
|
||||
|
||||
def finished(evidence: str) -> str:
|
||||
return f"{FINISHED}\n\nEvidence:\n{quoted(evidence)}"
|
||||
def finished(worker: str, evidence: str) -> str:
|
||||
return f"{FINISHED}\n\nEvidence:\n{quoted(evidence)}\n\n{WORKER}{worker}"
|
||||
|
||||
|
||||
def unfinished(evidence: str) -> str:
|
||||
return f"Verdict: unfinished\n\nEvidence:\n{quoted(evidence)}"
|
||||
def unfinished(worker: str, evidence: str) -> str:
|
||||
return f"Verdict: unfinished\n\nEvidence:\n{quoted(evidence)}\n\n{WORKER}{worker}"
|
||||
|
||||
|
||||
def interrupted(question: str, evidence: str) -> str:
|
||||
@@ -23,5 +27,22 @@ def answered(question: str, sources: str, answer: str, evidence: str) -> str:
|
||||
return f"Verdict: quoted answer\n\n{question}\n\n{sources} says:\n{quoted(answer)}\n\nEvidence:\n{quoted(evidence)}"
|
||||
|
||||
|
||||
def stopped(why: str) -> str:
|
||||
return f"Stopped: {why}"
|
||||
def stepped_up(worker: str) -> str:
|
||||
"""The rung the next cycle steps up to, archived as it steps, so a rerun after any exit resumes on it."""
|
||||
return f"Stepped up\n\n{WORKER}{worker}"
|
||||
|
||||
|
||||
def stopped(worker: str, why: str) -> str:
|
||||
return f"Stopped: {why}\n\n{WORKER}{worker}"
|
||||
|
||||
|
||||
def workers(outcomes: str) -> list[tuple[str, str]]:
|
||||
"""The UTC stamp and the worker of each outcome that names one, in the order they were written."""
|
||||
starts = list(OUTCOME_AT.finditer(outcomes))
|
||||
found = []
|
||||
for start, after in zip(starts, [*starts[1:], None]):
|
||||
outcome = outcomes[start.end():after.start() if after else len(outcomes)].rstrip("\n")
|
||||
last = outcome.rsplit("\n", 1)[-1]
|
||||
if outcome.startswith(NAMES_A_WORKER) and last.startswith(WORKER):
|
||||
found.append((start[1], last.removeprefix(WORKER)))
|
||||
return found
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import dataclasses
|
||||
import pathlib
|
||||
|
||||
from transcript_archive.location import chunk_folder
|
||||
from failure import Failure
|
||||
from transcript_archive.outcomes import FINISHED_AT
|
||||
from transcript_archive.outcomes import FINISHED_AT, workers
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
@@ -33,3 +34,22 @@ def read_threads(chunk: FinishedChunk) -> list[bytes] | Failure:
|
||||
return [path.read_bytes() for path in (chunk.transcript, *chunk.subagents)]
|
||||
except OSError as failure:
|
||||
return Failure(f"could not read {failure.filename}: {failure.strerror}")
|
||||
|
||||
|
||||
def latest_worker(project_archive: pathlib.Path, transcript: str) -> str | Failure | None:
|
||||
"""The worker named by the latest outcome archived for the transcript's chunk; None before any outcome names one.
|
||||
|
||||
Stamps run to the second, so the newest outcomes file decides, and in it the last outcome naming a worker.
|
||||
"""
|
||||
try:
|
||||
with open(transcript, "rb") as handle:
|
||||
folder = chunk_folder(project_archive, handle.read())
|
||||
if not folder.is_dir():
|
||||
return None
|
||||
files = sorted((path for path in folder.iterdir() if path.name.endswith(".outcomes.md")), key=lambda path: path.stat().st_mtime_ns, reverse=True)
|
||||
for path in files:
|
||||
if found := workers(path.read_text(encoding="utf-8")):
|
||||
return found[-1][1]
|
||||
except (OSError, UnicodeDecodeError) as failure:
|
||||
return Failure(f"could not read the archive {project_archive}: {failure}")
|
||||
return None
|
||||
|
||||
@@ -8,7 +8,8 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from transcript_archive import outcomes, write
|
||||
from failure import Failure
|
||||
from transcript_archive import outcomes, read, write
|
||||
from transcript_archive.location import project_archive
|
||||
from project_key import project_key
|
||||
from utc_stamp import utc
|
||||
@@ -23,11 +24,11 @@ class Archive(unittest.TestCase):
|
||||
transcript = pathlib.Path(root, "t.jsonl")
|
||||
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.unfinished("working"), NOW))
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.unfinished("m", "working"), NOW))
|
||||
(kept_outcomes,) = kept.glob("*/*.outcomes.md")
|
||||
kept_outcomes.unlink()
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.stopped("Ctrl-C"), NOW))
|
||||
self.assertEqual(kept_outcomes.read_text(), "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n")
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.stopped("m", "Ctrl-C"), NOW))
|
||||
self.assertEqual(kept_outcomes.read_text(), "Task: Do it.\n\n2026-09-21T14:13:20Z Stopped: Ctrl-C\n\nWorker: m\n")
|
||||
|
||||
|
||||
class CopyFor(unittest.TestCase):
|
||||
@@ -44,7 +45,44 @@ class CopyFor(unittest.TestCase):
|
||||
|
||||
class Outcomes(unittest.TestCase):
|
||||
def test_the_archive_stamps_what_it_reads_back(self):
|
||||
self.assertRegex(f"{utc(NOW)} {outcomes.finished('merged')}", outcomes.FINISHED_AT)
|
||||
self.assertRegex(f"{utc(NOW)} {outcomes.finished('m', 'merged')}", outcomes.FINISHED_AT)
|
||||
|
||||
def test_reads_back_the_worker_of_each_outcome_that_closes_a_cycle(self):
|
||||
kept = "Task: Do it.\n\n" + "\n".join(f"{utc(NOW + offset)} {outcome}\n" for offset, outcome in enumerate([
|
||||
outcomes.unfinished("w1", "Worker: not this"),
|
||||
outcomes.interrupted("Which one?\nWorker: nor this", "e"),
|
||||
outcomes.stopped("w2", "the judge failed\nthree times"),
|
||||
outcomes.finished("w3", "merged"),
|
||||
]))
|
||||
self.assertEqual(outcomes.workers(kept), [(utc(NOW), "w1"), (utc(NOW + 2), "w2"), (utc(NOW + 3), "w3")])
|
||||
|
||||
def test_reads_no_worker_from_an_outcome_written_before_the_worker_line(self):
|
||||
kept = f"Task: Do it.\n\n{utc(NOW)} Stopped: Ctrl-C\n\n{utc(NOW)} Verdict: unfinished\n\nEvidence:\n> Worker: no\n"
|
||||
self.assertEqual(outcomes.workers(kept), [])
|
||||
self.assertEqual(outcomes.workers(f"{utc(NOW)} {outcomes.stepped_up('w')}\n"), [(utc(NOW), "w")])
|
||||
|
||||
def test_takes_the_outcome_written_last_within_one_second(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
kept = pathlib.Path(root, "archive")
|
||||
transcript = pathlib.Path(root, "t.jsonl")
|
||||
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
for worker in ("z-first", "a-last"):
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.unfinished(worker, "working"), NOW))
|
||||
self.assertEqual(read.latest_worker(kept, str(transcript)), "a-last")
|
||||
|
||||
def test_names_the_worker_the_latest_outcome_of_the_transcripts_chunk_records(self):
|
||||
with tempfile.TemporaryDirectory() as root:
|
||||
kept = pathlib.Path(root, "archive")
|
||||
transcript = pathlib.Path(root, "t.jsonl")
|
||||
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n')
|
||||
self.assertIsNone(read.latest_worker(kept, str(transcript)))
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.unfinished("w1", "working"), NOW))
|
||||
transcript.write_text('{"timestamp": "2026-10-03T00:00:00Z"}\n{"more": 1}\n')
|
||||
self.assertIsNone(write.archive(kept, str(transcript), "Do it.", outcomes.stopped("w2", "Ctrl-C"), NOW + 1))
|
||||
self.assertEqual(read.latest_worker(kept, str(transcript)), "w2")
|
||||
self.assertIsInstance(read.latest_worker(kept, str(pathlib.Path(root, "gone.jsonl"))), Failure)
|
||||
|
||||
|
||||
class Location(unittest.TestCase):
|
||||
|
||||
@@ -3,8 +3,8 @@ import os
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
from transcript_archive.location import chunk_folder
|
||||
from failure import Failure
|
||||
from claude_code.transcript import first_timestamped_line
|
||||
from file_writes import make_private, replace_file
|
||||
from claude_code.session import session_files
|
||||
from utc_stamp import utc
|
||||
@@ -14,8 +14,7 @@ def archive(project_archive: pathlib.Path, path: str, task: str, outcome: str, a
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
raw = handle.read()
|
||||
# Each chunk's first timestamped line is its own: a finished chunk's session is deleted, so the next chunk's transcript starts afresh.
|
||||
folder = project_archive / hashlib.sha256(first_timestamped_line(raw)).hexdigest()
|
||||
folder = chunk_folder(project_archive, raw)
|
||||
make_private(folder)
|
||||
except OSError as failure:
|
||||
return unarchived(failure.filename, failure.strerror)
|
||||
|
||||
@@ -1613,3 +1613,28 @@ Inherited architect, would restructure first:
|
||||
- `bin/reading_cost/measure.py:16` `Reading`: "places" also names comprehension places, and "record" and "outcome" each carry three meanings
|
||||
- `bin/loop_claude/cycle/judge.py:16` `JUDGE_RULES`: says "the agent" where the code says "worker", and restates the working loop unchecked
|
||||
- `bin/loop_claude/loop/reload.py:38` `fingerprint`: covers every `bin/**/*.py`, so editing `statusline.py` reloads the loop
|
||||
|
||||
## 2026-10-14T21:34:44Z, PR #262 at f52228d, against e17168e
|
||||
|
||||
Ruling: worse
|
||||
|
||||
| Seat | Navigation | Locality | Shape | Self-sufficiency | Overall |
|
||||
|---|---|---|---|---|---|
|
||||
| Mid A | worse | worse | worse | same | worse |
|
||||
| Inherited architect | worse | worse | worse | same | worse |
|
||||
|
||||
Mid A, decided by:
|
||||
|
||||
- Locality, Overall: `bin/loop_claude/routing.py:10` `ROUTED`: a chunk's rung survives a rerun only as prose that `Rung.__str__` and `Route.__str__` write and a regex in another package parses back; rewording one string, or a new outcome kind, silently breaks resume three files away
|
||||
- Locality, Shape: `bin/loop_claude/loop/main.py:164` `routed`: `run_loop` now threads a `Route` through `LoopRun` with two lazy lambdas, and following one step-up means holding `stepped`, the judge's `bundled`, `after_verdict` and `archive_step` together
|
||||
- Locality: `bin/loop_claude/loop/reload.py:41` `decoded_as`: the flat key and type check became a recursive decoder, and the comment above `LoopRun` still says a field change stops a running loop
|
||||
- Navigation: `bin/model_routing/ladder.py:1` `LADDER`: routing is split across two modules named for routing plus five helpers in `main.py`, and README → Tools still says `bin/`'s shared modules serve more than one tool
|
||||
- Shape: `bin/loop_claude/cycle/judge.py:54` `Reply.bundled`: the name says neither R + C nor highest, and `kept` and `routed` make a reader open their bodies
|
||||
|
||||
Inherited architect, decided by:
|
||||
|
||||
- Locality, Navigation: `bin/loop_claude/routing.py:10` `ROUTED`: a string contract spread over four files, `Rung.__str__`, `Route.__str__`, `NAMES_A_WORKER` and the regex; a reword makes `route_from` return None and the chunk routes afresh with no error
|
||||
- Shape, Navigation: `bin/loop_claude/loop/main.py:129` `run_loop`: once a flat sequence, it now threads an anchor, a route, a before and after pair and two lambdas, and its imports grew from 13 modules to 21
|
||||
- Shape: `bin/loop_claude/loop/main.py:180` `archive_step`: `archive` is a function and a Path, and `project_archive` a function and a parameter, a few lines apart
|
||||
- Locality, Shape: `bin/loop_claude/loop/reload.py:41` `decoded_as`: a recursive decoder with a sentinel, and the comment above `LoopRun` half-contradicts the one inside it
|
||||
- Shape: `bin/loop_claude/routing.py:59` `stepped`: does two jobs, raising to a bundled item's rung and counting towards a one-rung step, and its name covers only the second
|
||||
|
||||
@@ -169,3 +169,15 @@ The reviewer limits each run to the changed lines, so its cost grows with the ch
|
||||
The run has no time cap. Limiting it to the changed lines already bounds it by the chunk's size, and no published standard gives a cap to copy. Each run's wall time in the reviewer's report shows when the premise below fails.
|
||||
|
||||
Valid while a mutation run on a chunk's changed lines stays cheap beside the review itself.
|
||||
|
||||
## routing: one ladder of (model, effort) rungs, ordered by the Artificial Analysis Intelligence Index with Fable 5.1 at max on top; a task's R + C picks a rung by fixed bands, counted from the cheapest rung of the project's model, and a chunk's worker never runs below that rung
|
||||
|
||||
2026-10-14, lilleman. Serves goal 1.4, and goal 1.3: one ladder, the same in every project, read by loop-claude now, and by larv-review and the worker once items 90 and 92 land. `bin/model_routing/ladder.py` holds the rungs and the bands.
|
||||
|
||||
The rungs, least competent first: Haiku 5.5 at low, medium, high, xhigh and max; Sonnet 5.5 at high; Opus 5.5 at medium, high, xhigh and max; Fable 5.1 at max. The order and the cut come from the Artificial Analysis Intelligence Index (https://artificialanalysis.ai/leaderboards/models, read 2026-10-14), which scores each model at each effort and gives a cost per task. A rung stays only where no cheaper rung scores as high, so a step up always buys more score (goal 1.4). That cut drops Sonnet 5.5 at low, medium, xhigh and max, Opus 5.5 at low, and every Fable 5.1 rung, and every tie in the index falls among the dropped rungs. lilleman put Fable 5.1 at max back on top: the index scores it under Opus 5.5 at high, but Anthropic's model catalog names Fable the most capable model, and for the hardest tasks lilleman wants the highest quality at any price.
|
||||
|
||||
A chunk's worker never runs below the project's rung, until `todo.md` item 93 shows a cheaper worker takes no more review rounds (goal 1.1). A project's rung is its model's cheapest rung, which for Opus 5.5 is its default effort, medium. lilleman chose medium knowing `settings.json` runs Opus 5.5 at xhigh, so a routed worker runs below the effort an interactive session takes. A model off the ladder routes nothing: claude runs it at the effort its settings give. R + C steps a task from the project's rung: 2–4 three rungs down, 5–6 two down, 7–8 one down, 9–10 none, 11–12 one up, 13–14 two up, 15–16 three up, 17 and above four up. A step past either end stops there. The bands spread the R + C that `todo.md`'s items held on 2026-10-14, 2 to 12, across the rungs, so most tasks would step down; a chunk's worker does not, until item 93 lands. S never moves a rung, so a huge task with low R and C runs cheap. The comprehension panel and the security pass stay on the project's model. `score-outcomes` had too few finished chunks to ground the bands; `todo.md` item 94 measures the routes' outcomes, and a change to the ladder or the bands is lilleman's.
|
||||
|
||||
The worker changes rung only as the chunk starts, after every second unfinished verdict, or when the judge reports a taken-in item whose R + C picks a higher rung, because a model switch inside a session discards its prompt cache. loop-claude routes on the first row of the next release in `todo.md`'s table that waits on no question. It reads neither an order the plan states in prose nor the task, so a chunk may start on another item's rung: a higher one costs money, and a lower one holds until the judge reports the item the worker took in, after the first cycle. `todo.md` item 136 ends this, routing on the item the worker says it takes.
|
||||
|
||||
Valid while the index ranks these models, and while no newer model joins the catalog.
|
||||
|
||||
@@ -248,3 +248,20 @@ Read most (opens and searches):
|
||||
- `agents/principles-auditor.md` 1
|
||||
|
||||
## Reading cost reset, 2026-10-13T15:08:08Z
|
||||
|
||||
## Reading cost, chunk d288bf3f6ee6, finished 2026-10-14T20:22:39Z
|
||||
|
||||
80214 new input tokens before the first commit.
|
||||
|
||||
Read most (opens and searches):
|
||||
|
||||
- `todo.md` 5
|
||||
- `docs/reading-cost.md` 4
|
||||
- `CHANGELOG.md` 3
|
||||
- `CLAUDE.md` 3
|
||||
- `docs/decisions.md` 3
|
||||
- `bin/rank_todo.py` 2
|
||||
- `skills/larv-review/SKILL.md` 2
|
||||
- `AGENTS.md` 1
|
||||
- `README.md` 1
|
||||
- `agents/stability-reviewer.md` 1
|
||||
|
||||
@@ -18,4 +18,4 @@ We follow a simplified variant:
|
||||
|
||||
All logging in our applications goes to OTLP or stdout.
|
||||
|
||||
In our server environments, nothing is saved to disk on the servers. Neither in Docker nor on the host. Everything is sent to Grafana Cloud via OTLP.
|
||||
In our server environments, every log, trace and metric ends up in the environment's remote observability platform, such as Grafana Cloud or a self-hosted Grafana. A file on the server's disk is fine on the way there, such as a log a shipper tails, as long as it is bounded in size. It is never the only place a record lives.
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
`Bar = 9`
|
||||
|
||||
`Next ID = 136`
|
||||
`Next ID = 138`
|
||||
|
||||
| Goal | W |
|
||||
|---|---|
|
||||
@@ -26,13 +26,12 @@ Release 0.2.0 takes items 125, 114 and 124 first, whatever their scores: Shape i
|
||||
|
||||
| ID | Release | Exempt | Item | R | S | C | A | G | Goals | Score |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 93 | 0.1.0 | | **Let a chunk's worker start below the project's model, once the archive shows chunks on a lower rung took no more review rounds than chunks on the project's model.** | 3 | 2 | 6 | 6 | 7 | 1.1, 1.4 | 22.4 |
|
||||
| 94 | 0.1.0 | | **Every 20th cycle, have loop-claude measure token spend and routing outcomes, file them here, and interrupt the human only when they call for a routing change.** | 3 | 5 | 6 | 6 | 7 | 1.1, 1.4 | 21.6 |
|
||||
| 88 | 0.1.0 | | **Define one routing ladder of (model, effort) rungs ordered by a published ranking of their competence, and map a task's R + C to a rung relative to the project's model.** | 3 | 4 | 5 | 6 | 7 | 1.4 | 21.4 |
|
||||
| 93 | 0.1.0 | decision | **Let a chunk's worker start below the project's model, once the archive shows chunks on a lower rung took no more review rounds than chunks on the project's model.** | 3 | 2 | 6 | 6 | 7 | 1.1, 1.4 | 22.4 |
|
||||
| 94 | 0.1.0 | decision | **Every 20th cycle, have loop-claude measure token spend and routing outcomes, file them here, and interrupt the human only when they call for a routing change.** | 3 | 5 | 6 | 6 | 7 | 1.1, 1.4 | 21.6 |
|
||||
| 90 | 0.1.0 | | **Route each reviewer by the chunk's R + C, once a replay of past chunks shows the routed reviewer finds the 🔴 and 🟡 findings on record.** | 4 | 5 | 8 | 6 | 7 | 1.4 | 20.2 |
|
||||
| 91 | 0.1.0 | | **Start each chunk's worker on the rung its item's R + C picks, never below the project's model, and step it one rung up after two unfinished verdicts on one chunk.** | 4 | 5 | 6 | 6 | 7 | 1.4 | 20.2 |
|
||||
| 92 | 0.1.0 | | **Have the worker route each subagent by its task's R + C, and delegate a task only when the worker needs just its conclusion.** | 2 | 2 | 5 | 5 | 6 | 1.4 | 19.0 |
|
||||
| 89 | 0.1.0 | | **Run the judge on the rung its chunk's R + C picks, once a replay of archived transcripts shows it returns the verdicts on record.** | 3 | 4 | 7 | 5 | 6 | 1.4 | 17.5 |
|
||||
| 136 | 0.1.0 | decision | **Route a chunk on the item its worker says it takes, in place of the next release's first row.** | 2 | 3 | 4 | 4 | 5 | 1.4 | 14.8 |
|
||||
| 105 | 0.2.0 | decision | **Compare review rounds per pass and reading cost for the five chunks before and after the `worker:` decision moved the principles out of the worker's context, and file an item if either rose.** | 1 | 2 | 3 | 5 | 6 | 1.4 | 20.0 |
|
||||
| 112 | 0.2.0 | | **Count an item as shipped only where its chunk built it, so an item a chunk dropped from the plan gets no outcome.** | 2 | 3 | 5 | 5 | 6 | 1.1 | 19.1 |
|
||||
| 110 | 0.2.0 | | **Count, for each finished chunk, the later changes that fixed code it wrote, and show them beside its guessed R.** | 2 | 4 | 6 | 5 | 6 | 1.1, 1.4 | 18.9 |
|
||||
@@ -55,6 +54,7 @@ Release 0.2.0 takes items 125, 114 and 124 first, whatever their scores: Shape i
|
||||
| 124 | 0.2.0 | decision | **Group the shared modules at `bin/`'s root into packages named by purpose, so the root holds only launchers and stand-alone tools.** | 3 | 6 | 3 | 3 | 5 | 1.1 | 11.4 |
|
||||
| 118 | 0.2.0 | | **Have `score-outcomes` count review rounds only from chunks merged after 2026-10-13, when review passes were narrowed, so C's medians compare chunks reviewed under one rule.** | 2 | 2 | 3 | 3 | 4 | 1.4 | 11.2 |
|
||||
| 130 | 0.2.0 | | **Move the vote counting out of `bin/larv_review/ruling.py`'s `votes`, so reading a votes file decides nothing.** | 2 | 3 | 3 | 3 | 4 | 1.1 | 11.2 |
|
||||
| 137 | 0.2.0 | | **Archive a chunk's route as a record loop-claude reads back field by field, in place of the `Worker:` prose `ROUTED` parses, and name `stepped` and the judge's `bundled` by what they do.** | 2 | 3 | 3 | 3 | 4 | 1.1 | 11.2 |
|
||||
| 129 | 0.2.0 | | **Explain `bin/rule_hook/rules.py`'s `OVERRIDE` pattern beside it: why the reason is lazy, what the lookahead splits and why `-->` is stripped.** | 1 | 1 | 1 | 2 | 4 | 1.1 | 10.7 |
|
||||
| 135 | 0.2.0 | | **Fingerprint only the modules loop-claude imports, so editing another tool's code no longer reloads a running loop.** | 3 | 3 | 4 | 3 | 4 | 1.1 | 10.2 |
|
||||
| 1 | 0.3.0 | question | **Say in README → Install how the transcript archive is backed up, once bender-infra backs up ai-sandbox's data volume.** | 3 | 5 | 2 | 8 | 9 | 3, 4 | 27.0 |
|
||||
@@ -91,41 +91,27 @@ Release 0.2.0 takes items 125, 114 and 124 first, whatever their scores: Shape i
|
||||
|
||||
Goal 1.1 asks for proven quality, so stepping down waits on the archive's evidence.
|
||||
|
||||
Needs 91.
|
||||
|
||||
### 94. Every 20th cycle, have loop-claude measure token spend and routing outcomes, file them here, and interrupt the human only when they call for a routing change.
|
||||
|
||||
Count the cycles across all projects in loop-claude's archive, which holds every project's transcripts on the machine. Measure in code, as `bin/reading-cost` does, so the check itself costs no model run (goal 1.4): tokens by project, subagent type, rung and tool, and per routed chunk its review rounds and unfinished verdicts. A loop on another project cannot commit here, so it writes the figures to the archive, and this repo's next chunk files or refreshes one plan item from them. The ladder's thresholds and the delegation rule are decisions, so a change to them is the human's. Interrupt when chunks on a rung took more review rounds or unfinished verdicts than chunks on the rung above, or when subagent reports fill more of the worker's peak context than the searches they replaced would have; never twice for the same figures (goal 1.2).
|
||||
|
||||
Needs 88, 91, 92.
|
||||
|
||||
### 88. Define one routing ladder of (model, effort) rungs ordered by a published ranking of their competence, and map a task's R + C to a rung relative to the project's model.
|
||||
|
||||
One ladder, the same in every project (goal 1.3), read by loop-claude, larv-review and the worker. A rung is a model and an effort; the rungs run from Haiku at low effort to the strongest model at max effort. Order the rungs by a published source, cited in the decision recording the ladder; where no source orders two rungs, such as Sonnet at xhigh against Opus at medium, ask the human. R + C moves a task up or down from the rung holding the project's model at its default effort: a low sum steps down, a high sum steps up, the thresholds set in the decision. S never moves a rung: a huge task with low R and C runs cheap. Leave out the comprehension panel and the security pass, which stay on the project model. Record each route in loop-claude's archive, so the outcomes can tune the thresholds.
|
||||
Needs 92.
|
||||
|
||||
### 90. Route each reviewer by the chunk's R + C, once a replay of past chunks shows the routed reviewer finds the 🔴 and 🟡 findings on record.
|
||||
|
||||
larv-review passes the rung through the Agent tool's `model` and `effort`. A reviewer never steps down when the worker has stepped down on the same chunk, so a cheap worker always meets a reviewer on the project model or above.
|
||||
|
||||
Needs 88.
|
||||
|
||||
### 91. Start each chunk's worker on the rung its item's R + C picks, never below the project's model, and step it one rung up after two unfinished verdicts on one chunk.
|
||||
|
||||
loop-claude reads the top item of the next release with `rank_todo.py` before starting a chunk's first cycle. Change the rung only when a chunk starts or steps up: every cycle resumes the chunk's session, and a model switch discards its prompt cache. When the worker bundles an item scoring a higher R + C than the one routed on, it says so in its transcript, and the judge's verdict steps the next cycle up.
|
||||
|
||||
Needs 88.
|
||||
|
||||
### 92. Have the worker route each subagent by its task's R + C, and delegate a task only when the worker needs just its conclusion.
|
||||
|
||||
State in `CLAUDE.md` when the worker delegates: a search or read that spans many files whose conclusion is all it needs, and work independent enough to run in parallel. Edits and anything the worker must verify line by line stay in the main thread, since a delegated diff is read again to check it. The worker scores R and C for each delegated task as it would for an item, and passes the rung's model and effort. In item 73's measurement, reports cost a median 8% of the worker's peak context, and `sed -n` and `grep` output made up 41% of its Bash results, so delegating searches saves the most context. The Agent tool sets a subagent's effort only when instructions ask, so the rule names both model and effort.
|
||||
|
||||
Needs 88.
|
||||
|
||||
### 89. Run the judge on the rung its chunk's R + C picks, once a replay of archived transcripts shows it returns the verdicts on record.
|
||||
|
||||
loop-claude starts the judge itself (`bin/loop_claude/cycle/judge.py`), so it routes without any skill change. Replay the judge on archived cycles at the candidate rung, and switch only when its interrupts and verdicts match those on record.
|
||||
|
||||
Needs 88.
|
||||
### 136. Route a chunk on the item its worker says it takes, in place of the next release's first row.
|
||||
|
||||
loop-claude routes on the first row of the next release, while the worker takes the item the plan's prose orders first, or a prerequisite, as the chunk that shipped 88 took it over 93. A first row of higher R + C pays for a dearer rung all chunk; one of lower R + C holds until the judge reports the item taken in. The judge reports the item the worker takes only after the first cycle, so routing the first cycle right needs the item before the worker starts.
|
||||
|
||||
### 105. Compare review rounds per pass and reading cost for the five chunks before and after the `worker:` decision moved the principles out of the worker's context, and file an item if either rose.
|
||||
|
||||
@@ -221,6 +207,10 @@ Since 2026-10-13 (`docs/decisions.md` → "review: stability and product-owner r
|
||||
|
||||
Mid A of PR #257's scoring run named `bin/larv_review/ruling.py:8` `votes`. It calls `counted` while validating, to reject the maintainability senior's and Junior A's lines when Mid A and the inherited architect already agree overall: a ruling hidden in a parser. The counting rule is spread over `seats.py`, `ruling.py`, `comprehension_history.place_dimensions` and the help. Make `votes` easier to read.
|
||||
|
||||
### 137. Archive a chunk's route as a record loop-claude reads back field by field, in place of the `Worker:` prose `ROUTED` parses, and name `stepped` and the judge's `bundled` by what they do.
|
||||
|
||||
PR #262's paired ruling found the project reads worse: a rerun resumes on its rung only by parsing `Rung.__str__`'s and `Route.__str__`'s wording back with `bin/loop_claude/routing.py`'s `ROUTED`, a string contract over four files, where a reword silently routes afresh. Write the route as a structured line, such as JSON after `Worker:`, that `transcript_archive` reads into fields. `stepped` both raises to a bundled item's rung and counts towards a one-rung step; `Reply.bundled` names neither R + C nor highest. Make `bin/loop_claude/loop/main.py`'s `run_loop` easier to read where it threads the route.
|
||||
|
||||
### 129. Explain `bin/rule_hook/rules.py`'s `OVERRIDE` pattern beside it: why the reason is lazy, what the lookahead splits and why `-->` is stripped.
|
||||
|
||||
The maintainability senior of PR #257's scoring run named `bin/rule_hook/rules.py:49` `OVERRIDE`: nothing in the code explains the pattern. Make `OVERRIDE` easier to read.
|
||||
|
||||
Reference in New Issue
Block a user