Answer stability round 1: 1 🔴, 0 🟡 (0 misread); weigh the epic's achieve goals the same in every repo's chain

This commit is contained in:
2026-11-18 16:51:37 +01:00
parent c63c3b3467
commit e6b9de55cc
4 changed files with 88 additions and 45 deletions
+50 -27
View File
@@ -1,8 +1,8 @@
"""The order do-epic works an epic's items in: by score, then each blocker above what it blocks (docs/decisions.md → `do-epic: drives one Jira epic …`).
An item scores by the epic's formula over one chain of goals: the epic's goals, then code health, then the goals of the item's repo. plan/goals_file.py's goal_weights weighs the chain as it weighs one project's goals, so a goal of the epic outweighs every goal of a repo. An item without a repo has only the epic's goals.
An item scores by the epic's formula over one chain of goals: the epic's goals, then code health, then the goals of the item's repo. An item without a repo has only the epic's goals. Each achieve goal weighs 0.97 of the one above it in the chain, and the epic's achieve goals weigh the same in every repo, so an achieve goal of the epic outweighs every achieve goal of a repo, whichever repos the two items are in. chain_scoring says what code health and a maintain goal weigh.
The epic and each repo number their goals from 1, so an item's Moves names a goal by its scope: `epic 1.1 by 8, code health by 5, repo 2 by 4`."""
An item's Moves names a goal by its scope, as plan/todo_table.py's EPIC_GOAL says."""
import ast
import dataclasses
import decimal
@@ -10,17 +10,18 @@ import decimal
from plan.blockers import blockers_first
from plan.todo_table import CODE_HEALTH, EPIC_GOAL, Item, REPO_GOAL, SHAPE
from failure import Failure
from plan.goals_file import goal_weights, Mark, numbered, ranked_goals
from plan.goals_file import goal_weights, GOAL_STEP, Mark, numbered, ranked_goals
from review_rules.comprehension_history import latest_overall
from plan.formula import parse_formula
from plan import ranking
from plan_store.jira_epic import SET_SCORES
MARK_IT = "is marked neither **Maintain:** nor **Achieve:**; mark it, as ~/.claude/rules/goals-and-audience.md says, so the epic's items can be scored"
@dataclasses.dataclass(frozen=True)
class RepoPremises:
"""What a repo's own plan is ranked by: its goals file, its decision log and comprehension history, each as text, and its readability floor."""
class RepoRanking:
"""What a repo's own plan is ranked by: its goals file, its decision log and its comprehension history, each as text, and its readability floor."""
goals_text: str
decisions: str
history: str
@@ -29,13 +30,13 @@ class RepoPremises:
@dataclasses.dataclass(frozen=True)
class EpicItem:
"""One of an epic's open items. An epic item has no release, so every item of one epic holds the same one. repo is the name of the repo it is worked in; None where it has none."""
"""One of an epic's open items. An Item names a release and an epic item has none: give every item of one epic the same release, so that no blocker sits in a later release than what it blocks. repo is the name of the repo the item is worked in; None where it has none."""
item: Item
repo: str | None
def ranked(items: list[EpicItem], formula: str, epic_goals: str, repos: dict[str, RepoPremises]) -> list[tuple[Item, float]] | Failure:
"""Each item with its score: a question first, then a security finding, then by score from highest, then by ID, and then each blocker above every item it blocks. formula is the epic's, epic_goals the text under its ## Goals, and repos the premises of each repo an item is in, by name."""
def ranked(items: list[EpicItem], formula: str, epic_goals: str, repos: dict[str, RepoRanking]) -> list[tuple[Item, float]] | Failure:
"""Each item with its score: a question first, then a security finding, then by score from highest, then by ID, and then each blocker above every item it blocks. formula is the epic's, epic_goals the text under its ## Goals, and repos holds, by name, each repo an item is in. An item without a score fails, so leave out an issue a person filed and nobody scored."""
parsed = parse_formula(formula, ranking.variables(SHAPE))
if isinstance(parsed, Failure):
return parsed
@@ -50,7 +51,7 @@ def ranked(items: list[EpicItem], formula: str, epic_goals: str, repos: dict[str
if isinstance(scoring, Failure):
return scoring
scorings[each.repo] = scoring
value = unnamed_goal(each, scorings[each.repo], name) or ranking.item_score(each.item, scorings[each.repo], name)
value = unscored(each.item, name) or misnamed_goal(each, scorings[each.repo], name) or ranking.item_score(each.item, scorings[each.repo], name)
if isinstance(value, Failure):
return value
scored.append((each.item, value))
@@ -58,48 +59,70 @@ def ranked(items: list[EpicItem], formula: str, epic_goals: str, repos: dict[str
return blockers_first(scored)
def chain_scoring(formula: ast.Expression, epic_goals: str, repo: str | None, premises: RepoPremises | None) -> ranking.Scoring | Failure:
"""How an item in repo scores: the chain's goals, each achieve goal at its weight. The bar is 0, since the bar decides what is filed, not the order of what was."""
goals = chain(epic_goals, repo, premises)
def chain_scoring(formula: ast.Expression, epic_goals: str, repo: str | None, own: RepoRanking | None) -> ranking.Scoring | Failure:
"""How an item in repo scores: the chain's goals, and what each achieve goal weighs. The bar is 0, since a bar decides what is filed, and the order holds every item that was.
goal_weights scales a chain's weights to sum to 1. Here they are scaled so that the epic's achieve goals sum to 1 instead, in every chain, which makes the scores of items in different repos comparable.
Code health and a maintain goal weigh by the rules every project's plan follows, which an epic doesn't change: code health weighs four times its place while its repo is unmeasured or under its readability floor, and plan/ranking.py's weight_of weighs a maintain goal. Either may then outweigh an achieve goal of the epic. "The epic wins" in the decision settles the epic's goals against a repo's, and code health is neither's: it is every project's, and a repo that is hard to read costs each epic item worked in it."""
goals = chain(epic_goals, repo, own)
if isinstance(goals, Failure):
return goals
if unmarked := next((goal for goal, mark in goals if mark is None), None):
whose, _, number = unmarked.partition(" ")
return Failure(f"{'the epic' if whose == 'epic' else repo}'s goal {number} {MARK_IT}")
overall = latest_overall(premises.history) if premises and CODE_HEALTH in dict(goals) else None
overall = latest_overall(own.history) if own and CODE_HEALTH in dict(goals) else None
if isinstance(overall, Failure):
return Failure(f"{repo}: {overall.reason}")
# goal_weights fails only on a goal without a mark, which the check above already named by whose it is.
weights = goal_weights(goals, overall, premises.floor if premises else decimal.Decimal(0))
return weights if isinstance(weights, Failure) else ranking.Scoring(SHAPE, formula, 0, 0, dict(goals), weights)
# goal_weights fails only on a goal without a mark, and the check above has named that goal as the epic's or the repo's.
weights = goal_weights(goals, overall, own.floor if own else decimal.Decimal(0))
if isinstance(weights, Failure):
return weights
achieve = [goal for goal, mark in goals if mark == "Achieve"]
of_epic = len([goal for goal in achieve if goal.startswith(EPIC_GOAL)])
# An epic without an achieve goal leaves each chain summing to 1.
scale = total(len(achieve)) / total(of_epic) if of_epic else 1
return ranking.Scoring(SHAPE, formula, 0, 0, dict(goals), {goal: weight * scale for goal, weight in weights.items()})
def chain(epic_goals: str, repo: str | None, premises: RepoPremises | None) -> list[tuple[str, Mark]] | Failure:
"""Every goal an item may name, in rank order, each with its mark: the epic's goals as `epic <number>`, then, for an item in repo, which has these premises, code health unless the repo's goals file voids it, and the repo's goals as `repo <number>`."""
def total(goals: int) -> float:
"""What this many achieve goals weigh together before goal_weights scales them, the first 1 and each 0.97 of the one above it."""
return sum(GOAL_STEP**place for place in range(goals))
def chain(epic_goals: str, repo: str | None, own: RepoRanking | None) -> list[tuple[str, Mark]] | Failure:
"""Every goal an item in repo may name, in rank order, each with its mark. The epic's goals come first, as `epic <number>`. For an item with a repo, whose ranking is own, code health follows unless the repo's goals file voids it, then the repo's goals as `repo <number>`."""
epic = numbered(epic_goals)
if isinstance(epic, Failure):
return Failure(epic.reason.replace("the goals file's", "the epic's"))
scoped = [(f"{EPIC_GOAL}{number}", mark) for number, mark in epic]
if premises is None:
if own is None:
return scoped
own = ranked_goals(premises.goals_text, premises.decisions)
if isinstance(own, Failure):
return Failure(f"{repo}: {own.reason}")
return [*scoped, *((goal if goal == CODE_HEALTH else f"{REPO_GOAL}{goal}", mark) for goal, mark in own)]
goals = ranked_goals(own.goals_text, own.decisions)
if isinstance(goals, Failure):
return Failure(f"{repo}: {goals.reason}")
return [*scoped, *((goal if goal == CODE_HEALTH else f"{REPO_GOAL}{goal}", mark) for goal, mark in goals)]
def unnamed_goal(each: EpicItem, scoring: ranking.Scoring, name: str) -> Failure | None:
def unscored(item: Item, name: str) -> Failure | None:
"""Why the item cannot be scored, such as an issue a person filed and nobody scored; None where it holds each score."""
missing = next((score for score in SET_SCORES if score not in item.scores), None)
return Failure(f"{name} has no {missing}: score it on {', '.join(SET_SCORES[:-1])} and {SET_SCORES[-1]} before it is ranked") if missing else None
def misnamed_goal(each: EpicItem, scoring: ranking.Scoring, name: str) -> Failure | None:
"""Why the item may not name one of the goals it names, in the epic's terms; None where its chain holds each."""
for goal, _ in each.item.moves:
if goal in scoring.goals:
continue
if not goal.startswith((EPIC_GOAL, REPO_GOAL)) and goal != CODE_HEALTH:
return Failure(f"{name} names goal {goal} without saying whose it is; write `{EPIC_GOAL}{goal}` for the epic's goal or `{REPO_GOAL}{goal}` for its repo's")
if goal.startswith(EPIC_GOAL):
return Failure(f"{name} names {goal}, which the epic's ## Goals does not number")
if each.repo is None:
return Failure(f"{name} names {goal}, but has no repo whose {'code' if goal == CODE_HEALTH else 'goal'} that could be; name only the epic's goals, as in `{EPIC_GOAL}1 by 5`")
scoped = f"; write `{EPIC_GOAL}{goal}` for the epic's goal" if goal[0].isdigit() else f"; name only the epic's goals, as in `{EPIC_GOAL}1 by 5`"
return Failure(f"{name} names {goal}, but isn't worked in a repo{scoped}")
if goal == CODE_HEALTH:
return Failure(f"{name} names {CODE_HEALTH}, which {each.repo}'s goals file voids")
if not goal.startswith(REPO_GOAL):
return Failure(f"{name} names goal {goal} without saying whose it is; write `{EPIC_GOAL}{goal}` for the epic's goal or `{REPO_GOAL}{goal}` for its repo's")
return Failure(f"{name} names {goal}, which the ## Goals of {each.repo}'s goals file does not number")
return None
+36 -16
View File
@@ -6,17 +6,17 @@ import unittest
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from do_epic.ranking import chain, EpicItem, ranked, RepoPremises
from do_epic.ranking import chain, EpicItem, ranked, RepoRanking
from failure import Failure
from plan.todo_table import Item, parse_moves
EPIC_GOALS = "1. **Achieve:** Ship invoices.\n2. **Achieve:** Cut support mail.\n3. **Maintain:** Leak no customer data.\n"
VOIDED = RepoPremises("## Goals\n\n**Void:** code health, since it is a one-off script.\n\n1. **Achieve:** Fast builds.\n", "", "", decimal.Decimal("7.0"))
HEALTHY = RepoPremises("## Goals\n\n1. **Achieve:** Fast builds.\n2. **Maintain:** Stay on the LTS.\n", "", "", decimal.Decimal("7.0"))
VOIDED = RepoRanking("## Goals\n\n**Void:** code health, since it is a one-off script.\n\n1. **Achieve:** Fast builds.\n", "", "", decimal.Decimal("7.0"))
HEALTHY = RepoRanking("## Goals\n\n1. **Achieve:** Fast builds.\n2. **Maintain:** Stay on the LTS.\n", "", "", decimal.Decimal("7.0"))
# A scoring run of a repo whose overall mean is {overall}.
SCORED = "# Comprehension history\n\n## 2026-10-08T12:00:00Z, PR #1 at aaaaaaa\n\n| Seat | Overall |\n|---|---|\n| Mid A | 6 |\n| Mean | {overall} |\n"
# What the three achieve goals of the chain epic 1, epic 2, repo 1 weigh together before they are scaled to sum to 1, each 0.97 of the one above.
TOTAL = 1 + 0.97 + 0.97**2
# What the epic's two achieve goals weigh together before they are scaled to sum to 1, the second 0.97 of the first.
TOTAL = 1 + 0.97
def item(id_: int, moves: str, repo: str | None = "billing-api", exempt: str = "", blocked_by: tuple[int, ...] = ()) -> EpicItem:
@@ -25,7 +25,7 @@ def item(id_: int, moves: str, repo: str | None = "billing-api", exempt: str = "
return EpicItem(Item(id_, (0, 0, 0), exempt, f"**Item {id_}.**", {"L": 1, "E": 1, "I": 1, "S": 1}, parsed, blocked_by), repo) # type: ignore[arg-type]
def order(items: list[EpicItem], repos: dict[str, RepoPremises], formula: str = "V") -> list[int] | Failure:
def order(items: list[EpicItem], repos: dict[str, RepoRanking], formula: str = "V") -> list[int] | Failure:
found = ranked(items, formula, EPIC_GOALS, repos)
return found if isinstance(found, Failure) else [scored.id for scored, _ in found]
@@ -72,15 +72,36 @@ class RankedTest(unittest.TestCase):
def test_an_item_without_a_repo_scores_by_the_epics_goals_alone(self) -> None:
found = ranked([item(1, "epic 1 by 10", repo=None)], "V", EPIC_GOALS, {})
assert not isinstance(found, Failure), found
self.assertAlmostEqual(found[0][1], 10 / 1.97)
self.assertAlmostEqual(found[0][1], 10 / TOTAL)
def test_an_item_without_a_repo_may_not_name_a_repos_goal_or_code_health(self) -> None:
self.assertEqual(order([item(4, "repo 1 by 5", repo=None)], {}), Failure("item 4 names repo 1, but has no repo whose goal that could be; name only the epic's goals, as in `epic 1 by 5`"))
self.assertEqual(order([item(4, "code health by 5", repo=None)], {}), Failure("item 4 names code health, but has no repo whose code that could be; name only the epic's goals, as in `epic 1 by 5`"))
self.assertEqual(order([item(4, "repo 1 by 5", repo=None)], {}), Failure("item 4 names repo 1, but isn't worked in a repo; name only the epic's goals, as in `epic 1 by 5`"))
self.assertEqual(order([item(4, "code health by 5", repo=None)], {}), Failure("item 4 names code health, but isn't worked in a repo; name only the epic's goals, as in `epic 1 by 5`"))
self.assertEqual(order([item(4, "2 by 5", repo=None)], {}), Failure("item 4 names 2, but isn't worked in a repo; write `epic 2` for the epic's goal"))
def test_a_goal_without_its_scope_is_refused(self) -> None:
self.assertEqual(order([item(4, "1 by 5")], {"billing-api": VOIDED}), Failure("item 4 names goal 1 without saying whose it is; write `epic 1` for the epic's goal or `repo 1` for its repo's"))
def test_an_epics_goal_outweighs_a_repos_goal_across_repos_with_chains_of_different_lengths(self) -> None:
long = RepoRanking("## Goals\n\n" + "".join(f"{place}. **Achieve:** Goal {place}.\n" for place in range(1, 10)), "", "", decimal.Decimal("7.0"))
found = ranked([item(1, "repo 1 by 10"), item(2, "epic 2 by 10", repo="web"), item(3, "epic 2 by 10", repo=None)], "V", EPIC_GOALS, {"billing-api": VOIDED, "web": long})
assert not isinstance(found, Failure), found
self.assertEqual([scored.id for scored, _ in found], [2, 3, 1])
for (_, value), expected in zip(found, (9.7 / TOTAL, 9.7 / TOTAL, 10 * 0.97**2 / TOTAL)):
self.assertAlmostEqual(value, expected)
def test_an_epic_without_an_achieve_goal_leaves_each_repos_weights_summing_to_1(self) -> None:
found = ranked([item(1, "repo 1 by 10")], "V", "1. **Maintain:** Leak no customer data.\n", {"billing-api": VOIDED})
assert not isinstance(found, Failure), found
self.assertAlmostEqual(found[0][1], 10)
def test_an_item_without_a_score_fails_naming_it(self) -> None:
unscored = EpicItem(dataclasses.replace(item(4, "").item, scores={}), None)
self.assertEqual(ranked([unscored], "V", EPIC_GOALS, {}), Failure("item 4 has no L: score it on L, E, I and S before it is ranked"))
def test_an_epic_that_numbers_a_goal_twice_fails(self) -> None:
self.assertEqual(chain("1. **Achieve:** A.\n1. **Achieve:** B.\n", None, None), Failure("the epic's ## Goals numbers 1, 1; number each goal by its position"))
def test_a_goal_the_chain_does_not_hold_is_refused(self) -> None:
self.assertEqual(order([item(4, "epic 9 by 5")], {"billing-api": VOIDED}), Failure("item 4 names epic 9, which the epic's ## Goals does not number"))
self.assertEqual(order([item(4, "repo 9 by 5")], {"billing-api": VOIDED}), Failure("item 4 names repo 9, which the ## Goals of billing-api's goals file does not number"))
@@ -90,27 +111,26 @@ class RankedTest(unittest.TestCase):
self.assertEqual(order([item(4, "epic 1 by 5", repo="ghost")], {}), Failure("item 4 is in the repo ghost, whose goals were not read"))
def test_a_goal_marked_neither_way_fails_naming_whose_it_is(self) -> None:
unmarked = RepoPremises("## Goals\n\n1. Fast builds.\n", "", "", decimal.Decimal("7.0"))
unmarked = RepoRanking("## Goals\n\n1. Fast builds.\n", "", "", decimal.Decimal("7.0"))
self.assertEqual(order([item(4, "epic 1 by 5")], {"billing-api": unmarked}), Failure("billing-api's goal 1 is marked neither **Maintain:** nor **Achieve:**; mark it, as ~/.claude/rules/goals-and-audience.md says, so the epic's items can be scored"))
found = ranked([item(4, "epic 1 by 5", repo=None)], "V", "1. Ship invoices.\n", {})
self.assertEqual(found, Failure("the epic's goal 1 is marked neither **Maintain:** nor **Achieve:**; mark it, as ~/.claude/rules/goals-and-audience.md says, so the epic's items can be scored"))
def test_a_repos_goals_file_that_cannot_be_read_fails_naming_the_repo(self) -> None:
self.assertEqual(order([item(4, "epic 1 by 5")], {"billing-api": RepoPremises("# Billing\n", "", "", decimal.Decimal("7.0"))}), Failure("billing-api: the goals file has no ## Goals heading"))
self.assertEqual(order([item(4, "epic 1 by 5")], {"billing-api": RepoRanking("# Billing\n", "", "", decimal.Decimal("7.0"))}), Failure("billing-api: the goals file has no ## Goals heading"))
def test_code_health_weighs_four_times_its_place_while_the_repo_is_unmeasured_or_under_its_floor(self) -> None:
total = 1 + 0.97 + 0.97**2 + 0.97**3
for history, times in (("", 4), (SCORED.format(overall="6.9"), 4), (SCORED.format(overall="7.0"), 1)):
with self.subTest(history=history):
found = ranked([item(1, "code health by 10", repo="web"), item(2, "repo 1 by 10", repo="web")], "V", EPIC_GOALS, {"web": dataclasses.replace(HEALTHY, history=history)})
assert not isinstance(found, Failure), found
self.assertAlmostEqual(found[0][1], times * 10 * 0.97**2 / total)
self.assertAlmostEqual(found[1][1], 10 * 0.97**3 / total)
self.assertAlmostEqual(found[0][1], times * 10 * 0.97**2 / TOTAL)
self.assertAlmostEqual(found[1][1], 10 * 0.97**3 / TOTAL)
def test_a_repo_under_its_own_floor_weighs_code_health_by_that_floor(self) -> None:
def test_code_health_weighs_four_times_its_place_under_a_floor_the_repo_sets_higher(self) -> None:
found = ranked([item(1, "code health by 10", repo="web")], "V", EPIC_GOALS, {"web": dataclasses.replace(HEALTHY, history=SCORED.format(overall="7.5"), floor=decimal.Decimal("8.0"))})
assert not isinstance(found, Failure), found
self.assertAlmostEqual(found[0][1], 4 * 10 * 0.97**2 / (1 + 0.97 + 0.97**2 + 0.97**3))
self.assertAlmostEqual(found[0][1], 4 * 10 * 0.97**2 / TOTAL)
def test_a_history_that_cannot_be_read_fails_naming_the_repo(self) -> None:
found = order([item(1, "code health by 10", repo="web")], {"web": dataclasses.replace(HEALTHY, history=SCORED.format(overall="high"))})
+1 -1
View File
@@ -22,7 +22,7 @@ BLOCKED_BY = "Blocked by"
MOVES = "Moves"
# The achieve goal every project shares, which ~/.claude/CLAUDE.md → "Shared goal" states, and a Moves cell names by these words, since it has no number in any goals file.
CODE_HEALTH = "code health"
# What opens a goal's name in the Moves of an epic's item, whose epic and repo both number their goals from 1: `epic 1.1 by 8, code health by 5, repo 2 by 4`. Three readers, seated as the human who reads and writes these cells, picked a scope word on both over one on either alone and over the repo's name, 3-0. do_epic/ranking.py holds the goals such an item may name; a todo.md's item may name neither.
# The scope that opens a goal's name in the Moves of an epic's item, since the epic and the item's repo both number their goals from 1: `epic 1.1 by 8, code health by 5, repo 2 by 4`. Three readers, seated as the human who writes these cells, picked a scope word on both goals, 3-0, over one on either alone and over the repo's name. A todo.md's item may name neither.
EPIC_GOAL = "epic "
REPO_GOAL = "repo "
# How the items table's header opens, which tells it from any other table in todo.md.
+1 -1
View File
@@ -97,7 +97,7 @@ Touches:
### 408. Score an epic's items by one weight chain, the epic's achieve goals, then code health, then the item's repo goals, and order them by score, then by Blocked by.
Where the epic and a repo disagree, the epic wins (`docs/decisions.md` → `do-epic: drives one Jira epic …`). An item without a repo weighs only the epic's goals. Build the chain as one list for `goal_weights`, score with `ranking.item_score` and order with `blockers_first`. `plan/ranking.py` `score` is a place the seats found hard, so the combiner must not add a branch to it.
Where the epic and a repo disagree, the epic wins (`docs/decisions.md` → `do-epic: drives one Jira epic …`). An item without a repo weighs only the epic's goals. Build the chain as one list for `goal_weights`, score with `ranking.item_score` and order with `blockers_first`. `plan/ranking.py` `score` is a place the seats found hard, so the scorer must not add a branch to it.
The epic's and a repo's goal numbers collide, so a Moves cell needs a spelling for a repo's goal beside the epic's; a panel of readers picks it before it ships (`~/.claude/CLAUDE.md` → "Working loop"). The scorer takes the items and each repo's goals as values; item 424 reads them from the tracker, and item 425 re-ranks Jira by the order.