Forbid a test that decides at run time whether to check anything
CI / tests (pull_request) Successful in 3m53s

This commit is contained in:
2026-11-18 16:13:37 +01:00
parent e898fc0f03
commit eb49d62e36
3 changed files with 36 additions and 0 deletions
+1
View File
@@ -125,6 +125,7 @@ Every PR gets at least this sweep.
- Was a test deleted while the behaviour it pinned remains? Flag it; restore the test.
- Were tests and implementation changed together in a way that could mask a regression — e.g. an assertion weakened to make a test pass? Flag it; these should be separate changes.
- Does each test earn its keep, as the technical principles' test bullet says? Look at where its expected values come from, what it asserts, what it mocks, what wording it pins, whether another test already pins the behaviour, and whether it sleeps or reads the wall clock.
- Does a test decide at run time whether to check anything, by a conditional skip, early return or catch around its assertions, as the technical principles' test bullet forbids? Flag it.
- Are there untestable side effects that should be refactored?
+34
View File
@@ -2691,3 +2691,37 @@ Read most (opens and searches):
- `bin/transcript_archive/outcomes.py` 2
- `bin/judge_replay/replay.py` 1
- `bin/rung_replay/archive.py` 1
## Reading cost, chunk f6452ae1f4d1, finished 2026-11-18T15:04:37Z
164779 new input tokens before the first commit.
Read most (opens and searches):
- `README.md` 5
- `CHANGELOG.md` 2
- `bin/claude_code/reply.py` 2
- `bin/do_epic/backend/contract.py` 2
- `bin/do_epic/backend/jira.py` 2
- `bin/do_epic/backend/test_jira.py` 2
- `docs/decisions.md` 2
- `bin/atlassian/client.py` 1
- `bin/do_epic/backend/fake.py` 1
- `bin/do_epic/backend/protocol.py` 1
## Reading cost, chunk a1d1adafc713, finished 2026-11-18T15:13:10Z
257245 new input tokens before the first commit.
Read most (opens and searches):
- `docs/decisions.md` 11
- `bin/review_replay/test_review_replay.py` 9
- `bin/review_replay/replay.py` 7
- `README.md` 6
- `bin/review_replay/change_comparison.py` 5
- `bin/rule_hook/test_rule_hook.py` 4
- `bin/loop_claude/cycle/run.py` 2
- `bin/loop_claude/test_loop_claude.py` 2
- `bin/review_replay/main.py` 2
- `bin/rule_hook/main.py` 2
+1
View File
@@ -46,6 +46,7 @@
- Assert the outcome — returned values, stored state, emitted output — not how the code reached it. Use real collaborators or fakes; mock only slow, nondeterministic or external edges. BANNED: asserting which internal function ran, or scanning the source with `ast` or a regex. GOOD: `assertEqual(saved_row, {...})` against a fake store; a fake clock.
- Pin wording only where the wording is the contract: a message a consumer parses, a published format. For prompts, help and error text, assert the facts the text must carry — an item ID, a model argument, an error code — not the sentence. BANNED: `assertIn("<a 600-character prompt sentence>", prompt)`. GOOD: `assertIn("model opus, effort high", prompt)`.
- One behaviour per test, named for it, tested once, at the lowest level that reproduces it. No loops or branches in a test body; parametrize the cases.
- A test doesn't decide at run time whether to check anything: no conditional skip, early return, or catch around its assertions. A test that needs a state creates it, or fails and names what is missing. BANNED: `test.skip(created.every((count) => count === 0))`, since a broken payroll also creates zero rows. GOOD: check the precondition before the action, failing with its cause, then `expect(created).toEqual([1, 2, 1])`.
- Every escaped bug becomes a test that fails without its fix.
- Write no test that no real bug could fail: a constant's value, a list's length, the framework's own behaviour.
- A test neither sleeps nor asserts on the wall clock, and seeds any randomness.