Commit Graph

4027 Commits

Author SHA1 Message Date
lilleman a40eafa336 Answer prose round 2: 0 🔴, 0 🟡 (0 misread); label the pooled change and the mean once, and say what the filed items are for
CI / tests (pull_request) Has been cancelled
2026-11-22 21:35:58 +01:00
lilleman e25b4fc401 Merge remote-tracking branch 'origin/main' into item-516
CI / tests (pull_request) Has been cancelled
# Conflicts:
#	docs/decisions.md
2026-11-22 21:34:45 +01:00
lilleman 98754352e5 Answer prose round 1: 0 🔴, 3 🟡 (0 misread); make the turns' and reads' figures one reading each 2026-11-22 21:34:09 +01:00
lilleman e47e71167b Record the spread of the reads and the turns beside the tokens' in the task-bench decision
CI / tests (pull_request) Has been cancelled
2026-11-22 21:32:48 +01:00
lilleman ab08c3c238 Finish item 514 in the plan
CI / tests (pull_request) Successful in 5m24s
2026-11-22 17:49:53 +01:00
lilleman a95dd360c8 Merge remote-tracking branch 'origin/main' into item-514
CI / tests (pull_request) Successful in 8m13s
2026-11-22 17:32:09 +01:00
lilleman b65710dca7 Answer prose round 1: 0 🔴, 9 🟡 (0 misread); reword the docstrings, help and decision, and item 516's detail
CI / tests (pull_request) Has been cancelled
2026-11-22 17:32:04 +01:00
lilleman 4612f71592 Answer product-owner round 1: 0 🔴, 0 🟡 (0 misread); the help calls the noise figures an All change, and the changelog says the row's meaning changed 2026-11-22 17:32:04 +01:00
lilleman f5a691ff07 Answer stability round 1: 0 🔴, 2 🟡 (0 misread); count the passes from the bugs' runs and drop change's duplicate parameters 2026-11-22 17:32:04 +01:00
lilleman f9f96f69ae Build each column's mean change without a no-bugs branch
CI / tests (pull_request) Successful in 5m25s
2026-11-22 17:10:03 +01:00
lilleman 0b6cc71520 Finish item 513 in the plan
CI / tests (pull_request) Successful in 7m50s
2026-11-22 17:09:32 +01:00
lilleman 72c481696d Answer stability round 1: 0 🔴, 0 🟡 (0 misread); approved without findings
CI / tests (pull_request) Has been cancelled
2026-11-22 17:09:21 +01:00
lilleman 5b792514a4 Answer security round 1: 0 🔴, 0 🟡 (0 misread); nothing found 2026-11-22 17:09:21 +01:00
lilleman e38b6c344b Print the mean of the bugs' own changes as task-bench's All change
CI / tests (pull_request) Successful in 4m28s
2026-11-22 16:52:54 +01:00
lilleman 993295b486 Stop the bench with a Failure on a run that ended on an api_error without a status
CI / tests (pull_request) Successful in 5m21s
2026-11-22 16:50:51 +01:00
lilleman e687860824 Name decision titles in the plan's citations without the heading marker
CI / tests (pull_request) Successful in 4m53s
2026-11-22 16:40:22 +01:00
lilleman 82511b3d0f Finish item 503 in the plan
CI / tests (pull_request) Failing after 6m49s
2026-11-22 16:32:13 +01:00
lilleman f45e06fe9d Merge remote-tracking branch 'origin/main' into item-503
CI / tests (pull_request) Has been cancelled
# Conflicts:
#	todo.md
2026-11-22 16:31:57 +01:00
lilleman e8c123ced0 Record the comprehension check: scoring run at 16d9b3a0, overall 6.13, Locality lowest at 5.63; refresh floor item 300 2026-11-22 16:30:21 +01:00
lilleman 16d9b3a059 Answer prose round 1: 0 🔴, 10 🟡 (0 misread); rewrite the decision conclusions first with corrected figures, and clarify the help's noise paragraph 2026-11-22 16:24:07 +01:00
lilleman 5c3ac81ed7 Calibrate task-bench on fejkdata and record how many tasks and runs a bench takes
CI / tests (pull_request) Failing after 5m1s
2026-11-22 16:16:00 +01:00
lilleman 378dafeef3 Record the reading cost of the chunk before this one 2026-11-22 16:16:00 +01:00
lilleman 0c2f2a5fd1 Cut release 0.15.0
CI / tests (pull_request) Successful in 4m49s
v0.15.0
2026-11-22 15:52:21 +01:00
lilleman ec7a161339 Move item 503 to 0.15.1, since 0.15.0 is cut before its release review
CI / tests (pull_request) Successful in 4m16s
2026-11-22 15:50:23 +01:00
lilleman 2a190c7700 File item 512: make stability-reviewer check that composed lookup keys cannot collide
CI / tests (pull_request) Successful in 5m40s
2026-11-21 13:28:23 +01:00
lilleman ed07d6a002 Limit goal 3's backup of a bench's transcripts to the time before its gather
CI / tests (pull_request) Successful in 4m50s
2026-11-21 13:01:59 +01:00
lilleman 78ed3819aa Finish item 502 in the plan
CI / tests (pull_request) Successful in 4m50s
2026-11-21 12:41:35 +01:00
lilleman e2fc5dbfb2 Answer stability round 2: 0 🔴, 0 🟡 (0 misread); test that a gather before any review makes the review folders private; the digest's errors="replace" stays untested, as it takes a lone surrogate in a stream
CI / tests (pull_request) Successful in 5m6s
2026-11-21 12:22:46 +01:00
lilleman de5802e603 Answer prose round 1: 0 🔴, 8 🟡 (0 misread); reword the help, prompt, docstrings and README entries in the commit above
CI / tests (pull_request) Has been cancelled
2026-11-21 12:18:41 +01:00
lilleman d265e242f8 Answer product-owner round 1: 0 🔴, 3 🟡 (0 misread); in the commit above, name the commits and trees in the digest, keep the folder when claude ends in an error, and say the next bench clears it 2026-11-21 12:18:40 +01:00
lilleman 101c8a7c3d Answer stability round 1: 0 🔴, 5 🟡 (1 misread); keep locks out of compress-archives, count only a search's hits, show an unfinished failing run, and test the walk and the home 2026-11-21 12:18:40 +01:00
lilleman 27e5014322 Test the median pick by tokens, the name of a silent claude and the environment git runs in
CI / tests (pull_request) Successful in 4m13s
2026-11-21 11:57:43 +01:00
lilleman b474ec2f5a Test the digest's cuts, the median pick and gather's refusals, and drop project_rung's checkout parameter 2026-11-21 04:24:52 +01:00
lilleman 7a568f80af Answer security round 1: 0 🔴, 0 🟡 (0 misread); nothing to fix 2026-11-21 03:57:17 +01:00
lilleman ad11c72016 Distil each task-bench run into a digest and gather plan items from it
CI / tests (pull_request) Failing after 4m49s
2026-11-21 03:56:18 +01:00
lilleman d813fdad5a Finish item 501 in the plan
CI / tests (pull_request) Successful in 6m11s
2026-11-21 03:40:18 +01:00
lilleman 68152c5dec Merge remote-tracking branch 'origin/main' into item-501
CI / tests (pull_request) Has been cancelled
2026-11-21 03:37:59 +01:00
lilleman 1500c52d69 Answer stability round 2: 0 🔴, 0 🟡 (0 misread); take a failed subtest's message over what its parent logged 2026-11-21 03:37:59 +01:00
lilleman 0c17559efe Test a line printed before any test's mark
CI / tests (pull_request) Successful in 3m51s
2026-11-21 03:09:34 +01:00
lilleman 488f1fd698 Answer prose round 1: 0 🔴, 14 🟡 (0 misread); reword the help, docstrings and README entries in the commit above
CI / tests (pull_request) Successful in 3m57s
2026-11-21 02:41:37 +01:00
lilleman 76131e1ee6 Answer product-owner round 1: 0 🔴, 1 🟡 (0 misread); keep the finished runs and say Bug, the host's gremlins and the reads' median in the commit above 2026-11-21 02:41:36 +01:00
lilleman 8c1edf552a Answer stability round 1: 0 🔴, 10 🟡 (0 misread); take the failed test's last message, fail on a docker run that never started, keep testdata as planted, show finished runs when the API refuses one, and test five survivors 2026-11-21 02:41:36 +01:00
lilleman 1e96a397af Test the value after an equals sign, -h, a line next to a change and the proxy's refusals
CI / tests (pull_request) Successful in 3m54s
2026-11-21 02:04:37 +01:00
lilleman ebbe926ab7 Test task-bench's docker calls, command line and planter where mutants survived
CI / tests (pull_request) Successful in 4m38s
2026-11-21 01:39:12 +01:00
lilleman fed84cc510 Answer security round 1: 0 🔴, 0 🟡 (0 misread); nothing to change 2026-11-21 01:07:18 +01:00
lilleman 089e9b683b Add task-bench, which has Haiku fix planted bugs at a base and a head and prints what each side took
CI / tests (pull_request) Failing after 4m9s
2026-11-21 01:05:50 +01:00
lilleman 6953dcb8e0 Finish item 511 in the plan
CI / tests (pull_request) Successful in 5m22s
2026-11-21 00:54:24 +01:00
lilleman 96ee03260e Merge remote-tracking branch 'origin/main' into item-511 2026-11-21 00:54:23 +01:00
lilleman 457c789165 Answer prose round 1: 0 🔴, 5 🟡 (0 misread); reword the claimed-output paragraph and both reviewer checks in the commit above
CI / tests (pull_request) Has been cancelled
2026-11-21 00:51:07 +01:00
lilleman a558908d83 Answer stability round 1: 0 🔴, 3 🟡 (1 misread); name the claimed-output file per round, run as the project runs tools, leave files untouched 2026-11-21 00:51:07 +01:00