Lilleman
|
a75826d35a
|
Load no MCP server in the bench's headless claude calls that are denied tools
CI / tests (pull_request) Successful in 3m45s
|
2026-11-20 18:02:12 +01:00 |
|
lilleman
|
762614353c
|
Answer principles round 2: 0 🔴, 2 🟡 (0 misread); workflow-bench hands its effects in as arguments, and no Python file of the repo patches a global
CI / tests (pull_request) Successful in 3m46s
|
2026-11-19 12:55:35 +01:00 |
|
lilleman
|
4875e07f98
|
Answer stability round 1: 0 🔴, 2 🟡 (0 misread); refuse a failed compaction and count the starts' turns as claude does
|
2026-11-17 03:58:52 +01:00 |
|
lilleman
|
b4490b4a8c
|
Refuse a compact arm whose compaction failed, and state the bench as it ran
|
2026-11-17 03:56:25 +01:00 |
|
lilleman
|
24800c156b
|
Assert that the start's saved diff holds an untracked file
|
2026-11-17 03:51:01 +01:00 |
|
lilleman
|
7da82b4748
|
Hold the handoff bench's prompts, saved change and steps to their tests
|
2026-11-17 03:48:46 +01:00 |
|
lilleman
|
1bc2ce89e7
|
Net a resumed arm's cost of the start it resumes
|
2026-11-17 03:44:53 +01:00 |
|
lilleman
|
e42eb5b288
|
Bench a model step's handoff: resume, compact or a fresh session with a brief
|
2026-11-17 02:57:22 +01:00 |
|
lilleman
|
14e6756a53
|
Answer prose round 1: 0 🔴, 6 🟡 (0 misread); rewrite each <verb> nothing to the negation rule, tighten the new docstrings and comments
|
2026-11-12 20:33:32 +01:00 |
|
lilleman
|
2556fdace9
|
Answer prose round 1: 0 🔴, 7 🟡 (0 misread); the decision's scope, the changelog bullet and the prompts read once
CI / tests (pull_request) Successful in 2m30s
|
2026-11-12 05:30:15 +01:00 |
|
lilleman
|
6817b074bd
|
Answer stability round 1: 0 🔴, 6 🟡 (0 misread); the bench tests only the packages the clone holds, refuses a run that ended in an error, and pins a worker's committed change
|
2026-11-12 05:30:15 +01:00 |
|
lilleman
|
e1e99645be
|
Pin the advisor bench's argv, kwargs, diff exclusions and failures in its tests
|
2026-11-12 05:20:39 +01:00 |
|
lilleman
|
45fd66d3ab
|
Test the advisor bench's run, grade, main and rows_of against a throwaway repo, and kill the mutants of its pure functions
|
2026-11-12 05:18:44 +01:00 |
|
lilleman
|
66b9651d68
|
Bench the advisor tool on six archived L 1 and L 2 chunks: Sonnet alone passes as often as Opus at medium for less, so workers stay without it
CI / tests (pull_request) Successful in 2m18s
|
2026-11-12 04:27:48 +01:00 |
|
lilleman
|
a52ebb94bf
|
Run mutation tests on a chunk's changed lines before review with bin/mutate-changes, and hand stability-reviewer the survivors
CI / tests (pull_request) Successful in 2m14s
|
2026-11-11 18:58:31 +01:00 |
|
lilleman
|
c74ec6fc0a
|
Answer stability round 1: 0 🔴, 4 🟡 (0 misread); test main's continuing past a failed run, name the denied commits in decided's tests, check the replay puts premises before records
|
2026-11-08 22:58:52 +01:00 |
|
lilleman
|
268200de23
|
Pass the judge a Brief, decide read_seats from its inputs and return a failed bench run as a line
|
2026-11-08 22:51:27 +01:00 |
|
lilleman
|
53e87d1051
|
Answer security round 5: 0 🔴, 1 🟡; load no settings source at all, and answer stability round 5's nit with a test that no config or hook program runs
|
2026-10-16 22:54:18 +02:00 |
|
lilleman
|
57cd5f76a4
|
Answer security round 4: 1 🔴, 1 🟡; refuse the format options that start gpg, and load no settings from the case's clone
|
2026-10-16 22:42:11 +02:00 |
|
lilleman
|
b30c91aafc
|
Answer stability round 4: 2 🔴, 1 🟡; the wrapper allows only listed options, takes the clone from the environment, and skips only a message its flag takes whole
|
2026-10-16 22:30:24 +02:00 |
|
lilleman
|
cf4072f69f
|
Answer security round 3: 0 🔴, 0 🟡; the answer holds
|
2026-10-16 20:53:27 +02:00 |
|
lilleman
|
4625b805eb
|
Record in bench.py why the clone's settings need no deny
|
2026-10-16 20:32:42 +02:00 |
|
lilleman
|
95aa2b3477
|
Answer security round 2: 1 🔴, 0 🟡; deny a git argument that names a path outside the clone, in place of listing options
|
2026-10-16 20:32:36 +02:00 |
|
lilleman
|
58252f8948
|
Answer security round 1: 1 🔴, 1 🟡; allow only the git commands a review needs, deny the options that name a path, and load no user settings
|
2026-10-16 20:24:02 +02:00 |
|
lilleman
|
14c8e7f012
|
Grant the bench's runs only the tools a review needs, in place of skipping every permission
|
2026-10-16 20:17:42 +02:00 |
|
lilleman
|
dbe7bd3abd
|
Answer stability, product-owner, prose and architecture round 3: count only a commit's last history entry toward the base, merge the default branch in before each ruling, judge the scoring run's trigger on the plan as the chunk leaves it, count a vote naming no place as same, say in the changelog what a worse ruling and the floor do to the owner, and correct item 29
|
2026-10-08 20:50:19 +02:00 |
|
lilleman
|
26826685e2
|
Answer architecture round 1: rule against the last measured commit, launch the third and fourth seats together, name the floor's chunk by the plan, skip the whole check in the bench, and correct the history's writer and the paired brief
|
2026-10-08 20:36:25 +02:00 |
|
lilleman
|
7b38aed614
|
Answer prose round 1: precedent names what a pass skips on, the bench and floor-decision prose reads on one pass
|
2026-10-03 18:21:33 +02:00 |
|
lilleman
|
d3613f45d4
|
Answer architecture round 1: Failure and the file modules get truthful homes, the interrupt question is a pure function, the judge grounds on what it saw
|
2026-10-03 18:06:56 +02:00 |
|
lilleman
|
5f7d34b87d
|
Give record, verdict and judge one meaning each
|
2026-10-03 17:53:25 +02:00 |
|
lilleman
|
31ec74498c
|
Answer round two: an answer quoted without elisions, unambiguous referents in --help, the changelog and the security line
|
2026-10-03 15:56:54 +02:00 |
|
lilleman
|
61b7b44243
|
Answer round one: the bench runs the security pass in a subagent, an unavailable skill is reported, the answer's strictness explained where it is checked, prose nits
|
2026-10-03 15:54:42 +02:00 |
|
lilleman
|
43aa6ec4df
|
Answer the second review round
|
2026-10-02 08:47:00 +02:00 |
|
lilleman
|
c729a130a6
|
Answer the first review round
|
2026-10-02 08:45:25 +02:00 |
|
lilleman
|
765705e3ca
|
Keep handoff material in the repo, and move the bench scripts the plan points at into workflow-bench/
|
2026-10-02 08:43:20 +02:00 |
|