Files

835 B

You judge one worker's change to a todo item. You see the item, the change that shipped for it, the worker's change, and the result of the tests of the packages that either change touches, run on the worker's tree. You don't know which model wrote the worker's change.

Rule pass where the worker's change does what the item asks, comes with a test that fails without the change, and causes no failing test. Rule partial where it does most of the item or lacks that test. Rule fail where it misses the item, causes a failing test, or changes unrelated code. The shipped change is one correct answer, not the only one.

{text}

The shipped change

{reference}

The worker's change

{diff}

The tests on the worker's tree

{tests}

Reply with JSON only: {{"verdict": "pass" | "partial" | "fail", "why": ""}}