Name the judge's transcript floor in loop-claude's help and sort the effects test's imports
This commit is contained in:
@@ -101,6 +101,7 @@ class JudgeFailures(unittest.TestCase):
|
||||
("run_judge", "verdict.reason", field),
|
||||
("run_judge", "checked.reason", "quotes a passage that neither the task nor the transcript holds"),
|
||||
("misquoted_evidence", "f'the judge quoted {verdict.evidence!r}, which is not in the task or the transcript", "quotes a passage that neither the task nor the transcript holds"),
|
||||
("judge_prompt", "f\"the premises, the records and the task leave the transcript", "leave it room for less than the last 50,000 characters of the transcript"),
|
||||
("claude_reply", "f'could not start claude:", started),
|
||||
("claude_reply", "proc.stderr.strip() or f'{who} exited", exited),
|
||||
("claude_reply", 'f"{who} printed no json reply', unparsed),
|
||||
|
||||
@@ -186,10 +186,11 @@ Exit 1, saying why, when:
|
||||
handover
|
||||
- the worker or the judge fails three times running, retried a minute apart within the cycle. A
|
||||
worker fails when claude cannot start, exits non-zero, is killed, or ends without a result event;
|
||||
a judge when claude cannot start, exits non-zero or prints no json reply, or its reply misses or
|
||||
mistypes a field, interrupts or quotes an answer without a question, or quotes a passage that
|
||||
neither the task nor the transcript holds, where each fragment of an elided quote must appear in
|
||||
order, in the task then the transcript.
|
||||
a judge when the premises, the records and the task leave it room for less than the last 50,000
|
||||
characters of the transcript, when claude cannot start, exits non-zero or prints no json reply, or
|
||||
when its reply misses or mistypes a field, interrupts or quotes an answer without a question, or
|
||||
quotes a passage that neither the task nor the transcript holds, where each fragment of an elided
|
||||
quote must appear in order, in the task then the transcript.
|
||||
A usage limit is not a failure, and restarts the count of three: the worker or the judge is
|
||||
retried 2 minutes after the limit resets, or every 10 minutes when the reset time is unknown or
|
||||
more than 2 minutes past. The judge's reply gives its reset time only as text, so the judge is
|
||||
|
||||
@@ -13,8 +13,8 @@ import unittest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from failure import Failure
|
||||
from loop_effects import child_shells
|
||||
from failure import Failure
|
||||
from loop_effects.effects import run_python, say, start_timer, StreamingClaude, system_effects
|
||||
|
||||
NOW = 1790000000
|
||||
|
||||
Reference in New Issue
Block a user