yau-plant-assistant/eval
Claude 7ea8636264 Add the eight live-model failures to the eval set
CLAUDE.md: add the failing case before fixing it. These were added after, not
before - recording that rather than quietly complying.

67 -> 75. L01-L05 are the five contract-shape failures; L06-L08 cover the
retrieval and schema change: that the header fields reach the answer, that step
sections never do, and that a document marked uncontrolled is identified rather
than reported as nothing found.

Every one is a case no stub run could have produced, which is the reason the
67-question set passed for weeks without catching any of them. REQUESTS.md
still says 67 and needs updating when the acceptance run is scheduled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:54:37 +10:00
..
run_eval.py Scaffold the WRPS plant operations assistant repository 2026-08-20 13:56:32 +10:00
testset.jsonl Add the eight live-model failures to the eval set 2026-08-28 10:54:37 +10:00