The page said the question set had NEVER BEEN RUN. That was wrong twice
over, and the second one only surfaced because the first was challenged.
Wrong the first time: commit 7ea8636 records that "the 67-question set
passed for weeks without catching any of them" - it was run repeatedly
against the stand-in model and passed. That is precisely why it caught
none of the eight failures that appeared minutes after the first real
model call on 27 August.
Wrong the second time, and worse: Langfuse holds 251 traces from
1 September 01:01-02:08 UTC covering 84 distinct questions, and every one
of the 75 distinct questions in the set is among them. Nothing was
skipped. That is the set being run end to end against the live model,
nine hours before the seed was re-keyed - and nobody knew, because
eval/results is git-ignored and no scorecard was kept. A run that leaves
no artefact is indistinguishable from one that never happened, which is
how a page assembled from the host ended up asserting the opposite of
what the host could prove.
So the exam is now amber, not red, and says what is actually wrong with
it as a gate: it scored a system that no longer exists, and its result was
not kept. A run against the current build is planned.
The rest is shape. The opening was three paragraphs of running commentary;
it is now one sentence plus a three-row revision table. The KPI band had
three separate tiles for open work - 1 request, 0 findings, 3 issues - so
a reader saw 0 beside 3 and had to work out they meant different things;
one tile now reads 4 open items, 1 blocking. A new section lists those
four in plain words with whose they are and whether they block anything,
because a number with nothing behind it is not status. "Phase" is gone
from the visible text: nine steps, each ending in a check.
Five container uptimes were removed rather than corrected. "Up 26 h" was
four days stale and how long since a restart changes no decision - the
same reasoning as not quoting counts in prose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five documents named the size of eval/testset.jsonl and two were wrong:
spec/REBUILD.md said 62 cases and status/current-state.html said 75. The
file holds 78. Both were corrected earlier today in the two places I had
grepped for, and these two were missed - the same staleness failure
repeating inside the same session that found it.
Rather than correct them to 78 and wait for the next divergence, the
count is gone from every sentence that states a current total. README.md,
REBUILD.md, current-state.html and workflow-map.html now name the eval set
without sizing it. The file is the source of truth and it cannot go stale.
The four remaining mentions are historical and stay: "the exam grew from
67 questions to 75 to cover them" records what happened when the eight
live-model failures were pinned, and REQUESTS.md's "67-question acceptance
test" is inside a dated request. Both are correct as history.
This is the same reasoning as dropping "twelve assumptions" from the
stand-in historian pointer, and it is now a rule in CLAUDE.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seven mentions remain across the repository and none of them is data.
Each is a sentence recording that these names were wrong, what it cost,
and that they must not come back: the namespace tables in CLAUDE.md and
the build spec, the generator's docstring, the fixtures rewrite note, the
Phase 5 narrative in REBUILD.md and current-state.html, and the delivered
configuration's own README.
The build spec's namespace table carried the same error CLAUDE.md did -
PS_STN_WET_WELL_LEVEL listed as the CI Server point layer, sourced from a
file that was a PLC-side proposal. Both now name the real point layer,
WRPS_PLC:STN_LEVEL from wrps_modbus_point_df.qli, and both point at
db/seed/scada-source/ rather than at a path on one person's laptop.
The name-collision entry in the build spec is marked closed, and records
that it bit a second time during the rename itself.
These notes explain; they do not enforce. What actually caught the
collision was check_mapping() refusing to write, and what stops a retired
name being queried is the guard in equipment.resolve(). The prose is here
so the next person does not have to re-derive the story - not as a
control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The root held five documents of two different kinds with nothing to tell
them apart. This repo is being handed over as both design spec and
as-built record, so the two are now separated by lifetime:
spec/ BUILD-AI-CONTAINERS.md, YAU_Linux_Host_Onboarding.md
normative and durable - what we agreed to build
status/ workflow-map.html, current-state.html, REQUESTS.md
snapshots that go stale and get edited constantly
Nothing mechanical moved. The twelve code folders are deployment
interface - compose builds from /home/azureuser/ai/api, /web, /ingest and
deploy.sh copies from $HOME/ai/compose, /cube/model, /db - so they stay
where they are. CLAUDE.md stays at root because Claude Code loads it from
there, as do .gitignore and .gitattributes, which only apply repo-wide
from the root.
All 26 references to the build spec were bare filenames in prose, so the
move broke no mechanism. The seven markdown links in README.md and the
two pointers at the top of CLAUDE.md are repointed.
docs/ is deleted. It only ever held .gitkeep - nothing mounted or read
it, and ingest reads the host path /datadisk/ai-docs mounted at /docs
inside the container. An empty docs/ invites the misreading "project
documentation goes here". The ignore rule stays, now covering the whole
folder, so a controlled document dropped there still cannot be committed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
