The page said the question set had NEVER BEEN RUN. That was wrong twice
over, and the second one only surfaced because the first was challenged.
Wrong the first time: commit 7ea8636 records that "the 67-question set
passed for weeks without catching any of them" - it was run repeatedly
against the stand-in model and passed. That is precisely why it caught
none of the eight failures that appeared minutes after the first real
model call on 27 August.
Wrong the second time, and worse: Langfuse holds 251 traces from
1 September 01:01-02:08 UTC covering 84 distinct questions, and every one
of the 75 distinct questions in the set is among them. Nothing was
skipped. That is the set being run end to end against the live model,
nine hours before the seed was re-keyed - and nobody knew, because
eval/results is git-ignored and no scorecard was kept. A run that leaves
no artefact is indistinguishable from one that never happened, which is
how a page assembled from the host ended up asserting the opposite of
what the host could prove.
So the exam is now amber, not red, and says what is actually wrong with
it as a gate: it scored a system that no longer exists, and its result was
not kept. A run against the current build is planned.
The rest is shape. The opening was three paragraphs of running commentary;
it is now one sentence plus a three-row revision table. The KPI band had
three separate tiles for open work - 1 request, 0 findings, 3 issues - so
a reader saw 0 beside 3 and had to work out they meant different things;
one tile now reads 4 open items, 1 blocking. A new section lists those
four in plain words with whose they are and whether they block anything,
because a number with nothing behind it is not status. "Phase" is gone
from the visible text: nine steps, each ending in a check.
Five container uptimes were removed rather than corrected. "Up 26 h" was
four days stale and how long since a restart changes no decision - the
same reasoning as not quoting counts in prose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLAUDE.md requires a failure to be pinned before it is fixed, and names
two places to pin one. Neither covers a shell script, a SQL migration or a
seed CSV: api/tests holds six files and all six are Python-level.
Three defects on 1 September showed what that costs. deploy.sh loaded
historian_items positionally, so a migrated column order put a poll group
into modbus_address - it failed loudly only because a text value hit an
integer column, and two columns of the same type would have swapped
silently. The PS_ rename mapped the high level alarm STATUS BIT onto the
alarm SETPOINT, which would have made "how many high level alarms last
week" count setpoint changes. Neither could be pinned anywhere. The third,
a TypeError in the retired-name guard, had a home in api/tests and still
was not pinned.
The second was caught only because gen_historian_items.py's
check_mapping() refuses to write when an item resolves to nothing. That
check exists because the same ambiguity caused a Phase 5 finding in
August. It has now caught the same class of defect twice and it is the
only guard of its kind in the repository.
The issue records three things worth considering - a test over the seed
CSVs asserting unique keys and resolving foreign keys, migrations applied
to a throwaway Postgres rather than to lin001, and an honest question
about whether deploy.sh is worth testing at all for a demo - without
deciding any of them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five documents named the size of eval/testset.jsonl and two were wrong:
spec/REBUILD.md said 62 cases and status/current-state.html said 75. The
file holds 78. Both were corrected earlier today in the two places I had
grepped for, and these two were missed - the same staleness failure
repeating inside the same session that found it.
Rather than correct them to 78 and wait for the next divergence, the
count is gone from every sentence that states a current total. README.md,
REBUILD.md, current-state.html and workflow-map.html now name the eval set
without sizing it. The file is the source of truth and it cannot go stale.
The four remaining mentions are historical and stay: "the exam grew from
67 questions to 75 to cover them" records what happened when the eight
live-model failures were pinned, and REQUESTS.md's "67-question acceptance
test" is inside a dated request. Both are correct as history.
This is the same reasoning as dropping "twelve assumptions" from the
stand-in historian pointer, and it is now a rule in CLAUDE.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
OI-02: the second of the three lines this system does not cross - no
recommended setpoints or operating parameters - is expected to come up as
a feature request. An Advisory answer gives evidence, ranges, outcomes and
documented limits, then defers to a competent person, and never returns a
number as the answer. There is a foreseeable case for allowing it, and the
conversation should start from what is built rather than from scratch.
The entry records where the rule actually lives, because it is a code path
and not a prompt instruction, so relaxing it is a change in four places at
once: AdvisoryAnswer's recommendation_given: Literal[False] and required
deferral in api/contracts.py, the classifier routing anything
partly-advisory to the class that refuses to advise, 14 Advisory cases in
the eval set, and 15 tests. It also records what has to be decided before
any of that is touched - who is accountable for the number, what evidence
is sufficient when the historian shows what happened rather than what the
plant can safely do now, and how a recommendation is told apart from a
documented limit on screen.
It is blocked by the OT/safety review of section 2 of the build spec,
which is still outstanding. That review will have the strongest opinion on
this rule of anything in the document, so the rule should not be relaxed
before it happens.
The register's header said it held work we "intend to do", and its
boundary table admitted only defects. OI-02 is neither: the rule works as
designed. Rather than let the register's rules and its contents disagree -
which is the failure this register exists to prevent - the header now
covers decisions we expect to revisit, and the table has a row for them
requiring the entry to be marked "not a defect".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seven mentions remain across the repository and none of them is data.
Each is a sentence recording that these names were wrong, what it cost,
and that they must not come back: the namespace tables in CLAUDE.md and
the build spec, the generator's docstring, the fixtures rewrite note, the
Phase 5 narrative in REBUILD.md and current-state.html, and the delivered
configuration's own README.
The build spec's namespace table carried the same error CLAUDE.md did -
PS_STN_WET_WELL_LEVEL listed as the CI Server point layer, sourced from a
file that was a PLC-side proposal. Both now name the real point layer,
WRPS_PLC:STN_LEVEL from wrps_modbus_point_df.qli, and both point at
db/seed/scada-source/ rather than at a path on one person's laptop.
The name-collision entry in the build spec is marked closed, and records
that it bit a second time during the rename itself.
These notes explain; they do not enforce. What actually caught the
collision was check_mapping() refusing to write, and what stops a retired
name being queried is the guard in equipment.resolve(). The prose is here
so the next person does not have to re-derive the story - not as a
control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
status/OPEN-ISSUES.md is a fourth register, so it opens by saying what
does NOT belong in it. Work blocked on other people stays in REQUESTS.md;
shortcuts we consciously accepted stay in BUILD-AI-CONTAINERS.md 14 and
are closed by decision, not to be re-raised here; what is running stays
in current-state.html. What had no home until now is a defect we own and
have not fixed - those were living in commit messages. Closed issues move
to the bottom rather than being deleted: this repo is an as-built record,
and an issue with no trace of how it closed is worth less than one that
was never raised.
OI-01: the OpenPLC Editor is not installed anywhere. It is a desktop tool
published by OpenPLC, and it is what authors and compiles the IEC 61131-3
program openplc-runtime executes. Without it there is no reviewable
source for the control logic in or beside this repository - the program
exists only inside the container.
Raising it exposed two wrong statements in the build spec, both calling
port 8443 the "OpenPLC Runtime web UI". Probed read-only on the host:
Server: Werkzeug/3.1.8 Python/3.11.2
/ /login /index.html /programs /status /runtime -> 404
/api/v1 -> 401 Unauthorized
It is an authenticated REST API with no browser interface at all. Section
4 now says so with the evidence, and 14's entry is restated: 8443 is
contained by the same 10.0.0.17 binding as 502, but unlike 502 it is not
anonymous, which is a better position than that section recorded. The
binding is still the control that matters.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The exam: eval/testset.jsonl holds 78 cases (A10 H28 L8 N5 P10 R10 T5 U2).
workflow-map.html called it a 75-question exam in the two places it states
the current total. Corrected. The two sentences describing how the exam
"grew from 67 questions to 75" are left alone - that step is historically
correct: 67 plus the eight live-model failures is 75, and the Phase 5
findings H26, H27 and H31 took it to 78 afterwards.
The env files: .env.example said TWO 0600 files under ~/ai/ and listed
pg-ai.env and api.env, but its own line 105 refers to langfuse.env, and
README.md, scripts/deploy.sh and compose/langfuse-compose.yml all use
three. The header was simply wrong; langfuse.env is now named in it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The root held five documents of two different kinds with nothing to tell
them apart. This repo is being handed over as both design spec and
as-built record, so the two are now separated by lifetime:
spec/ BUILD-AI-CONTAINERS.md, YAU_Linux_Host_Onboarding.md
normative and durable - what we agreed to build
status/ workflow-map.html, current-state.html, REQUESTS.md
snapshots that go stale and get edited constantly
Nothing mechanical moved. The twelve code folders are deployment
interface - compose builds from /home/azureuser/ai/api, /web, /ingest and
deploy.sh copies from $HOME/ai/compose, /cube/model, /db - so they stay
where they are. CLAUDE.md stays at root because Claude Code loads it from
there, as do .gitignore and .gitattributes, which only apply repo-wide
from the root.
All 26 references to the build spec were bare filenames in prose, so the
move broke no mechanism. The seven markdown links in README.md and the
two pointers at the top of CLAUDE.md are repointed.
docs/ is deleted. It only ever held .gitkeep - nothing mounted or read
it, and ingest reads the host path /datadisk/ai-docs mounted at /docs
inside the container. An empty docs/ invites the misreading "project
documentation goes here". The ignore rule stays, now covering the whole
folder, so a controlled document dropped there still cannot be committed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
