Five documents named the size of eval/testset.jsonl and two were wrong:
spec/REBUILD.md said 62 cases and status/current-state.html said 75. The
file holds 78. Both were corrected earlier today in the two places I had
grepped for, and these two were missed - the same staleness failure
repeating inside the same session that found it.
Rather than correct them to 78 and wait for the next divergence, the
count is gone from every sentence that states a current total. README.md,
REBUILD.md, current-state.html and workflow-map.html now name the eval set
without sizing it. The file is the source of truth and it cannot go stale.
The four remaining mentions are historical and stay: "the exam grew from
67 questions to 75 to cover them" records what happened when the eight
live-model failures were pinned, and REQUESTS.md's "67-question acceptance
test" is inside a dated request. Both are correct as history.
This is the same reasoning as dropping "twelve assumptions" from the
stand-in historian pointer, and it is now a rule in CLAUDE.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
status/OPEN-ISSUES.md is a fourth register, so it opens by saying what
does NOT belong in it. Work blocked on other people stays in REQUESTS.md;
shortcuts we consciously accepted stay in BUILD-AI-CONTAINERS.md 14 and
are closed by decision, not to be re-raised here; what is running stays
in current-state.html. What had no home until now is a defect we own and
have not fixed - those were living in commit messages. Closed issues move
to the bottom rather than being deleted: this repo is an as-built record,
and an issue with no trace of how it closed is worth less than one that
was never raised.
OI-01: the OpenPLC Editor is not installed anywhere. It is a desktop tool
published by OpenPLC, and it is what authors and compiles the IEC 61131-3
program openplc-runtime executes. Without it there is no reviewable
source for the control logic in or beside this repository - the program
exists only inside the container.
Raising it exposed two wrong statements in the build spec, both calling
port 8443 the "OpenPLC Runtime web UI". Probed read-only on the host:
Server: Werkzeug/3.1.8 Python/3.11.2
/ /login /index.html /programs /status /runtime -> 404
/api/v1 -> 401 Unauthorized
It is an authenticated REST API with no browser interface at all. Section
4 now says so with the evidence, and 14's entry is restated: 8443 is
contained by the same 10.0.0.17 binding as 502, but unlike 502 it is not
anonymous, which is a better position than that section recorded. The
binding is still the control that matters.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Counted on lin001: 28 running, none stopped. Seven are ours (ai-api,
ai-web, pg-ai, cube, cubestore, langfuse, lf-db), so 21 were already
there. Four documents gave four different numbers because they were
counting different things at different times.
BUILD-AI-CONTAINERS.md (21 pre-existing) and current-state.html (28
total) were already right and are untouched. README.md said 22 where it
meant 21. The host brief said 20 in three places while naming only 19,
and it is the one with a real gap behind it:
openplc-runtime appears NOWHERE in the 351-line host brief. Not in the
stack table, not in the data flow, not in the rules - while CLAUDE.md
calls it live control that must never be restarted, and the build spec
names it as the reason port 502 is the one exception to no-published-
ports on this host. The document a new engineer reads first to learn what
runs here omitted the only container doing live control. wireguard was
missing from the same table, though mentioned in three other sections.
Both are now in §3. §6's published-ports list was missing 502/tcp and
8443/tcp for the same reason; verified against docker ps, they bind to
10.0.0.17 rather than 0.0.0.0, and that binding is what keeps
unauthenticated Modbus off the internet, so it is recorded as a control
and not as a detail. §12's "confirm 20 containers up" no longer names a
number - that is how it went stale.
The brief's stamp moves to 2026-09-01 with a note saying what was amended
and what was left alone, so Dan's authorship of the rest stays clear.
Separately, README.md listed ai-docs-worker among the containers as
though it were running. It was never deployed: Phase 9 shipped with the
document screens served by ai-api, and the worker sits behind a
--profile worker flag. compose and current-state.html both say so
already; the README table now does too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README.md was 481 lines, 60% of which was a nine-phase deployment runbook.
A new team member had to scroll past every deploy command to reach what
the folders are. The runbook moves out whole to spec/REBUILD.md - content
unchanged apart from heading levels and seven link targets repointed for
the new depth. README.md is now 258 lines.
Two additions:
"Start here" gives a reading order - README, then the workflow map, then
the current state, then CLAUDE.md - and a table routing to the runbook,
the build spec, the host brief or REQUESTS.md by what the reader came to
do. It also names the three accesses needed before starting, none of
which are in this repository.
The architecture diagram is redrawn because the old one was wrong. It
showed Authelia in front of ai-web. Since 28 August that hostname carries
no Authelia at all - it is matched on source IP. The README said so
correctly, 170 lines below the picture. The new diagram shows both
parties arriving at the same Caddy and then branching: ai.yokogawa.tech
to the @scada remote_ip matcher, the other three through import authelia.
The operator is inside the Caddy/Authelia environment; they take a branch
Authelia does not sit on. The answer path behind ai-api is drawn
separately, because it is the same whichever door the question came in
through. Structure follows caddy/ai-routes.caddy:190-203.
The layout block gains REBUILD.md and the eval count is corrected to 78
in passing, that line being part of the block rewritten here.
Every markdown link in all nine documentation files was checked after the
move: none broken.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The root held five documents of two different kinds with nothing to tell
them apart. This repo is being handed over as both design spec and
as-built record, so the two are now separated by lifetime:
spec/ BUILD-AI-CONTAINERS.md, YAU_Linux_Host_Onboarding.md
normative and durable - what we agreed to build
status/ workflow-map.html, current-state.html, REQUESTS.md
snapshots that go stale and get edited constantly
Nothing mechanical moved. The twelve code folders are deployment
interface - compose builds from /home/azureuser/ai/api, /web, /ingest and
deploy.sh copies from $HOME/ai/compose, /cube/model, /db - so they stay
where they are. CLAUDE.md stays at root because Claude Code loads it from
there, as do .gitignore and .gitattributes, which only apply repo-wide
from the root.
All 26 references to the build spec were bare filenames in prose, so the
move broke no mechanism. The seven markdown links in README.md and the
two pointers at the top of CLAUDE.md are repointed.
docs/ is deleted. It only ever held .gitkeep - nothing mounted or read
it, and ingest reads the host path /datadisk/ai-docs mounted at /docs
inside the container. An empty docs/ invites the misreading "project
documentation goes here". The ignore rule stays, now covering the whole
folder, so a controlled document dropped there still cannot be committed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three open Phase 5 findings were one defect: the stand-in was keyed on
CI Server POINT names (PS_STN_WET_WELL_LEVEL) when the historian is keyed on
CI Server ITEM names (AID.WRPS.STN.LEVEL). Modbus carries register numbers,
not names, so those two layers are free to differ - and do. Reconciling
against the register map, as planned, would only have proved the first three
namespaces agreed with each other.
Rebuilt from WRPS/05-scada/modbus, so item names, sample rates, retention and
timestamp semantics come from the machine rather than from a guess.
(a) Level tag does not join. PS_STN_WET_WELL_LEVEL becomes a tag row in its
own right; LIT-101 is marked NOT HISTORISED - a field input on %IW0 that
never reaches SCADA. It was the only seed row carrying two addresses.
public.historian_items holds the item-to-tag mapping, generated by
scripts/gen_historian_items.py and enforced non-empty at generate, at
deploy and at verify.
(b) first_alarm/last_alarm returned UTC. Converted inside the measure, so it
stays in Cube and happens once. Aggregate first, convert after - the other
order picks the wrong row across a DST fall-back. Returned as a formatted
string with a companion site_timezone measure. Storage being UTC is now
confirmed, not assumed: all 49 points carry TIME_ZONE "Date+time GMT" and
every history group CORRECT_DAYLIGHT=0. This answers Phase 4 task 4.
(c) High level alarm filed against the wrong equipment. Both sides were right
about different things; the defect was asserting equipment twice. The
history now carries no equipment column at all - faithful, since CI
Server's section tree stops at the station and three pumps. Equipment is
reached bit -> tag -> equipment via public.alarm_bits.
Alarms are derived, not stored: CI Server's ALARM_HISTORY group is empty
because every item imports with alarming off. Decomposing the alarm word needs
no configuration that does not exist.
Three things the SCADA config changed that were never filed as faults:
- retention is 7 days, not 30. The advisory path was reporting a month of
evidence drawn from a week of data
- the analogue rate is 5 s, not 60. Two measures multiplied sample counts by
a hardcoded 60 - a twelvefold overstatement that read as plausible
- the deadband warning in process_values.yml was wrong and was steering
people away from the correct measure
db/002_fixtures.sql now asserts its own counts at load and cross-checks the
alarm derivation against two independent signals. Those prove the pipeline,
not the plant.
db/README-standin-historian.md documents removal: the seam between generation
and contract, and twelve assumptions about imh that are NOT confirmed. Two of
them fail silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 9's operator path, built ahead of Phase 8 at the customer's direction and
live at api.yokogawa.tech/documents. Upload, convert, review, approve, withdraw
and restore. The pool screen is explicitly out of scope.
Served by ai-api rather than ai-web, and mounted at /documents rather than
/docs. ai.yokogawa.tech is SCADA-only since 2026-08-28 and passes through no
Authelia, so it has no identity to record; publishers arrive on
api.yokogawa.tech where the forward-auth headers still do. /docs stays with
Swagger, which the customer is keeping - two things under one prefix with two
different access policies is what gets misread during a later edit.
Conversion is text extraction, not document parsing: pypdf, python-docx and
openpyxl. Docling would be better at this and pulls torch, which lin001 has
neither the memory to install nor the business running next to the demo plant's
PLC. The cost is real - no layout, no table structure, and a scan cannot be read
at all, so it is refused rather than stored empty. It is acceptable only because
the converted text is shown to a person before the document can be cited, which
is the same safety net the design already required for the header. convert.py is
the one file to change if that stops being true.
Chunking is mirrored from ingest.py rather than shared, because the two live in
different images. They must stay identical: if they drift, the same document
chunks differently depending on who loaded it, and the assistant answers or
fails to answer depending on that. The step-sequence rule is locked by a test.
Identity is self-asserted for the demo - the actor is typed on the form, which
section 16 forbids, and the publisher list is one name with no password. Rows are
written as `demo:<name>` with actor_groups = 'DEMO-UNVERIFIED' so that when real
auth goes on, a name somebody typed stays tellable from a name Authelia proved.
doc_actions cannot be deleted from, so an ambiguity there would be permanent.
Two rules the code enforces rather than documents: uploading is open to anyone
who reaches the page, because uploading changes nothing an operator can see -
approving does, and that is what is gated; and an empty publisher list means
nobody, not everybody.
Verified on the host end to end: withdraw as a non-publisher 403s, with a short
reason 400s, and as admin flips 5 chunks and writes a complete audit row;
restore puts them back and keeps both rows. The corpus is unchanged afterwards.
Requirements are split so the document dependencies install in their own layer -
a change there costs four small wheels instead of re-resolving fastapi,
langgraph and langfuse on a 2 vCPU shared host.
The five divergences from section 16 are recorded in section 14. The one with
teeth: files published through the UI stay in the inbox, so `ai-ingest --all`
cannot see them and the two paths must not be used on the same document.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An operator standing at the SCADA console should not complete a Duo push to
ask a question, and nobody outside the plant should reach the assistant at
all. The ai.yokogawa.tech Caddy block now admits remote_ip 10.0.0.21
(yau-poc-cicore1, static) and returns 403 to everything else. Applied on
lin001 2026-08-28; snapshot at ~/Caddyfile.bak-ai-scadaonly-20260828.
This also settles why the console could not reach the assistant at all:
auth.yokogawa.tech has no pinpoint record on the DC, so a LAN browser got a
correct 302 to the portal and then died on DNS. It went unnoticed because the
device agents write to Influx over the /api/v2/write MFA bypass and never
touch the portal - no browser had ever hit Authelia from inside the VNet.
Verified before applying that an IP matcher can work here: Caddy sees real
client addresses, and WireGuard peers arrive masqueraded as 172.19.0.6 so
they do not match and are refused along with the internet.
What this costs, recorded in section 14 as a shortcut and not as a security
control: it is an IP allowlist on a flat network with no OT/IT boundary, so
anything that can take 10.0.0.21 inherits unauthenticated access; Langfuse
traces are now anonymous, so there is no record of who asked what; and the
assistant is out of browser reach over the VPN. It is in scope for the
section 2 OT/safety review, which is still outstanding.
api.yokogawa.tech is unchanged and still fully gated - Phase 9 publishing
depends on Remote-User/Remote-Groups and stays there. The now-inert
ai.yokogawa.tech entry in the Authelia rule is deliberately left in place so
restoring the gate is a Caddy reload rather than an Authelia restart that
logs out every user on the host.
verify.sh treats 403-from-lin001 as the pass for the deny arm and states
plainly that the allow arm can only be proved from cicore1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
api.yokogawa.tech has a public A record but no pinpoint record on the DC, so
it does not resolve from inside the VNet at all. The browser called it by
hostname, which means an operator on cicore1 would have loaded the page and
had every question fail on DNS - the exact gap Phase 7's gate exists to catch,
and one an engineer's laptop cannot see.
Caddy now routes /ask under ai.yokogawa.tech to ai-api, inside a route block
so import authelia still runs first: forward_auth sorts after handle in the
default directive order, and outside a route the handles would be terminal and
the gate would never run. Only /ask is routed - the Phase 9 publisher rule is
scoped to api.yokogawa.tech and a wider route here would leave it inert.
Also fixes the fallback it replaces. The build arg defaults to "", and
`?? "https://api.yokogawa.tech"` does not catch an empty string, so the
documented real-deployment build resolved the API base to "" and posted /ask
at ai-web, which 404s it. verify.sh and deploy.sh now check both the route and
whether the built bundle carries the hostname.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both were found hand-verifying the measures on fixtures, and both are left
unfixed deliberately: fixing either now means guessing at what imh contains,
and a guess baked into the seed data or the models is harder to find later than
an open finding. Flagged at Phase 4, in the README, in the model files at the
point of use, and as eval cases that fail until they are settled.
(a) The wet well level tag does not join, and fails as "no records found".
History is keyed PS_STN_WET_WELL_LEVEL, which process_values.yml hardcodes
in seconds_above_high_level_alarm and seconds_above_lshh. tags.csv carries
that name only as an ALIAS of LIT-101, so public.tags has no row with that
tag_id and all 43,201 level rows - a third of the history, on the most
important tag at this station - are unreachable from a tag-level lookup.
Resolve "wet well" -> WW-101 -> LIT-101 -> filter history on LIT-101 ->
zero rows -> "no records found".
That is the failure mode worth being loudest about: it is the safety
behaviour, produced by a key mismatch rather than by an absence of data,
and indistinguishable from the real thing on screen. Filtering by
equipment_id works, so whether a level question fails depends on which path
the agent takes.
The two flow tags use the opposite, self-consistent convention -
PS_STN_INFLOW is a row in its own right and FIT-201 is marked NOT
HISTORISED - so applying that to level is the likely fix. It still waits
for the register map and for imh to say what CI Server historises the point
as. Seed, hardcoded model names and 002_fixtures.sql change together.
(b) alarms.first_alarm and last_alarm return UTC, not SITE_TIMEZONE.
Cube converts time dimensions to the query timezone; a min/max measure over
a timestamp comes back unconverted. The Sydney day bucket for 2026-08-01
returns 2026-07-31T20:00:35 - the right instant, ten hours and one calendar
day out, beside a bucket label that IS in site time.
This breaks "convert to SITE_TIMEZONE exactly once, in Cube", and the fix
has to stay in Cube - the API compensating with timezone arithmetic is the
thing that rule exists to prevent. Which fix is right depends on whether
imh stores UTC or local, which is Phase 4 task 4. Until then these two must
not be quoted to an operator as a clock time.
Phase 4's gate gains an item for each. Everything else verified in this pass -
alarms, operations, the equipment join, alias resolution and the Sydney/UTC day
boundaries - matched hand-written SQL exactly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ingest_file() inserted every chunk with superseded = FALSE, so replacing a
document's chunks reset its withdrawal. One `ai-ingest --all` made every
superseded revision citable again - including the old revision of a procedure -
silently, and only after a bulk run, so a supersede survived exactly until the
next full ingest.
- ingest_file() reads the existing state with superseded_state() before the
delete and carries it through the insert. A withdrawn document is
re-ingested as withdrawn, and logs that it did so.
- --all skips withdrawn documents, so a bulk run does not spend an embeddings
call on a document that will not be cited either way. --include-superseded
overrides it; the chunks still come back withdrawn.
- --restore DOC_NUMBER REVISION is the counterpart to --supersede, refusing
while another revision of the same document is live. Without it the
conservative default would be a dead end - mark_superseded() only ever sets
TRUE, so there was no way back.
- Rule 5 in the module docstring, beside the other four.
Corrects a claim in the Phase 9 design: 16.10 said moving the withdrawn file
out of /datadisk/ai-docs "is not tidying" because otherwise --all resurrects it.
That was true when written and is not now. The guard belongs in the ingest code,
because a rule that depends on somebody remembering to move a file is not a
rule. The move stays as archival housekeeping and is documented as such in
16.10, db/005_doc_actions.sql and README.md.
Phase 3's gate gains the proof: supersede a revision, run --all again, ask the
question that used to cite it.
Not executed anywhere - no Postgres or Docker on this machine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ai-ingest built its DSN from PGUSER/PGPASSWORD and takes its environment from
~/ai/api.env, where PGUSER=agent_ro - SELECT and nothing else, deliberately,
because it is what the answer path runs as. So
docker compose -f ~/ai-compose.yml run --rm ai-ingest --all
connected as a role that cannot INSERT INTO doc_chunks, and Phase 3 was
unrunnable exactly as the README documents it. Nothing had reached Phase 3 yet,
so nobody had hit it.
The failure would also have landed at the worst possible moment: at the final
INSERT, after the Docling parse, after a person had typed the header
confirmations for every file, and after a billed embeddings call - with a
permission error naming no cause.
- ingest_rw moves to 003_roles.sql, at Phase 1 with the other roles. It is
not a Phase 9 concept; ingestion has needed a writing role since Phase 3
and never had one. 004 keeps only its grants on the upload queue, and its
idempotent role creation so it still applies to an older database.
- ingest.py connects through INGEST_DB_USER / INGEST_DB_PASSWORD, falling
back to PGUSER only for a local shell where one pair is set.
- require_write_access() checks INSERT, UPDATE and DELETE on doc_chunks
before anything is parsed or embedded, and fails with the fix in the
message. Falling back to PGUSER cannot smuggle agent_ro past it.
- Keyword connection parameters rather than a URL: a generated password
containing @ or / breaks a DSN string silently.
- A missing doc_chunks now says "apply 001_schema.sql" instead of raising
UndefinedTable.
Phase 1's gate gains the check that would have caught this: ingest_rw must be
able to write doc_chunks. An ingestion role that cannot write is the same class
of failure as an API role that can - it just surfaces two phases later.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A single-page explainer for people who will not read the build spec: how a
question becomes an answer, the four lanes and why they are separate, the four
tools the assistant may reach for, where the knowledge comes from, and what is
live against what is only built. Status snapshot as observed on the host,
21 August 2026.
Kept at the repository root beside the other narrative documents rather than in
docs/, which is gitignored and holds controlled plant documents. Listed in the
layout sections of README.md and the build spec so it is findable.
Note: it predates Phase 9 and so does not mention operator document upload,
withdrawal or pool curation. Its step table stops at 8.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operators cannot add a document today: ingestion is CLI-only, needs a host
login and a TTY for confirm_header(), /datadisk/ai-docs is mounted read-only,
ai-api has no identity, and nothing in the stack has a role that can write
doc_chunks. This designs the way in, the way out, and control over what is in
the retrieval pool. Design and schema only - no router, worker or UI code yet.
Documents in (16.1-16.9, db/004):
upload -> pre-scan -> review -> approve -> published, with the header
confirmation moved from a terminal prompt to a review screen and recorded
rather than discarded. A CHECK constraint refuses an approved row without a
confirmed number, revision and effective date, so an API bug cannot skip it.
Three roles: agent_ro unchanged, uploads_rw writes the queue only, ingest_rw
writes doc_chunks and has no HTTP surface.
Documents out (16.10-16.11, db/005):
--supersede needs a revision to keep, so a cancelled procedure cannot be
withdrawn at all. Adds withdraw (immediate, reversible, audited), restore
(refused while another revision is live) and purge (off by default). A
column grant plus a trigger let the web-facing role make a document less
citable and never more.
The pool (16.13-16.15, db/006):
pool_enabled, orthogonal to superseded: one is a claim about the document,
the other about the corpus. Retrieval requires both, so re-enabling a
withdrawn document does not make it citable. Named profiles and a
per-request override let a demo trim the corpus without mutating state on a
shared live host, and every reduced-pool answer carries a banner with the
document count, following the used_fixture_data precedent.
Two existing defects found and documented while designing this:
- ai-ingest takes PGUSER=agent_ro from api.env, a SELECT-only role, so the
Phase 3 command in the README cannot write doc_chunks (16.1).
- ingest_file() always inserts superseded = FALSE, so `--all` re-ingests a
superseded revision as live. --supersede survives only until the next bulk
run (16.10).
One commit rather than three: the upload, withdrawal and pool designs
interleave in the same spec, README and compose files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Build spec and host brief carried in from C:\Claude and WRPS/02-env; the
plant model (equipment, tags, alarm bitmask, enums, unit conversions) is
derived from WRPS/04-plc/register-map.csv, WRPS/05-scada/modbus/scada-points.csv
and WRPS-CTL-003.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
