The point list handed over in August named the points PS_STN_WET_WELL_LEVEL
and the like. Those were never CI Server names. They came from
gen_scada_points.py on the PLC side, which describes its own output as "a
starting point for CI Server configuration, not a CI Server import file",
and the CI Server generator ignored them entirely - it names points from
its own address-keyed table. The names appear in no .qli export, and
nowhere else in the WRPS project at all.
This is the corrected delivery. ci-server-points.csv is keyed on the CI
Server ITEM each point feeds, with the Modbus station and the point name
beside it, so all four namespaces are visible in one row:
LIT-101 instrument tag
%QW0 PLC symbol and address
WRPS_PLC:STN_LEVEL CI Server point - station, colon, point name
AID.WRPS.STN.LEVEL CI Server item - what the historian is keyed on
The 49 rows were joined onto the .qli exports by function code and Modbus
address: 49 matched, 0 unmatched. The item and point names are read from
the exports, not invented here.
The six .qli files come too, unchanged. gen_historian_items.py needs them,
and until now they existed only on one laptop behind a --wrps path - so
the one file the rules say must never be hand-edited could be regenerated
by exactly one person. 84 KB buys the team that back.
These are uncontrolled documents: no revision, no effective date, no
approval. The README says so, says how to ask for a fresh set, and records
the two things that bite - that PS_ survives legitimately as four poll
group names, and that his_group.qli and the live export disagree about the
sample rates, with the live one winning.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reverts 038cdc5. The source itself was reachable and the demo data was
sound, but the design questions underneath it were not settled: whether it
belongs in Cube or in plain SQL once Cube is repointed at imh at Phase 4,
whether it is append-only, how an entry is authored and authenticated, and
what a real query window over it looks like. Better out than half-committed.
Kept from 038cdc5:
H02 the 'last 3 days' wording. That change was asked for on its own and
has nothing to do with the shift log. It still fails, on the
_contains_quantity false positive.
Restored:
N04 back to the shift log question. With no such source it tests what it
was written to test again: "no such source" is not "no records found".
S01, S02 removed with the feature.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A shift log is continuously updated, which rules out both stores we already
have. Not doc_chunks: that store is built on doc_number + revision +
effective_date and nothing in it is citable until a human confirms the header,
so a record changing every shift would wear out the gate that makes procedure
citations trustworthy. Not the historian: we hold read-only on imh and may not
write to it at all, it is keyed on the CI Server item and a log entry has no
item, and its seven-day retention would expire the one source that could
outlive it.
So a table in pg-ai, read on the historical lane because that lane's contract
already fits it - a time window, rows, and "no records found" when there are
none. Entries are rows, NOT citations: a citation is a controlled document with
a confirmed revision and an operator's note is not one however true it is. The
Citation contract is untouched.
Routed on the question text rather than a classifier label. Adding a sixth
class to the five-way classifier - the most safety-relevant component in the
stack - to reach one demo source would be a poor trade. Case S01 pins the
trigger.
Demo entries are anchored to the start of today in site local time, so a
re-run always lands them inside the rolling seven days and the shift label
always matches the clock. The alarm fixtures are absolute and have gone stale;
this cannot. Future-dated entries are dropped, and the load asserts the window
and the count rather than trusting them.
Eval, 78 -> 80 cases:
S01, S02 the shift log, whole and filtered by equipment
N04 repointed at the environmental discharge licence. The old wording
asked about the shift log, which now exists, so it had stopped
testing "no such source" - a different answer from "no records
found", and the distinction is the point of the case.
H02 window changed to a relative one. It still fails, on a false
positive in _contains_quantity: "the last 3 days" reads as a
fabricated figure, so the correct zero-row answer is rejected.
Left for its own change.
80 cases: 97.5% overall, 100% classification, p95 5257 ms. One contract
violation (H02), so the Phase 8 gate is still not met.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The query window is a rolling seven days by design, and stays that way against
the real SQL historian - it is not parsed out of the question. Two things I
wrote assumed otherwise.
H29 was wrong in two ways. It demanded the words "retention" and "seven days",
and it BANNED "no records found" - which contradicts CLAUDE.md, where that is
the required wording for zero rows. With a fixed one-week window June genuinely
has zero rows in what was queried, so the phrase is correct rather than evasive.
It also described the missing piece as question-window parsing, which is not a
gap but the design.
Rewritten to pin the risk that actually exists: SUBSTITUTION. A question naming
June must never be answered with this week's figure wearing June's label.
Answering "there were 14" would be exactly that, and nothing downstream could
catch it. The observed answer refuses the substitution and states the window it
used, so the case now passes on its merits rather than being red by default.
must_not is the only half run_eval enforces, so the banned phrases are ones
that appear only on a substitution.
outside_retention is reframed as what it is: a GUARD, always false while every
caller asks for seven days, there so a caller who later passes a longer window
cannot get an empty result that reads as "nothing happened".
REQUESTS.md and the removal guide now say that extending retention on its own
is inert - the window would still ask for a week of a longer history. Both
halves are needed, or neither.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three open Phase 5 findings were one defect: the stand-in was keyed on
CI Server POINT names (PS_STN_WET_WELL_LEVEL) when the historian is keyed on
CI Server ITEM names (AID.WRPS.STN.LEVEL). Modbus carries register numbers,
not names, so those two layers are free to differ - and do. Reconciling
against the register map, as planned, would only have proved the first three
namespaces agreed with each other.
Rebuilt from WRPS/05-scada/modbus, so item names, sample rates, retention and
timestamp semantics come from the machine rather than from a guess.
(a) Level tag does not join. PS_STN_WET_WELL_LEVEL becomes a tag row in its
own right; LIT-101 is marked NOT HISTORISED - a field input on %IW0 that
never reaches SCADA. It was the only seed row carrying two addresses.
public.historian_items holds the item-to-tag mapping, generated by
scripts/gen_historian_items.py and enforced non-empty at generate, at
deploy and at verify.
(b) first_alarm/last_alarm returned UTC. Converted inside the measure, so it
stays in Cube and happens once. Aggregate first, convert after - the other
order picks the wrong row across a DST fall-back. Returned as a formatted
string with a companion site_timezone measure. Storage being UTC is now
confirmed, not assumed: all 49 points carry TIME_ZONE "Date+time GMT" and
every history group CORRECT_DAYLIGHT=0. This answers Phase 4 task 4.
(c) High level alarm filed against the wrong equipment. Both sides were right
about different things; the defect was asserting equipment twice. The
history now carries no equipment column at all - faithful, since CI
Server's section tree stops at the station and three pumps. Equipment is
reached bit -> tag -> equipment via public.alarm_bits.
Alarms are derived, not stored: CI Server's ALARM_HISTORY group is empty
because every item imports with alarming off. Decomposing the alarm word needs
no configuration that does not exist.
Three things the SCADA config changed that were never filed as faults:
- retention is 7 days, not 30. The advisory path was reporting a month of
evidence drawn from a week of data
- the analogue rate is 5 s, not 60. Two measures multiplied sample counts by
a hardcoded 60 - a twelvefold overstatement that read as plausible
- the deadband warning in process_values.yml was wrong and was steering
people away from the correct measure
db/002_fixtures.sql now asserts its own counts at load and cross-checks the
alarm derivation against two independent signals. Those prove the pipeline,
not the plant.
db/README-standin-historian.md documents removal: the seam between generation
and contract, and twelve assumptions about imh that are NOT confirmed. Two of
them fail silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ProcedureIdentity requires a title and an authorising role, and neither was
stored anywhere. The answer writer was asked for both, read them off whatever
chunk retrieval happened to return, and returned "" whenever the header chunk
was not among them.
They belong in the row for the same reason doc_number and revision do: they
are facts about the controlled document, established once when a human
confirms the header, not something to re-derive per question from whatever
text was retrieved. Denormalised onto every chunk exactly as the existing
header fields are - ingest replaces every chunk of a source_file in one
transaction, so they cannot drift within a document.
complete() deliberately still requires only doc_number, revision and
effective_date. A missing title makes an answer less useful; a wrong revision
sends somebody to the wrong document. --assume-yes must keep refusing on the
second and tolerate the first.
controlled_copy_location is NOT in the schema. It is a site fact, identical on
every row, and the one field where an invented value sends a person to a place
that does not exist. It is CONTROLLED_COPY_LOCATION in api.env, defaulting to
a string that names who to ask.
The authorising-role pattern requires the colon: without it the lazy gap
swallowed the field name and captured "role: Station Maintenance Supervisor"
as the value, which the first run caught.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ingest_file() inserted every chunk with superseded = FALSE, so replacing a
document's chunks reset its withdrawal. One `ai-ingest --all` made every
superseded revision citable again - including the old revision of a procedure -
silently, and only after a bulk run, so a supersede survived exactly until the
next full ingest.
- ingest_file() reads the existing state with superseded_state() before the
delete and carries it through the insert. A withdrawn document is
re-ingested as withdrawn, and logs that it did so.
- --all skips withdrawn documents, so a bulk run does not spend an embeddings
call on a document that will not be cited either way. --include-superseded
overrides it; the chunks still come back withdrawn.
- --restore DOC_NUMBER REVISION is the counterpart to --supersede, refusing
while another revision of the same document is live. Without it the
conservative default would be a dead end - mark_superseded() only ever sets
TRUE, so there was no way back.
- Rule 5 in the module docstring, beside the other four.
Corrects a claim in the Phase 9 design: 16.10 said moving the withdrawn file
out of /datadisk/ai-docs "is not tidying" because otherwise --all resurrects it.
That was true when written and is not now. The guard belongs in the ingest code,
because a rule that depends on somebody remembering to move a file is not a
rule. The move stays as archival housekeeping and is documented as such in
16.10, db/005_doc_actions.sql and README.md.
Phase 3's gate gains the proof: supersede a revision, run --all again, ask the
question that used to cite it.
Not executed anywhere - no Postgres or Docker on this machine.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ai-ingest built its DSN from PGUSER/PGPASSWORD and takes its environment from
~/ai/api.env, where PGUSER=agent_ro - SELECT and nothing else, deliberately,
because it is what the answer path runs as. So
docker compose -f ~/ai-compose.yml run --rm ai-ingest --all
connected as a role that cannot INSERT INTO doc_chunks, and Phase 3 was
unrunnable exactly as the README documents it. Nothing had reached Phase 3 yet,
so nobody had hit it.
The failure would also have landed at the worst possible moment: at the final
INSERT, after the Docling parse, after a person had typed the header
confirmations for every file, and after a billed embeddings call - with a
permission error naming no cause.
- ingest_rw moves to 003_roles.sql, at Phase 1 with the other roles. It is
not a Phase 9 concept; ingestion has needed a writing role since Phase 3
and never had one. 004 keeps only its grants on the upload queue, and its
idempotent role creation so it still applies to an older database.
- ingest.py connects through INGEST_DB_USER / INGEST_DB_PASSWORD, falling
back to PGUSER only for a local shell where one pair is set.
- require_write_access() checks INSERT, UPDATE and DELETE on doc_chunks
before anything is parsed or embedded, and fails with the fix in the
message. Falling back to PGUSER cannot smuggle agent_ro past it.
- Keyword connection parameters rather than a URL: a generated password
containing @ or / breaks a DSN string silently.
- A missing doc_chunks now says "apply 001_schema.sql" instead of raising
UndefinedTable.
Phase 1's gate gains the check that would have caught this: ingest_rw must be
able to write doc_chunks. An ingestion role that cannot write is the same class
of failure as an API role that can - it just surfaces two phases later.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operators cannot add a document today: ingestion is CLI-only, needs a host
login and a TTY for confirm_header(), /datadisk/ai-docs is mounted read-only,
ai-api has no identity, and nothing in the stack has a role that can write
doc_chunks. This designs the way in, the way out, and control over what is in
the retrieval pool. Design and schema only - no router, worker or UI code yet.
Documents in (16.1-16.9, db/004):
upload -> pre-scan -> review -> approve -> published, with the header
confirmation moved from a terminal prompt to a review screen and recorded
rather than discarded. A CHECK constraint refuses an approved row without a
confirmed number, revision and effective date, so an API bug cannot skip it.
Three roles: agent_ro unchanged, uploads_rw writes the queue only, ingest_rw
writes doc_chunks and has no HTTP surface.
Documents out (16.10-16.11, db/005):
--supersede needs a revision to keep, so a cancelled procedure cannot be
withdrawn at all. Adds withdraw (immediate, reversible, audited), restore
(refused while another revision is live) and purge (off by default). A
column grant plus a trigger let the web-facing role make a document less
citable and never more.
The pool (16.13-16.15, db/006):
pool_enabled, orthogonal to superseded: one is a claim about the document,
the other about the corpus. Retrieval requires both, so re-enabling a
withdrawn document does not make it citable. Named profiles and a
per-request override let a demo trim the corpus without mutating state on a
shared live host, and every reduced-pool answer carries a banner with the
document count, following the used_fixture_data precedent.
Two existing defects found and documented while designing this:
- ai-ingest takes PGUSER=agent_ro from api.env, a SELECT-only role, so the
Phase 3 command in the README cannot write doc_chunks (16.1).
- ingest_file() always inserts superseded = FALSE, so `--all` re-ingests a
superseded revision as live. --supersede survives only until the next bulk
run (16.10).
One commit rather than three: the upload, withdrawal and pool designs
interleave in the same spec, README and compose files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
agent_ro could still create temporary tables: Postgres grants TEMP on every
database to PUBLIC, and revoking it from the role alone does not remove what
PUBLIC grants. Revoke it from PUBLIC too. postgres is a superuser and keeps
its temp tables, so the seed load is unaffected.
Nothing but the superuser could read schema fixture: 003_roles.sql grants only
on schema public, and the fixture schema is created afterwards. Grant read to
agent_ro and cube_rw. The grants live in 002_fixtures.sql because that file
opens with DROP SCHEMA fixture CASCADE, which would destroy grants held
anywhere else on every fixture reload.
Both re-verified on pg-ai: 11/11 privilege tests and 10/10 fixture access
tests pass, reads work for both roles, writes are rejected for both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Build spec and host brief carried in from C:\Claude and WRPS/02-env; the
plant model (equipment, tags, alarm bitmask, enums, unit conversions) is
derived from WRPS/04-plc/register-map.csv, WRPS/05-scada/modbus/scada-points.csv
and WRPS-CTL-003.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
