45 of the 65 rows in db/seed/tags.csv had a PS_* name as their PRIMARY
KEY, and historian_items.csv, alarm_bits.csv, the fixtures, two Cube
models and the exam all referenced them. They are gone. The tag seed is
now keyed on the CI Server item for everything the historian carries -
AID.WRPS.STN.LEVEL - and on the instrument tag for the 20 field devices
that never reach SCADA.
gen_historian_items.py reads db/seed/scada-source/ by default, so it runs
for anyone with a clone: --wrps is now --source. The tag match is the item
name itself, an identity lookup, and the TAG_FOR_ITEM special case is
deleted - all 49 item names are unique, which the old point names were
not. scada_point in the output is replaced by ci_station, ci_point and
poll_group; 001_schema.sql and deploy.sh's upsert follow.
Regenerated, and it comes out the same shape it went in: 49 items, 45
answerable, 4 deliberately excluded, three groups at 5 s, 30 s and on
change. Every historian_items.tag_id and alarm_bits.tag_id resolves to a
tags.csv row. No duplicate keys.
THE ONE NAME THAT WAS AMBIGUOUS, AND NEARLY COST US
PS_STN_HIGH_LEVEL_ALARM named two different things in the old delivery:
the high level alarm STATUS BIT on coil 10, and the alarm SETPOINT on
holding register 1032. Building the rename map from that file kept
whichever came last, so the status bit was silently renamed onto the
setpoint. check_mapping() refused to write and named the item that no
longer resolved - which is the only reason this is a paragraph in a commit
message rather than a defect.
Had it gone through, alarm bit 0 - wet well high level - would have
pointed at the setpoint. "How many high level alarms last week" would have
counted setpoint changes and returned a small, plausible, confident,
wrong number. That check exists because the same ambiguity caused the
first Phase 5 finding in August. It has now bitten twice.
NOT YET VERIFIED: pytest api/tests could not be run here - this machine
has neither fastapi nor psycopg. The tests are unchanged and reference no
PS_ name, but they have not been run. pg-ai on lin001 still holds the old
keys and must be reloaded, Cube pre-aggregations rebuilt, and the Phase 1
gate re-run. eval/testset.jsonl changed, so the 78-case exam - never yet
run - should be run after.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three open Phase 5 findings were one defect: the stand-in was keyed on
CI Server POINT names (PS_STN_WET_WELL_LEVEL) when the historian is keyed on
CI Server ITEM names (AID.WRPS.STN.LEVEL). Modbus carries register numbers,
not names, so those two layers are free to differ - and do. Reconciling
against the register map, as planned, would only have proved the first three
namespaces agreed with each other.
Rebuilt from WRPS/05-scada/modbus, so item names, sample rates, retention and
timestamp semantics come from the machine rather than from a guess.
(a) Level tag does not join. PS_STN_WET_WELL_LEVEL becomes a tag row in its
own right; LIT-101 is marked NOT HISTORISED - a field input on %IW0 that
never reaches SCADA. It was the only seed row carrying two addresses.
public.historian_items holds the item-to-tag mapping, generated by
scripts/gen_historian_items.py and enforced non-empty at generate, at
deploy and at verify.
(b) first_alarm/last_alarm returned UTC. Converted inside the measure, so it
stays in Cube and happens once. Aggregate first, convert after - the other
order picks the wrong row across a DST fall-back. Returned as a formatted
string with a companion site_timezone measure. Storage being UTC is now
confirmed, not assumed: all 49 points carry TIME_ZONE "Date+time GMT" and
every history group CORRECT_DAYLIGHT=0. This answers Phase 4 task 4.
(c) High level alarm filed against the wrong equipment. Both sides were right
about different things; the defect was asserting equipment twice. The
history now carries no equipment column at all - faithful, since CI
Server's section tree stops at the station and three pumps. Equipment is
reached bit -> tag -> equipment via public.alarm_bits.
Alarms are derived, not stored: CI Server's ALARM_HISTORY group is empty
because every item imports with alarming off. Decomposing the alarm word needs
no configuration that does not exist.
Three things the SCADA config changed that were never filed as faults:
- retention is 7 days, not 30. The advisory path was reporting a month of
evidence drawn from a week of data
- the analogue rate is 5 s, not 60. Two measures multiplied sample counts by
a hardcoded 60 - a twelvefold overstatement that read as plausible
- the deadband warning in process_values.yml was wrong and was steering
people away from the correct measure
db/002_fixtures.sql now asserts its own counts at load and cross-checks the
alarm derivation against two independent signals. Those prove the pipeline,
not the plant.
db/README-standin-historian.md documents removal: the seam between generation
and contract, and twelve assumptions about imh that are NOT confirmed. Two of
them fail silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lin001 is 2 vCPU with 3.8 GiB and no swap, shared with 27 other containers.
Cube was rebuilding three pre-aggregations over static fixture data on 10, 10
and 30 minute keys, and its own log shows what that cost:
"Previous interval #19593 was not finished with 30000 interval"
"Interval #19594 finished after 00:02:50"
A refresh taking 2m50s scheduled every 30 seconds, at interval #19594 -
overlapping and never catching up, for days. Every rebuild produced a
byte-identical result, because the fixtures do not change.
24 hours until imh makes the data genuinely live, at which point these get
tuned deliberately rather than left at a number that was never chosen. Cube
also suggests fewer partitions via rollup_lambda; that is a modelling change,
not a demo-day one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both were found hand-verifying the measures on fixtures, and both are left
unfixed deliberately: fixing either now means guessing at what imh contains,
and a guess baked into the seed data or the models is harder to find later than
an open finding. Flagged at Phase 4, in the README, in the model files at the
point of use, and as eval cases that fail until they are settled.
(a) The wet well level tag does not join, and fails as "no records found".
History is keyed PS_STN_WET_WELL_LEVEL, which process_values.yml hardcodes
in seconds_above_high_level_alarm and seconds_above_lshh. tags.csv carries
that name only as an ALIAS of LIT-101, so public.tags has no row with that
tag_id and all 43,201 level rows - a third of the history, on the most
important tag at this station - are unreachable from a tag-level lookup.
Resolve "wet well" -> WW-101 -> LIT-101 -> filter history on LIT-101 ->
zero rows -> "no records found".
That is the failure mode worth being loudest about: it is the safety
behaviour, produced by a key mismatch rather than by an absence of data,
and indistinguishable from the real thing on screen. Filtering by
equipment_id works, so whether a level question fails depends on which path
the agent takes.
The two flow tags use the opposite, self-consistent convention -
PS_STN_INFLOW is a row in its own right and FIT-201 is marked NOT
HISTORISED - so applying that to level is the likely fix. It still waits
for the register map and for imh to say what CI Server historises the point
as. Seed, hardcoded model names and 002_fixtures.sql change together.
(b) alarms.first_alarm and last_alarm return UTC, not SITE_TIMEZONE.
Cube converts time dimensions to the query timezone; a min/max measure over
a timestamp comes back unconverted. The Sydney day bucket for 2026-08-01
returns 2026-07-31T20:00:35 - the right instant, ten hours and one calendar
day out, beside a bucket label that IS in site time.
This breaks "convert to SITE_TIMEZONE exactly once, in Cube", and the fix
has to stay in Cube - the API compensating with timezone arithmetic is the
thing that rule exists to prevent. Which fix is right depends on whether
imh stores UTC or local, which is Phase 4 task 4. Until then these two must
not be quoted to an operator as a clock time.
Phase 4's gate gains an item for each. Everything else verified in this pass -
alarms, operations, the equipment join, alias resolution and the Sydney/UTC day
boundaries - matched hand-written SQL exactly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Hand-verifying the measures against the fixtures on lin001, per the Phase 5
gate. Two of them had never executed anywhere, and both failed outright rather
than returning a wrong number - which is the good version of this, but they
failed at the point an operator asks a question, not at review.
- time_weighted_avg put LEAD() inside SUM(). Postgres rejects that flatly:
"aggregate function calls cannot contain window function calls". The per
sample duration now comes from the cube's source query, which changes
sql_table to sql, and the measure just sums value * duration over duration.
The last sample of each tag gets a NULL duration and SUM skips it, which is
correct - how long it stood is not yet known.
This is the measure that matters most later. On the regular one-minute
fixtures it agrees with avg_value to thirteen decimal places
(42.45934027777778 against 42.45934027777775), which proves it is wired up
and proves nothing about imh, where the deadband makes samples irregular
and the two will not agree. Re-verify it there.
- p95_value applied the quality filter through a Cube measure `filters:`
block, which lands outside the aggregate and cannot work on an ordered-set
aggregate: "column process_values.quality must appear in the GROUP BY
clause". Folded into the CASE inside PERCENTILE_CONT, whose NULL handling
does the exclusion.
Also: the priority dimension said only SPILL and PUMP_TRIP are priority 1,
while the data has LEVEL_SIGNAL_FAULT at priority 1 too - correctly, losing the
level signal on a well that can spill is a priority 1 condition. That comment is
the line an engineer reads when checking a priority_1_count, so it disagreeing
with the data matters more than its length suggests.
eval cases H24 and H25 record the two failures, added before the fix.
Verified against hand-written SQL on the same pinned windows: p95_value
61.104999999999976 and time_weighted_avg 42.45934027777778 both match to the
floating point tail, as do sample_count, avg_value, max_value, min_value,
seconds_above_high_level_alarm (7680 = 128 samples x 60) and every measure in
alarms and operations.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Build spec and host brief carried in from C:\Claude and WRPS/02-env; the
plant model (equipment, tags, alarm bitmask, enums, unit conversions) is
derived from WRPS/04-plc/register-map.csv, WRPS/05-scada/modbus/scada-points.csv
and WRPS-CTL-003.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
