YAU.POC/yau-plant-assistant
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
yau-plant-assistant/db/002_fixtures.sql
Claude 3d19b186da Key the whole seed on the CI Server item name 
45 of the 65 rows in db/seed/tags.csv had a PS_* name as their PRIMARY
KEY, and historian_items.csv, alarm_bits.csv, the fixtures, two Cube
models and the exam all referenced them. They are gone. The tag seed is
now keyed on the CI Server item for everything the historian carries -
AID.WRPS.STN.LEVEL - and on the instrument tag for the 20 field devices
that never reach SCADA.

gen_historian_items.py reads db/seed/scada-source/ by default, so it runs
for anyone with a clone: --wrps is now --source. The tag match is the item
name itself, an identity lookup, and the TAG_FOR_ITEM special case is
deleted - all 49 item names are unique, which the old point names were
not. scada_point in the output is replaced by ci_station, ci_point and
poll_group; 001_schema.sql and deploy.sh's upsert follow.

Regenerated, and it comes out the same shape it went in: 49 items, 45
answerable, 4 deliberately excluded, three groups at 5 s, 30 s and on
change. Every historian_items.tag_id and alarm_bits.tag_id resolves to a
tags.csv row. No duplicate keys.

THE ONE NAME THAT WAS AMBIGUOUS, AND NEARLY COST US

PS_STN_HIGH_LEVEL_ALARM named two different things in the old delivery:
the high level alarm STATUS BIT on coil 10, and the alarm SETPOINT on
holding register 1032. Building the rename map from that file kept
whichever came last, so the status bit was silently renamed onto the
setpoint. check_mapping() refused to write and named the item that no
longer resolved - which is the only reason this is a paragraph in a commit
message rather than a defect.

Had it gone through, alarm bit 0 - wet well high level - would have
pointed at the setpoint. "How many high level alarms last week" would have
counted setpoint changes and returned a small, plausible, confident,
wrong number. That check exists because the same ambiguity caused the
first Phase 5 finding in August. It has now bitten twice.

NOT YET VERIFIED: pytest api/tests could not be run here - this machine
has neither fastapi nor psycopg. The tests are unchanged and reference no
PS_ name, but they have not been run. pg-ai on lin001 still holds the old
keys and must be reloaded, Cube pre-aggregations rebuilt, and the Phase 1
gate re-run. eval/testset.jsonl changed, so the 78-case exam - never yet
run - should be run after.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 19:00:38 +10:00

856 lines
43 KiB
PL/PgSQL
Raw Blame History

-- =============================================================================
-- 002_fixtures.sql
--
-- ############################################################
-- ## FIXTURE DATA. NOT REAL PLANT HISTORY. DO NOT QUOTE. ##
-- ############################################################
--
-- Interim stand-in for `imh` (yau-sls-poc-imh), which is not built yet.
-- Everything in the `fixture` schema is generated. A number produced from
-- these tables is a test result about the pipeline, never a fact about the
-- station.
--
-- WHY THIS FILE WAS REWRITTEN (2026-08-31)
-- ----------------------------------------
-- The previous version was keyed on names delivered as PS_STN_* - which were
-- never CI Server names at all, but a proposal from the PLC register map. The
-- historian is keyed on CI Server ITEM names (AID.WRPS.*). Four namespaces
-- describe the same measurement and only the last is what `imh` stores:
--
-- LIT-101 instrument tag WRPS/01-design-doc
-- %QW0 PLC symbol WRPS/04-plc/register-map.csv
-- AID.WRPS.STN.LEVEL CI Server point WRPS/05-scada/modbus/scada-points.csv
-- AID.WRPS.STN.LEVEL CI Server ITEM WRPS/05-scada/modbus/wrps_item_df.qli
--
-- Being keyed two layers off the real one is what produced all three of the
-- Phase 5 findings. They are not patched here; the substitution that caused
-- them is removed. See the notes on each below.
--
-- WHAT IS NOW TAKEN FROM THE SCADA CONFIGURATION RATHER THAN INVENTED
-- * item names, units, gains, sections wrps_item_df.qli, scada-points.csv
-- * which items are historised at all item_his.qli
-- * sample rates and retention export_his_group.qli (the LIVE one)
-- * timestamp semantics wrps_modbus_point_df.qli
--
-- All of it is pinned into db/seed/historian_items.csv by
-- scripts/gen_historian_items.py and loaded into public.historian_items by
-- scripts/deploy.sh. THIS FILE HARDCODES NO ITEM NAME IT DID NOT GET FROM
-- THERE, and no sample interval at all.
--
-- THREE PROPERTIES OF THE REAL HISTORIAN THAT THIS ONE COPIES
-- ----------------------------------------------------------
-- 1. RETENTION IS SEVEN DAYS. Every WRPS history group carries
-- LIFE_TIME "1 weeks". The old fixtures generated thirty days, which is
-- why questions about last month worked here and would have failed on the
-- real thing. They now fail here too. That is the point: see REQUESTS.md,
-- which asks the historian owner to extend it, because a plant assistant
-- that cannot answer "last month" is of limited use.
--
-- 2. SAMPLES ARE REGULAR, NOT DEADBAND-COMPRESSED. Every group has
-- DATA_COMP = 0 and every item STORE_DEADBAND = 0, with
-- COL_STOR_TYPE "Scan/Time". The previous comment in
-- cube/model/process_values.yml warned that real history would be
-- irregular and that a plain average would therefore be biased. That is
-- true only of WRPS_EVENT, which is Event/Item and genuinely on-change.
--
-- 3. TIMESTAMPS ARE UTC. Not an assumption: all 49 Modbus points carry
-- TIME_ZONE "Date+time GMT" in wrps_modbus_point_df.qli, and every WRPS
-- history group carries CORRECT_DAYLIGHT = 0. Store UTC, convert once in
-- Cube. Finding (b) is fixed on that basis.
--
-- WHAT IS STILL A GUESS, AND MUST BE CONFIRMED AT THE PHASE 4 GATE
-- * the SQL Server table and column names `imh` exposes these items as.
-- The SHAPE below is now right - one item-keyed history table, alarms and
-- operations derived rather than stored - but the names are ours.
-- * whether `imh` exposes CI Server's built-in ALARM_HISTORY group at all.
-- It is configured on the server but every item imports with alarming OFF
-- and limits at 0 (05-scada/modbus/README.md), so it is empty. Alarms
-- here are derived from the alarm word instead, which needs no SCADA
-- configuration that does not exist.
--
-- >>> REMOVING THIS: db/README-standin-historian.md <<<
-- It carries the seam between generation and contract, an inventory of what to
-- delete versus repoint, and a table of TWELVE ASSUMPTIONS about imh that are
-- NOT confirmed - table names, column types, how quality is expressed, whether
-- values arrive in engineering units. Two of them (A5 quality codes, A10 the
-- 32767 sentinel) fail SILENTLY, producing a plausible wrong number rather than
-- an error. Do not connect imh without working through that table.
--
-- HOW IT IS SWITCHED OFF: USE_FIXTURES=true in ~/ai/api.env points Cube here.
-- Set it false and repoint CUBEJS_DB_* at imh. Every Cube model file carries a
-- fixture/real note at the top; check them all when you flip it.
--
-- Every fixture table carries an is_fixture column, defaulted TRUE, so a row
-- that reaches the UI can be traced back to here. The API surfaces it as a
-- banner. Do not remove it as tidying-up.
-- =============================================================================
DROP SCHEMA IF EXISTS fixture CASCADE;
CREATE SCHEMA fixture;
COMMENT ON SCHEMA fixture IS
'GENERATED FIXTURE DATA standing in for imh. Not real plant history. Keyed on '
'CI Server item names. See db/002_fixtures.sql.';
-- public.historian_items must be loaded before this file runs: it is what
-- says which items exist, which group each belongs to and how often each is
-- sampled. scripts/deploy.sh loads it with the other seeds.
DO $$
BEGIN
IF (SELECT count(*) FROM public.historian_items) = 0 THEN
RAISE EXCEPTION
'public.historian_items is empty - load db/seed/historian_items.csv '
'before running this file (scripts/deploy.sh does it for you)';
END IF;
END $$;
-- =============================================================================
-- The build clock.
--
-- Everything is generated relative to one origin, held in a table rather than
-- recomputed from now() in each statement. Two reasons, and the second is the
-- one that bites: a statement-by-statement now() drifts within a long load, so
-- the level series and the alarms derived from it would be computed against
-- slightly different clocks and the derived alarm count would not be
-- reproducible. Truncating to the hour also makes every count below EXACT
-- rather than approximate, which is what lets this file assert them.
-- =============================================================================
CREATE TABLE fixture.build_meta (
origin TIMESTAMPTZ NOT NULL, -- UTC, start of the retained window
horizon_days INT NOT NULL,
built_at TIMESTAMPTZ NOT NULL DEFAULT now(),
note TEXT
);
INSERT INTO fixture.build_meta (origin, horizon_days, note)
SELECT date_trunc('hour', now()) - interval '7 days', 7,
'Retention copied from CI Server: every WRPS history group is LIFE_TIME '
'"1 weeks". Questions older than this return no rows, here and on imh.';
-- =============================================================================
-- The generating functions.
--
-- The plant is PRESCRIBED, not simulated: the wet well level is a closed-form
-- function of time and everything else is derived from it. That is deliberate.
-- A simulation produces numbers nobody can check by hand; this produces
-- numbers an engineer can verify with a calculator, which is the only reason
-- the assertions at the foot of this file mean anything.
--
-- The shape is a 140-minute pump-down cycle - 55 minutes filling on inflow,
-- 85 minutes drawing down - riding on a diurnal inflow, with a wet-weather
-- event in the middle of the week. Do not add realism. Realism in fixtures is
-- how a fixture number ends up in a slide.
-- =============================================================================
-- Peak level reached by pump-down number n, percent of the weir crest.
-- Cycles 31-46 are the wet-weather event: those peaks reach the high level
-- alarm at 86.7 %, most reach LSHH at 91.7 %, and two go over the weir.
-- Outside it nothing comes close to alarming.
CREATE FUNCTION fixture.f_peak(n int) RETURNS double precision AS $$
SELECT CASE WHEN n BETWEEN 31 AND 46
THEN 84.0 + (n % 7) * 3.0 -- 84.0 .. 102.0
ELSE 68.0 + (n % 5) * 1.4 -- 68.0 .. 73.6
END
$$ LANGUAGE sql IMMUTABLE;
-- Wet well level, percent of the weir crest, s seconds after the origin.
--
-- The level signal fault at bit 13 FREEZES the transmitter: the historian goes
-- on sampling and the samples go on reading the last good value. Modelled by
-- evaluating the level at the fault's start time for its duration, and the
-- samples are flagged BAD so that every Cube measure excludes them. This is
-- the only thing in the fixtures that exercises the quality filter.
CREATE FUNCTION fixture.f_level(s int) RETURNS double precision AS $$
SELECT CASE WHEN v.phase_min < 55
THEN 16.7 + (v.pk - 16.7) * v.phase_min / 55.0
ELSE v.pk - (v.pk - 16.7) * (v.phase_min - 55) / 85.0
END
FROM (
SELECT (e.eff % 8400) / 60.0 AS phase_min,
fixture.f_peak((e.eff / 8400)::int) AS pk
FROM (SELECT CASE WHEN s >= 558000 AND s < 561600
THEN 558000 ELSE s END AS eff) e
) v
$$ LANGUAGE sql IMMUTABLE;
-- Units running. Zero while filling; during the draw-down the cascade follows
-- the documented start levels - 66.7 % duty, 75.0 % assist 1, 83.3 % assist 2.
CREATE FUNCTION fixture.f_pumps(s int) RETURNS int AS $$
SELECT CASE
WHEN (s % 8400) / 60.0 < 55 THEN 0
WHEN fixture.f_level(s) >= 83.3 THEN 3
WHEN fixture.f_level(s) >= 75.0 THEN 2
ELSE 1
END
$$ LANGUAGE sql IMMUTABLE;
-- Station inflow, m3/h. Diurnal with a morning and an evening peak, lifted
-- through the wet-weather cycles.
CREATE FUNCTION fixture.f_inflow(s int) RETURNS double precision AS $$
SELECT round((
180.0
+ 90.0 * sin((s - 21600) * 2 * pi() / 86400.0)
+ 40.0 * sin(s * 4 * pi() / 86400.0)
+ CASE WHEN (s / 8400) BETWEEN 31 AND 46 THEN 420.0 ELSE 0.0 END
)::numeric, 1)::double precision
$$ LANGUAGE sql IMMUTABLE;
-- Total discharge, m3/h. Approximately 120 L/s per unit against the 22 m
-- static lift: 120 x 3.6 = 432 m3/h.
CREATE FUNCTION fixture.f_discharge(s int) RETURNS double precision AS $$
SELECT fixture.f_pumps(s) * 432.0
$$ LANGUAGE sql IMMUTABLE;
-- Duty unit for pump-down n. Duty rotates on lowest accumulated run hours,
-- which over a long window is round-robin.
CREATE FUNCTION fixture.f_duty(s int) RETURNS int AS $$
SELECT 1 + ((s / 8400) % 3)
$$ LANGUAGE sql IMMUTABLE;
-- =============================================================================
-- Injected discrete conditions.
--
-- The level-driven alarms - high level, LSHH, spill - fall out of f_level and
-- are not listed here. These are the ones that have no analogue signature: the
-- trips, the seal leak, the vibration alarm, the frozen level transmitter and
-- a rejected setpoint write.
--
-- BOTH TRIPS START ON A CYCLE BOUNDARY, the instant a pump-down ends, and are
-- reset well inside the following fill. That keeps a tripped unit from ever
-- overlapping a running one, so the discharge arithmetic stays exactly
-- checkable. It is a real scenario - a unit that trips as the set shuts down
-- and is reset before the next start - and it is a deliberate simplification.
-- =============================================================================
CREATE TABLE fixture.injected_condition (
bit INT NOT NULL REFERENCES public.alarm_bits(bit),
start_s INT NOT NULL,
end_s INT NOT NULL,
narrative TEXT
);
INSERT INTO fixture.injected_condition (bit, start_s, end_s, narrative) VALUES
( 5, 84000, 86700, 'PU-302 tripped - no flow 20 s after start (PIT-321 below 150 kPa). Reset by operator command 45 minutes later.'),
( 4, 277200, 279900, 'PU-301 tripped on motor thermal TE-312 during the wet weather event. Reset by operator command.'),
(10, 442800, 450000, 'PU-301 bearing vibration above the 7.1 mm/s alarm threshold for two hours. Below the 11.0 mm/s trip, so the unit kept running.'),
( 9, 453600, 518400, 'PU-303 seal leak MSE-333. Alarm only - availability is unaffected and the unit went on running, per WRPS-PRO-001 section 5.5.'),
(13, 558000, 561600, 'LIT-101 frozen - no change greater than 1 mm for 10 minutes with a pump running. Level samples are flagged BAD for the hour.'),
(15, 590400, 590460, 'Setpoint write rejected - start duty level above the spill weir crest; the previous value was retained.');
-- The alarm word, exactly as the PLC assembles it: the OR of every active
-- condition. Level-driven bits come from f_level, the rest from the table
-- above. Bit 0 is mirrored by AID.WRPS.STN.HIGH_LEVEL and bit 3 by
-- AID.WRPS.STN.SPILL_ACTIVE; the assertions at the foot of this file check
-- that the mirrors agree, which is a real test of the derivation.
CREATE FUNCTION fixture.f_alarm_word(s int) RETURNS int AS $$
SELECT (CASE WHEN fixture.f_level(s) >= 86.7 THEN 1 ELSE 0 END) -- bit0
+ (CASE WHEN fixture.f_level(s) >= 91.7 THEN 2 ELSE 0 END) -- bit1
+ (CASE WHEN fixture.f_level(s) >= 100.0 THEN 8 ELSE 0 END) -- bit3
+ COALESCE((SELECT sum(1 << c.bit)::int
FROM fixture.injected_condition c
WHERE s >= c.start_s AND s < c.end_s), 0)
$$ LANGUAGE sql STABLE;
-- =============================================================================
-- fixture.item_history - THE ONLY HISTORY TABLE.
--
-- One row per sample of one item. That is all CI Server holds, and it is all
-- this holds. Alarms and pump-down operations are DERIVED from it below,
-- because that is what will have to happen against imh too.
--
-- THERE IS NO equipment_id COLUMN, AND THERE MUST NEVER BE ONE. CI Server's
-- section tree stops at the station and the three pumps; it has no wet well,
-- no weir, no manifold, no switchboard. Equipment is asserted exactly once, in
-- public.tags.equipment_id, and reached from here through
-- public.historian_items. Finding (c) was a denormalised equipment column in
-- the history disagreeing with the tag seed about which asset an alarm
-- belonged to. With one assertion there is nothing left to disagree.
-- =============================================================================
CREATE TABLE fixture.item_history (
item_name TEXT NOT NULL REFERENCES public.historian_items(item_name),
sample_time TIMESTAMPTZ NOT NULL, -- UTC ("Date+time GMT")
value DOUBLE PRECISION,
quality TEXT NOT NULL DEFAULT 'GOOD', -- GOOD | BAD | UNCERTAIN
is_fixture BOOLEAN NOT NULL DEFAULT TRUE,
PRIMARY KEY (item_name, sample_time)
);
CREATE INDEX ON fixture.item_history (sample_time);
-- --- WRPS_ONE_SEC: the five 5-second analogues ------------------------------
-- The interval is read from public.historian_items, never written here. The
-- old fixtures hardcoded 60 seconds and two Cube measures hardcoded it again
-- to convert sample counts into durations; against a 5-second group that
-- arithmetic is wrong by a factor of twelve.
INSERT INTO fixture.item_history (item_name, sample_time, value, quality)
SELECT
'AID.WRPS.STN.LEVEL',
m.origin + make_interval(secs => s),
round(fixture.f_level(s)::numeric, 1)::double precision,
-- Flagged BAD for the hour the transmitter is frozen. Cube excludes
-- non-GOOD samples from every measure.
CASE WHEN s >= 558000 AND s < 561600 THEN 'BAD' ELSE 'GOOD' END
FROM fixture.build_meta m,
public.historian_items hi,
LATERAL generate_series(0, m.horizon_days * 86400 - hi.scan_interval_seconds,
hi.scan_interval_seconds) AS s
WHERE hi.item_name = 'AID.WRPS.STN.LEVEL';
INSERT INTO fixture.item_history (item_name, sample_time, value)
SELECT hi.item_name,
m.origin + make_interval(secs => s),
CASE hi.item_name
WHEN 'AID.WRPS.STN.INFLOW' THEN fixture.f_inflow(s)
WHEN 'AID.WRPS.STN.DISCHARGE' THEN fixture.f_discharge(s)
WHEN 'AID.WRPS.STN.NET_ACCUM' THEN fixture.f_inflow(s) - fixture.f_discharge(s)
-- Percent of 50 Hz, clamped at the documented 76 % floor: below
-- 38 Hz the 22 m static lift means no delivery.
WHEN 'AID.WRPS.STN.SPEED' THEN
CASE WHEN fixture.f_pumps(s) = 0 THEN 0.0
ELSE round(least(100.0, 76.0 + (fixture.f_level(s) - 16.7) * 0.3)::numeric, 1)::double precision
END
END
FROM fixture.build_meta m,
public.historian_items hi,
LATERAL generate_series(0, m.horizon_days * 86400 - hi.scan_interval_seconds,
hi.scan_interval_seconds) AS s
WHERE hi.his_group = 'WRPS_ONE_SEC'
AND hi.item_name <> 'AID.WRPS.STN.LEVEL';
-- --- WRPS_THIRTY_SEC: the four 30-second items ------------------------------
-- Volume and time to spill reconcile with the level through the plant
-- geometry - 120 m3 per metre, weir crest at 6000 mm - so an engineer can
-- check one against the other. That cross-check is the same one the WRPS team
-- used to confirm the Modbus address base.
INSERT INTO fixture.item_history (item_name, sample_time, value)
SELECT hi.item_name,
m.origin + make_interval(secs => s),
CASE hi.item_name
WHEN 'AID.WRPS.STN.PUMPS_RUNNING' THEN fixture.f_pumps(s)::double precision
WHEN 'AID.WRPS.STN.VOL_TO_SPILL' THEN
round(greatest(0.0, 720.0 - 7.2 * fixture.f_level(s))::numeric, 0)::double precision
-- 32767 is the SENTINEL for "drawing down or holding". It is not a
-- duration and every Cube measure excludes it. Do not remove that
-- filter to make a number look tidier.
WHEN 'AID.WRPS.STN.TIME_TO_SPILL' THEN
CASE WHEN fixture.f_inflow(s) - fixture.f_discharge(s) <= 0 THEN 32767.0
ELSE least(32766.0, round((greatest(0.0, 720.0 - 7.2 * fixture.f_level(s))
/ (fixture.f_inflow(s) - fixture.f_discharge(s)) * 3600.0)::numeric, 0)::double precision)
END
WHEN 'AID.WRPS.STN.TIME_TO_LSHH' THEN
CASE WHEN fixture.f_level(s) >= 91.7 THEN 0.0
WHEN fixture.f_inflow(s) - fixture.f_discharge(s) <= 0 THEN 32767.0
ELSE least(32766.0, round(((91.7 - fixture.f_level(s)) * 7.2
/ (fixture.f_inflow(s) - fixture.f_discharge(s)) * 3600.0)::numeric, 0)::double precision)
END
END
FROM fixture.build_meta m,
public.historian_items hi,
LATERAL generate_series(0, m.horizon_days * 86400 - hi.scan_interval_seconds,
hi.scan_interval_seconds) AS s
WHERE hi.his_group = 'WRPS_THIRTY_SEC';
-- --- WRPS_EVENT: on value change only ---------------------------------------
-- Event/Item storage writes a row only when the value changes. Everything
-- below is evaluated on the 5-second grid and then reduced to its transitions,
-- which is genuinely irregular history - the one place the deadband warning in
-- cube/model/process_values.yml actually applies.
CREATE TEMP TABLE _event_grid AS
SELECT s,
fixture.f_pumps(s) AS pumps,
fixture.f_level(s) AS level,
fixture.f_duty(s) AS duty,
fixture.f_alarm_word(s) AS alarm_word,
EXISTS (SELECT 1 FROM fixture.injected_condition c
WHERE c.bit = 4 AND s >= c.start_s AND s < c.end_s) AS trip1,
EXISTS (SELECT 1 FROM fixture.injected_condition c
WHERE c.bit = 5 AND s >= c.start_s AND s < c.end_s) AS trip2,
EXISTS (SELECT 1 FROM fixture.injected_condition c
WHERE c.bit = 6 AND s >= c.start_s AND s < c.end_s) AS trip3,
-- A trip clears only on the operator's reset command, never
-- automatically. Each trip therefore ends with a command word write and
-- an acknowledgement echoed back from the PLC - momentary, 60 seconds.
EXISTS (SELECT 1 FROM fixture.injected_condition c
WHERE c.bit IN (4, 5, 6)
AND s >= c.end_s AND s < c.end_s + 60) AS reset_pulse
FROM fixture.build_meta m,
LATERAL generate_series(0, m.horizon_days * 86400 - 5, 5) AS s;
CREATE INDEX ON _event_grid (s);
-- One row per (item, value) pair over the grid, reduced to changes.
CREATE TEMP TABLE _event_values AS
SELECT item_name, s, value FROM (
SELECT 'AID.WRPS.STN.ALARM_WORD' AS item_name, s, alarm_word::double precision AS value FROM _event_grid
UNION ALL SELECT 'AID.WRPS.STN.HIGH_LEVEL', s, (level >= 86.7)::int::double precision FROM _event_grid
UNION ALL SELECT 'AID.WRPS.STN.SPILL_ACTIVE', s, (level >= 100.0)::int::double precision FROM _event_grid
UNION ALL SELECT 'AID.WRPS.STN.IN_AUTO', s, 1.0::double precision FROM _event_grid
UNION ALL SELECT 'AID.WRPS.STN.DUTY_PUMP', s, CASE WHEN pumps = 0 THEN 0 ELSE duty END::double precision FROM _event_grid
-- Station state enum: 4 Emergency (LSHH) - 3 High level - 6 Fault -
-- 2 Pumping - 1 Idle. Report the label, never the bare number.
UNION ALL SELECT 'AID.WRPS.STN.STATE', s,
(CASE WHEN level >= 91.7 THEN 4 WHEN level >= 86.7 THEN 3
WHEN trip1 OR trip2 OR trip3 THEN 6
WHEN pumps > 0 THEN 2 ELSE 1 END)::double precision FROM _event_grid
-- The reset command and the PLC's acknowledgement of it. Both are
-- momentary: they pulse for 60 s as each trip is reset and are 0 the rest
-- of the time. Command word 1 is "reset trips" - see db/seed/tags.csv.
UNION ALL SELECT 'AID.WRPS.SP.CMD_WORD', s,
(CASE WHEN reset_pulse THEN 1 ELSE 0 END)::double precision FROM _event_grid
UNION ALL SELECT 'AID.WRPS.STN.CMD_ACK', s,
(CASE WHEN reset_pulse THEN 1 ELSE 0 END)::double precision FROM _event_grid
-- Per-unit signals. run_order 0 is the duty unit, then the other two in
-- ascending number; a unit runs when the cascade has called for its
-- position. A tripped unit never runs and is never available.
UNION ALL SELECT 'AID.WRPS.PU30' || p || '.RUN_CMD', s,
((p - duty + 3) % 3 < pumps)::int::double precision
FROM _event_grid, generate_series(1,3) p
UNION ALL SELECT 'AID.WRPS.PU30' || p || '.RUNNING', s,
(((p - duty + 3) % 3 < pumps)
AND NOT (p = 1 AND trip1 OR p = 2 AND trip2 OR p = 3 AND trip3))::int::double precision
FROM _event_grid, generate_series(1,3) p
UNION ALL SELECT 'AID.WRPS.PU30' || p || '.AVAILABLE', s,
(NOT (p = 1 AND trip1 OR p = 2 AND trip2 OR p = 3 AND trip3))::int::double precision
FROM _event_grid, generate_series(1,3) p
UNION ALL SELECT 'AID.WRPS.PU30' || p || '.TRIPPED', s,
(p = 1 AND trip1 OR p = 2 AND trip2 OR p = 3 AND trip3)::int::double precision
FROM _event_grid, generate_series(1,3) p
-- Pump state enum: 6 Tripped - 3 Running - 1 Available stopped.
UNION ALL SELECT 'AID.WRPS.PU30' || p || '.STATE', s,
(CASE WHEN p = 1 AND trip1 OR p = 2 AND trip2 OR p = 3 AND trip3 THEN 6
WHEN (p - duty + 3) % 3 < pumps THEN 3
ELSE 1 END)::double precision
FROM _event_grid, generate_series(1,3) p
) v;
-- Accumulated run hours. Published as whole hours, so the value changes once
-- per hour of runtime rather than every scan. A STEP DOWN in this trend is a
-- service - command word 5 resets it - not a data error.
INSERT INTO _event_values (item_name, s, value)
SELECT item_name, s,
(start_hours + floor(run_seconds / 3600.0))::double precision
FROM (
SELECT 'AID.WRPS.PU30' || p || '.RUN_HOURS' AS item_name,
g.s,
CASE p WHEN 1 THEN 3200 WHEN 2 THEN 3350 ELSE 2980 END AS start_hours,
sum(CASE WHEN ((p - g.duty + 3) % 3 < g.pumps)
AND NOT (p = 1 AND g.trip1 OR p = 2 AND g.trip2 OR p = 3 AND g.trip3)
THEN 5 ELSE 0 END)
OVER (PARTITION BY p ORDER BY g.s) AS run_seconds
FROM _event_grid g, generate_series(1,3) p
) x;
INSERT INTO fixture.item_history (item_name, sample_time, value)
SELECT v.item_name, m.origin + make_interval(secs => v.s), v.value
FROM (
SELECT item_name, s, value,
LAG(value) OVER (PARTITION BY item_name ORDER BY s) AS prev
FROM _event_values
) v, fixture.build_meta m
WHERE v.prev IS DISTINCT FROM v.value;
-- Setpoints and simulation controls. Constant across the window - the one
-- setpoint write in the period was REJECTED, so the previous value was
-- retained and no value row follows it. That is why the rejected write shows
-- up only as bit 15 of the alarm word.
INSERT INTO fixture.item_history (item_name, sample_time, value)
SELECT hi.item_name, m.origin,
CASE hi.item_name
WHEN 'AID.WRPS.SP.MODE' THEN 1.0 -- auto
WHEN 'AID.WRPS.SP.CMD_PARAM' THEN 0.0
WHEN 'AID.WRPS.SP.LEVEL_SP' THEN 70.0 -- 4200 mm
WHEN 'AID.WRPS.SP.START_DUTY' THEN 66.7 -- 4000 mm
WHEN 'AID.WRPS.SP.START_P2' THEN 75.0 -- 4500 mm
WHEN 'AID.WRPS.SP.START_P3' THEN 83.3 -- 5000 mm
WHEN 'AID.WRPS.SP.STOP_ALL' THEN 16.7 -- 1000 mm
WHEN 'AID.WRPS.SP.HIGH_ALARM' THEN 86.7 -- 5200 mm
WHEN 'AID.WRPS.SP.MIN_SPEED' THEN 76.0 -- 38 Hz
WHEN 'AID.WRPS.SP.SERVICE_HRS' THEN 4000.0
WHEN 'AID.WRPS.SIM.INFLOW' THEN 0.0
WHEN 'AID.WRPS.SIM.SCENARIO' THEN 1.0 -- diurnal dry weather
WHEN 'AID.WRPS.SIM.RESET' THEN 0.0
WHEN 'AID.WRPS.SIM.TIME_SCALE' THEN 1.0
END
FROM fixture.build_meta m, public.historian_items hi
WHERE hi.section IN ('SP', 'SIM')
-- CMD_WORD is momentary and is generated with the event items above, not
-- held at rest here.
AND hi.item_name <> 'AID.WRPS.SP.CMD_WORD';
DROP TABLE _event_grid;
DROP TABLE _event_values;
-- =============================================================================
-- fixture.process_value_history - what Cube reads for analogue history.
--
-- A view, not a table. It resolves the item onto its tag through
-- public.historian_items and exposes nothing else: no equipment column, and
-- no row for an item that is deliberately unanswerable.
--
-- AT PHASE 4 this view is what gets repointed at imh. Everything above it is
-- fixture generation and goes away; everything below it is contract.
-- =============================================================================
CREATE VIEW fixture.process_value_history AS
SELECT
h.sample_time,
h.item_name,
hi.tag_id,
h.value,
hi.eng_unit AS engineering_unit,
h.quality,
hi.his_group,
hi.scan_interval_seconds,
h.is_fixture
FROM fixture.item_history h
JOIN public.historian_items hi USING (item_name)
WHERE hi.his_group IN ('WRPS_ONE_SEC', 'WRPS_THIRTY_SEC')
AND hi.tag_id IS NOT NULL;
COMMENT ON VIEW fixture.process_value_history IS
'Scan/Time analogue history, item-keyed. Regular samples - CI Server has '
'DATA_COMP=0 and STORE_DEADBAND=0 on these groups. scan_interval_seconds is '
'carried so no measure has to assume a rate.';
-- =============================================================================
-- fixture.alarm_history - DERIVED, not stored.
--
-- Every alarm at this station is a bit of the PLC alarm word, historised as
-- AID.WRPS.STN.ALARM_WORD. An ALARM IS A TRANSITION INTO THE ACTIVE STATE - a
-- 0 -> 1 on one bit. The 1 -> 0 is the return to normal for the activation
-- that preceded it, not a second alarm. Counting every row roughly doubles
-- every answer, which is the single most likely way "how many times last week"
-- returns a wrong number confidently.
--
-- WHY DERIVED. CI Server's built-in ALARM_HISTORY group exists on the server
-- but every WRPS item imports with alarming OFF and limits at 0
-- (05-scada/modbus/README.md, "one thing left for you to set"), so it holds
-- nothing. Deriving from the alarm word needs no configuration that does not
-- exist, and it is the same derivation that will run against imh.
--
-- EQUIPMENT comes from public.alarm_bits -> tags -> equipment. A bitmask packs
-- several units' alarms into one item, so the item alone cannot say which pump
-- a seal leak belongs to; the bit can. This is the reason the bit map is
-- reference data rather than a constant inside a Cube model.
-- =============================================================================
CREATE VIEW fixture.alarm_history AS
WITH word AS (
-- READ THE ALARM WORD AS UNSIGNED. Bit 15 (setpoint rejected) does not fit
-- a signed 16-bit integer, so any source that hands it back signed turns
-- the whole word NEGATIVE exactly when bit 15 sets - and a negative int
-- shifted right in Postgres sign-extends, which would report every higher
-- bit as active at once. The fixtures store it as a positive double and
-- never trip this, but imh is SQL Server and SMALLINT there IS signed.
--
-- The modulo below normalises both cases and is a no-op on a value that is
-- already unsigned. Do not "simplify" it away when repointing at imh; that
-- is precisely when it starts mattering. See db/README-standin-historian.md,
-- assumption A11.
SELECT sample_time,
((value::int % 65536) + 65536) % 65536 AS mask,
LAG(((value::int % 65536) + 65536) % 65536)
OVER (ORDER BY sample_time) AS prev_mask,
is_fixture
FROM fixture.item_history
WHERE item_name = 'AID.WRPS.STN.ALARM_WORD'
), bits AS (
SELECT w.sample_time, w.is_fixture, b.bit,
(w.mask >> b.bit) & 1 AS active,
COALESCE((w.prev_mask >> b.bit) & 1, 0) AS prev_active
FROM word w CROSS JOIN public.alarm_bits b
)
SELECT
row_number() OVER (ORDER BY b.sample_time, b.bit) AS alarm_id,
b.sample_time AS event_time,
'AID.WRPS.STN.ALARM_WORD'::text AS item_name,
b.bit,
ab.alarm_type,
ab.priority,
ab.tag_id,
CASE WHEN b.active = 1 THEN 'ACTIVE' ELSE 'RTN' END AS state,
-- The process value at the transition, for the alarms that have one. A
-- pump trip has no level reading worth quoting; a high level alarm does.
CASE WHEN ab.tag_id IN ('AID.WRPS.STN.HIGH_LEVEL', 'LSHH-102', 'LSLL-103',
'AID.WRPS.STN.SPILL_ACTIVE', 'LIT-101')
THEN lv.value END AS value,
CASE WHEN ab.tag_id IN ('AID.WRPS.STN.HIGH_LEVEL', 'LSHH-102', 'LSLL-103',
'AID.WRPS.STN.SPILL_ACTIVE', 'LIT-101')
THEN '%' END AS engineering_unit,
ab.alarm_text,
COALESCE(ic.narrative, ab.description) AS description,
b.is_fixture
FROM bits b
JOIN public.alarm_bits ab USING (bit)
LEFT JOIN fixture.item_history lv
ON lv.item_name = 'AID.WRPS.STN.LEVEL'
AND lv.sample_time = b.sample_time
LEFT JOIN fixture.build_meta m ON true
LEFT JOIN fixture.injected_condition ic
ON ic.bit = b.bit
AND m.origin + make_interval(secs => ic.start_s) = b.sample_time
WHERE b.active IS DISTINCT FROM b.prev_active;
COMMENT ON VIEW fixture.alarm_history IS
'Alarm activations and returns, derived from bit transitions of '
'AID.WRPS.STN.ALARM_WORD. state = ACTIVE is an activation; RTN is the '
'return to normal of the one before it. Count ACTIVE only.';
-- =============================================================================
-- fixture.operation_history - DERIVED pump-downs.
--
-- A pump-down starts where AID.WRPS.STN.PUMPS_RUNNING goes from 0 to non-zero
-- and ends where it returns to 0. Its max level is the maximum level over that
-- span plus the ten minutes before it, because the peak is usually just before
-- the pumps catch up. Operations shorter than five minutes are start/stop
-- noise and are discarded.
--
-- That heuristic is the one written into cube/model/operations.yml, and it is
-- deliberately simple: a heuristic nobody can explain is not evidence, and
-- this is the table that answers the advisory question with evidence.
--
-- MATERIALISED because lin001 is a shared 2 vCPU host and this scans the
-- 5-second level series once per operation. Fixtures are static, so it is
-- built once here and never refreshed.
-- =============================================================================
CREATE MATERIALIZED VIEW fixture.operation_history AS
WITH pr AS (
SELECT sample_time, value,
LAG(value) OVER (ORDER BY sample_time) AS prev
FROM fixture.item_history
WHERE item_name = 'AID.WRPS.STN.PUMPS_RUNNING'
), grouped AS (
SELECT sample_time, value,
sum(CASE WHEN value > 0 AND COALESCE(prev, 0) = 0 THEN 1 ELSE 0 END)
OVER (ORDER BY sample_time) AS run_id
FROM pr
), runs AS (
SELECT run_id,
min(sample_time) AS start_time,
max(sample_time) AS end_time,
max(value)::int AS peak_pumps_running
FROM grouped
WHERE value > 0
GROUP BY run_id
HAVING max(sample_time) - min(sample_time) >= interval '5 minutes'
)
SELECT
r.run_id AS operation_id,
'PUMP_DOWN'::text AS operation_type,
r.start_time,
r.end_time,
r.peak_pumps_running,
lvl.start_level_pct,
lvl.end_level_pct,
lvl.max_level_pct,
flow.avg_inflow_m3h,
flow.avg_discharge_m3h,
duty.duty_pump,
lvl.max_level_pct >= 86.7 AS high_level_alarm,
lvl.max_level_pct >= 100.0 AS spill,
TRUE AS is_fixture
FROM runs r
CROSS JOIN LATERAL (
-- max over the operation PLUS the ten minutes before it: the peak is
-- usually just before the pumps catch up. start and end are read at the
-- exact boundary samples, which line up because the 30-second group the
-- operation is derived from is a multiple of the 5-second level group.
SELECT max(value) AS max_level_pct,
max(value) FILTER (WHERE sample_time = r.start_time) AS start_level_pct,
max(value) FILTER (WHERE sample_time = r.end_time) AS end_level_pct
FROM fixture.item_history
WHERE item_name = 'AID.WRPS.STN.LEVEL'
AND quality = 'GOOD'
AND sample_time BETWEEN r.start_time - interval '10 minutes' AND r.end_time
) lvl
CROSS JOIN LATERAL (
SELECT avg(value) FILTER (WHERE item_name = 'AID.WRPS.STN.INFLOW') AS avg_inflow_m3h,
avg(value) FILTER (WHERE item_name = 'AID.WRPS.STN.DISCHARGE') AS avg_discharge_m3h
FROM fixture.item_history
WHERE item_name IN ('AID.WRPS.STN.INFLOW', 'AID.WRPS.STN.DISCHARGE')
AND quality = 'GOOD'
AND sample_time BETWEEN r.start_time AND r.end_time
) flow
CROSS JOIN LATERAL (
-- The duty unit in force when the set started. DUTY_PUMP is an on-change
-- item, so take the last value at or before the start rather than an
-- aggregate over a window - two different duty values can fall inside any
-- window wide enough to be safe, and max() would silently pick the higher.
SELECT 'PU-30' || value::int AS duty_pump
FROM fixture.item_history
WHERE item_name = 'AID.WRPS.STN.DUTY_PUMP'
AND sample_time <= r.start_time
AND value > 0
ORDER BY sample_time DESC
LIMIT 1
) duty;
CREATE UNIQUE INDEX ON fixture.operation_history (operation_id);
CREATE INDEX ON fixture.operation_history (start_time);
COMMENT ON MATERIALIZED VIEW fixture.operation_history IS
'Pump-downs derived from AID.WRPS.STN.PUMPS_RUNNING. No equipment column - '
'the station is the only equipment a pump-down belongs to, and duty_pump '
'names the unit that led it.';
-- =============================================================================
-- Grants. These live HERE and not in 003_roles.sql on purpose: this file
-- starts with DROP SCHEMA fixture CASCADE, which destroys every grant on it.
-- Grants belong with the object they are granted on, so a fixture reload keeps
-- them. Read-only for both roles - nothing writes fixtures except this file.
-- =============================================================================
GRANT USAGE ON SCHEMA fixture TO agent_ro, cube_rw;
GRANT SELECT ON ALL TABLES IN SCHEMA fixture TO agent_ro, cube_rw;
ALTER DEFAULT PRIVILEGES IN SCHEMA fixture GRANT SELECT ON TABLES TO agent_ro, cube_rw;
-- =============================================================================
-- Assertions.
--
-- These are not a comment suggesting what to check. They FAIL THE LOAD.
--
-- The counts are exact rather than approximate because the origin is truncated
-- to the hour and the plant is prescribed rather than simulated. They were
-- derived independently before this file was written, not read back out of it,
-- which is what makes them a test rather than a restatement.
--
-- If one of these fires after an intentional change to the generating
-- functions, recompute it - do not relax it.
-- =============================================================================
DO $$
DECLARE
n BIGINT;
m BIGINT;
txt TEXT;
BEGIN
-- Analogue row counts follow from the rates in public.historian_items:
-- 5 items x 7 days / 5 s, plus 4 items x 7 days / 30 s.
SELECT count(*) INTO n FROM fixture.process_value_history;
IF n <> 685440 THEN
RAISE EXCEPTION 'process_value_history has % rows, expected 685440', n;
END IF;
-- Every historised, answerable item must actually have history. This is
-- the assertion that would have caught finding (a) on the day it was
-- introduced: the level item existed, the tag existed, and nothing joined.
SELECT count(*) INTO n
FROM public.historian_items hi
WHERE hi.his_group IS NOT NULL
AND hi.tag_id IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM fixture.item_history h
WHERE h.item_name = hi.item_name);
IF n <> 0 THEN
SELECT string_agg(hi.item_name, ', ') INTO txt
FROM public.historian_items hi
WHERE hi.his_group IS NOT NULL AND hi.tag_id IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM fixture.item_history h
WHERE h.item_name = hi.item_name);
RAISE EXCEPTION 'answerable items with no history: %', txt;
END IF;
-- Alarm activations, by bit. Derived from the level series and the
-- injected conditions; see the prototype in the commit that added this.
SELECT count(*) INTO n FROM fixture.alarm_history WHERE state = 'ACTIVE';
IF n <> 32 THEN
RAISE EXCEPTION 'expected 32 alarm activations, found %', n;
END IF;
SELECT count(*) INTO n FROM fixture.alarm_history
WHERE state = 'ACTIVE' AND alarm_type = 'HIGH_LEVEL';
IF n <> 14 THEN
RAISE EXCEPTION 'expected 14 high level activations, found %', n;
END IF;
SELECT count(*) INTO n FROM fixture.alarm_history
WHERE state = 'ACTIVE' AND alarm_type = 'SPILL';
IF n <> 2 THEN
RAISE EXCEPTION 'expected 2 spill activations, found %', n;
END IF;
SELECT count(*) INTO n FROM fixture.alarm_history
WHERE state = 'ACTIVE' AND priority = 1;
IF n <> 15 THEN
RAISE EXCEPTION 'expected 15 priority 1 activations, found %', n;
END IF;
-- THE CROSS-CHECK THAT MATTERS. Bit 0 of the alarm word and the discrete
-- item AID.WRPS.STN.HIGH_LEVEL are two independent records of the same
-- condition. If the derivation is right they agree exactly. This is the
-- check the old fixtures had no way to express, because the alarm table
-- was written by hand rather than derived from the signal.
SELECT count(*) INTO n FROM fixture.alarm_history
WHERE state = 'ACTIVE' AND bit = 0;
SELECT count(*) INTO m FROM (
SELECT value, LAG(value) OVER (ORDER BY sample_time) AS prev
FROM fixture.item_history
WHERE item_name = 'AID.WRPS.STN.HIGH_LEVEL'
) t WHERE value = 1 AND COALESCE(prev, 0) = 0;
IF n <> m THEN
RAISE EXCEPTION
'alarm word bit 0 shows % high level activations but the discrete '
'item AID.WRPS.STN.HIGH_LEVEL shows % - the derivation disagrees '
'with the signal it is derived from', n, m;
END IF;
-- Same cross-check for the spill bit against its own discrete item.
SELECT count(*) INTO n FROM fixture.alarm_history
WHERE state = 'ACTIVE' AND bit = 3;
SELECT count(*) INTO m FROM (
SELECT value, LAG(value) OVER (ORDER BY sample_time) AS prev
FROM fixture.item_history
WHERE item_name = 'AID.WRPS.STN.SPILL_ACTIVE'
) t WHERE value = 1 AND COALESCE(prev, 0) = 0;
IF n <> m THEN
RAISE EXCEPTION
'alarm word bit 3 shows % spill activations but AID.WRPS.STN.'
'SPILL_ACTIVE shows %', n, m;
END IF;
-- Pump-downs, and how many of them alarmed.
SELECT count(*) INTO n FROM fixture.operation_history;
IF n <> 72 THEN
RAISE EXCEPTION 'expected 72 pump-downs, found %', n;
END IF;
SELECT count(*) INTO n FROM fixture.operation_history WHERE high_level_alarm;
IF n <> 14 THEN
RAISE EXCEPTION 'expected 14 pump-downs reaching the high level alarm, found %', n;
END IF;
-- Every pump-down that reached the alarm setpoint must have produced an
-- alarm activation, and vice versa. Two derivations from two different
-- items agreeing is worth more than either on its own.
SELECT count(*) INTO n FROM fixture.operation_history WHERE spill;
SELECT count(*) INTO m FROM fixture.alarm_history
WHERE state = 'ACTIVE' AND alarm_type = 'SPILL';
IF n <> m THEN
RAISE EXCEPTION 'operations show % spills, the alarm word shows %', n, m;
END IF;
-- The frozen transmitter. One hour of 5-second samples, flagged BAD, and
-- excluded from every Cube measure.
SELECT count(*) INTO n FROM fixture.item_history
WHERE item_name = 'AID.WRPS.STN.LEVEL' AND quality = 'BAD';
IF n <> 720 THEN
RAISE EXCEPTION 'expected 720 BAD level samples, found %', n;
END IF;
-- Geometry. Volume remaining to spill must reconcile with the level
-- through the plant's own dimensions - 120 m3 per metre, crest at 6000 mm
-- - at every sample where both were recorded. This is the check the WRPS
-- team used to confirm the Modbus address base, applied continuously.
SELECT count(*) INTO n
FROM fixture.item_history lv
JOIN fixture.item_history vs
ON vs.item_name = 'AID.WRPS.STN.VOL_TO_SPILL'
AND vs.sample_time = lv.sample_time
WHERE lv.item_name = 'AID.WRPS.STN.LEVEL'
AND lv.value < 100.0
AND abs(vs.value - (6.0 - lv.value * 0.06) * 120.0) > 1.0;
IF n <> 0 THEN
RAISE EXCEPTION
'% samples where volume remaining to spill does not reconcile with '
'the level through the plant geometry', n;
END IF;
-- Nothing here is real, and every row must say so.
IF NOT (SELECT bool_and(is_fixture) FROM fixture.item_history) THEN
RAISE EXCEPTION 'a row in fixture.item_history is not flagged as fixture data';
END IF;
RAISE NOTICE 'fixture load OK: % analogue rows, % alarm activations, % pump-downs',
(SELECT count(*) FROM fixture.process_value_history),
(SELECT count(*) FROM fixture.alarm_history WHERE state = 'ACTIVE'),
(SELECT count(*) FROM fixture.operation_history);
END $$;
Reference in a new issue View git blame Copy permalink