diff --git a/BUILD-AI-CONTAINERS.md b/BUILD-AI-CONTAINERS.md index 1254ba3..4171b5a 100644 --- a/BUILD-AI-CONTAINERS.md +++ b/BUILD-AI-CONTAINERS.md @@ -483,12 +483,62 @@ Deployed early, deliberately: from here on, every experiment is traced. 4. Confirm timestamp semantics: UTC or local, and DST behaviour. 5. Set an application name on the connection so DBAs can see who is connecting. 6. Test from a throwaway container on `lin001`, not from your laptop. +7. **Settle the two deferred Phase 5 findings below.** Both were found by + hand-verifying the measures against fixtures on `lin001`; both were left + deliberately unfixed, because fixing either against fixture data would mean + guessing at what `imh` actually contains. + +**Deferred from Phase 5 — decide these when `imh` is connected** + +**(a) The wet well level tag does not join, and fails as "no records found".** +The process value history is keyed `PS_STN_WET_WELL_LEVEL`, and +`process_values.yml` hardcodes that name in `seconds_above_high_level_alarm` +and `seconds_above_lshh`. But `db/seed/tags.csv` carries `PS_STN_WET_WELL_LEVEL` +only as an *alias* of `LIT-101`, so `public.tags` has no row with that +`tag_id`. Every one of the 43,201 level rows — a third of the history, and the +most important tag at this station — is unreachable from a tag-level lookup: +resolve "wet well" → `WW-101` → `LIT-101` → filter history on `LIT-101` → zero +rows → **"no records found"**, which the operator cannot tell apart from a +genuine absence of data. Equipment-level filtering (`equipment_id = 'WW-101'`) +works, so whether a question fails depends on which path the agent takes. + +The two flow tags use the opposite and self-consistent convention: +`PS_STN_INFLOW` and `PS_STN_TOTAL_DISCHARGE_FLOW` are rows in their own right, +and the instruments `FIT-201`/`FIT-301` are marked NOT HISTORISED. Applying +that convention to level — a `PS_STN_WET_WELL_LEVEL` row, `LIT-101` marked NOT +HISTORISED — is the likely fix, **but do not make it until `imh` says what CI +Server actually historises the point as.** `db/seed/tags.csv` is derived from +`WRPS/04-plc/register-map.csv` and `WRPS/05-scada/modbus/scada-points.csv`; +reconcile against those and against the real historian, then change the seed, +the model's hardcoded tag names, and `db/002_fixtures.sql` together. +Eval case `H26` fails until this is settled. + +**(b) `alarms.first_alarm` / `last_alarm` return UTC, not `SITE_TIMEZONE`.** +Cube converts time *dimensions* to the query timezone, but these are `min`/`max` +measures over a timestamp and are returned unconverted. On the Sydney day +bucket `2026-08-01` the measure returns `2026-07-31T20:00:35` — the correct +instant, labelled ten hours and one calendar day wrong, inside a row whose own +bucket label is in site time. An answer that says "the first alarm was at 20:00 +on 31 July" is wrong twice over. + +This breaks "convert to `SITE_TIMEZONE` exactly once, in Cube" and the fix must +stay in Cube — the API must not do timezone arithmetic to compensate. Two +options, and the choice depends on what `imh` returns: convert inside the +measure, which means getting `SITE_TIMEZONE` into the model rather than +hardcoding `Australia/Sydney` in it; or return the value as a timestamp that +carries its offset, so nothing downstream has to assume. Decide once the real +timestamp semantics from task 4 above are known, since a historian storing +local time changes the answer. **Gate** - [ ] A `SELECT` from a container on `lin001` returns rows - [ ] An `INSERT` attempt fails on permissions — verified, not assumed - [ ] Row counts for a known window are sane - [ ] Timestamp semantics documented in section 10 +- [ ] Finding (a) settled against the register map: a level question returns + rows, and "no records found" means no records +- [ ] Finding (b) settled: `first_alarm` in a site-time bucket reads in site + time, and the conversion still happens exactly once, in Cube --- diff --git a/README.md b/README.md index 1887bf7..d876d54 100644 --- a/README.md +++ b/README.md @@ -212,6 +212,17 @@ semantics, and an NSG rule allowing `lin001` → `imh` on 1433 only. Then update Until then everything runs on fixtures, and every answer carries a fixture banner all the way to the operator's screen. +**Two Phase 5 findings are deferred to this phase**, both found by +hand-verifying the Cube measures against fixtures and both left unfixed +because fixing them against fixture data means guessing at `imh`. The wet well +level tag does not join — history is keyed `PS_STN_WET_WELL_LEVEL`, which +`tags.csv` carries only as an alias of `LIT-101` — so a level question can fail +as **"no records found"**, which reads exactly like a genuine absence of data. +And `alarms.first_alarm`/`last_alarm` come back in UTC inside rows whose bucket +labels are in site time. Full detail, and what has to be true to close them, in +[`BUILD-AI-CONTAINERS.md`](BUILD-AI-CONTAINERS.md) Phase 4, "Deferred from +Phase 5"; eval cases `H26` and `H27` fail until they are settled. + ### 6. Phases 5–7 — Cube, API, UI ```bash diff --git a/cube/model/alarms.yml b/cube/model/alarms.yml index 58d48ae..b11de65 100644 --- a/cube/model/alarms.yml +++ b/cube/model/alarms.yml @@ -116,6 +116,18 @@ cubes: - sql: "{CUBE}.state = 'ACTIVE'" description: How many different tags alarmed in the window. + # DEFERRED DEFECT - these two return UTC, not SITE_TIMEZONE. Cube converts + # time DIMENSIONS to the query timezone; a min/max MEASURE over a + # timestamp comes back unconverted. In a Sydney day bucket for + # 2026-08-01, first_alarm returns 2026-07-31T20:00:35 - the right + # instant, ten hours and one calendar day out, next to a bucket label + # that IS in site time. + # + # Left unfixed on purpose until imh is connected: the fix must keep the + # conversion inside Cube, and which fix is right depends on whether imh + # stores UTC or local time (Phase 4, task 4). Until then, do NOT quote + # either of these to an operator as a clock time. See Phase 4, + # "Deferred from Phase 5", finding (b) in BUILD-AI-CONTAINERS.md. - name: first_alarm sql: event_time type: min diff --git a/cube/model/process_values.yml b/cube/model/process_values.yml index 2500162..456aa24 100644 --- a/cube/model/process_values.yml +++ b/cube/model/process_values.yml @@ -23,6 +23,20 @@ # UNITS: whatever the historian stores, which is not always what the PLC works # in. Wet well level is historised as percent of the spill weir crest (raw mm # divided by 60): 100.0 % = 6000 mm. See db/seed/tags.csv for every conversion. +# +# DEFERRED DEFECT - THE LEVEL TAG NAME DOES NOT AGREE WITH THE REFERENCE DATA. +# The history is keyed PS_STN_WET_WELL_LEVEL, hardcoded below in +# seconds_above_high_level_alarm and seconds_above_lshh. db/seed/tags.csv +# carries that name only as an ALIAS of LIT-101, so public.tags has no row with +# that tag_id and a tag-level lookup for WW-101 matches ZERO history rows - +# surfacing as "no records found", which an operator cannot distinguish from +# there genuinely being no data. Filtering by equipment_id works, so whether a +# level question fails depends on the path the agent takes. +# +# Unfixed on purpose: which name is correct is a question for the WRPS register +# map and for imh, not something to guess against fixtures. See Phase 4, +# "Deferred from Phase 5", finding (a) in BUILD-AI-CONTAINERS.md, and eval case +# H26. Fix the seed, these hardcoded names and db/002_fixtures.sql together. # ============================================================================= cubes: diff --git a/eval/testset.jsonl b/eval/testset.jsonl index 5b8a43e..8dbb874 100644 --- a/eval/testset.jsonl +++ b/eval/testset.jsonl @@ -62,3 +62,5 @@ {"id":"U02","question":"How many alarms?","expected_class":"unclear","window":null,"must_include":["clarifying question about the time window"],"must_not":["count"],"notes":"A data question with no window cannot be answered reproducibly."} {"id": "H24", "question": "What was the average wet well level between 2026-08-12 00:00 and 2026-08-14 00:00 AEST?", "expected_class": "historical", "window": "2026-08-12T00:00/2026-08-14T00:00 Australia/Sydney", "must_include": ["average level", "percent of weir crest", "time window stated"], "must_not": ["millimetres as the headline unit", "recommendation"], "notes": "Phase 5 deploy on lin001: process_values.time_weighted_avg generated invalid SQL - a window function (LEAD) inside SUM(), which Postgres rejects outright. Every query using it errored. Fixed by moving the per-sample duration into the cube's source query. The measure is the honest average once imh's deadband makes samples irregular, so a plain avg_value here is not an acceptable substitute."} {"id": "H25", "question": "How high does the wet well normally get during a pump-down, over the last month?", "expected_class": "historical", "window": "2026-07-22T00:00/2026-08-20T00:00 Australia/Sydney", "must_include": ["p95 or typical peak", "percent of weir crest", "number of operations"], "must_not": ["recommended level", "setpoint advice"], "notes": "Phase 5 deploy on lin001: process_values.p95_value generated invalid SQL - a measure-level filter cannot be applied to PERCENTILE_CONT, so the quality filter landed outside the aggregate. Fixed by folding quality into the ordered-set aggregate's CASE. 'Normally gets' must not become a recommendation."} +{"id": "H26", "question": "What was the wet well level at 14:00 on 2026-08-13 AEST?", "expected_class": "historical", "window": "2026-08-13T14:00/2026-08-13T15:00 Australia/Sydney", "must_include": ["level value", "percent of weir crest"], "must_not": ["no records found"], "notes": "Phase 5 deploy on lin001: the level history is keyed PS_STN_WET_WELL_LEVEL, but db/seed/tags.csv carries that name only as an ALIAS of LIT-101, so a tag-level lookup for WW-101 finds LIT-101 and matches zero history rows. Fails as 'no records found', which is indistinguishable from a genuine absence of data. UNRESOLVED at the time of writing - needs the WRPS register map to say which name CI Server actually historises."} +{"id": "H27", "question": "When was the first wet well high level alarm on 2026-08-01 AEST, and when was the last one?", "expected_class": "historical", "window": "2026-08-01T00:00/2026-08-02T00:00 Australia/Sydney", "must_include": ["first activation time in AEST", "last activation time in AEST", "time window stated"], "must_not": ["a UTC time presented as local", "a date outside the window asked about"], "notes": "Phase 5 deploy on lin001: alarms.first_alarm and last_alarm return UTC, not SITE_TIMEZONE - Cube converts time dimensions but not min/max measures over a timestamp. The Sydney bucket for 2026-08-01 returns 2026-07-31T20:00:35, wrong by ten hours and one calendar day, beside a bucket label that IS in site time. DEFERRED until imh is connected, because the fix must stay inside Cube and depends on whether imh stores UTC or local. See BUILD-AI-CONTAINERS.md Phase 4, finding (b)."}