yau-plant-assistant/cube/model/operations.yml
Claude 156d2a6d43 Refresh pre-aggregations daily, not every ten minutes
lin001 is 2 vCPU with 3.8 GiB and no swap, shared with 27 other containers.
Cube was rebuilding three pre-aggregations over static fixture data on 10, 10
and 30 minute keys, and its own log shows what that cost:

    "Previous interval #19593 was not finished with 30000 interval"
    "Interval #19594 finished after 00:02:50"

A refresh taking 2m50s scheduled every 30 seconds, at interval #19594 -
overlapping and never catching up, for days. Every rebuild produced a
byte-identical result, because the fixtures do not change.

24 hours until imh makes the data genuinely live, at which point these get
tuned deliberately rather than left at a number that was never chosen. Cube
also suggests fewer partitions via rollup_lambda; that is a modelling change,
not a demo-day one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:53:51 +10:00

176 lines
5.9 KiB
YAML

# =============================================================================
# operations.yml — pump-down operations.
#
# THIS IS THE FILE THAT ANSWERS THE ADVISORY QUESTION WITH EVIDENCE RATHER THAN
# OPINION. When an operator asks what rate to run the station at, the honest
# answer is a table of what has actually been run, what happened each time, and
# what the documented limits are - then a deferral. Everything needed for that
# is a measure here.
#
# SOURCE: fixture.operation_history while USE_FIXTURES=true.
#
# IF imh HAS NO OPERATIONS TABLE - and it probably does not - derive it here.
# The heuristic, kept deliberately simple so it can be explained to the person
# checking the number:
#
# A PUMP-DOWN starts at the sample where PS_STN_PUMPS_RUNNING goes from 0 to
# non-zero, and ends at the next sample where it returns to 0. Its max level
# is the maximum PS_STN_WET_WELL_LEVEL over that span plus the 10 minutes
# before it, because the peak is usually just before the pumps catch up.
# Operations shorter than 5 minutes are discarded as start/stop noise.
#
# Do not make this cleverer. A heuristic nobody can explain is not evidence,
# and this cube's whole job is to produce evidence.
#
# ON THE WORD "FILL": the generic spec calls these fills. WRPS is a pump
# station, so the operation is a pump-down - the well fills passively on inflow
# and the station draws it back down. Same shape, opposite sign. The measures
# keep the pump-down naming because that is what an operator here would say.
# =============================================================================
cubes:
- name: operations
sql_table: fixture.operation_history # -> derived from imh at Phase 4
description: >
One row per pump-down at the Waterloo Road Pump Station: when it ran,
how high the well got, how much came in, how much was pumped, which unit
was duty, and whether it alarmed or spilled.
joins:
- name: equipment
sql: "{CUBE}.equipment_id = {equipment}.equipment_id"
relationship: many_to_one
dimensions:
- name: operation_id
sql: operation_id
type: number
primary_key: true
- name: start_time
sql: start_time
type: time
description: Stored UTC, presented in SITE_TIMEZONE.
- name: end_time
sql: end_time
type: time
- name: equipment_id
sql: equipment_id
type: string
- name: operation_type
sql: operation_type
type: string
description: PUMP_DOWN. Reserved for future manual or wash-down operations.
- name: duty_pump
sql: duty_pump
type: string
description: >
The unit that led the operation. Duty rotates on lowest accumulated
run hours, service-due units ranked last, ties by ascending pump
number - so an uneven distribution over a long window is a finding,
not a rotation fault.
- name: peak_pumps_running
sql: peak_pumps_running
type: number
- name: high_level_alarm
sql: high_level_alarm
type: boolean
description: Did this pump-down reach the high level alarm setpoint.
- name: spill
sql: spill
type: boolean
description: >
Did the well go over the weir crest. A spill is an environmental
reportable event; report the count plainly and never round it.
- name: is_fixture
sql: is_fixture
type: boolean
measures:
- name: pump_down_count
type: count
description: Number of pump-down operations in the window.
- name: avg_discharge_rate
sql: avg_discharge_m3h
type: avg
description: >
Mean discharge rate across operations, m3/h. Evidence of what has
been run - NOT a recommendation of what to run.
- name: min_discharge_rate
sql: avg_discharge_m3h
type: min
- name: max_discharge_rate
sql: avg_discharge_m3h
type: max
- name: avg_inflow_rate
sql: avg_inflow_m3h
type: avg
- name: max_level_reached
sql: max_level_pct
type: max
description: Highest wet well level reached, percent of the weir crest.
- name: avg_max_level
sql: max_level_pct
type: avg
- name: high_alarm_count
type: count
filters:
- sql: "{CUBE}.high_level_alarm = TRUE"
- name: high_alarm_rate
sql: >
COUNT(*) FILTER (WHERE {CUBE}.high_level_alarm)::float
/ NULLIF(COUNT(*), 0)
type: number
description: >
Fraction of pump-downs that reached the high level alarm. Pair it
with pump_down_count in the answer - 1 in 2 and 50 in 100 are not the
same evidence, and a rate quoted without its denominator invites the
reader to treat a small sample as a trend.
- name: spill_count
type: count
filters:
- sql: "{CUBE}.spill = TRUE"
- name: avg_duration_minutes
sql: "EXTRACT(EPOCH FROM ({CUBE}.end_time - {CUBE}.start_time)) / 60"
type: avg
pre_aggregations:
- name: ops_by_day
measures:
- pump_down_count
- avg_discharge_rate
- max_level_reached
- high_alarm_count
- spill_count
dimensions: [equipment_id, duty_pump, operation_type]
time_dimension: start_time
granularity: day
partition_granularity: month
refresh_key:
# 24h, not minutes: the fixtures are static, so a shorter
# interval rebuilds a byte-identical result on a 2-vCPU host
# and was a standing CPU load for no gain. Tune this back
# deliberately when imh makes the data genuinely live.
every: 24 hours
build_range_start:
sql: "SELECT now() - interval '365 days'"
build_range_end:
sql: "SELECT now()"