YAU.POC/yau-plant-assistant
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Load historian_items by column name, not by position

Browse source 
The seed load broke on lin001 the first time it ran after migration 008:

  ERROR: invalid input syntax for type integer: "PS_STATUS_BITS"
  CONTEXT: COPY hi_stage, line 2, column modbus_address

hi_stage is created LIKE historian_items, so it inherits the LIVE table's
column order. On a database built fresh from 001_schema.sql that matches
the CSV. On a database migrated by 008 it does not: ALTER TABLE ADD COLUMN
appends, so ci_station, ci_point and poll_group sit at the end of the
table while the CSV has them in the middle. \copy matches by position and
ignores the header, so it loaded the poll group into modbus_address.

It failed loudly here because a text value landed in an integer column.
Two columns of the same type would have loaded silently into each other's
places, which is the version of this defect worth designing against.

Both the \copy and the INSERT now name their columns, so the load no
longer depends on the two orders agreeing. The \copy line is long because
psql meta-commands cannot be wrapped across lines.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude 2026-09-01 20:28:55 +10:00
parent 6a64c64588
commit 421c585422
1 changed files with 9 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
scripts/deploy.sh
View file

@ -120,9 +120,16 @@ ON CONFLICT (tag_id) DO UPDATE SET
-- the only place an item name is joined to a tag - so it must load AFTER tags -- the only place an item name is joined to a tag - so it must load AFTER tags
-- (it references them) and BEFORE the fixtures (they are driven by it). -- (it references them) and BEFORE the fixtures (they are driven by it).
-- Regenerate from the WRPS repo with scripts/gen_historian_items.py. -- Regenerate from the WRPS repo with scripts/gen_historian_items.py.
-- Columns are NAMED, not positional. hi_stage is LIKE historian_items, so it
-- inherits the LIVE table's column order - and ALTER TABLE appends, so a
-- migrated database orders its columns differently from a freshly created one.
-- A positional copy then loads the wrong column into the wrong place, or fails
-- with a type error if you are lucky. Migration 008 made that concrete.
-- The \copy line is long because psql meta-commands cannot be wrapped.
CREATE TEMP TABLE hi_stage (LIKE historian_items EXCLUDING CONSTRAINTS); CREATE TEMP TABLE hi_stage (LIKE historian_items EXCLUDING CONSTRAINTS);
\copy hi_stage FROM '/tmp/db/seed/historian_items.csv' WITH (FORMAT csv, HEADER true, NULL '') \copy hi_stage (item_name,tag_id,exclusion_reason,section_path,section,attribute,section_description,description,eng_unit,value_format,conv_type,has_sign,phys_low,phys_high,eng_gain,raw_to_eng,his_group,scan_interval_seconds,life_time,ci_station,ci_point,poll_group,iec_address,modbus_kind,modbus_address,data_type,point_time_zone) FROM '/tmp/db/seed/historian_items.csv' WITH (FORMAT csv, HEADER true, NULL '')
INSERT INTO historian_items SELECT * FROM hi_stage INSERT INTO historian_items (item_name,tag_id,exclusion_reason,section_path,section,attribute,section_description,description,eng_unit,value_format,conv_type,has_sign,phys_low,phys_high,eng_gain,raw_to_eng,his_group,scan_interval_seconds,life_time,ci_station,ci_point,poll_group,iec_address,modbus_kind,modbus_address,data_type,point_time_zone)
SELECT item_name,tag_id,exclusion_reason,section_path,section,attribute,section_description,description,eng_unit,value_format,conv_type,has_sign,phys_low,phys_high,eng_gain,raw_to_eng,his_group,scan_interval_seconds,life_time,ci_station,ci_point,poll_group,iec_address,modbus_kind,modbus_address,data_type,point_time_zone FROM hi_stage
ON CONFLICT (item_name) DO UPDATE SET ON CONFLICT (item_name) DO UPDATE SET
tag_id=EXCLUDED.tag_id, exclusion_reason=EXCLUDED.exclusion_reason, tag_id=EXCLUDED.tag_id, exclusion_reason=EXCLUDED.exclusion_reason,
section_path=EXCLUDED.section_path, section=EXCLUDED.section, section_path=EXCLUDED.section_path, section=EXCLUDED.section,