OI-03: deployment and migration defects have nowhere to be pinned

CLAUDE.md requires a failure to be pinned before it is fixed, and names
two places to pin one. Neither covers a shell script, a SQL migration or a
seed CSV: api/tests holds six files and all six are Python-level.

Three defects on 1 September showed what that costs. deploy.sh loaded
historian_items positionally, so a migrated column order put a poll group
into modbus_address - it failed loudly only because a text value hit an
integer column, and two columns of the same type would have swapped
silently. The PS_ rename mapped the high level alarm STATUS BIT onto the
alarm SETPOINT, which would have made "how many high level alarms last
week" count setpoint changes. Neither could be pinned anywhere. The third,
a TypeError in the retired-name guard, had a home in api/tests and still
was not pinned.

The second was caught only because gen_historian_items.py's
check_mapping() refuses to write when an item resolves to nothing. That
check exists because the same ambiguity caused a Phase 5 finding in
August. It has now caught the same class of defect twice and it is the
only guard of its kind in the repository.

The issue records three things worth considering - a test over the seed
CSVs asserting unique keys and resolving foreign keys, migrations applied
to a throwaway Postgres rather than to lin001, and an honest question
about whether deploy.sh is worth testing at all for a demo - without
deciding any of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

status/OPEN-ISSUES.md

@@ -31,6 +31,44 @@ Three rules:
 
 ## Open
 
+### OI-03 · Deployment and migration defects have nowhere to be pinned
+
+**Raised** 2026-09-01 · **Owner** unassigned · **Affects** how defects are prevented from returning
+
+`CLAUDE.md` requires a failure to be pinned before it is fixed. There are two places to pin
+one — `eval/testset.jsonl` for a question the assistant answered wrongly, and `api/tests/`
+for anything provable without a model. **Neither covers a shell script, a SQL migration or a
+seed CSV.** `api/tests/` holds six files and all six are Python-level; nothing exercises
+`scripts/deploy.sh`, `scripts/verify.sh`, `db/*.sql` or the contents of `db/seed/`.
+
+**Three defects on 2026-09-01 demonstrated the gap.** Two of them had no test home at all:
+
+| Defect | Consequence if unnoticed | Could it be pinned? |
+|---|---|---|
+| `deploy.sh` loaded `historian_items` **positionally**, so a migrated column order put the poll group into `modbus_address` | Failed loudly only because a text value hit an integer column. Two columns of the same type would have loaded into each other's places **silently** | **No** |
+| The `PS_*` rename mapped the high level alarm **status bit** onto the alarm **setpoint**, because one retired name meant two things | Alarm bit 0 would have counted setpoint changes and reported them as alarms — a small, plausible, confident, wrong number | **No** |
+| The retired-name guard called `ContractViolation` with one argument, raising `TypeError` | The guard would have crashed every request it was written to explain | Yes, `api/tests/` — but nothing was pinned |
+
+The second was caught only because `gen_historian_items.py`'s `check_mapping()` refuses to
+write when an item resolves to nothing. That check exists because the same ambiguity caused a
+Phase 5 finding in August. **It has now caught the same class of defect twice, and it is the
+only guard of its kind in the repository.**
+
+**Worth considering when this is picked up** — not decided here:
+
+- A test that loads `db/seed/*.csv` and asserts what the seeds must satisfy: unique keys, every
+  foreign key resolving, no retired name patterns. This needs no database and would have caught
+  the rename defect.
+- A migration applied to a throwaway Postgres and checked, rather than to `lin001` and checked.
+  The 008 dry run inside a rolled-back transaction was a manual version of this.
+- Whether `deploy.sh` and `verify.sh` are worth testing at all, or whether the honest answer for
+  a demo is that they are exercised by being run.
+
+**Blocked by** nothing. **Blocking** nothing — but every deployment defect until then is
+recorded only in git history.
+
+---
+
 ### OI-02 · The no-setpoint rule may need to become optional
 
 **Raised** 2026-09-01 · **Owner** unassigned · **Affects** the product, not the build ·