From f852cb901f9f1ae5684076084a14b1102139eaf1 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 21:37:59 +1000 Subject: [PATCH] OI-03: deployment and migration defects have nowhere to be pinned CLAUDE.md requires a failure to be pinned before it is fixed, and names two places to pin one. Neither covers a shell script, a SQL migration or a seed CSV: api/tests holds six files and all six are Python-level. Three defects on 1 September showed what that costs. deploy.sh loaded historian_items positionally, so a migrated column order put a poll group into modbus_address - it failed loudly only because a text value hit an integer column, and two columns of the same type would have swapped silently. The PS_ rename mapped the high level alarm STATUS BIT onto the alarm SETPOINT, which would have made "how many high level alarms last week" count setpoint changes. Neither could be pinned anywhere. The third, a TypeError in the retired-name guard, had a home in api/tests and still was not pinned. The second was caught only because gen_historian_items.py's check_mapping() refuses to write when an item resolves to nothing. That check exists because the same ambiguity caused a Phase 5 finding in August. It has now caught the same class of defect twice and it is the only guard of its kind in the repository. The issue records three things worth considering - a test over the seed CSVs asserting unique keys and resolving foreign keys, migrations applied to a throwaway Postgres rather than to lin001, and an honest question about whether deploy.sh is worth testing at all for a demo - without deciding any of them. Co-Authored-By: Claude Opus 5 --- status/OPEN-ISSUES.md | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/status/OPEN-ISSUES.md b/status/OPEN-ISSUES.md index 7d052bc..cb4c6b2 100644 --- a/status/OPEN-ISSUES.md +++ b/status/OPEN-ISSUES.md @@ -31,6 +31,44 @@ Three rules: ## Open +### OI-03 · Deployment and migration defects have nowhere to be pinned + +**Raised** 2026-09-01 · **Owner** unassigned · **Affects** how defects are prevented from returning + +`CLAUDE.md` requires a failure to be pinned before it is fixed. There are two places to pin +one — `eval/testset.jsonl` for a question the assistant answered wrongly, and `api/tests/` +for anything provable without a model. **Neither covers a shell script, a SQL migration or a +seed CSV.** `api/tests/` holds six files and all six are Python-level; nothing exercises +`scripts/deploy.sh`, `scripts/verify.sh`, `db/*.sql` or the contents of `db/seed/`. + +**Three defects on 2026-09-01 demonstrated the gap.** Two of them had no test home at all: + +| Defect | Consequence if unnoticed | Could it be pinned? | +|---|---|---| +| `deploy.sh` loaded `historian_items` **positionally**, so a migrated column order put the poll group into `modbus_address` | Failed loudly only because a text value hit an integer column. Two columns of the same type would have loaded into each other's places **silently** | **No** | +| The `PS_*` rename mapped the high level alarm **status bit** onto the alarm **setpoint**, because one retired name meant two things | Alarm bit 0 would have counted setpoint changes and reported them as alarms — a small, plausible, confident, wrong number | **No** | +| The retired-name guard called `ContractViolation` with one argument, raising `TypeError` | The guard would have crashed every request it was written to explain | Yes, `api/tests/` — but nothing was pinned | + +The second was caught only because `gen_historian_items.py`'s `check_mapping()` refuses to +write when an item resolves to nothing. That check exists because the same ambiguity caused a +Phase 5 finding in August. **It has now caught the same class of defect twice, and it is the +only guard of its kind in the repository.** + +**Worth considering when this is picked up** — not decided here: + +- A test that loads `db/seed/*.csv` and asserts what the seeds must satisfy: unique keys, every + foreign key resolving, no retired name patterns. This needs no database and would have caught + the rename defect. +- A migration applied to a throwaway Postgres and checked, rather than to `lin001` and checked. + The 008 dry run inside a rolled-back transaction was a manual version of this. +- Whether `deploy.sh` and `verify.sh` are worth testing at all, or whether the honest answer for + a demo is that they are exercised by being run. + +**Blocked by** nothing. **Blocking** nothing — but every deployment defect until then is +recorded only in git history. + +--- + ### OI-02 · The no-setpoint rule may need to become optional **Raised** 2026-09-01 · **Owner** unassigned · **Affects** the product, not the build ·