yau-plant-assistant/status/OPEN-ISSUES.md
Claude f852cb901f OI-03: deployment and migration defects have nowhere to be pinned
CLAUDE.md requires a failure to be pinned before it is fixed, and names
two places to pin one. Neither covers a shell script, a SQL migration or a
seed CSV: api/tests holds six files and all six are Python-level.

Three defects on 1 September showed what that costs. deploy.sh loaded
historian_items positionally, so a migrated column order put a poll group
into modbus_address - it failed loudly only because a text value hit an
integer column, and two columns of the same type would have swapped
silently. The PS_ rename mapped the high level alarm STATUS BIT onto the
alarm SETPOINT, which would have made "how many high level alarms last
week" count setpoint changes. Neither could be pinned anywhere. The third,
a TypeError in the retired-name guard, had a home in api/tests and still
was not pinned.

The second was caught only because gen_historian_items.py's
check_mapping() refuses to write when an item resolves to nothing. That
check exists because the same ambiguity caused a Phase 5 finding in
August. It has now caught the same class of defect twice and it is the
only guard of its kind in the repository.

The issue records three things worth considering - a test over the seed
CSVs asserting unique keys and resolving foreign keys, migrations applied
to a throwaway Postgres rather than to lin001, and an honest question
about whether deploy.sh is worth testing at all for a demo - without
deciding any of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 21:37:59 +10:00

157 lines
8.5 KiB
Markdown

# Open issues — WRPS Plant Assistant
**Work we own, and decisions we expect to revisit.** One entry per issue, newest first.
Nothing here is assigned yet; owners are set at handover.
## What belongs here, and what does not
This repository has three other registers. Putting an item in the wrong one is how
a repository ends up saying two different things about the same fact.
| If it is... | It goes in | Not here |
|---|---|---|
| Waiting on somebody outside this project | [`REQUESTS.md`](REQUESTS.md) | ✗ |
| A shortcut we **consciously accepted** | [`BUILD-AI-CONTAINERS.md`](../spec/BUILD-AI-CONTAINERS.md) §14 | ✗ |
| Something already running, and its state | [`current-state.html`](current-state.html) | ✗ |
| **A defect or gap we own and have not fixed** | **here** | ✓ |
| **A design decision we expect to revisit** | **here**, marked *not a defect* | ✓ |
Three rules:
1. **Anything in §14 is closed by decision.** It was weighed and accepted. Do not
re-open it here — `caddy/ai-routes.caddy` carries one such decision explicitly
marked *"do not re-raise this as a task"*.
2. **If an issue is a defect in something that already passed a phase gate**, fixing
it means re-running that gate. `CLAUDE.md` requires it; say so in the issue.
3. **Closed issues move to the bottom, they are not deleted.** The register is part
of the as-built record, and an issue with no trace of how it closed is worth less
than one that was never raised.
---
## Open
### OI-03 · Deployment and migration defects have nowhere to be pinned
**Raised** 2026-09-01 · **Owner** unassigned · **Affects** how defects are prevented from returning
`CLAUDE.md` requires a failure to be pinned before it is fixed. There are two places to pin
one — `eval/testset.jsonl` for a question the assistant answered wrongly, and `api/tests/`
for anything provable without a model. **Neither covers a shell script, a SQL migration or a
seed CSV.** `api/tests/` holds six files and all six are Python-level; nothing exercises
`scripts/deploy.sh`, `scripts/verify.sh`, `db/*.sql` or the contents of `db/seed/`.
**Three defects on 2026-09-01 demonstrated the gap.** Two of them had no test home at all:
| Defect | Consequence if unnoticed | Could it be pinned? |
|---|---|---|
| `deploy.sh` loaded `historian_items` **positionally**, so a migrated column order put the poll group into `modbus_address` | Failed loudly only because a text value hit an integer column. Two columns of the same type would have loaded into each other's places **silently** | **No** |
| The `PS_*` rename mapped the high level alarm **status bit** onto the alarm **setpoint**, because one retired name meant two things | Alarm bit 0 would have counted setpoint changes and reported them as alarms — a small, plausible, confident, wrong number | **No** |
| The retired-name guard called `ContractViolation` with one argument, raising `TypeError` | The guard would have crashed every request it was written to explain | Yes, `api/tests/` — but nothing was pinned |
The second was caught only because `gen_historian_items.py`'s `check_mapping()` refuses to
write when an item resolves to nothing. That check exists because the same ambiguity caused a
Phase 5 finding in August. **It has now caught the same class of defect twice, and it is the
only guard of its kind in the repository.**
**Worth considering when this is picked up** — not decided here:
- A test that loads `db/seed/*.csv` and asserts what the seeds must satisfy: unique keys, every
foreign key resolving, no retired name patterns. This needs no database and would have caught
the rename defect.
- A migration applied to a throwaway Postgres and checked, rather than to `lin001` and checked.
The 008 dry run inside a rolled-back transaction was a manual version of this.
- Whether `deploy.sh` and `verify.sh` are worth testing at all, or whether the honest answer for
a demo is that they are exercised by being run.
**Blocked by** nothing. **Blocking** nothing — but every deployment defect until then is
recorded only in git history.
---
### OI-02 · The no-setpoint rule may need to become optional
**Raised** 2026-09-01 · **Owner** unassigned · **Affects** the product, not the build ·
**Not a defect** — the rule works as designed. Logged because it is a design decision that
is expected to be revisited.
The second of the three lines this system does not cross is **no recommended setpoints or
operating parameters**. An Advisory answer gives evidence, ranges, outcomes and documented
limits, then defers explicitly to a competent person. It never returns a number as the answer.
There is a foreseeable case for allowing it — a recommendation is the thing an operator
actually wants, and withholding it has a cost. This entry exists so that conversation starts
from what is built rather than from scratch.
**Where the rule lives.** It is not a prompt instruction, so relaxing it is a code change in
several places at once:
| Enforcement point | What it does |
|---|---|
| `api/contracts.py``AdvisoryAnswer` | `recommendation_given: Literal[False]` — the contract cannot express a recommendation. A `deferral` string is required, and a scope banner is attached to every Advisory answer |
| `api/classifier.py` | Advisory beats Historical; partly-advisory is advisory. Anything that could be read as advice is routed to the class that refuses to advise |
| `eval/testset.jsonl` | 14 Advisory cases assert the refusal |
| `api/tests/` | 15 tests across contracts and classifier rules |
**What would have to be decided before it changes** — none of this is a coding question:
- **Who is accountable for a number the assistant produces**, once it stops deferring. Today the
deferral is what keeps that answer unambiguous.
- **What evidence is sufficient.** The rule exists because "best" depends on equipment condition
and concurrent operations this system cannot see — the historian shows what happened, not what
the plant can safely do now.
- **How a recommendation is distinguished on screen** from evidence, so it cannot be misread as a
documented limit.
- **Whether it applies to all parameters or a named subset**, and who approves that list.
- **§2 of the build spec is still awaiting an OT/safety review.** This rule is the largest single
thing that review will have an opinion on. Do not relax it beforehand.
**Blocked by** the OT/safety review of `spec/BUILD-AI-CONTAINERS.md` §2, which is outstanding.
**Blocking** nothing.
---
### OI-01 · OpenPLC Editor is not installed
**Raised** 2026-09-01 · **Owner** unassigned · **Affects** the demo plant, not the assistant
The **OpenPLC Editor** — a desktop tool published by the OpenPLC project, installed on a
workstation — is the application used to author and compile the IEC 61131-3 program that
`openplc-runtime` executes. It is not installed anywhere. It talks to the running container
over its REST API on port `8443`, bound to `10.0.0.17`.
**Why it matters.** `openplc-runtime` is live control for this demo: `CLAUDE.md` and
[`BUILD-AI-CONTAINERS.md`](../spec/BUILD-AI-CONTAINERS.md) §4 both forbid reconfiguring it
as a side effect of other work. Without the Editor there is no way to author, review or
compile the control logic — and no reviewable source for it in or beside this repository.
The program exists only inside the container. If the container is lost, so is the logic.
**What port 8443 is** — probed read-only on the host 2026-09-01, because two earlier
statements in the build spec called it a web UI and were wrong:
```
Server: Werkzeug/3.1.8 Python/3.11.2
/ /login /index.html /programs /status /runtime -> 404
/api/v1 -> 401 Unauthorized
```
An authenticated REST API. No browser interface. Both build-spec statements were corrected
in the commit that raised this issue.
**Open questions to settle when this is picked up.**
- **Which workstation.** Not `lin001` — the Editor is a desktop application. Not `cicore1`
`CLAUDE.md` forbids installing anything on it. That leaves an engineering workstation with
VPN or LAN reach to `10.0.0.17:8443`.
- **Credentials for `/api/v1`**, which currently answers `401`. Not held by this project.
- **Whether the PLC program goes under version control**, and where. This is the part that
closes the "logic exists only in the container" gap, and it is the reason this issue is
worth more than "install a tool".
**Blocked by** nothing. **Blocking** nothing today — the demo runs.
---
## Closed
*None yet. Closed issues move here with the commit that closed them.*