yau-plant-assistant/REQUESTS.md
Claude 41c4cbbbdc Correct H29 and the retention framing: the query window is always one week
The query window is a rolling seven days by design, and stays that way against
the real SQL historian - it is not parsed out of the question. Two things I
wrote assumed otherwise.

H29 was wrong in two ways. It demanded the words "retention" and "seven days",
and it BANNED "no records found" - which contradicts CLAUDE.md, where that is
the required wording for zero rows. With a fixed one-week window June genuinely
has zero rows in what was queried, so the phrase is correct rather than evasive.
It also described the missing piece as question-window parsing, which is not a
gap but the design.

Rewritten to pin the risk that actually exists: SUBSTITUTION. A question naming
June must never be answered with this week's figure wearing June's label.
Answering "there were 14" would be exactly that, and nothing downstream could
catch it. The observed answer refuses the substitution and states the window it
used, so the case now passes on its merits rather than being red by default.
must_not is the only half run_eval enforces, so the banned phrases are ones
that appear only on a substitution.

outside_retention is reframed as what it is: a GUARD, always false while every
caller asks for seven days, there so a caller who later passes a longer window
cannot get an empty result that reads as "nothing happened".

REQUESTS.md and the removal guide now say that extending retention on its own
is inert - the window would still ask for a week of a longer history. Both
halves are needed, or neither.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 16:49:02 +10:00

251 lines
13 KiB
Markdown

# Outstanding requests — WRPS Plant Assistant
**One thing is still needed from other people, of the three originally raised.** Nothing else
is blocking: as at **28 August 2026** the assistant runs end to end on `lin001` with a real
model — the question box, name resolution, the data translator, document search, all four
answer lanes, the rulebook, the working panel, the logbook and the document library are all
live and were exercised by hand on the host.
**Items 1 and 2 are both delivered and closed** — they are kept below as a record of what was
asked for and what arrived. What is still missing is real plant data. That is the request
furthest outside this project's control, and it cannot be hurried at the end.
Item 3 needs the owner of the `imh` historian, who is not yet identified.
Contact for items 1 and 2, both closed: **Daniel Watson** (daniel.watson@yokogawa.com).
| # | Request | Needed for | Blocks |
|---|---|---|---|
| 1 | ~~Azure OpenAI account, 3 deployments~~ | The two AI steps — classifying the question, wording the answer | **Delivered 27 Aug 2026 — closed** |
| 2 | ~~Three public DNS records + DC pinpoint records~~ | Reaching the assistant by name from a control-room PC | **Delivered 27 Aug 2026 — closed** |
| 3 | Read-only login to the `imh` historian | Real plant figures instead of stand-ins | Phases 4, 5; every data answer |
---
## 1. Azure OpenAI account — DELIVERED
**Closed 27 August 2026.** The resource is live at `yau-dem-oai.openai.azure.com`, API version
`2024-10-21`, and `lin001` reaches it outbound over HTTPS. `NO_LLM_STUB=false` on the host:
both AI steps — classifying the question and wording the answer — run against the real model,
and the documents were indexed with real embeddings the same day. Switching it on surfaced
eight failures the stand-in had hidden; all eight are fixed and pinned by cases in
`eval/testset.jsonl`, which grew from 67 to 75.
**Two things are worth recording, and the second needs a decision.**
- **The deployments as built are not the deployments as requested.** The host runs
`CHAT_DEPLOYMENT=gpt-4o` and `CHEAP_DEPLOYMENT=gpt-4o` — the *same model on both lanes*
with `text-embedding-3-small` for embeddings as asked. The request was for a
GPT-5-mini-class model on the cheap lane specifically so that classification, entity
extraction and tool selection would not bill at flagship rates.
- **The cost estimate below therefore no longer holds.** ≈US$35/month at 3,000 questions was
computed on three calls per question with two of them on a cheap model. With both chat lanes
on `gpt-4o`, the same traffic bills materially higher. Either deploy a cheap-lane model and
repoint `CHEAP_DEPLOYMENT`, or re-cost the budget ask before the demo period. **The US$50
monthly budget alert should be confirmed as actually set**, since it is now the thing that
catches this rather than the estimate.
The original request follows, unchanged.
---
### Action
Create an Azure OpenAI resource in the existing subscription and deploy three models on it:
| Deployment | Model |
|---|---|
| Small chat | GPT-5-mini class or equivalent |
| Flagship chat | GPT-5 class or equivalent |
| Embeddings | `text-embedding-3-small` |
Hand over, via a secure channel: the **endpoint URL**, an **API key**, and the **API
version**. Confirm outbound HTTPS from `lin001` to the endpoint is permitted.
Pay-as-you-go. **No fine-tuning and no provisioned throughput** — both bill a fixed hourly
amount whether used or not, and neither is needed.
### Cost
Consumption-based, billed on usage only. No licence, no commitment, no minimum term,
cancellable at any time. Estimated on three model calls per question at published list
rates:
| Usage | Questions / month | Estimated |
|---|---|---|
| Occasional demos | 500 | ≈ US$6 |
| **Expected — build, testing and demos** | **3,000** | **≈ US$35** |
| Heavy daily use | 10,000 | ≈ US$120 |
One-off cost to index the document set: under US$1.
**Requested: a US$50 per month budget, approximately US$250 for the demo period**, with an
Azure budget alert set at that figure.
Rates are published list prices and should be confirmed at approval. This is a
demonstration figure; an operational plant deployment would be sized and costed separately.
### What it is for
The assistant runs end to end today except for the two steps that need a model: classifying
each question, and wording the answer. Without the account those two cannot be built,
tested or reviewed, and the 67-question acceptance test cannot be run at all.
*(As delivered: both steps now run. The acceptance test is runnable and has grown to 75
cases, but has not yet been run as a formal Phase 8 gate — that is a task inside this
project, not a request of anybody.)*
---
## 2. Three public DNS records, plus pinpoint records on the DC — DELIVERED
**Closed 27 August 2026.** All three A records resolve to `20.211.144.151`, and the DC
carries the pinpoint record `ai.yokogawa.tech → 10.0.0.17`. Nothing further is needed from
Dan for this item. Two things are worth recording:
- **`api.yokogawa.tech` and `cube.yokogawa.tech` have no pinpoint record**, so they do not
resolve from inside the VNet at all. That would have broken the operator demo, because
the browser called the API by that hostname. Rather than ask for a fourth record, Caddy
now routes `/ask` under `ai.yokogawa.tech` to `ai-api` and the page is same-origin. No
further DNS is needed for the demo. `cube.yokogawa.tech` is for engineers on the public
side only; `ai-api` reaches Cube container-to-container and never by hostname.
- **`auth.yokogawa.tech` has no pinpoint record either**, and that was the defect this item
missed. Confirmed 28 August 2026: it does not resolve from inside the VNet. Every
Authelia-gated service redirects there, so a LAN browser reached `ai.yokogawa.tech`, got a
correct 302 to the portal, and then died on DNS — the name resolved, the service answered,
and the operator still saw "this site can't be reached". It went unnoticed because the
device agents write to Influx over the `/api/v2/write` MFA bypass and never touch the
portal; no browser had ever hit Authelia from inside the VNet. **This is no longer needed
for the operator path** — on 28 August the console was made unauthenticated from
`cicore1` only (README §6, `BUILD-AI-CONTAINERS.md` §14) — but any *other* gated service
browsed from the LAN will hit exactly this wall, so it is worth asking Dan for regardless.
- **The names resolving is not the same as the services answering.** The Caddyfile blocks
and the Authelia rules for `ai`, `api` and `cube` were applied on the host the same day;
all four AI hostnames now return 302 to the auth portal. What is still unproven is the
part no check here can reach: an operator on `cicore1` — since 28 August, with no sign-in
at all — getting an
answer end to end.
The original request follows, unchanged.
### What is needed
Three A records under `yokogawa.tech`, each → **20.211.144.151**, the same treatment
`lf.yokogawa.tech` already has:
| Name | Points at | For |
|---|---|---|
| `ai.yokogawa.tech` | 20.211.144.151 | The operator's screen |
| `api.yokogawa.tech` | 20.211.144.151 | The assistant behind it |
| `cube.yokogawa.tech` | 20.211.144.151 | The data translator |
**And — separately, on the domain controller — a pinpoint record for `ai.yokogawa.tech`
`10.0.0.17`.** Azure hairpin means LAN hosts cannot reach the VM's public IP from inside
the VNet, so without this an operator sitting at `cicore1` cannot resolve the assistant at
all. `influx.yokogawa.tech` already has exactly this treatment.
### What it is for
All three services are running right now and none of them can be reached by name. Today
they are opened down an engineer's SSH tunnel, which is fine for development and useless
for a demonstration. No control-room PC can open the assistant until these records exist.
Caddy already holds the site blocks, each with `import authelia`, so sign-in and TLS are
handled the moment the names resolve; certificates are issued automatically on first
request. Access is governed by the existing `HTTPS_UserAccess` group — no new AD group is
needed for this item.
### Why raise it now
This is the item most likely to be forgotten, because **nothing fails until the day
somebody tries.** Every container is healthy, every check passes, and the gap only appears
when a person outside this project opens a browser. It also depends on two different people
— public DNS and the DC are not the same change.
---
## 3. A read-only login to the `imh` historian
### What is needed
1. **The server.** `yau-sls-poc-imh:1433` is configured and waiting; the host is still
being built.
2. **`svc_agent_ro`** created with `SELECT` on the agreed tables only — no DDL, no write
access, no `xp_` procedures. Password over a secure channel.
3. **The database name**, and the agreed **table names and key columns**.
4. **NSG rule:** `lin001``imh` on 1433 only.
5. **Timestamp semantics confirmed:** UTC or local, and DST behaviour. This one is not a
formality — see below.
6. Permission to set an application name on the connection, so DBAs can see who is
connecting.
### What it is for
Every figure the assistant produces today is a stand-in. Cube — the data translator — is
running and answering questions against generated tables (32 alarm activations, 72 pump-downs,
about 685,000 analogue samples over seven days), and every answer built on one carries a
warning label all the way to the operator's screen. The definitions Cube uses are the real
ones and will not change when the historian arrives; only the source will.
**The stand-in is now keyed the same way the historian is.** It was rebuilt on 2026-08-31
against `WRPS/05-scada/modbus` — CI Server item names, the real historisation groups, the
real sample rates and the real retention. Three defects that had been left open closed as a
result, and the schema conversation below is correspondingly narrower than it was: the
*shape* is now right, and what remains is names.
### Why the schema conversation is still needed
- **What `imh` calls these tables and columns.** One item-keyed history table is the right
shape; `fixture.item_history` is our guess at its name. Everything else — alarms,
pump-down operations — is derived from it, so this is the only naming question that
actually blocks anything.
- **Whether `imh` exposes CI Server's `ALARM_HISTORY` group at all.** It exists on the
server and is empty: every WRPS item was imported with alarming off and limits at 0, which
`05-scada/modbus/README.md` records as engineering judgement nobody has made yet. Alarms
are derived from the PLC alarm word instead, which needs no configuration that does not
exist. If someone does configure item alarm limits, we would rather use them.
- **An engineer must confirm Cube's first real numbers by hand** before anybody trusts one.
The stand-in asserts its own counts at load and cross-checks them two ways, but that
proves the pipeline, not the plant. This is a person's time, not just a login.
### A separate request: extend the historian's retention
**Every WRPS history group is `LIFE_TIME "1 weeks"`.** The assistant therefore cannot answer
a question about anything older than seven days — not badly, but at all. "How did last month
compare with this one", "when did we last spill", "how many trips this quarter" are all
outside reach, and they are among the questions an operator is most likely to ask.
The system now reports this honestly: a window reaching past retention is answered as *"the
historian does not go back that far"*, never as *"no records found"*, because those are
different answers and only one is true. But honest is not the same as useful.
**Asked for:** raise `LIFE_TIME` on `WRPS_ONE_SEC`, `WRPS_THIRTY_SEC` and `WRPS_EVENT` to at
least 90 days, or confirm that seven days is a deliberate constraint we should design around
and stop asking. Storage is the trade: the 5-second group is five items, which is roughly
120,000 samples per item per week.
> [!NOTE]
> **Extending retention on its own changes nothing.** The assistant's query window is a
> rolling seven days by design and is not parsed out of the question, so it would go on
> asking for a week of a longer history. Both halves are needed to answer "last month":
> the historian has to hold it, **and** the query window has to become variable. Raise them
> together or neither — and if seven days is confirmed as the intended constraint, the
> window work should be dropped too rather than left as a half-built capability.
**Also worth a decision while someone is in there:** the repository's `his_group.qli` and the
live server's `export_his_group.qli` disagree about sample rates — the file says 1 s and
60 s, the server runs 5 s and 30 s. The live server was taken as authoritative. The
repository does not currently describe the running system, and one of the two should change.
### What it unblocks
Phase 4 and the remainder of Phase 5.
---
## Related, but not a request
**Phase 9 — operator document management** will need a second AD group,
`AI_DocPublishers`, with **direct** membership (Authelia resolves direct membership only; a
user nested inside a group is silently denied). That is deliberately held until Phase 8
passes and is **not** being asked for now — noted here only so it is not a surprise later.