885d8e31e2
Azure OpenAI is pending and imh is pending, so POST /ask could not return
anything at all - which left the entire chain either side of the model
unproven: the browser, the API, entity resolution, Cube, retrieval, the
contracts, the banners, the error paths. All of it is testable now, and waiting
for a key to find out whether it works is a choice to find out later.
NO_LLM_STUB=true substitutes the two steps that need a model and nothing else.
- Classification: the caller supplies the class, from a dropdown in the UI.
NOT a keyword classifier. A crude keyword classifier produces a PLAUSIBLE
label, and a plausible wrong label is the exact failure this system exists
to prevent - "how do I reset it" landing in Historical is how a synthesised
procedure reaches an operator. Choosing by hand is honest about what is
happening and drives each branch deliberately. apply_safety_rules() still
runs over the result.
- Prose: a fixed placeholder per class, in stub.py.
Everything else is the real path. This is possible because generate() already
kept the factual fields away from the model: rows, counts, citations, the
fixture flag and the class are attached from evidence, and only prose comes
from the generator. Splitting that into _generate_prose() and _assemble() makes
the seam explicit - the stub feeds _assemble() exactly as the model does, so
this is a fair test of the assembly path rather than a mock of it.
The contracts are the point. A stub payload goes through enforce_contract()
unchanged, and it FAILED first time on two classes: the "nothing found" wording
did not match the not-found detectors, so Reference and Procedural returned 422
rather than an uncited answer. That is the contract doing its job against text
no model wrote. Retries are pointless on deterministic output, and a 422 is a
real result here, not a stub bug.
Retrieval is lexical (retrieval.lexical_search), because embedding the question
needs the model. Kept beside search() and never called on the normal path, so
nobody reads a trace and mistakes a lexical hit for a semantic one. It matches
what the operator typed, not what they meant.
What it does not prove: whether the classifier would have labelled correctly -
a person did; whether retrieval finds the RIGHT chunk; and nothing about prose.
It also cannot fill prerequisites_verbatim - extracting them with a regex would
be the "synthesised from fragments" failure the Procedural contract forbids, so
the list is empty and the answer says so.
Every answer carries stub_mode: true in the contract, not decorated on by the
UI, and a banner beside the fixture banner. Same reasoning: an answer nobody
generated must not be indistinguishable from one that was.
Also here:
- demo/ai-docs: three fabricated documents, numbered WRPS-DEMO-00x so header
extraction is genuinely exercised against a number no real WRPS document
can have. Their setpoints contradict tags.csv on purpose.
- VITE_API_BASE build arg, for a tunnelled build before DNS exists. The
tunnel origin is allowed in CORS only while NO_LLM_STUB is on, so it
disappears with the flag. Proxying /api through ai-web's nginx would have
been easier and was rejected: it creates a second route to the API that
bypasses the api.yokogawa.tech Caddy block, where the Phase 9 publisher
rule lives.
Verified on lin001 with no Azure key set at all: all five classes return 200
through the real UI in a browser, over an SSH tunnel, with citations from the
demo documents, real Cube numbers, and both banners showing.
Turning it off: NO_LLM_STUB=false in ~/ai/api.env, restart ai-api.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
39 lines
1.6 KiB
Markdown
39 lines
1.6 KiB
Markdown
# DEMO DOCUMENT — NOT A CONTROLLED DOCUMENT — DO NOT USE ON PLANT
|
|
|
|
Document number: WRPS-DEMO-003
|
|
Title: Station Design Basis Extract — Discharge and Storage
|
|
Revision: 0
|
|
Effective: 2026-01-01
|
|
|
|
## 1. Scope
|
|
|
|
A fabricated design basis extract for the Waterloo Road Pump Station STN-001,
|
|
written so that the Advisory branch of the plant assistant has documented
|
|
limits to retrieve and cite during a no-LLM demonstration. Every figure below
|
|
is invented. Where it contradicts db/seed/tags.csv, the seed data is the one
|
|
derived from the register map and this one is fiction.
|
|
|
|
## 2. Discharge
|
|
|
|
The discharge manifold MAN-301 is described in this fabricated extract as
|
|
having a stated hydraulic capacity against the 22 m static lift. The three
|
|
pumps PU-301, PU-302 and PU-303 share a common speed reference, and this
|
|
fabricated extract states that the drives are clamped at both ends of their
|
|
range rather than being free to run to zero.
|
|
|
|
## 3. Storage and spill
|
|
|
|
This fabricated extract states that the wet well WW-101 provides storage
|
|
between the stop-all level and the spill weir crest, that the weir crest is the
|
|
point at which flow leaves the site, and that the volume between the current
|
|
level and the crest is the quantity of interest when assessing how much time is
|
|
available.
|
|
|
|
## 4. Limits stated in this fabricated document
|
|
|
|
- The station is not to be operated with the level held above the high level
|
|
alarm setpoint as a normal operating state.
|
|
- Continuous operation of all three pumps is described as a short-duration
|
|
condition, not a normal duty.
|
|
- Any change to a level setpoint is described as requiring assessment against
|
|
the storage remaining to the weir crest.
|