7d6cc8985c
Five documents named the size of eval/testset.jsonl and two were wrong: spec/REBUILD.md said 62 cases and status/current-state.html said 75. The file holds 78. Both were corrected earlier today in the two places I had grepped for, and these two were missed - the same staleness failure repeating inside the same session that found it. Rather than correct them to 78 and wait for the next divergence, the count is gone from every sentence that states a current total. README.md, REBUILD.md, current-state.html and workflow-map.html now name the eval set without sizing it. The file is the source of truth and it cannot go stale. The four remaining mentions are historical and stay: "the exam grew from 67 questions to 75 to cover them" records what happened when the eight live-model failures were pinned, and REQUESTS.md's "67-question acceptance test" is inside a dated request. Both are correct as history. This is the same reasoning as dropping "twelve assumptions" from the stand-in historian pointer, and it is now a rule in CLAUDE.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
742 lines
49 KiB
HTML
742 lines
49 KiB
HTML
<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
<title>WRPS Plant Assistant — Workflow Map</title>
|
|
<style>
|
|
:root{
|
|
--bg:#f6f7f9; --panel:#ffffff; --ink:#1a1d21; --muted:#5d6672; --line:#dfe3e8;
|
|
--live:#1a7f4b; --live-bg:#e6f4ec; --built:#1f5fa8; --built-bg:#e8f0fa;
|
|
--blocked:#a86400; --blocked-bg:#fdf1de; --none:#6b7280; --none-bg:#eef0f2;
|
|
--accent:#8b4a9c; --stop:#b03030; --stop-bg:#fbebeb;
|
|
}
|
|
@media (prefers-color-scheme: dark){
|
|
:root:not([data-theme="light"]){
|
|
--bg:#14161a; --panel:#1c1f25; --ink:#e9ecef; --muted:#9aa4b2; --line:#2c313a;
|
|
--live:#5cd39a; --live-bg:#15311f; --built:#7fb2ee; --built-bg:#16283d;
|
|
--blocked:#e6b567; --blocked-bg:#332713; --none:#98a1ad; --none-bg:#23262c;
|
|
--accent:#cf9ade; --stop:#f08a8a; --stop-bg:#3a1c1c;
|
|
}
|
|
}
|
|
*{box-sizing:border-box}
|
|
body{margin:0;background:var(--bg);color:var(--ink);
|
|
font:16px/1.55 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;}
|
|
.wrap{max-width:1160px;margin:0 auto;padding:32px 20px 80px}
|
|
h1{font-size:28px;margin:0 0 6px;line-height:1.25}
|
|
h2{font-size:20px;margin:46px 0 4px;padding-top:18px;border-top:1px solid var(--line)}
|
|
.sub{color:var(--muted);margin:0 0 18px;max-width:72ch}
|
|
.stamp{color:var(--muted);font-size:13px;margin-bottom:24px}
|
|
|
|
.legend{display:flex;flex-wrap:wrap;gap:10px;margin:18px 0 8px}
|
|
.key{display:inline-flex;align-items:center;gap:7px;font-size:13px;color:var(--muted);
|
|
background:var(--panel);border:1px solid var(--line);border-radius:20px;padding:5px 12px}
|
|
.dot{width:10px;height:10px;border-radius:50%;flex:none;display:inline-block}
|
|
.d-live{background:var(--live)} .d-built{background:var(--built)}
|
|
.d-blocked{background:var(--blocked)} .d-none{background:var(--none)}
|
|
.d-partial{background:var(--accent)}
|
|
|
|
.flow{display:flex;flex-direction:column}
|
|
.step{display:grid;grid-template-columns:118px minmax(0,1fr);gap:18px;align-items:start}
|
|
.stepnum{color:var(--muted);font-size:12.5px;text-align:right;padding-top:16px;
|
|
text-transform:uppercase;letter-spacing:.05em}
|
|
.card{background:var(--panel);border:1px solid var(--line);border-left:4px solid var(--none);
|
|
border-radius:10px;padding:14px 16px}
|
|
.card.live{border-left-color:var(--live)} .card.built{border-left-color:var(--built)}
|
|
.card.blocked{border-left-color:var(--blocked)}
|
|
.card.partial{border-left-color:var(--accent)}
|
|
.card h3{margin:0 0 5px;font-size:16.5px;display:flex;flex-wrap:wrap;align-items:center;gap:9px}
|
|
.card p{margin:0 0 6px;color:var(--muted);font-size:14.5px}
|
|
.card p:last-of-type{margin-bottom:0}
|
|
.card .ai{display:inline-block;font-size:11px;font-weight:600;letter-spacing:.03em;
|
|
padding:1px 7px;border-radius:4px;background:var(--none-bg);color:var(--muted);
|
|
text-transform:uppercase}
|
|
.tool{font-size:12.5px;color:var(--muted);margin-top:9px;padding-top:8px;
|
|
border-top:1px dashed var(--line)}
|
|
.tool b{color:var(--ink);font-weight:600;
|
|
font-family:ui-monospace,SFMono-Regular,Consolas,monospace;font-size:12.5px}
|
|
.badge{font-size:11.5px;font-weight:600;letter-spacing:.02em;padding:2px 9px;border-radius:20px;
|
|
white-space:nowrap}
|
|
.b-live{color:var(--live);background:var(--live-bg)}
|
|
.b-built{color:var(--built);background:var(--built-bg)}
|
|
.b-blocked{color:var(--blocked);background:var(--blocked-bg)}
|
|
.b-none{color:var(--none);background:var(--none-bg)}
|
|
.b-partial{color:var(--accent);background:var(--none-bg)}
|
|
.arrow{grid-column:2;color:var(--line);padding:5px 0 5px 22px;font-size:19px;line-height:1}
|
|
.arrow span{color:var(--muted);font-size:12.5px;padding-left:10px}
|
|
|
|
.lanes{display:grid;grid-template-columns:repeat(auto-fit,minmax(232px,1fr));gap:12px;margin-top:6px}
|
|
.lane{background:var(--panel);border:1px solid var(--line);border-top:3px solid var(--accent);
|
|
border-radius:10px;padding:13px 14px}
|
|
.lane h4{margin:0 0 3px;font-size:14.5px}
|
|
.lane .q{font-style:italic;font-size:13.5px;margin:0 0 8px;opacity:.85}
|
|
.lane p{margin:0;color:var(--muted);font-size:13.5px}
|
|
.lane .uses{margin-top:9px;font-size:12.5px;color:var(--muted);
|
|
border-top:1px dashed var(--line);padding-top:8px}
|
|
.lane .uses b{color:var(--ink);font-weight:600}
|
|
.lane.unclear{border-top-color:var(--muted)}
|
|
|
|
.cols{display:grid;grid-template-columns:repeat(auto-fit,minmax(320px,1fr));gap:14px;margin-top:14px}
|
|
|
|
table{width:100%;border-collapse:collapse;margin-top:14px;font-size:14.5px}
|
|
th,td{text-align:left;padding:10px 12px;border-bottom:1px solid var(--line);vertical-align:top}
|
|
th{font-size:12.5px;text-transform:uppercase;letter-spacing:.05em;color:var(--muted);font-weight:600}
|
|
td.ph{white-space:nowrap;color:var(--muted);font-variant-numeric:tabular-nums}
|
|
.tblwrap{overflow-x:auto}
|
|
ul.next{margin:12px 0 0;padding-left:20px;color:var(--muted)}
|
|
ul.next li{margin-bottom:8px}
|
|
ul.next b{color:var(--ink)}
|
|
.note{background:var(--panel);border:1px solid var(--line);border-radius:10px;
|
|
padding:14px 16px;margin-top:16px;color:var(--muted);font-size:14.5px}
|
|
.note.stop{border-left:4px solid var(--stop);background:var(--stop-bg)}
|
|
.note b{color:var(--ink)}
|
|
.foot{color:var(--muted);font-size:13px;margin-top:34px;padding-top:14px;
|
|
border-top:1px solid var(--line)}
|
|
@media (max-width:640px){
|
|
.step{grid-template-columns:1fr;gap:4px}
|
|
.stepnum{text-align:left;padding-top:10px}
|
|
.arrow{grid-column:1;padding-left:6px}
|
|
}
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<div class="wrap">
|
|
|
|
<h1>The plant assistant — how a question becomes an answer</h1>
|
|
<p class="sub">Waterloo Road Pump Station. An operator types a question in plain English; the
|
|
assistant answers it from plant records and controlled documents, and shows its evidence. This page
|
|
follows one question through the assistant, names the piece of software doing each job, and marks
|
|
what already exists.</p>
|
|
<p class="stamp">Status as observed on the live host, 28 August 2026, end of day.
|
|
Every status below was read from the running containers and probed by hand, not taken from the plan.
|
|
Where this page and the build documents disagree, this page is the later reading.</p>
|
|
|
|
<div class="note stop"><b>What changed in the last week:</b> two of the three things this project
|
|
was waiting on arrived. The network names came on 27 August, and the AI model account the same day.
|
|
<b>Both AI steps now run</b> — the assistant sorts the question itself and writes the answer
|
|
in its own words — and the documents are indexed so search works by meaning rather than
|
|
keyword. The dropdown and the placeholder wording are gone.
|
|
<p style="margin:.7em 0 0">Switching the model on was not the end of the work. <b>Five faults
|
|
surfaced within the hour, and every one had been hidden by the stand-in</b> that replaced the model.
|
|
The most important: asked how to lift an interlock, the assistant was handing the answer writer
|
|
<em>the step list</em> — the one thing it must never reproduce — while withholding the
|
|
title block it actually needed. <b>Three more followed that afternoon</b>, when the first real
|
|
document was uploaded and found faults no fabricated one could. All eight are fixed, each is pinned
|
|
by a test, and the exam grew from 67 questions to 75 to cover them.</p>
|
|
<p style="margin:.7em 0 0"><b>Two more things changed later the same day, both at the
|
|
customer’s direction, and both are worth understanding before the next demo.</b>
|
|
The operator’s screen <b>no longer asks anyone to sign in</b> — it now admits the SCADA
|
|
machine by its network address alone and refuses everything else. And <b>document management was
|
|
built and put live</b>, out of order, ahead of the exam it was meant to follow. Operators can now
|
|
add and withdraw documents from a screen. Both are explained in section 3b.</p>
|
|
<p style="margin:.7em 0 0">One thing remains outstanding, and it is the one furthest outside this
|
|
project’s control: <b>a read-only login to the plant historian.</b> Until it exists every
|
|
figure on screen is a stand-in, and says so. A fresh instance of that gap surfaced today —
|
|
see “the question that cannot be answered” in section 2.</p></div>
|
|
|
|
<div class="legend">
|
|
<span class="key"><i class="dot d-live"></i> Running now</span>
|
|
<span class="key"><i class="dot d-partial"></i> Running, on stand-in data</span>
|
|
<span class="key"><i class="dot d-blocked"></i> Waiting on someone else</span>
|
|
</div>
|
|
|
|
<h2>1 · The question’s journey</h2>
|
|
<p class="sub">Seven stages. The important thing to notice: the assistant decides <em>what kind of
|
|
question</em> it is before it looks anything up, and that decision fixes both which sources it may
|
|
use and what it is allowed to say.</p>
|
|
|
|
<div class="flow">
|
|
|
|
<div class="step">
|
|
<div class="stepnum">Ask</div>
|
|
<div class="card live">
|
|
<h3>The question box <span class="badge b-live">Running now</span></h3>
|
|
<p>A web page with a question box and an answer pane, opened in an ordinary browser on the
|
|
control-room PC. Nothing to install. It is running on the host and has been driven by hand in
|
|
a browser.</p>
|
|
<p>It cannot yet be reached by its name: <b>ai.yokogawa.tech</b> does not exist as a network
|
|
record, so today it is opened down an engineer’s secure tunnel. Three names are outstanding —
|
|
the screen, the assistant behind it, and the data translator.</p>
|
|
<div class="tool">Application: <b>ai-web</b> — the operator’s screen</div>
|
|
</div>
|
|
</div>
|
|
<div class="arrow">↓</div>
|
|
|
|
<div class="step">
|
|
<div class="stepnum">Sort</div>
|
|
<div class="card live">
|
|
<h3>What kind of question is this? <span class="ai">AI step</span>
|
|
<span class="badge b-live">Running now</span></h3>
|
|
<p>A small, fast AI model reads the question and labels it as one of four kinds — or says it is
|
|
unsure. It does no answering; it only labels. This is the most safety-relevant step in the
|
|
whole assistant, because the label decides everything downstream.</p>
|
|
<p>Two rules are built in: if it is not confident, it asks the operator to rephrase rather than
|
|
guessing; and if two labels are close, it takes the more cautious one — a question that is
|
|
partly advice is treated as advice.</p>
|
|
<p><b>This step now runs.</b> The model account arrived on 27 August 2026 and all four kinds
|
|
are labelled correctly with high confidence. The dropdown it replaced is gone.</p>
|
|
<p>Switching it on immediately exposed a fault nothing else could have found. The worked
|
|
examples the model is shown returned only the label, not the confidence figure beside it — so
|
|
the model copied that shape, every question scored zero confidence, and <em>every</em>
|
|
question fell below the caution threshold and came back as “unsure”. The stand-in had been
|
|
supplying its own confidence all along, which hid the fault completely. That is the lesson
|
|
worth keeping: a stand-in proves the plumbing, never the step it stands in for.</p>
|
|
<div class="tool">Application: <b>ai-api</b> · the classifier, using a small AI model</div>
|
|
</div>
|
|
</div>
|
|
<div class="arrow">↓</div>
|
|
|
|
<div class="step">
|
|
<div class="stepnum">Translate</div>
|
|
<div class="card live">
|
|
<h3>What is “Pump 02”, exactly? <span class="badge b-live">Running now</span></h3>
|
|
<p>Operators say “Pump 02” or “the wet well”. The plant records call
|
|
those PU-302 and WW-101. This step converts everyday names into the plant’s official
|
|
ones by looking them up in a list — no AI, no guessing. When it gets a name wrong, the fix is
|
|
to add the nickname to the list.</p>
|
|
<p>Live on the host with 8 pieces of equipment and 56 measurement points loaded, and exercised
|
|
by real questions travelling the chain.</p>
|
|
<div class="tool">Application: <b>ai-api</b> · the name resolver, reading the equipment
|
|
and tag list in <b>pg-ai</b></div>
|
|
</div>
|
|
</div>
|
|
<div class="arrow">↓<span>into one of four lanes — or back to the operator</span></div>
|
|
|
|
<div class="lanes">
|
|
<div class="lane">
|
|
<h4>Historical — what happened?</h4>
|
|
<p class="q">“How many high level alarms last week?”</p>
|
|
<p>Counts and measurements taken from the plant’s recorded history. If there are no
|
|
records, the answer says so — it never estimates a figure.</p>
|
|
<div class="uses">Looks in: <b>Cube</b> → the plant historian.
|
|
<br>Proven on the host against stand-in figures — 145 alarm records, 309 pump
|
|
operations and about 130,000 level readings.</div>
|
|
</div>
|
|
<div class="lane">
|
|
<h4>Reference — what does this mean?</h4>
|
|
<p class="q">“What does the level fault alarm mean?”</p>
|
|
<p>Explains a signal, alarm or piece of equipment from the manuals and alarm records, quoting
|
|
the document it came from.</p>
|
|
<div class="uses">Looks in: <b>document search</b> + the equipment and tag list.
|
|
<br>Proven on the host against four real documents and three fabricated ones.</div>
|
|
</div>
|
|
<div class="lane">
|
|
<h4>Procedural — how do I do it?</h4>
|
|
<p class="q">“How do I lift the interlock on Pump 02?”</p>
|
|
<p>Finds the approved procedure and identifies it — document number, revision, date — and
|
|
quotes its prerequisites word for word. It never writes the steps itself.</p>
|
|
<div class="uses">Looks in: <b>document search</b>, procedures only.
|
|
<br>Proven on the host: it locates the demo procedure and identifies it. It cannot yet quote
|
|
prerequisites word for word — pulling those out of a document needs the model.</div>
|
|
</div>
|
|
<div class="lane">
|
|
<h4>Advisory — what should we do?</h4>
|
|
<p class="q">“What discharge rate avoids a spill?”</p>
|
|
<p>Shows what has been run before, what happened when, and the documented limits — then hands
|
|
the decision back to a competent person. It never gives a number as the answer.</p>
|
|
<div class="uses">Looks in: <b>Cube</b> and <b>document search</b> together.
|
|
<br>Proven on the host, with the deferral banner in place.</div>
|
|
</div>
|
|
<div class="lane unclear">
|
|
<h4>Unclear — I need more</h4>
|
|
<p class="q">“Is the pump alright?”</p>
|
|
<p>When the sorting step is not confident, the assistant stops and asks a clarifying question.
|
|
No sources are searched and no answer is drafted.</p>
|
|
<div class="uses">Looks in: <b>nothing</b> — it asks instead.
|
|
<br>Now exercisable, and it was: for a short period after the model was switched on, a fault
|
|
in the sorting step scored <em>every</em> question at zero confidence, so every question
|
|
landed in this lane. Wrong, but it failed to the cautious side — which is how it was designed
|
|
to fail.</div>
|
|
</div>
|
|
</div>
|
|
<div class="arrow" style="grid-column:1/-1">↓<span>evidence gathered</span></div>
|
|
|
|
<div class="step">
|
|
<div class="stepnum">Word it</div>
|
|
<div class="card live">
|
|
<h3>Turn the evidence into a readable answer <span class="ai">AI step</span>
|
|
<span class="badge b-live">Running now</span></h3>
|
|
<p>The larger AI model is used here and only here: to put the gathered evidence into plain
|
|
English. It cannot go looking for anything else — it only sees what its lane collected. It
|
|
supplies wording, not facts.</p>
|
|
<p><b>This step now runs</b>, and the split it was designed around held under test. Three
|
|
faults appeared the moment a real model wrote the words, and every one was the same mistake:
|
|
the model had been asked for a <em>fact</em> that should have come from the evidence — which
|
|
revision of a procedure it was, what the document was called, which document a limit came
|
|
from. Each time it supplied something empty or invented, and each time the rulebook refused
|
|
the answer rather than showing it.</p>
|
|
<p>The fix in all three was the same, and it was not a better instruction: those facts now
|
|
come from the retrieved document itself, so the wording step cannot influence them at all. It
|
|
supplies English and nothing else — which is what it was always meant to do, now enforced
|
|
rather than asked for.</p>
|
|
<div class="tool">Application: <b>ai-api</b> · the answer writer, using the larger AI model</div>
|
|
</div>
|
|
</div>
|
|
<div class="arrow">↓</div>
|
|
|
|
<div class="step">
|
|
<div class="stepnum">Check</div>
|
|
<div class="card built">
|
|
<h3>Is this answer allowed to leave? <span class="badge b-live">Running now</span></h3>
|
|
<p>Every draft is checked against a fixed rulebook before the operator sees it: no invented
|
|
steps for safety-critical work, no recommended setpoint, nothing beyond the evidence actually
|
|
found, and every claim carrying its citation. A draft that fails is rewritten once and then
|
|
refused — it is never shown.</p>
|
|
<p>These are ordinary software checks written in code, not instructions given to the AI. That
|
|
distinction is the point: an AI can be talked out of an instruction, and cannot be talked out
|
|
of a check.</p>
|
|
<p>The rulebook earned its keep today, on a real question. Asked how many high level alarms
|
|
there had been last week, the assistant produced an answer with a number in it — and the
|
|
check refused to let it out, twice, because no records had come back to support it. The
|
|
operator saw an error rather than a figure nobody could stand behind. <b>A refusal is a
|
|
correct result, not a fault</b>; what it exposed was a fault one layer below, in how the alarm
|
|
is filed. See section 2.</p>
|
|
<div class="tool">Application: <b>ai-api</b> · the answer contracts and guardrails</div>
|
|
</div>
|
|
</div>
|
|
<div class="arrow">↓</div>
|
|
|
|
<div class="step">
|
|
<div class="stepnum">Show</div>
|
|
<div class="card built">
|
|
<h3>The answer, and its working <span class="badge b-live">Running now</span></h3>
|
|
<p>Beside the answer the operator sees how it was reached: which kind of question it was judged
|
|
to be, what data was counted and over what period, how many records were found, and which
|
|
documents were cited with their revision and date. Procedural and advisory answers carry a
|
|
visible banner stating plainly what the assistant did <em>not</em> do.</p>
|
|
<p>Two further banners are on screen today: one saying the figures are stand-ins rather than
|
|
real plant data, and one saying no AI model was involved in the wording. Neither can be
|
|
switched off from the screen.</p>
|
|
<div class="tool">Application: <b>ai-web</b> · the “show working” panel</div>
|
|
</div>
|
|
</div>
|
|
<div class="arrow">↓</div>
|
|
|
|
<div class="step">
|
|
<div class="stepnum">Record</div>
|
|
<div class="card live">
|
|
<h3>Everything is written down <span class="badge b-live">Running now</span></h3>
|
|
<p>Each question, the kind it was judged to be, the sources used, the rulebook result and how
|
|
long it took are all recorded, so any answer can be reviewed afterwards and any mistake traced
|
|
to the step that caused it. Live since early on, and recording the trial questions being put
|
|
through the chain now.</p>
|
|
<div class="tool">Application: <b>Langfuse</b> — the assistant’s logbook</div>
|
|
</div>
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div class="note stop"><b>Why four lanes instead of one clever assistant:</b> the four kinds of
|
|
question need different sources, and — more importantly — different limits on what may be said. One
|
|
general-purpose pipeline answering all four is the main way a project like this hurts somebody: it
|
|
is how a bypass procedure gets reassembled from fragments nobody approved, or a setpoint gets
|
|
suggested and typed into a control system. The assistant cannot wander between lanes, because the
|
|
lane is fixed before any searching starts.</div>
|
|
|
|
<h2>2 · The four tools it can reach for</h2>
|
|
<p class="sub">The assistant has no free access to anything. These four, and nothing else.</p>
|
|
|
|
<div class="cols">
|
|
<div class="card partial">
|
|
<h3>Cube — the data translator <span class="badge b-partial">Running on stand-in figures</span></h3>
|
|
<p>Turns “high level alarms last week” into an exact query over the plant’s
|
|
recorded history. The definitions that make an answer right — what counts as an alarm, what
|
|
“last week” means, what counts as a fill — are written down in files an engineer can
|
|
read and check, instead of being invented per question. It also keeps ready-made summaries so
|
|
common questions never hammer the live historian.</p>
|
|
<p>The assistant never writes database code itself. It fills in a request form; Cube does the
|
|
rest.</p>
|
|
<p>It is running on the host and answering the data lane’s questions. It reads stand-in figures,
|
|
because the real historian is still not available — so no number it produces today means anything
|
|
about the plant, and every answer built on one says so on the operator’s screen. The definitions
|
|
it uses are the real ones and will not change when the historian arrives; only the source will.</p>
|
|
<p><b>The question that cannot be answered.</b> Asked today, on the running system:
|
|
<em>“how many wet well high level alarms were there between 1 and 7 August?”</em>
|
|
— the most obvious question anyone would ask at this station. It came back as an error.
|
|
The alarm is filed in the name list against <em>the station</em>; every one of its recorded
|
|
occurrences is filed against <em>the wet well</em>. The two never meet, so the count comes out
|
|
as nothing at all. Counted by equipment, that week holds 100 alarms.</p>
|
|
<p>What did work is the part that matters most: the assistant had written an answer with a
|
|
number in it, and <b>the rulebook stopped it leaving</b>, because there were no records behind
|
|
it. The operator got an error rather than a figure nobody could stand behind. This is the third
|
|
place the same underlying mismatch has appeared, and none of the three can be fixed here —
|
|
the stand-in tables were built from the same name list, so “correcting” either side
|
|
would only make the two agree with each other and prove nothing. All three wait on the real
|
|
historian.</p>
|
|
<div class="tool">Reads: today the stand-in tables in <b>pg-ai</b> — 145 alarms, 129,603 level
|
|
readings, 309 pump operations. Eventually the <b>imh</b> plant historian, read-only. Definitions
|
|
live in <b>cube/model/</b> — alarms, process values, operations, equipment. Cube runs as two
|
|
containers, <b>cube</b> and <b>cubestore</b>; the second holds the ready-made summaries and the
|
|
first will not start without it. Those summaries are rebuilt daily, and are not yet being
|
|
written into <b>pg-ai</b> as the design specifies.</div>
|
|
</div>
|
|
|
|
<div class="card partial">
|
|
<h3>Document search <span class="badge b-live">Searching real documents by meaning</span></h3>
|
|
<p>Searches the controlled documents by meaning rather than keyword, so “lift the
|
|
interlock” finds the right procedure even when the document says “interlock
|
|
override”. Two rules are permanently on: withdrawn revisions are never returned, and every
|
|
result carries its document number, revision, date, page and section — a passage that cannot be
|
|
cited cannot be used.</p>
|
|
<p>Search by meaning works, and it now has real material to search. <b>Four real documents
|
|
were loaded on 28 August</b> — three control descriptions and an instrument document
|
|
— alongside the three fabricated demo ones that were written so the document lanes had
|
|
something to find at all. The demo documents carry deliberately impossible document numbers and
|
|
made-up figures, and say so on every passage, so nothing in them can be mistaken for plant
|
|
content. What is outstanding is the rest of the real set, not the ability to hold it.</p>
|
|
<p>Procedures are now handled differently from everything else, and deliberately so. Ranking a
|
|
procedure’s sections by how closely they match “how do I lift the interlock”
|
|
returned <em>the step list</em> — the one part that must never be reproduced — and
|
|
left out the title block. So the assistant now identifies <em>which</em> procedure governs the
|
|
question first, then returns that document’s heading, purpose and prerequisites in order,
|
|
and withholds its step sections entirely. The wording step never receives them. The rulebook
|
|
still refuses instruction language independently; withholding is the first line, not the
|
|
only one.</p>
|
|
<div class="tool">Reads: document sections stored in <b>pg-ai</b> — 92 passages across
|
|
7 documents, 4 of them real. Withdrawn and superseded revisions are filtered out before ranking,
|
|
every time, and that filter is not something a question can switch off.</div>
|
|
</div>
|
|
|
|
<div class="card live">
|
|
<h3>The name list <span class="badge b-live">Running now</span></h3>
|
|
<p>Every piece of equipment and every measurement point, with the everyday names operators
|
|
actually use. Deliberately a plain list rather than something the AI works out, so it behaves the
|
|
same way every time and can be corrected by editing one row.</p>
|
|
<div class="tool">Reads: the equipment and tag tables in <b>pg-ai</b> — 8 pieces of
|
|
equipment, 56 measurement points, live and in use.</div>
|
|
</div>
|
|
|
|
<div class="card live">
|
|
<h3>The logbook <span class="badge b-live">Running now</span></h3>
|
|
<p>Not a source of answers — the record of them. Every question and every step it took is stored
|
|
so the assistant’s behaviour can be audited and improved from evidence rather than
|
|
impressions.</p>
|
|
<div class="tool">Application: <b>Langfuse</b></div>
|
|
</div>
|
|
</div>
|
|
|
|
<h2>3 · Where the knowledge comes from</h2>
|
|
<p class="sub">Two supply lines feed those tools. Neither is finished.</p>
|
|
|
|
<div class="cols">
|
|
<div class="card partial">
|
|
<h3>Documents in <span class="badge b-live">Proven on real documents</span></h3>
|
|
<p>Procedures, manuals, alarm rationalisation and design documents are read, split into sections,
|
|
and converted into a searchable form. Three habits matter: a numbered step sequence is never cut
|
|
in half; the document’s type comes from the folder it was filed in, not from a guess; and
|
|
a person confirms the document number, revision and date before anything is stored. A wrong
|
|
revision on a procedure is a safety problem, not a tidiness one.</p>
|
|
<p>The whole path has been run on the host end to end, first at a terminal on three
|
|
fabricated demo documents, and since 28 August through the new upload screen on <b>four real
|
|
ones</b>.</p>
|
|
<p><b>The first real document was worth more than the entire demo set</b>, and this is the part
|
|
worth repeating. An eight-page control philosophy exposed three faults in a single upload, none
|
|
of which any test or any demo document had caught, and every one of which left the screen
|
|
looking perfectly correct. The whole document had become <em>one</em> passage, because the PDF
|
|
reader produces no blank lines and the splitter had nothing to split on — eight pages
|
|
behind a single search entry, every citation reading “untitled, page 1”. Every
|
|
passage was labelled with the document’s revision instead of its title, because a cover
|
|
page collapses into a column of labels and the matcher ran past the line break. And publishing
|
|
the same document twice quietly duplicated it, leaving the same passage quotable twice with
|
|
nothing on screen to show it. All three are fixed; the document now splits into 19 sections that
|
|
match its own headings.</p>
|
|
<p>The lesson is not that the code was careless. It is that <b>fabricated documents are shaped
|
|
like what the code expects</b>. Real ones are not, and only a real one could have shown it.</p>
|
|
<div class="tool">Two ways in, and they must not both be used on the same document:
|
|
<b>ai-ingest</b> at a terminal, and the upload screen added at step 9. Both run the same
|
|
splitting code, now held identical by a test rather than by intention. Stored in <b>pg-ai</b>.
|
|
The demo documents stay marked so they can be removed in one step.</div>
|
|
</div>
|
|
|
|
<div class="card blocked">
|
|
<h3>Plant history in <span class="badge b-blocked">Waiting on someone else</span></h3>
|
|
<p>The station’s control system records levels, pump runs and alarms into the plant
|
|
historian. The assistant reads that history through Cube — read-only, never writing to it, and
|
|
never touching the control system itself.</p>
|
|
<div class="tool">Source: the <b>imh</b> historian. That server is still being built. A login
|
|
name has been proposed and the connection settings are in place waiting for it, but the password,
|
|
the database name and the agreed table names are all still outstanding.</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="note"><b>Worth knowing:</b> of the two supply lines, one is now connected and one is
|
|
not. The documents are real. <b>The figures are not</b> — until the historian is available the
|
|
data lanes run on stand-ins, and every answer built on one carries a label saying so, on the
|
|
operator’s screen, all the way through. That label is not something an operator can turn off.
|
|
The placeholder wording is gone: the assistant has written in its own words since 27 August.</div>
|
|
|
|
<h2 id="waiting">3a · What was being waited on</h2>
|
|
<p class="sub">Two of the three arrived in the last week. One remains, and it is the one furthest
|
|
outside this project’s control. Nobody outside this project is holding up anything else.</p>
|
|
|
|
<div class="cols">
|
|
<div class="card live">
|
|
<h3>An AI model account <span class="badge b-live">Arrived 27 August 2026</span></h3>
|
|
<p>Both AI steps now run, and the documents are indexed for search by meaning. Switching them
|
|
on was not the end of the work: five separate faults surfaced within the hour, and three more
|
|
followed that afternoon when the first real document was uploaded. Every one of the eight was
|
|
invisible while the stand-in was in place; all eight are now fixed and pinned by tests. The
|
|
parts a safety reviewer most needs to examine can now actually be examined.</p>
|
|
<p><b>One difference from what was asked for.</b> Two model deployments were provided rather
|
|
than three — there is no small, cheap model, so the labelling step runs on the large one.
|
|
It works correctly, but it costs roughly ten times the estimate per question, because labelling
|
|
is by far the most frequent call. A second deployment on the same account fixes it; no new
|
|
account and no new key.</p>
|
|
<div class="tool">Have: <b>gpt-4o</b> and <b>text-embedding-3-small</b>. Still wanted: a small
|
|
model deployment, so the cost figures hold.</div>
|
|
</div>
|
|
|
|
<div class="card blocked">
|
|
<h3>A read-only login to the historian <span class="badge b-blocked">Waiting on someone else</span></h3>
|
|
<p>Until this exists every figure the assistant produces is a stand-in. The data translator is
|
|
running and its definitions are written down, so the switch-over is a change of source rather
|
|
than a rebuild — but an engineer must independently confirm its first real numbers before
|
|
anybody trusts one.</p>
|
|
<div class="tool">Needed: the <b>imh</b> server, a read-only login, and the agreed table names.</div>
|
|
</div>
|
|
|
|
<div class="card live">
|
|
<h3>Three network name records <span class="badge b-live">Arrived 27 August 2026</span></h3>
|
|
<p>All three names resolve. The operator’s page and the assistant behind it were moved
|
|
onto a single address, because the assistant’s own name does not resolve from inside the
|
|
plant network and a control-room browser would have failed on every question.</p>
|
|
<p><b>Only the operator’s address works from inside the plant network.</b> The other
|
|
three — and, importantly, the sign-in page itself — have no internal record, so a
|
|
browser on the plant network resolves them, gets correctly handed to sign-in, and then fails
|
|
there. It went unnoticed for weeks because the equipment that talks to this host never opens a
|
|
browser. It no longer affects the operator, who does not sign in at all, but it will catch the
|
|
next protected service anyone tries to open from the plant floor.</p>
|
|
<p><b>The part no check from here could reach has now happened:</b> an operator sat at a
|
|
control-room PC and got an answer end to end. The screen’s own access log records the
|
|
control-room machine opening the page on 28 August and asking questions on 31 August. Names
|
|
resolving is not the same as a person succeeding — and this time the person succeeded.</p>
|
|
<div class="tool">Have: <b>ai</b>, <b>api</b> and <b>cube</b> under <b>yokogawa.tech</b>, plus
|
|
the internal record for <b>ai</b>.</div>
|
|
</div>
|
|
</div>
|
|
|
|
<h2>3b · Two decisions taken on 28 August, and what they cost</h2>
|
|
<p class="sub">Both were asked for by the customer, both are reversible, and both trade a
|
|
protection for convenience. Neither is a mistake — but neither should reach a wider audience
|
|
undescribed.</p>
|
|
|
|
<div class="cols">
|
|
<div class="card partial">
|
|
<h3>The operator no longer signs in <span class="badge b-partial">Applied 28 August 2026</span></h3>
|
|
<p>An operator at the console should not have to complete a phone prompt to ask a question, and
|
|
nobody outside the plant should reach the assistant at all. So the operator’s screen now
|
|
admits <b>one machine, by its network address</b>, and refuses everything else — plant
|
|
network, remote access and internet alike.</p>
|
|
<p><b>What it costs.</b> This is an address list on a flat network with no boundary between the
|
|
office and plant sides, so anything that can take that address inherits the same unquestioned
|
|
access. It is a demonstration convenience, not a security control, and it is the first thing
|
|
proper network separation would replace. Two further consequences are easy to miss: the logbook
|
|
can no longer record <em>who</em> asked a question, only that somebody did; and engineers can no
|
|
longer open the assistant in a browser over the remote connection.</p>
|
|
<p>The assistant behind the screen is unchanged and still fully protected — document
|
|
management depends on that and must stay there. Undoing this is a single configuration reload;
|
|
the sign-in rule was deliberately left in place.</p>
|
|
<div class="tool">In scope for the safety review, which is still outstanding.</div>
|
|
</div>
|
|
|
|
<div class="card partial">
|
|
<h3>Document management, built out of order <span class="badge b-partial">Live 28 August 2026</span></h3>
|
|
<p>Step 9 was designed to follow the exam, for a reason: adding a way for more documents to
|
|
arrive makes a wrong answer harder to diagnose, not easier. It was built and put live first, at
|
|
the customer’s direction. Upload, convert, review, approve, withdraw and restore all work,
|
|
and four real documents have been through it.</p>
|
|
<p><b>What it costs.</b> Four things, all deliberate and all reversible. <b>Nobody is checked:</b>
|
|
the publisher is a name typed into a box, checked against a one-entry list with no password, so
|
|
anyone who reaches the page can claim it — rows are permanently marked as unverified so
|
|
they stay tellable apart once real sign-in goes on. <b>Scanned documents cannot be read at all</b>
|
|
and are refused rather than stored empty, because the proper document reader needs software this
|
|
host must not run. <b>The two ways of loading a document must not be used on the same one</b>,
|
|
or the older path cannot see what the screen loaded. And the conversion happens inside the
|
|
request, so a large document holds up its own upload.</p>
|
|
<p>What has not moved is the judgement: nothing is quotable until a named person has confirmed
|
|
the document’s number, revision and date, and the database itself refuses an approved
|
|
document without them.</p>
|
|
<div class="tool">Application: the document screens inside <b>ai-api</b>, at the assistant’s
|
|
own protected address — not on the operator’s screen, which now carries no identity at all.</div>
|
|
</div>
|
|
</div>
|
|
|
|
<h2>4 · What exists, and the order the rest gets built</h2>
|
|
<p class="sub">Each step ends in a check that must pass before the next begins — so that when
|
|
an answer comes out wrong, there is one place to look, not four. <b>Two of those checks have never
|
|
been run, and step 9 was built before step 8 rather than after it.</b> Status below is what was read
|
|
off the running host on 28 August 2026, in the evening.</p>
|
|
|
|
<div class="tblwrap">
|
|
<table>
|
|
<thead><tr><th>Step</th><th>What it delivers</th><th>Status</th><th>Where it actually stands</th></tr></thead>
|
|
<tbody>
|
|
|
|
<tr><td class="ph">1</td>
|
|
<td><b>The store</b> — where documents, equipment names and ready-made summaries live</td>
|
|
<td><span class="badge b-live">Running now</span></td>
|
|
<td>Healthy for eight days. It holds 92 passages from seven documents, the 8 equipment items and 56
|
|
tags that turn “Pump 02” into a real tag, and — since step 9 — the upload
|
|
queue and a permanent audit trail that nothing in the system is allowed to delete from. Five
|
|
database logins with the split that matters: <b>the answer path can only read</b>, and only the
|
|
loading job can write documents. Checked by hand, not assumed: the reading login was proven unable
|
|
to write.</td></tr>
|
|
|
|
<tr><td class="ph">2</td>
|
|
<td><b>The logbook</b> — a record of every question, put in early on purpose so every later experiment is traceable</td>
|
|
<td><span class="badge b-live">Running now</span></td>
|
|
<td>Both parts healthy for seven days, and every question since has been recorded: how it was
|
|
sorted and with what confidence, what was retrieved, how long it took, and whether the rulebook
|
|
passed or refused it. Every refusal is kept with the wording that caused it. <b>One thing changed on
|
|
28 August:</b> now that the operator does not sign in, the logbook no longer knows <em>who</em>
|
|
asked — only that somebody did.</td></tr>
|
|
|
|
<tr><td class="ph">3</td>
|
|
<td><b>The documents</b> — the real procedures and manuals, read in and searchable</td>
|
|
<td><span class="badge b-live">Running on real documents</span></td>
|
|
<td><b>Four real documents have now been through it</b> — three control descriptions and an
|
|
instrument document — alongside the three fabricated demo ones, for 92 passages in total. The
|
|
first real document was worth more than all the testing before it: an eight-page control philosophy
|
|
exposed <b>three silent faults in one upload</b>, every one of which left the screen looking
|
|
perfectly correct. The whole document had become a single passage, because the PDF reader emits no
|
|
blank lines and the splitter had nothing to split on. Every passage was titled
|
|
“Revision”, because a cover page flattens into a column of labels and the title matcher
|
|
ran on past the line break. And re-publishing the same document quietly duplicated it, leaving the
|
|
same passage quotable twice. All three are fixed, and the two loading paths are now held identical
|
|
by a test rather than by good intentions. What is outstanding is the rest of the real document set,
|
|
and a named reviewer for each.</td></tr>
|
|
|
|
<tr><td class="ph">4</td>
|
|
<td><b>Access to plant history</b> — a read-only login to the historian</td>
|
|
<td><span class="badge b-blocked">Blocked — the only one</span></td>
|
|
<td><b>The only thing this project still needs from anybody.</b> The server is still being built and
|
|
the read-only login has not been created; the connection settings sit on the host waiting, with the
|
|
password and database name blank. Also outstanding: the agreed table names, the firewall rule, and
|
|
confirmation of whether the historian stores time as UTC or local. That last one is not a formality
|
|
— <b>three separate findings cannot be settled without the answer</b>, and one of them decides
|
|
whether an alarm time on screen is right or a day out.</td></tr>
|
|
|
|
<tr><td class="ph">5</td>
|
|
<td><b>Cube</b> — the data translator, with every definition written down</td>
|
|
<td><span class="badge b-partial">Running on stand-in figures</span></td>
|
|
<td>Running as two containers — the translator and its own store, which is a hard dependency
|
|
rather than an optimisation — and answering the data lane against 145 stand-in alarms, 129,603
|
|
level readings and 309 pump operations. Two of its definitions were found wrong and fixed while
|
|
proving it by hand, which is what proving it is for. <b>Three findings are deliberately left open</b>
|
|
until the real historian settles them, the newest found today: see section 2. Two further gate items
|
|
are ours, not anybody else’s — no number has been independently confirmed by an
|
|
engineer, and the ready-made summaries are not being written back into the store as the design
|
|
specifies.</td></tr>
|
|
|
|
<tr><td class="ph">6</td>
|
|
<td><b>The assistant itself</b> — the sorting step, the four lanes, the answer writer and the rulebook</td>
|
|
<td><span class="badge b-live">Running, with both AI steps</span></td>
|
|
<td>All four lanes answer end to end on the host with a real model, through the rulebook. Proven
|
|
again today: an interlock question came back in <b>3.96 seconds</b> with the governing
|
|
procedure’s identity, its prerequisites quoted word for word, its steps withheld entirely, and
|
|
the scope banner attached. <b>Eight faults have been found and pinned since the model went on</b>
|
|
— five within an hour of switching it on, and three more when the first real document went
|
|
through. Every one was invisible while the stand-in was in place. The exam grew from 67 questions to
|
|
75 to cover them.</td></tr>
|
|
|
|
<tr><td class="ph">7</td>
|
|
<td><b>The operator’s screen</b> — the question box, the answer, the working</td>
|
|
<td><span class="badge b-live">Used from the control room, end to end</span></td>
|
|
<td><b>The screen has been used from the SCADA machine, end to end.</b> It is live, and since 28
|
|
August it admits that machine by its network address with <b>no sign-in at all</b> — see
|
|
section 3b for what that costs. The refusing half of that rule was checked from the host and works:
|
|
this host, the remote connection and the internet are all shut out. <b>The admitting half has now
|
|
been confirmed the only way it could be</b> — from the console. The screen’s own record
|
|
shows the control-room machine opening the page on 28 August, the day the rule went in, and asking
|
|
three questions on 31 August, each answered.</td></tr>
|
|
|
|
<tr><td class="ph">8</td>
|
|
<td><b>The exam</b> — every engineer-checked question run end to end and scored</td>
|
|
<td><span class="badge b-blocked">Runnable, and never run</span></td>
|
|
<td><b>Nothing is blocking this and it has still not been done.</b> The model account arrived on 27
|
|
August, so both AI steps can be scored; no scorecard has ever been produced. The set covers 28
|
|
historical, 18 procedural, 14 advisory and 13 reference questions plus 2 that should be refused as
|
|
unclear, every data question with its time window pinned so the answer does not drift between runs.
|
|
Pass mark unchanged: 85% correct overall, 95% on the two dangerous kinds, and <b>zero</b> breaches of
|
|
the rulebook. Two questions fail by design until step 4 lands.</td></tr>
|
|
|
|
<tr><td class="ph">9</td>
|
|
<td><b>Document management by operators</b> — uploading and withdrawing documents from a screen instead of a terminal</td>
|
|
<td><span class="badge b-partial">Live, built ahead of step 8</span></td>
|
|
<td><b>Built on 28 August at the customer’s direction, before step 8 rather than after it.</b>
|
|
It was sequenced after the exam for a reason: adding a way for more documents to arrive makes a
|
|
wrong answer harder to diagnose, not easier. Upload, convert, review, approve, withdraw and restore
|
|
all work and have been used on four real documents. <b>Five deliberate shortcuts</b> come with it
|
|
— see section 3b — the most important being that the publisher is a name typed into a
|
|
box with no password. What has not moved is the judgement: nothing is quotable until a named person
|
|
has confirmed the document’s number, revision and date, and the database itself refuses an
|
|
approved document without them. The one part of the design deliberately left out is choosing which
|
|
documents are in the searchable set.</td></tr>
|
|
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div class="note stop"><b>Read the two red rows together.</b> Step 4 is the only thing anybody
|
|
outside this project owes us. Step 8 is the only thing on this list that nothing is blocking and
|
|
nobody has done — and step 9 went live in front of it, so people can now add documents to a
|
|
search path that has never been formally scored. That is a sequencing decision, taken deliberately;
|
|
it is not an accident, and it is reversible, but it is the sentence a reviewer should see first.</div>
|
|
|
|
<h2>5 · The short version</h2>
|
|
<ul class="next">
|
|
<li><b>Running today:</b> the store, the logbook, the name list, the data translator, document
|
|
search, all four lanes, the rulebook and the operator’s screen. A question typed in the box
|
|
comes back as an answer, with its evidence and its working, in well under a second.</li>
|
|
<li><b>Running, on stand-in data:</b> the figures are stand-ins, not plant data. That one is
|
|
labelled on screen, on every answer, and an operator cannot turn the label off. The documents are no
|
|
longer only demo ones — four real documents have been loaded through the new upload screen,
|
|
alongside the three fabricated ones. The two AI steps are no longer on this list; they run.</li>
|
|
<li><b>What switching the model on actually taught us.</b> Five faults appeared within the hour,
|
|
and not one of them could have been found any earlier, because the stand-in had been quietly
|
|
supplying the very things that turned out to be missing. Four were the same mistake in four places:
|
|
the answer writer was being asked for a <em>fact</em> — which revision, what the document is
|
|
called, which document a limit came from — and it answered with something empty or invented.
|
|
Every one is now taken from the retrieved document instead, so the wording step cannot influence it
|
|
at all.</li>
|
|
<li><b>And then the first real document found three more.</b> An eight-page control philosophy,
|
|
uploaded on 28 August, exposed three faults that every test and every demo document had missed
|
|
— and all three left the screen looking perfectly correct. The whole document had collapsed
|
|
into a single passage; every passage was titled “Revision”; and re-publishing one
|
|
quietly duplicated it. The lesson is not that the code was careless. It is that <b>fabricated
|
|
documents are shaped like what the code expects</b>, and one real file was worth more than the
|
|
entire demo set.</li>
|
|
<li><b>The one worth telling a safety reviewer.</b> Asked how to lift an interlock, the assistant
|
|
was ranking the procedure’s sections by similarity to the question — which handed the
|
|
answer writer <em>the step list</em>, the one thing it must never reproduce, while withholding the
|
|
title block it actually needed. It now identifies which procedure governs the question, then
|
|
returns that document’s heading, purpose and prerequisites and withholds the steps entirely.
|
|
The rulebook already refused instruction language and still does; this removes the temptation
|
|
rather than relying on catching it.</li>
|
|
<li><b>What it still does not prove:</b> whether the labelling and wording are <em>good enough</em>.
|
|
They work; they are not yet scored. The exam can now be run and has not been. And no
|
|
operator has yet sat at a control-room PC and got an answer — the one check that cannot be
|
|
done from here.</li>
|
|
<li><b>Found today, and worth repeating:</b> the most obvious question at this station —
|
|
how many high level alarms last week — cannot be answered at all, because the alarm and its
|
|
recorded occurrences are filed against different equipment. The rulebook caught it and returned an
|
|
error instead of a number. That is the system working; it is also a reminder that the stand-in data
|
|
is hiding this class of problem rather than proving it absent.</li>
|
|
<li><b>Two things were traded away on 28 August,</b> both deliberately and both reversible: the
|
|
operator’s screen no longer asks anyone to sign in, and document management went live before
|
|
the exam that was meant to precede it. Section 3b says what each one costs.</li>
|
|
<li><b>Needs other people:</b> just one thing now — the read-only historian login. The model
|
|
account and the network name records both arrived on 27 August. A small model deployment is still
|
|
wanted so the running-cost figures hold, but nothing is blocked without it.</li>
|
|
<li><b>Before any operator sees this:</b> what the assistant will and will not say must be reviewed
|
|
with an OT/safety representative. That review needs the model in place to be meaningful, which is
|
|
one more reason the account is the item to chase first.</li>
|
|
</ul>
|
|
|
|
<p class="foot">Sign-in, network routing and hosting are handled by platform services already running
|
|
on the demo host. They are deliberately left off this map — they protect the assistant, but they are
|
|
not part of how it answers a question.</p>
|
|
|
|
</div>
|
|
</body>
|
|
</html>
|