Commit graph

5 commits

Author SHA1 Message Date
Claude
fd85e62ebf Add the document library screens: upload, review, withdraw, restore
Phase 9's operator path, built ahead of Phase 8 at the customer's direction and
live at api.yokogawa.tech/documents. Upload, convert, review, approve, withdraw
and restore. The pool screen is explicitly out of scope.

Served by ai-api rather than ai-web, and mounted at /documents rather than
/docs. ai.yokogawa.tech is SCADA-only since 2026-08-28 and passes through no
Authelia, so it has no identity to record; publishers arrive on
api.yokogawa.tech where the forward-auth headers still do. /docs stays with
Swagger, which the customer is keeping - two things under one prefix with two
different access policies is what gets misread during a later edit.

Conversion is text extraction, not document parsing: pypdf, python-docx and
openpyxl. Docling would be better at this and pulls torch, which lin001 has
neither the memory to install nor the business running next to the demo plant's
PLC. The cost is real - no layout, no table structure, and a scan cannot be read
at all, so it is refused rather than stored empty. It is acceptable only because
the converted text is shown to a person before the document can be cited, which
is the same safety net the design already required for the header. convert.py is
the one file to change if that stops being true.

Chunking is mirrored from ingest.py rather than shared, because the two live in
different images. They must stay identical: if they drift, the same document
chunks differently depending on who loaded it, and the assistant answers or
fails to answer depending on that. The step-sequence rule is locked by a test.

Identity is self-asserted for the demo - the actor is typed on the form, which
section 16 forbids, and the publisher list is one name with no password. Rows are
written as `demo:<name>` with actor_groups = 'DEMO-UNVERIFIED' so that when real
auth goes on, a name somebody typed stays tellable from a name Authelia proved.
doc_actions cannot be deleted from, so an ambiguity there would be permanent.

Two rules the code enforces rather than documents: uploading is open to anyone
who reaches the page, because uploading changes nothing an operator can see -
approving does, and that is what is gated; and an empty publisher list means
nobody, not everybody.

Verified on the host end to end: withdraw as a non-publisher 403s, with a short
reason 400s, and as admin flips 5 chunks and writes a complete audit row;
restore puts them back and keeps both rows. The corpus is unchanged afterwards.

Requirements are split so the document dependencies install in their own layer -
a change there costs four small wheels instead of re-resolving fastapi,
langgraph and langfuse on a 2 vCPU shared host.

The five divergences from section 16 are recorded in section 14. The one with
teeth: files published through the UI stay in the inbox, so `ai-ingest --all`
cannot see them and the two paths must not be used on the same document.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 14:13:17 +10:00
Claude
40f32087a9 Take the facts back off the model
Four faults, all surfaced within an hour of the first live Azure OpenAI call
on 2026-08-27, all invisible under NO_LLM_STUB because the stub supplied the
very fields that turned out to be missing.

1. The classifier few-shot showed eight replies of {"question_class": ...}
   alone. A few-shot reply is a shape the model copies, so it omitted
   confidence, which defaulted to 0.0, fell below the 0.7 threshold, and EVERY
   non-procedural question downgraded to UNCLEAR. The replies now carry the
   complete payload the system prompt asks for. Confidences are varied and the
   traps carry alternatives: a constant teaches the model to emit that
   constant, and the tie rule in apply_safety_rules only has something to work
   with if the runners-up are populated.

2. procedure{} was the one part of the procedural payload not assembled from
   evidence, contrary to _assemble's own stated rule. The model returned
   effective_date "" - neither a date nor None - so ProceduralAnswer rejected
   the answer, the single regeneration failed identically, and every procedural
   question returned 422.

3. title and authorising_role came back "" for the same reason: the model was
   asked for header fields it had never been shown.

4. documented_limits[].citation arrived as the string "WRPS-DEMO-003, Section
   4" where a Citation was required, because the schema hint said only
   "documented_limits": [] and told the model nothing about the shape.

procedure_identity now takes no `generated` argument at all: there is no path
by which a model can name a revision an operator does not hold. documented_
limits attaches the real Citation by matching source_file against what was
actually retrieved, and DROPS a limit matching nothing - a limit carries the
authority of the document behind it, and misattributing one is worse than
omitting it.

Both live in contracts.py rather than agent.py because they are contract
rules, and because agent.py imports langgraph, which would make the test suite
unrunnable on a bare checkout.

The prompt also now separates two things it was conflating: retrieval
returning nothing (say so and stop) from retrieval returning a document marked
draft, demo or superseded (identify it, quote it, and state the marking).
Including the header chunk made the model read "NOT A CONTROLLED DOCUMENT" and
answer "no controlled procedure was retrieved" while citing one. The marking is
information the operator needs, not a reason to withhold what was found.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:54:29 +10:00
Claude
b3e47506b0 Identify the procedure first, then withhold its steps
Two retrieval faults, both only visible once a real model ran.

find_procedure ranked a procedure's chunks by similarity to the question. For
"how do I lift the interlock on Pump 02" the closest chunks ARE the step list -
so the branch whose entire purpose is not reproducing steps was handing the
answer writer nothing but steps, while omitting the header block carrying the
title and the authorising role. The model was being asked for a title it had
never been shown, and returned "".

Once the document is identified, WHICH document it is settles what to send:
the header and the prerequisites, in document order, never the steps.
STEP_SECTION_RE is a second line behind ProceduralAnswer's instruction-language
check, not a replacement for it - the contract still rejects instruction
language whatever arrives here. This removes the temptation rather than relying
on catching it.

Separately, rerank did 0.75 * chunk.similarity where similarity is NULL for a
chunk ingested with --no-embed: `1 - (NULL <=> vec)` is NULL, so it raised
TypeError and 500'd the whole question rather than ranking that chunk last. It
now degrades to the lexical half - an unembedded chunk is still findable, just
not by meaning - and Chunk.similarity is typed honestly as float | None.

find_procedure_lexical takes the same identify-then-expand shape, so the stub
keeps testing the shape it always did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:54:13 +10:00
Claude
e1499864d9 Pin the site timezone on every Cube query
Every Historical and Advisory answer stated a window in AEST and queried one
shifted by ten hours.

rolling_window() builds its boundary strings in SITE_TIMEZONE - that is the
whole point of it, and its docstring says so. metrics.run() then posted the
query to Cube with no timezone at all, and Cube defaults to UTC. So
"2026-08-14T15:22:13" meant 15:22 Sydney to the code that produced it and 15:22
UTC to the engine that ran it, and MetricResult.time_window reported
SITE_TIMEZONE from config rather than whatever the query actually used, so the
two could not disagree visibly.

Measured on the fixtures, same dateRange, one field changed:

    timezone UTC               8019 samples
    timezone Australia/Sydney  8619 samples

600 samples. One per minute, ten hours, exactly the offset.

Nothing about the answer looked wrong. The prose was right, the count was a
real count, the window description was correctly formatted and correctly named
AEST. It was only visible by reading the Cube query in the UI's "show working"
panel - which is an argument for that panel existing, and an argument for
looking at the thing in a browser rather than trusting curl against the API.

  - check_cube_query() now takes site_timezone and pins it onto the query, at
    the single point every Cube query passes through. Per-query-builder is the
    wrong place: "remember to set the timezone" is not a control, and this
    defect is what forgetting looks like. An explicit timezone already on the
    query is left alone.
  - time_window now reports capped["timezone"] - the timezone the query ran in,
    not the one it should have run in.

An unpinned timezone belongs in the same guardrail as an unpinned date range,
and for the same reason: both make an answer unreproducible. The difference is
that an unpinned date range is obvious in the query and an unpinned timezone
is invisible.

eval case H28 records it. Two unit tests: the timezone is pinned, and an
explicit one is not overridden.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 15:23:27 +10:00
Claude
34d2ccc576 Scaffold the WRPS plant operations assistant repository
Build spec and host brief carried in from C:\Claude and WRPS/02-env; the
plant model (equipment, tags, alarm bitmask, enums, unit conversions) is
derived from WRPS/04-plc/register-map.csv, WRPS/05-scada/modbus/scada-points.csv
and WRPS-CTL-003.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 13:56:32 +10:00