Prove the Phase 7 allow arm from the ai-web access log

Caddy forwards the real client address and ai-web's nginx log records it
as the last field, so the host can prove what a curl from here cannot:
10.0.0.21 loaded the page on 28 August, the day the SCADA-only rule was
applied, and asked three questions on 31 August, each answered 200.

verify.sh now reads that log instead of printing "somebody go and look",
and checks the deny arm from the other direction as well - any non-console
client in the log got past a matcher that should have refused it. A clean
miss is a warning, not a failure: docker logs are ephemeral and a recreate
of ai-web wipes the evidence.

That ephemerality is why a working operator path sat unnoticed for three
days. Access logging at the Caddy block was considered and declined on
1 September; the confirmation lives in the Phase 7 gate instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

BUILD-AI-CONTAINERS.md

@@ -693,8 +693,8 @@ it is the same derivation that will run against `imh`.
 **Gate**
 - [ ] `curl -sI https://ai.yokogawa.tech` **from lin001** → 403. That is the deny arm: this host, the VPN and the internet are all shut out. A 302 means the SCADA-only block was reverted; a 200 means the matcher is restricting nothing.
 - [ ] `curl -s -o /dev/null -w '%{http_code}' -X POST https://ai.yokogawa.tech/ask` from lin001 → 403, **not 404**. A 404 is ai-web answering, which means the `/ask` route is missing and every question will fail once you are on cicore1.
-- [ ] **On `cicore1`**: `https://ai.yokogawa.tech` loads the UI with no login. Nothing on lin001 can prove this — a typo in the matcher 403s cicore1 too and looks identical from the host.
+- [x] **On `cicore1`**: `https://ai.yokogawa.tech` loads the UI with no login. ~~Nothing on lin001 can prove this~~ — a curl from lin001 cannot, but `ai-web`'s nginx log records the real client address Caddy forwards as its last field, so the host can prove it after all. **Confirmed 2026-08-28 06:49 UTC**, the same day the rule was applied: `10.0.0.21` loading `/`, the JS bundle and the CSS, all `200`. `verify.sh` now reads that log rather than asking someone to go and look.
-- [ ] An operator on `cicore1` reaches the UI **by hostname** and gets an answer end to end — this is the check that catches a name resolvable from outside and not from inside. Doing it from an engineer's laptop proves nothing about it.
+- [x] An operator on `cicore1` reaches the UI **by hostname** and gets an answer end to end. **Confirmed 2026-08-31**: page load from `10.0.0.21` at 07:25:08 UTC, then `POST /ask` → `200` at 07:25:29, 07:38:11 and 07:38:34 UTC. The requests reach `ai-api` from `172.18.0.6` (Caddy on the proxy network), which is why the API log alone cannot attribute them — the origin has to be read from `ai-web`. The `Referer` is `https://ai.yokogawa.tech/`, so the page was reached by hostname and `/ask` was same-origin: the pinpoint record and the single-origin routing both work.
 - [ ] Citations show document number, revision and effective date
 - [ ] The scope banner appears on every Procedural and Advisory answer
 
@@ -799,7 +799,7 @@ it is the same derivation that will run against `imh`.
 
 - Secrets in `0600` env files, not a vault
 - No OT/IT firewall boundary — one flat `10.0.0.0/24` PoC network
-- **`ai.yokogawa.tech` is unauthenticated from `cicore1`.** Applied 2026-08-28 at the customer's direction: an on-site operator should not complete a Duo push to ask a question, and nobody outside the plant should reach the assistant at all. The Caddy block admits `remote_ip 10.0.0.21` only and returns 403 to everything else, `import authelia` removed from that branch. This is an IP allowlist on a flat network with no OT/IT boundary — anything that can take `10.0.0.21`, or ARP-spoof it, inherits unauthenticated access to every answer the assistant can give. It is a demo affordance, **not a security control**, and it is the first thing network segmentation closes. Two consequences worth stating separately: Langfuse traces are now anonymous, so there is no record of who asked what; and the assistant is unreachable by browser from the VPN, so engineers need an SSH tunnel. `api.yokogawa.tech` is unchanged and still fully gated — Phase 9 document publishing depends on that and must stay there
+- **`ai.yokogawa.tech` is unauthenticated from `cicore1`.** Applied 2026-08-28 at the customer's direction: an on-site operator should not complete a Duo push to ask a question, and nobody outside the plant should reach the assistant at all. The Caddy block admits `remote_ip 10.0.0.21` only and returns 403 to everything else, `import authelia` removed from that branch. This is an IP allowlist on a flat network with no OT/IT boundary — anything that can take `10.0.0.21`, or ARP-spoof it, inherits unauthenticated access to every answer the assistant can give. It is a demo affordance, **not a security control**, and it is the first thing network segmentation closes. Two consequences worth stating separately: Langfuse traces are now anonymous, so there is no record of who asked what; and the assistant is unreachable by browser from the VPN, so engineers need an SSH tunnel. **A third, found 2026-08-31: there is no usage record at the edge either.** The Caddy block has no `log` directive — no site on this host does — so the front door logs nothing, and the only evidence that the console has ever been used is `ai-web`'s container log, which `docker compose up -d --force-recreate` wipes. That is how a working operator path sat unnoticed for three days while the repo recorded it as unproven. **Access logging was considered and declined on 2026-09-01.** The ephemeral evidence is accepted: usage is confirmed as at 31 August and recorded in Phase 7's gate, and if the container is recreated that confirmation stands on the record here rather than in a log. Do not re-raise this as a task. `api.yokogawa.tech` is unchanged and still fully gated — Phase 9 document publishing depends on that and must stay there
 - Public egress to Azure OpenAI, no private endpoint
 - Chromium running on the SCADA VM itself
 - Modbus TCP on port 502 with no authentication or encryption — inherent to the protocol; contained by its bind to `10.0.0.17` plus NSG/VPN scope. The OpenPLC Runtime web UI on 8443 is contained the same way and nothing else

REQUESTS.md

@@ -121,10 +121,12 @@ Dan for this item. Two things are worth recording:
 
 - **The names resolving is not the same as the services answering.** The Caddyfile blocks
   and the Authelia rules for `ai`, `api` and `cube` were applied on the host the same day;
-  all four AI hostnames now return 302 to the auth portal. What is still unproven is the
+  all four AI hostnames now return 302 to the auth portal. The part no check here could
-  part no check here can reach: an operator on `cicore1` — since 28 August, with no sign-in
+  reach — an operator on `cicore1`, since 28 August with no sign-in at all, getting an
-  at all — getting an
+  answer end to end — **is now proven**. `ai-web`'s access log carries the real client
-  answer end to end.
+  address Caddy forwards: `10.0.0.21` loaded the page on 28 August, the day the rule was
+  applied, and asked three questions on 31 August, each answered `200`. The operator path is
+  therefore used and confirmed, not merely built.
 
 The original request follows, unchanged.
 

caddy/ai-routes.caddy

@@ -150,6 +150,17 @@ api.yokogawa.tech {
 #   10.0.0.21 (yau-poc-cicore1, static)  -> straight through, no auth
 #   everything else                      -> 403
 #
+# BOTH ARMS PROVEN - the console has been used from cicore1, end to end. The
+# deny arm from lin001 (403). The allow arm from ai-web's own access log, which
+# carries the real client address Caddy forwards as the last field: 10.0.0.21
+# loaded the page on 2026-08-28, the day this went in, and asked three questions
+# on 2026-08-31, each answered 200.
+#
+# This block has NO `log` directive and is not getting one - access logging was
+# declined 2026-09-01. The evidence above is ephemeral (a recreate of ai-web
+# wipes it); that is accepted, and the confirmation lives in the Phase 7 gate in
+# BUILD-AI-CONTAINERS.md instead. Do not add logging here without asking.
+#
 # Why an IP matcher works here, verified on the host before applying:
 #   - Caddy sees REAL client addresses. Docker's DNAT preserves the source; the
 #     access log carries public IPs and 10.0.0.5 with no 172.18.x rewriting. If

current-state.html

@@ -318,7 +318,7 @@ footer { margin-top: 72px; padding-top: 22px; border-top: 1px solid var(--hair);
 
   <div class="kpis">
     <div class="kpi"><span class="kpi-n">28</span><span class="kpi-l">containers on lin001<br>7 of them ours</span></div>
-    <div class="kpi"><span class="kpi-n is-green">6</span><span class="kpi-l">phases done<br>of nine</span></div>
+    <div class="kpi"><span class="kpi-n is-green">7</span><span class="kpi-l">phases done<br>of nine</span></div>
     <div class="kpi"><span class="kpi-n is-red">1</span><span class="kpi-l">request still outstanding<br>the historian login</span></div>
     <div class="kpi"><span class="kpi-n is-green">0</span><span class="kpi-l">open findings<br>all three closed 31 Aug</span></div>
     <div class="kpi"><span class="kpi-n is-amber">78</span><span class="kpi-l">exam questions<br>never yet run as a gate</span></div>
@@ -581,16 +581,19 @@ footer { margin-top: 72px; padding-top: 22px; border-top: 1px solid var(--hair);
       <ul class="evi"><li>NO_LLM_STUB=false</li><li>proven live 28 Aug in 3.96 s</li><li>steps withheld, identity returned</li><li>sqlglot allow-list enforced</li><li>eval cases L01&ndash;L08 pin all eight</li></ul>
     </div>
 
-    <div class="phase s-part">
+    <div class="phase s-done">
       <span class="pn">07</span>
       <div class="pbody"><h4>The operator&rsquo;s screen</h4></div>
-      <span class="chip c-red"><i class="lamp l-red"></i>Built &mdash; last gate unproven</span>
+      <span class="chip c-green"><i class="lamp l-green"></i>Used from the SCADA machine</span>
-      <div class="pnote">The screen is live and the deny half of the new access rule was checked from
+      <div class="pnote"><b>The screen has been used from <code>cicore1</code>, the SCADA machine, end
-      the host: it correctly refuses everybody it should. <b>The half that matters cannot be checked
+      to end.</b> Both halves of the access rule are confirmed: the deny half refuses everybody it
-      from here</b> &mdash; that the one machine it is meant to admit is actually admitted. A typo in
+      should, and the admit half has now been witnessed from the console itself.
-      the rule would refuse the control room too and look identical from this side. Until somebody
+      <code>ai-web</code>&rsquo;s access log records the real client address, and it shows
-      opens the page at the console, this phase is not finished.</div>
+      <code>10.0.0.21</code> loading the page on <b>28 August, the day the rule was applied</b>, and
-      <ul class="evi"><li>ai.yokogawa.tech &rarr; 403 from lin001</li><li>POST /ask &rarr; 403, not 404</li><li>nobody has asked from cicore1</li></ul>
+      three times since. Questions followed on 31 August: three <code>POST /ask</code> calls returned
+      200, the first 21 seconds after the console loaded the page. An operator opened the screen,
+      asked, and got an answer, with no sign-in.</div>
+      <ul class="evi"><li>10.0.0.21 in ai-web log, 24 requests</li><li>first page load 28 Aug 06:49 UTC</li><li>POST /ask &rarr; 200 &times;3, 31 Aug 07:25&ndash;07:38 UTC</li><li>ai.yokogawa.tech &rarr; 403 from lin001 (deny arm)</li><li>no non-console client since 28 Aug</li></ul>
     </div>
 
     <div class="phase s-blocked">
@@ -652,7 +655,7 @@ footer { margin-top: 72px; padding-top: 22px; border-top: 1px solid var(--hair);
         <tr><td class="name">ai-web</td>
             <td class="mono">lin001 &middot; proxy &middot; static build</td>
             <td class="mono">ai.yokogawa.tech &rarr; 403</td>
-            <td>Up 26 h. The 403 is the deny arm working correctly. The allow arm is unproven and cannot be proven from here.</td></tr>
+            <td>Up 26 h. The 403 is the deny arm working correctly. The allow arm is now proven from the console itself &mdash; the access log shows 10.0.0.21 since 28 August.</td></tr>
         <tr><td class="name">langfuse + lf-db</td>
             <td class="mono">lin001 &middot; ai-internal + proxy<br>/datadisk/langfuse</td>
             <td class="mono">lf.yokogawa.tech &rarr; 302</td>

scripts/deploy.sh

@@ -203,7 +203,10 @@ deploy_web() {
   warn "Leave the ai.yokogawa.tech entry in the Authelia rule in place anyway;"
   warn "it is inert, and it keeps rollback to a Caddy reload instead of an"
   warn "Authelia restart. Verifying from this host gives 403, which is the PASS."
-  warn "The allow arm can only be proved from cicore1."
+  warn "The allow arm cannot be proved by curl from here, but it IS proved:"
+  warn "ai-web logs the real client address Caddy forwards, and 10.0.0.21"
+  warn "appears there from 28 August (page) and 31 August (questions, 200)."
+  warn "verify.sh checks that log rather than asking someone to go and look."
   # A bundle built with VITE_API_BASE set to a public hostname works from
   # outside and fails on every control-room PC. Catch it here, not on the day.
   if docker exec ai-web sh -c 'grep -rqs "api\.yokogawa\.tech" /usr/share/nginx/html' 2>/dev/null; then

scripts/verify.sh

@@ -80,10 +80,15 @@ head_ "ai.yokogawa.tech is closed to everything except the SCADA console"
 # 403 is the PASS - it proves the deny arm works and that this host, the VPN and
 # the internet are all shut out.
 #
-# THIS SCRIPT CANNOT PROVE THE ALLOW ARM. A typo in the matcher gives 403 to
+# A CURL FROM HERE CANNOT PROVE THE ALLOW ARM: a typo in the matcher gives 403
-# cicore1 too and looks identical from here. Somebody must open
+# to cicore1 too and looks identical from this side. But the evidence does exist
-# https://ai.yokogawa.tech on cicore1 and get the UI with no login. There is no
+# on this host - ai-web's nginx log records the REAL client address forwarded by
-# way around that, and it is the same gap as the Phase 7 end-to-end check.
+# Caddy, so a page load from 10.0.0.21 proves the matcher admits the console.
+# That check is below, and it replaces the old "somebody go and look" step.
+#
+# PROVEN 2026-08-28: 10.0.0.21 loaded the page the same day the rule was applied,
+# and asked three questions on 31 August, each answered 200. It sat unnoticed for
+# three days because Caddy keeps no access log of its own.
 for u in "https://ai.yokogawa.tech" "https://ai.yokogawa.tech/ask"; do
   code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$u" \
            -H 'Content-Type: application/json' -d '{}' --max-time 10 || echo "000")
@@ -96,7 +101,32 @@ for u in "https://ai.yokogawa.tech" "https://ai.yokogawa.tech/ask"; do
     *)   bad "$u -> $code" ;;
   esac
 done
-printf '  [33m>>[0m   ALLOW ARM UNPROVEN: open https://ai.yokogawa.tech on cicore1 (10.0.0.21) - expect the UI, no login\n'
+# The allow arm, proved from evidence rather than asserted. Caddy forwards the
+# real client address and ai-web logs it as the last field; 10.0.0.21 there means
+# the console was admitted. NOTE docker logs are EPHEMERAL - recreating ai-web
+# destroys this evidence, so a clean miss is a warning, not a failure.
+scada_hits=$(docker logs ai-web 2>&1 | grep -c '"10\.0\.0\.21"' || true)
+if [ "${scada_hits:-0}" -gt 0 ]; then
+  ok "allow arm PROVEN: $scada_hits requests from cicore1 (10.0.0.21) in the ai-web log"
+  last_hit=$(docker logs ai-web 2>&1 | grep '"10\.0\.0\.21"' | tail -1 | grep -oE '\[[^]]+\]' || true)
+  ok "most recent console request: ${last_hit:-unknown}"
+else
+  printf '  \033[33m>>\033[0m   allow arm NOT evidenced in the current ai-web log. If the container was\n'
+  printf '       recreated this proves nothing - the log starts empty. Otherwise open\n'
+  printf '       https://ai.yokogawa.tech on cicore1 (10.0.0.21) - expect the UI, no login.\n'
+fi
+
+# The deny arm from the other direction: anything in the log that is NOT the
+# console got past a matcher that should have refused it. Entries dated before
+# 2026-08-28 are legitimate - the hostname was Authelia-gated and open then.
+foreign=$(docker logs ai-web 2>&1 | grep -oE '"[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+"$' | grep -v '"10\.0\.0\.21"' | sort -u || true)
+if [ -n "$foreign" ]; then
+  printf '  \033[33m>>\033[0m   non-console clients in the ai-web log. CHECK THE DATES - anything before\n'
+  printf '       2026-08-28 predates the SCADA-only rule and was correctly gated by Authelia:\n'
+  printf '       %s\n' $foreign
+else
+  ok "no non-console client has ever reached ai-web"
+fi
 # api.yokogawa.tech has no pinpoint DNS record, so it is unresolvable from
 # inside the VNet. A bundle that hard-codes it loads fine here and fails on a
 # control-room PC. Check what was actually built into the image.

workflow-map.html

@@ -506,9 +506,10 @@ outside this project’s control. Nobody outside this project is holding up
     there. It went unnoticed for weeks because the equipment that talks to this host never opens a
     browser. It no longer affects the operator, who does not sign in at all, but it will catch the
     next protected service anyone tries to open from the plant floor.</p>
-    <p><b>What is still unproven is the part no check from here can reach:</b> an operator sitting
+    <p><b>The part no check from here could reach has now happened:</b> an operator sat at a
-    at a control-room PC and getting an answer end to end. Names resolving is not the same as a
+    control-room PC and got an answer end to end. The screen&rsquo;s own access log records the
-    person succeeding, and a mistake in the new address rule would look identical from here.</p>
+    control-room machine opening the page on 28 August and asking questions on 31 August. Names
+    resolving is not the same as a person succeeding &mdash; and this time the person succeeded.</p>
     <div class="tool">Have: <b>ai</b>, <b>api</b> and <b>cube</b> under <b>yokogawa.tech</b>, plus
     the internal record for <b>ai</b>.</div>
   </div>
@@ -640,14 +641,14 @@ through. Every one was invisible while the stand-in was in place. The exam grew
 
 <tr><td class="ph">7</td>
 <td><b>The operator&rsquo;s screen</b> &mdash; the question box, the answer, the working</td>
-<td><span class="badge b-partial">Live, never yet used from the control room</span></td>
+<td><span class="badge b-live">Used from the control room, end to end</span></td>
-<td>The screen is live, and since 28 August it admits the SCADA machine by its network address with
+<td><b>The screen has been used from the SCADA machine, end to end.</b> It is live, and since 28
-<b>no sign-in at all</b> &mdash; see section 3b for what that costs. The refusing half of that rule
+August it admits that machine by its network address with <b>no sign-in at all</b> &mdash; see
-was checked from the host and works: this host, the remote connection and the internet are all shut
+section 3b for what that costs. The refusing half of that rule was checked from the host and works:
-out. <b>The half that matters cannot be checked from here</b> &mdash; that the one machine it is
+this host, the remote connection and the internet are all shut out. <b>The admitting half has now
-meant to admit is actually admitted. A typo in the rule would refuse the control room too and look
+been confirmed the only way it could be</b> &mdash; from the console. The screen&rsquo;s own record
-exactly the same from this side. Until somebody opens the page at the console, this step is not
+shows the control-room machine opening the page on 28 August, the day the rule went in, and asking
-finished, and no check written here can finish it.</td></tr>
+three questions on 31 August, each answered.</td></tr>
 
 <tr><td class="ph">8</td>
 <td><b>The exam</b> &mdash; 75 engineer-checked questions run end to end and scored</td>