diff --git a/BUILD-AI-CONTAINERS.md b/BUILD-AI-CONTAINERS.md index 7bb37a1..77a0286 100644 --- a/BUILD-AI-CONTAINERS.md +++ b/BUILD-AI-CONTAINERS.md @@ -195,7 +195,9 @@ ai.yokogawa.tech { **Azure hairpin:** LAN hosts cannot reach the VM's public IP from inside the VNet. For an operator on `cicore1` to reach `ai.yokogawa.tech` by hostname, the DC needs a pinpoint record → `10.0.0.17`, the same treatment `influx.yokogawa.tech` already has. **Raise this early** — it is a dependency on someone else and it will not surface until Phase 7. -**Done 2026-08-27** for `ai.yokogawa.tech` only. `api` and `cube` have public A records but **no pinpoint record**, so they do not resolve inside the VNet at all. The operator UI therefore does not call `api.yokogawa.tech`: Caddy routes `/ask` under `ai.yokogawa.tech` to `ai-api` and the page is same-origin. Only `/ask` — the Phase 9 publisher rule is scoped to `api.yokogawa.tech`, and a wider route there would make it inert. See `caddy/ai-routes.caddy`. +**Done 2026-08-27** for `ai.yokogawa.tech` only. `api` and `cube` have public A records but **no pinpoint record**, so they do not resolve inside the VNet at all. + +**`auth.yokogawa.tech` has no pinpoint record either** — confirmed 2026-08-28, it does not resolve from inside the VNet. Every Authelia-gated service redirects there, so before 2026-08-28 a LAN browser reached `ai.yokogawa.tech`, got a correct 302 to the portal, and then failed on DNS. Nobody had hit it because the device agents write to Influx over the `/api/v2/write` MFA bypass and never touch the portal. The operator path no longer needs that record (§14, the `cicore1` bypass); anything else gated and browsed from the LAN still would. The operator UI therefore does not call `api.yokogawa.tech`: Caddy routes `/ask` under `ai.yokogawa.tech` to `ai-api` and the page is same-origin. Only `/ask` — the Phase 9 publisher rule is scoped to `api.yokogawa.tech`, and a wider route there would make it inert. See `caddy/ai-routes.caddy`. --- @@ -597,13 +599,14 @@ local time changes the answer. **Tasks** 1. React + Vite: question box, answer pane, **"show working"** panel (class, query, row count, citations with revision and effective date). 2. Procedural and Advisory answers carry a visible scope banner stating what the assistant did *not* do. Operators must not infer this from tone. -3. `ai-web` behind Caddy + Authelia at `ai.yokogawa.tech`, with `/ask` routed to `ai-api` under the same hostname so the page is same-origin. Build `ai-web` with `VITE_API_BASE` empty; a bundle carrying `api.yokogawa.tech` works from outside and fails on every control-room PC. +3. `ai-web` behind Caddy at `ai.yokogawa.tech`, with `/ask` routed to `ai-api` under the same hostname so the page is same-origin. Build `ai-web` with `VITE_API_BASE` empty; a bundle carrying `api.yokogawa.tech` works from outside and fails on every control-room PC. 4. ~~**Pinpoint DNS on the DC** → `10.0.0.17` so `cicore1` can resolve it (Azure hairpin). Ask Dan.~~ **Done 2026-08-27.** -5. Confirm the operator's AD account is a **direct** member of `HTTPS_UserAccess` and Duo-enrolled. +5. ~~Confirm the operator's AD account is a **direct** member of `HTTPS_UserAccess` and Duo-enrolled.~~ **Superseded 2026-08-28** — the operator no longer signs in. The block admits `remote_ip 10.0.0.21` only and 403s everything else; `import authelia` is gone from it. See §14, and `caddy/ai-routes.caddy` for how to put the gate back. Still required for `api.yokogawa.tech`, which is unchanged. **Gate** -- [ ] `curl -sI https://ai.yokogawa.tech` → 302 to the auth portal -- [ ] `curl -s -o /dev/null -w '%{http_code}' -X POST https://ai.yokogawa.tech/ask` → 302, **not 404**. A 404 is ai-web answering, which means the `/ask` route is missing and every question will fail. +- [ ] `curl -sI https://ai.yokogawa.tech` **from lin001** → 403. That is the deny arm: this host, the VPN and the internet are all shut out. A 302 means the SCADA-only block was reverted; a 200 means the matcher is restricting nothing. +- [ ] `curl -s -o /dev/null -w '%{http_code}' -X POST https://ai.yokogawa.tech/ask` from lin001 → 403, **not 404**. A 404 is ai-web answering, which means the `/ask` route is missing and every question will fail once you are on cicore1. +- [ ] **On `cicore1`**: `https://ai.yokogawa.tech` loads the UI with no login. Nothing on lin001 can prove this — a typo in the matcher 403s cicore1 too and looks identical from the host. - [ ] An operator on `cicore1` reaches the UI **by hostname** and gets an answer end to end — this is the check that catches a name resolvable from outside and not from inside. Doing it from an engineer's laptop proves nothing about it. - [ ] Citations show document number, revision and effective date - [ ] The scope banner appears on every Procedural and Advisory answer @@ -709,6 +712,7 @@ local time changes the answer. - Secrets in `0600` env files, not a vault - No OT/IT firewall boundary — one flat `10.0.0.0/24` PoC network +- **`ai.yokogawa.tech` is unauthenticated from `cicore1`.** Applied 2026-08-28 at the customer's direction: an on-site operator should not complete a Duo push to ask a question, and nobody outside the plant should reach the assistant at all. The Caddy block admits `remote_ip 10.0.0.21` only and returns 403 to everything else, `import authelia` removed from that branch. This is an IP allowlist on a flat network with no OT/IT boundary — anything that can take `10.0.0.21`, or ARP-spoof it, inherits unauthenticated access to every answer the assistant can give. It is a demo affordance, **not a security control**, and it is the first thing network segmentation closes. Two consequences worth stating separately: Langfuse traces are now anonymous, so there is no record of who asked what; and the assistant is unreachable by browser from the VPN, so engineers need an SSH tunnel. `api.yokogawa.tech` is unchanged and still fully gated — Phase 9 document publishing depends on that and must stay there - Public egress to Azure OpenAI, no private endpoint - Chromium running on the SCADA VM itself - Modbus TCP on port 502 with no authentication or encryption — inherent to the protocol; contained by its bind to `10.0.0.17` plus NSG/VPN scope. The OpenPLC Runtime web UI on 8443 is contained the same way and nothing else diff --git a/CLAUDE.md b/CLAUDE.md index d1b3558..a60256b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,6 +26,9 @@ Advisory beats Historical. Partly-advisory is advisory. through Caddy. Some existing containers do publish (`caddy`, `wireguard`, `mosquitto`, `chirpstack-gateway-bridge`, `openplc-runtime`) — they carry non-HTTP protocols. We do not. - **Never omit `import authelia`** from a Caddyfile block — it silently makes a service public. + One deliberate exception, applied 2026-08-28 at the customer's direction: `ai.yokogawa.tech` + admits `remote_ip 10.0.0.21` (`cicore1`) unauthenticated and 403s everything else. It does + not generalise — do not copy it, and do not widen it. `api.yokogawa.tech` stays gated. - `~/authelia/configuration.yml` is **root-owned**: `sudo`, back up as `.bak--`. Restarting Authelia **logs out every active user** — announce it first. - AD group membership must be **DIRECT**. Nested membership silently fails. diff --git a/README.md b/README.md index a6afd34..38d014e 100644 --- a/README.md +++ b/README.md @@ -239,6 +239,26 @@ Each prints the manual DNS/Caddy/Authelia steps. rules for the three names still have to be applied on the host — until they are, the names resolve and nothing answers. +**The operator console is unauthenticated, and only from the SCADA machine.** +Applied 28 August 2026 at the customer's direction. The `ai.yokogawa.tech` Caddy +block admits `remote_ip 10.0.0.21` (`yau-poc-cicore1`, static) and returns 403 +to everything else — LAN, VPN and internet alike. An operator at the console +should not complete a Duo push to ask a question; nobody outside the plant +should reach the assistant at all. + +Read this before relying on it. It is an **IP allowlist on a flat network with +no OT/IT boundary**: anything that can take `10.0.0.21` inherits unauthenticated +access. It is a demo affordance, not a security control, and it is listed as +such in [`BUILD-AI-CONTAINERS.md`](BUILD-AI-CONTAINERS.md) §14. It also makes +Langfuse traces anonymous — there is no longer a record of who asked what — and +it puts the assistant out of browser reach over the VPN, so engineers need an +SSH tunnel. `api.yokogawa.tech` is unchanged and still fully gated; Phase 9 +document publishing depends on that and must stay there. Rollback is a Caddy +reload — the Authelia rule was left in place on purpose. See +[`caddy/ai-routes.caddy`](caddy/ai-routes.caddy). + +**This is in scope for the §2 OT/safety review**, which is still outstanding. + **`api.yokogawa.tech` has no pinpoint record and does not resolve inside the VNet.** That is why the Phase 7 Caddy block routes `/ask` under `ai.yokogawa.tech` to `ai-api` and the page is same-origin: a cross-origin diff --git a/REQUESTS.md b/REQUESTS.md index 1c73afe..f022134 100644 --- a/REQUESTS.md +++ b/REQUESTS.md @@ -80,10 +80,22 @@ Dan for this item. Two things are worth recording: now routes `/ask` under `ai.yokogawa.tech` to `ai-api` and the page is same-origin. No further DNS is needed for the demo. `cube.yokogawa.tech` is for engineers on the public side only; `ai-api` reaches Cube container-to-container and never by hostname. +- **`auth.yokogawa.tech` has no pinpoint record either**, and that was the defect this item + missed. Confirmed 28 August 2026: it does not resolve from inside the VNet. Every + Authelia-gated service redirects there, so a LAN browser reached `ai.yokogawa.tech`, got a + correct 302 to the portal, and then died on DNS — the name resolved, the service answered, + and the operator still saw "this site can't be reached". It went unnoticed because the + device agents write to Influx over the `/api/v2/write` MFA bypass and never touch the + portal; no browser had ever hit Authelia from inside the VNet. **This is no longer needed + for the operator path** — on 28 August the console was made unauthenticated from + `cicore1` only (README §6, `BUILD-AI-CONTAINERS.md` §14) — but any *other* gated service + browsed from the LAN will hit exactly this wall, so it is worth asking Dan for regardless. + - **The names resolving is not the same as the services answering.** The Caddyfile blocks and the Authelia rules for `ai`, `api` and `cube` were applied on the host the same day; all four AI hostnames now return 302 to the auth portal. What is still unproven is the - part no check here can reach: an operator on `cicore1`, signed in with Duo, getting an + part no check here can reach: an operator on `cicore1` — since 28 August, with no sign-in + at all — getting an answer end to end. The original request follows, unchanged. diff --git a/authelia/access-rules.md b/authelia/access-rules.md index df886ce..6197aab 100644 --- a/authelia/access-rules.md +++ b/authelia/access-rules.md @@ -16,7 +16,22 @@ Four hostnames join the existing `HTTPS_UserAccess` `two_factor` rule in | `lf.yokogawa.tech` | 2 | Langfuse — traces, prompts, eval runs | | `cube.yokogawa.tech` | 5 | Cube semantic layer, playground and REST API — **not essential**, see below | | `api.yokogawa.tech` | 6 | `ai-api` FastAPI | -| `ai.yokogawa.tech` | 7 | `ai-web` operator UI | +| `ai.yokogawa.tech` | 7 | `ai-web` operator UI — **rule is INERT as of 2026-08-28, see below** | + +> **`ai.yokogawa.tech` no longer passes through Authelia.** On 2026-08-28 the +> Caddy block for that hostname dropped `import authelia`: the SCADA console +> (`10.0.0.21`) reaches the assistant unauthenticated and every other source +> gets 403. Caddy never issues the forward-auth subrequest, so this domain +> entry in `configuration.yml` is now dead weight. +> +> **It was deliberately left in place. Do not remove it.** While the entry +> stands, restoring the gate is a one-line Caddyfile edit and a `caddy reload` +> — no Authelia restart, no all-user logout. Deleting it converts that rollback +> into a config edit plus a restart that logs out everyone on the host, +> including whoever is mid-demo, at exactly the moment somebody has decided the +> assistant needs to be gated again. +> +> `api.yokogawa.tech` is unchanged and still gated. Phase 9 depends on it. Phase 9 adds no hostname. It adds one **path-scoped rule** on an existing one — see "Phase 9: the document publisher rule" below. diff --git a/caddy/ai-routes.caddy b/caddy/ai-routes.caddy index 76a216f..b89d1a8 100644 --- a/caddy/ai-routes.caddy +++ b/caddy/ai-routes.caddy @@ -140,14 +140,55 @@ api.yokogawa.tech { # AFTER handle in Caddy's default directive order - outside a route block the # handles would be terminal and the gate would never run, silently serving the # UI to anyone. Inside `route`, directives run in written order. +# APPLIED 2026-08-28: THIS BLOCK NO LONGER IMPORTS AUTHELIA. +# +# At the customer's direction, the operator console is unauthenticated and the +# assistant is unreachable from anywhere else. Two reasons, both theirs: an +# operator standing at the SCADA console should not complete a Duo push to ask +# a question, and nobody outside the plant should reach the assistant at all. +# +# 10.0.0.21 (yau-poc-cicore1, static) -> straight through, no auth +# everything else -> 403 +# +# Why an IP matcher works here, verified on the host before applying: +# - Caddy sees REAL client addresses. Docker's DNAT preserves the source; the +# access log carries public IPs and 10.0.0.5 with no 172.18.x rewriting. If +# that ever stops being true this block fails CLOSED (403 for everyone), +# which is the safe direction but will look like an outage. +# - WireGuard peers do NOT match. The wg container sits on a bridge at +# 172.19.0.6 and MASQUERADEs outbound, so a peer reaching 10.0.0.17:443 +# arrives as 172.19.0.6. VPN users get 403 - including engineers. Use an +# SSH tunnel, or add a second matcher, deliberately. +# +# WHAT THIS COSTS, and it is not small: +# - ai-api no longer learns who is asking. Langfuse traces are anonymous and +# there is no record of who asked what. +# - It is an IP allowlist on a flat network with NO OT/IT boundary. Anything +# that can take 10.0.0.21 inherits unauthenticated access. Documented in +# BUILD-AI-CONTAINERS.md S14 as a shortcut, NOT as a security control. +# - It is in scope for the S2 OT/safety review, which is still outstanding. +# +# DO NOT put /docs here. Phase 9 publishing needs Remote-User/Remote-Groups and +# stays on api.yokogawa.tech, which is unchanged and still fully gated. +# +# TO RESTORE THE GATE: delete the @scada matcher and the two handle wrappers, +# put `import authelia` back as the first directive inside `route`. That is a +# Caddy reload only - the ai.yokogawa.tech entry in the Authelia rule was left +# in place precisely so this does not need an Authelia restart. See +# authelia/access-rules.md. ai.yokogawa.tech { + @scada remote_ip 10.0.0.21 route { - import authelia - handle /ask* { - reverse_proxy ai-api:8000 + handle @scada { + handle /ask* { + reverse_proxy ai-api:8000 + } + handle { + reverse_proxy ai-web:80 + } } handle { - reverse_proxy ai-web:80 + respond "Not available from this network" 403 } } } diff --git a/scripts/deploy.sh b/scripts/deploy.sh index cbcc55d..cf9f9e3 100644 --- a/scripts/deploy.sh +++ b/scripts/deploy.sh @@ -152,6 +152,14 @@ deploy_web() { warn "resolve inside the VNet. Do not widen that route past /ask - the" warn "Phase 9 publisher rule is scoped to api.yokogawa.tech. See" warn "caddy/ai-routes.caddy." + warn "" + warn "SCADA-ONLY, applied 2026-08-28: the ai.yokogawa.tech block does NOT" + warn "import authelia. It admits remote_ip 10.0.0.21 (cicore1) and 403s" + warn "everything else - so step 3 below does not apply to this hostname." + warn "Leave the ai.yokogawa.tech entry in the Authelia rule in place anyway;" + warn "it is inert, and it keeps rollback to a Caddy reload instead of an" + warn "Authelia restart. Verifying from this host gives 403, which is the PASS." + warn "The allow arm can only be proved from cicore1." # A bundle built with VITE_API_BASE set to a public hostname works from # outside and fails on every control-room PC. Catch it here, not on the day. if docker exec ai-web sh -c 'grep -rqs "api\.yokogawa\.tech" /usr/share/nginx/html' 2>/dev/null; then diff --git a/scripts/verify.sh b/scripts/verify.sh index d2003e9..1f376d9 100644 --- a/scripts/verify.sh +++ b/scripts/verify.sh @@ -58,7 +58,7 @@ head_ "Public endpoints - expect 302 to the auth portal" # and cube do not. So on this host those three do not resolve at all, and that # says nothing about whether they work from a browser. The check reports the # two cases separately rather than calling both a failure. -for host in lf.yokogawa.tech cube.yokogawa.tech api.yokogawa.tech ai.yokogawa.tech; do +for host in lf.yokogawa.tech cube.yokogawa.tech api.yokogawa.tech; do if ! getent hosts "$host" >/dev/null 2>&1; then printf ' ?? %s does not resolve FROM THIS HOST (no DC pinpoint record) - check it from outside the VNet ' "$host" @@ -74,19 +74,29 @@ for host in lf.yokogawa.tech cube.yokogawa.tech api.yokogawa.tech ai.yokogawa.te esac done -head_ "The operator page and the API share an origin" -# The browser posts to /ask on ai.yokogawa.tech. If the route is missing, ai-web -# serves the SPA's 404 and every question fails - with the UI itself looking -# perfectly healthy. Unauthenticated this must be the auth portal redirect, NOT -# a 404 from ai-web. -code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "https://ai.yokogawa.tech/ask" -H 'Content-Type: application/json' -d '{}' --max-time 10 || echo "000") -case "$code" in - 302|303) ok "ai.yokogawa.tech/ask -> $code (routed to ai-api, gated)" ;; - 403) bad "ai.yokogawa.tech/ask -> 403 - Authelia has no rule for this hostname yet; the route cannot be proved until it does" ;; - 404) bad "ai.yokogawa.tech/ask -> 404 - the /ask route is missing from the Caddy block; ai-web is answering" ;; - 000) bad "ai.yokogawa.tech/ask unreachable" ;; - *) bad "ai.yokogawa.tech/ask -> $code" ;; -esac +head_ "ai.yokogawa.tech is closed to everything except the SCADA console" +# Applied 2026-08-28: the block admits remote_ip 10.0.0.21 (cicore1) only and +# 403s everything else. lin001 is NOT 10.0.0.21 as Caddy sees it, so from here +# 403 is the PASS - it proves the deny arm works and that this host, the VPN and +# the internet are all shut out. +# +# THIS SCRIPT CANNOT PROVE THE ALLOW ARM. A typo in the matcher gives 403 to +# cicore1 too and looks identical from here. Somebody must open +# https://ai.yokogawa.tech on cicore1 and get the UI with no login. There is no +# way around that, and it is the same gap as the Phase 7 end-to-end check. +for u in "https://ai.yokogawa.tech" "https://ai.yokogawa.tech/ask"; do + code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$u" \ + -H 'Content-Type: application/json' -d '{}' --max-time 10 || echo "000") + case "$code" in + 403) ok "$u -> 403 from this host (deny arm working)" ;; + 302|303) bad "$u -> $code - still going through Authelia; the SCADA-only block was not applied or was reverted" ;; + 200) bad "$u -> 200 FROM THIS HOST - the matcher is not restricting anything. The assistant is open to the LAN, the VPN and, via the public A record, the internet" ;; + 404) bad "$u -> 404 - the /ask route is missing from the Caddy block; ai-web is answering" ;; + 000) bad "$u unreachable - Caddy has no certificate, or nothing is listening" ;; + *) bad "$u -> $code" ;; + esac +done +printf ' [33m>>[0m ALLOW ARM UNPROVEN: open https://ai.yokogawa.tech on cicore1 (10.0.0.21) - expect the UI, no login\n' # api.yokogawa.tech has no pinpoint DNS record, so it is unresolvable from # inside the VNet. A bundle that hard-codes it loads fine here and fails on a # control-room PC. Check what was actually built into the image.