diff --git a/BUILD-AI-CONTAINERS.md b/BUILD-AI-CONTAINERS.md index e27c057..2a5c509 100644 --- a/BUILD-AI-CONTAINERS.md +++ b/BUILD-AI-CONTAINERS.md @@ -62,7 +62,7 @@ These are **code paths, not prompt instructions**. Prompts are advisory and mode |---|---|---| | `yau-poc-cicore1` | SCADA server. Operator Chromium runs here. Holds the **raw historian**. Acts as **Modbus master**, polling the PLC on `lin001`. | **Built** | | `yau-sls-poc-imh` | SQL Server. Holds a **copy of the raw SCADA historian** — safe to query directly with no impact on the live system. | **⚠ Pending setup** | -| `yau-sls-poc-lin001` | Ubuntu 22.04 Docker host, `10.0.0.17`. 22 containers already running, **including `openplc-runtime`** — the PLC for this demo, serving Modbus TCP on host port 502. **Everything we build goes here.** | **Built** | +| `yau-sls-poc-lin001` | Ubuntu 22.04 Docker host, `10.0.0.17`. 21 containers already running, **including `openplc-runtime`** — the PLC for this demo, serving Modbus TCP on port 502. **Everything we build goes here.** | **Built** | ### `openplc-runtime` — added after the host brief was written @@ -70,8 +70,27 @@ The PLC for this demo runs as a container on `lin001`. SCADA on `cicore1` polls Consequences worth knowing: -- **It is the one deliberate exception to the no-published-ports rule** (host brief §10.6). Modbus is not HTTP and cannot go through Caddy. That exception is justified; it does not generalise to anything we build. -- **Port 502 has no authentication and no encryption.** Modbus never has. It must be reachable from the LAN and VPN only — confirm the Azure NSG does not expose it to the internet. This is worth checking now rather than assuming. +- **It is not the only published port on this host** — an earlier draft of this + document said it was. Verified on the host 2026-08-20: `caddy` publishes 80 and 443, + `wireguard` 443/udp, `mosquitto` 1883, and `chirpstack-gateway-bridge` 1700/udp, all on + `0.0.0.0`. The no-published-ports rule (host brief §10.6) is about **new web services**, + which belong on the `proxy` network behind Caddy. It still applies in full to + everything we build. Do not read the precedent more widely than that. +- **Port 502 is bound to `10.0.0.17`, not `0.0.0.0`** — verified on the host 2026-08-20: + + ``` + ports=map[502/tcp:[{10.0.0.17 502}] 8443/tcp:[{10.0.0.17 8443}]] + ss -lntp → LISTEN 10.0.0.17:502 + ``` + + So it is published on the VNet interface only and is not internet-reachable at the + Docker level, whatever the NSG says. That is a stronger position than this document + originally assumed, and it is the reason the NSG item in §15 is now a confirmation + rather than an open risk. **`openplc-runtime` also publishes 8443** — the OpenPLC + Runtime web UI — on the same private address; earlier drafts did not mention it. +- **Port 502 still has no authentication and no encryption.** Modbus never has. The + binding above is what contains it, so anything that changes the binding to `0.0.0.0`, + or any NSG rule that exposes the VNet address, removes the only control on it. - **`lin001` is now in the control path for the demo.** Restarting Caddy or Authelia doesn't touch Modbus, but a host-level problem — disk full, OOM, reboot — now stops the simulated plant as well as the web stack. Weigh that before any disruptive work, and announce it. - **Do not add `openplc-runtime` to Watchtower's update list**, and do not restart it casually while a demo is running. @@ -97,7 +116,7 @@ What local Postgres (`pg-ai`) is still for: `lin001` is **shared and live** — it runs customer-facing demos. From the host brief, §10: 1. **Growing data goes on `/datadisk`, never `/`.** Root is 62 GB and has hit 100% before, killing Grafana. -2. **No published host ports.** New services join the external `proxy` network and are reached through Caddy. +2. **No published host ports for anything we build.** New services join the external `proxy` network and are reached through Caddy. Some existing containers do publish ports — `caddy`, `wireguard`, `mosquitto`, `chirpstack-gateway-bridge`, `openplc-runtime` — because they carry non-HTTP protocols that cannot go through a reverse proxy. Nothing in the AI stack is in that category. 3. **Never bypass Authelia.** Omitting `import authelia` silently makes a service public. 4. **`~/authelia/configuration.yml` is root-owned.** Edit with `sudo`, back up first (`.bak--`), and know that restarting Authelia **logs out every active user**. 5. **AD group membership must be DIRECT** — nested membership silently fails. @@ -106,7 +125,7 @@ What local Postgres (`pg-ai`) is still for: 8. **Announce restarts of Caddy or Authelia** — they interrupt everyone. 9. **No secrets in Git or in compose files.** The Grafana admin password sitting in `~/docker-compose.yml` is a known defect, not a pattern to copy. Use a `0600` env file, following `~/authelia/authelia.env`. 10. Orphan-container warnings are expected (shared Compose project name) — ignore them. -11. **`openplc-runtime` is live control for the demo.** Do not restart, update or reconfigure it as a side effect of AI work. Do not reuse its published-port pattern for anything we build. +11. **`openplc-runtime` is live control for the demo.** Do not restart, update or reconfigure it as a side effect of AI work. Do not reuse its published-port pattern for anything we build, and do not change its `10.0.0.17` binding to `0.0.0.0` — that binding is what keeps unauthenticated Modbus off the internet. --- @@ -389,7 +408,7 @@ Each phase ends in a gate. Gates are not suggestions. ### Phase 1 — Compose scaffold and `pg-ai` **Tasks** -1. `mkdir -p /datadisk/pg-ai /datadisk/ai-docs` — **check `df -h /datadisk` first** (43% used, InfluxDB owns 52 GB). +1. `mkdir -p /datadisk/pg-ai /datadisk/ai-docs` — **check `df -h /datadisk` first** (46% used as at 2026-08-20, InfluxDB owns 55 GB and is growing). 2. Write `~/ai-compose.yml` with `pg-ai` only. Internal network, no published ports, log rotation, healthcheck. 3. `~/ai/pg-ai.env` at `0600`. 4. Apply `001_schema.sql`, `003_roles.sql`. Load `equipment.csv` and `tags.csv` with alias arrays. @@ -569,7 +588,7 @@ Deployed early, deliberately: from here on, every experiment is traced. ## 13. Cost and capacity -- **`/datadisk` is 128 GB and 43% used**, with InfluxDB at 52 GB and growing. Check `df -h` before every phase that writes data. The Grafana disk alert is **UI-only — nobody gets notified.** +- **`/datadisk` is 128 GB and 46% used** as at 2026-08-20 (65 GB free), with InfluxDB at 55 GB and growing — it is effectively the only consumer. It was 43%/52 GB when this document was first written, so budget for roughly 1 GB a week of InfluxDB growth on top of whatever the AI stack adds. `/` is 62 GB and 24% used (48 GB free). Check `df -h` before every phase that writes data. The Grafana disk alert is **UI-only — nobody gets notified.** - `CHEAP_DEPLOYMENT` for the classifier, entity extraction and tool selection; `CHAT_DEPLOYMENT` for final prose only. - Cap output tokens — output bills several times higher than input. - Keep system prompts byte-identical between calls so prompt caching applies. @@ -586,7 +605,7 @@ Deployed early, deliberately: from here on, every experiment is traced. - No OT/IT firewall boundary — one flat `10.0.0.0/24` PoC network - Public egress to Azure OpenAI, no private endpoint - Chromium running on the SCADA VM itself -- Modbus TCP on port 502 with no authentication or encryption — inherent to the protocol; contained by NSG/VPN scope only +- Modbus TCP on port 502 with no authentication or encryption — inherent to the protocol; contained by its bind to `10.0.0.17` plus NSG/VPN scope. The OpenPLC Runtime web UI on 8443 is contained the same way and nothing else - Single host, no HA — `lin001` is now a single point of failure for **both** the demo estate and the simulated plant's PLC - Shared `azureuser` login; no per-person audit trail on the host - Shared service account to `imh`; no per-operator row-level security @@ -605,7 +624,8 @@ Production closes these in the order: network segmentation → secrets → SQL g - [ ] `pg-ai` included in a backup routine and a restore tested once - [ ] Eval scorecard committed, broken down by question class - [ ] `/datadisk` headroom checked and recorded -- [ ] Azure NSG confirmed to expose port 502 to LAN/VPN only, never the internet +- [x] Port 502 confirmed bound to `10.0.0.17`, not `0.0.0.0` — verified on the host 2026-08-20, so it is not internet-reachable at the Docker level. Still worth confirming the NSG agrees, and worth re-checking after any change to `openplc-runtime` +- [ ] Port 22 on the VM is open to the internet (`20.211.144.151:22`). Key-only auth, but review it in the same NSG pass and restrict to office and VPN ranges - [ ] **Section 2 reviewed with an OT/safety representative before any operator sees a demo** - [ ] Section 14 reviewed and confirmed as still-accurate shortcuts - [ ] The new services added to the host documentation, following the existing change-log convention diff --git a/CLAUDE.md b/CLAUDE.md index 0b9160d..902c380 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -22,14 +22,18 @@ Advisory beats Historical. Partly-advisory is advisory. ## Host rules — inherited, non-negotiable - Growing data goes on **`/datadisk`**, never `/`. Root is 62 GB and has hit 100% before. -- **No published host ports.** Join the external `proxy` network, reach it through Caddy. +- **No published host ports for anything we build.** Join the external `proxy` network, reach it + through Caddy. Some existing containers do publish (`caddy`, `wireguard`, `mosquitto`, + `chirpstack-gateway-bridge`, `openplc-runtime`) — they carry non-HTTP protocols. We do not. - **Never omit `import authelia`** from a Caddyfile block — it silently makes a service public. - `~/authelia/configuration.yml` is **root-owned**: `sudo`, back up as `.bak--`. Restarting Authelia **logs out every active user** — announce it first. - AD group membership must be **DIRECT**. Nested membership silently fails. - `pg-ai` and `cube` are pinned — **do not add them to Watchtower's update list**. - **`openplc-runtime` is live control for this demo.** Never restart, update or reconfigure it as - a side effect of AI work. Do not copy its published-port pattern. + a side effect of AI work. Do not copy its published-port pattern, and never change its + `10.0.0.17` binding to `0.0.0.0` — that binding is what keeps unauthenticated Modbus off the + internet. - Never install on, write to or restart `cicore1`. Never exceed read-only on `imh`. - **No secrets in Git, compose files, logs or error messages.** `0600` env files under `~/ai/`.