wrps-demo-kit/02-environment/MIGRATION.md

# Migrating `openplc-runtime` to another Docker host (air-gapped)

> [!NOTE]
> **COMPLETED 2026-08-19. This is the as-built record of the migration, kept as a
> runbook for repeating it.** The PLC now runs on `yau-sls-poc-lin001`
> (`10.0.0.17`) and CI Server on `yau-poc-cicore1` (`10.0.0.21`) polls it live.
> The retired source host `dev-ubuntu` is no longer part of this project.
>
> **Part E was not executed as written.** The service was deployed as its own
> compose file on its own network rather than appended to the shared stack — see
> the "What was actually done" note in Part E. The as-built configuration is
> `openplc-compose.yml` and `openplc-container.md`.

Source: `dev-ubuntu` (SSH, now retired) · Destination: `yau-sls-poc-lin001`,
Portainer UI + SSH · both x86-64. Written 2026-08-18 against the live container.

Why `docker commit` and not `docker pull` on the far side: the compiled PLC
program lives in the container's **writable layer** (`/workdir/build/libplc_*.so`,
`/workdir/core/generated/`), not in the volume. Only a commit carries it.

Two artifacts move:

- `openplc-image.tar.gz` — committed image (~400–500 MB gz, 1.08 GB virtual)
- `openplc-vol.tar.gz` — the 20 KB volume (`restapi.db` = users + program record, `.env` = JWT secret)

---

## PART A — On the SOURCE host (SSH: `ssh dev-ubuntu`) — ✅ DONE 2026-08-19

Executed in the order A1 → A2 → A4 → A5 → A3 → A6 (volume tar and restart moved
ahead of the image save) to cut PLC downtime to ~10 s. The commit at A2 has
already frozen the layer, so the artifacts are identical either way.

**Result — files on `dev-ubuntu:/home/dev-admin/`:**

| File | Size | SHA-256 |
|---|---|---|
| `openplc-image.tar.gz` | 354 MB | `cbf2d2512d75acb5cce5640213d609383852f087bff6f4614397521ae7347d83` |
| `openplc-vol.tar.gz` | 746 B (mode 0600) | `a5a5011335a37f8f01fd0cae57bd8e135ec2e80fced30b723d8d08662d636628` |

Verified: committed image contains `/workdir/build/libplc_1786668930820554523.so`
and `/workdir/core/generated/`; volume tarball contains `.env` + `restapi.db`;
source PLC restarted clean — `MODBUS_SLAVE Server listening on 0.0.0.0:502`,
69 located vars, reachable from Windows.



**A1. Stop the container** — for a consistent `restapi.db` snapshot (SQLite). The PLC stops controlling; expected.

    docker stop openplc-runtime

**A2. Commit the container to an image** — freezes the writable layer (your compiled program).

    docker commit openplc-runtime openplc-runtime-migrated:v4.1.10

**A3. Save the image to a file** — one portable file. Takes 1–3 min.

    cd ~ && docker save openplc-runtime-migrated:v4.1.10 | gzip > openplc-image.tar.gz

**A4. Back up the volume.** `socket ignored` warnings are normal — runtime sockets, recreated on start.

    docker run --rm -v openplc-runtime-data:/data -v ~:/backup alpine \
      tar czf /backup/openplc-vol.tar.gz -C /data .

**A5. Restart the source container** — leaves the source working.

    docker start openplc-runtime

**A6. Record checksums + sizes.** Write the two hashes down; you verify them on the far side.

    cd ~ && ls -lh openplc-image.tar.gz openplc-vol.tar.gz && sha256sum openplc-*.tar.gz

---

## PART B — Transfer (from your Windows PC, PowerShell)

**B1. Pull down from source**

    scp dev-ubuntu:~/openplc-image.tar.gz dev-ubuntu:~/openplc-vol.tar.gz C:\Temp\

**B2. Push up to destination.** Use a dedicated subfolder — D3 bind-mounts this
directory into a container, and mounting the whole home directory would expose
`.ssh` and everything else to it. Not `/tmp`: cleared on reboot.

    ssh <user>@<dest-host> "mkdir -p ~/openplc-migration"
    scp C:\Temp\openplc-image.tar.gz C:\Temp\openplc-vol.tar.gz <user>@<dest-host>:~/openplc-migration/

Portainer's *Images → Import* upload also works for the image, but a 500 MB browser upload is failure-prone. Use scp.

---

## PART C — Pre-flight on the DESTINATION (SSH) — protects the existing 20 containers

Run all of these BEFORE changing anything. Every one must come back clean.

**C1. Verify the files arrived intact** — must match A6. If not, re-transfer.

    cd ~/openplc-migration && sha256sum openplc-image.tar.gz openplc-vol.tar.gz

**C2. Architecture + disk space** — `uname -m` must print `x86_64`; need ≥ 3 GB free.

    uname -m
    df -h /var/lib/docker

**C3. Name collisions — all three must return NOTHING**

    docker ps -a  --format '{{.Names}}'      | grep -x openplc-runtime
    docker volume ls --format '{{.Name}}'    | grep -x azureuser-openplc-runtime-data
    docker images --format '{{.Repository}}' | grep -x openplc-runtime-migrated

If any hits, stop and rename — see Appendix 1.

**C4. Port 502 free — must return NOTHING**

    sudo ss -lntp | grep -w 502
    docker ps --format '{{.Names}} {{.Ports}}' | grep -w 502

If taken, pick another host port — see Appendix 1.

---

## PART D — Load image + restore data (DESTINATION, SSH)

**D1. Load the image.** Expect `Loaded image: openplc-runtime-migrated:v4.1.10`. The existing 25 images are untouched.

    docker load -i ~/openplc-migration/openplc-image.tar.gz

**D2. Create the volume**

    docker volume create azureuser-openplc-runtime-data

**D3. Restore the data into it**

    docker run --rm -v azureuser-openplc-runtime-data:/data -v ~/openplc-migration:/backup alpine \
      tar xzf /backup/openplc-vol.tar.gz -C /data

**D4. Confirm the restore.** Expect `.env` and `restapi.db`. Sockets are absent — correct, they get recreated.

    docker run --rm -v azureuser-openplc-runtime-data:/data alpine ls -la /data

---

## PART E — Deploy the service on the destination

> [!IMPORTANT]
> **What was actually done (2026-08-19), and why it differs from E1–E6 below.**
> Rather than appending the service to the host's existing shared stack, it was
> deployed from a **separate compose file**, `~/openplc-compose.yml`, on its
> **own network** `openplc-net`. This is safer and is the recommended route:
> nothing else on the host is touched, there is no shared `volumes:`/`networks:`
> block to merge, rollback is `docker compose -f ~/openplc-compose.yml down`, and
> the PLC is isolated from the other ~20 containers.
>
> Two further differences from the draft below: ports are published on
> **`10.0.0.17:502` and `10.0.0.17:8443`** — the LAN address, not `0.0.0.0` — and
> `ufw` is **inactive** on this host (F6 does not apply; the Azure NSG governs).
>
> The live file is copied verbatim to `openplc-compose.yml`.
> E1–E6 below describe the shared-stack alternative and are retained for a
> destination host where a separate file is not an option.

### Alternative — appending to an existing shared stack (NOT the route taken)

The existing stack owns ~20 running containers. Updating it runs
`docker compose up -d` over the whole file. Compose only recreates services whose
config hash changed, so adding one service normally leaves the rest running — but
follow E1 and E5 exactly, they are what keeps that true.

**E0. Confirm the stack is editable.** Portainer → **Stacks** → click the stack.
If it shows a **Git repository** section (with *Pull and redeploy*) instead of a
*Web editor*, STOP — the change must be committed to that git repo instead; a UI
edit gets reverted on the next sync.

**E1. Back up the current stack file.** Open the stack → **Editor** tab →
select all → copy → paste into a local file (e.g. `C:\Temp\dest-stack-backup.yml`).
This is your rollback. Do not skip it.

**E2. Note what is running now**, so you can prove nothing else restarted:

    docker ps --format '{{.Names}}\t{{.Status}}' | sort > ~/before.txt

**E3. Append the service.** In the **Editor**, add this under the existing
top-level `services:` key, at the same indent as the sibling services:

```yaml
  openplc-runtime:
    image: openplc-runtime-migrated:v4.1.10
    container_name: openplc-runtime
    restart: unless-stopped
    cap_add:
      - SYS_NICE
      - SYS_RESOURCE
    ports:
      - "<dest-lan-ip>:502:502"
      - "<dest-lan-ip>:8443:8443"
    volumes:
      - azureuser-openplc-runtime-data:/var/run/runtime
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"
```

`cap_add` is required for the runtime's real-time scheduling.
`<dest-lan-ip>` — the destination host's LAN address (here `10.0.0.17`), so the
OpenPLC Editor and CI Server on **`yau-poc-cicore1` (10.0.0.21)** can reach it.
Binding the specific NIC rather than `0.0.0.0` keeps **both** 502 and 8443 off
every other interface. On a host with a public IP this is the only thing keeping
unauthenticated Modbus off the internet — never publish on `0.0.0.0`.

**E4. Merge the volume declaration.** The file already has a top-level `volumes:`
block — add this entry INSIDE it. Do not add a second `volumes:` key; a duplicate
top-level key is a YAML error or silently discards one block.

```yaml
  azureuser-openplc-runtime-data:
    external: true
```

`external: true` is load-bearing: without it Compose creates a NEW empty volume
named `azureuser_azureuser-openplc-runtime-data` and your restored data is ignored.

**E5. Deploy.** Click **Update the stack**. In the confirmation dialog:

- **Re-pull image — leave OFF.** Turning it on re-pulls every service in the
  stack; anything on a moving tag gets recreated.
- **Prune services — leave OFF.**

**E6. Confirm nothing else moved.**

    docker ps --format '{{.Names}}\t{{.Status}}' | sort > ~/after.txt
    diff ~/before.txt ~/after.txt

The only expected difference is the new `openplc-runtime` line. If other services
show fresh uptimes, they were recreated — see Rollback in Part G.

### Note: shared stack network

The service joins the existing stack's default network, so it can reach — and be
reached by — the other ~20 containers by service name. If you want it isolated,
give it its own network instead:

```yaml
    networks:
      - openplc-net
```

plus a top-level `openplc-net:` entry merged into the existing `networks:` block.
Port publishing to the host works either way.

---

## PART F — Verify

**F1.** Portainer → **Containers** → `openplc-runtime` shows **running**, green.

**F2. Logs** — Portainer container view → **Logs**, or by SSH:

    docker logs --tail 50 openplc-runtime

Look for the runtime starting and the `modbus_slave` plugin loading. No repeating crash loop.

**F3. Port is listening** (SSH on destination)

    sudo ss -lntp | grep -w 502

**F4. The PLC program came across** (SSH on destination) — a `libplc_*.so` must be present.

    docker exec openplc-runtime ls -la /workdir/build/ | grep libplc

**F5. Modbus read from your Windows PC** — the real acceptance test. Must match the register map, same as against the old host.

    cd C:\Claude\wrps-demo-kit\05-tests
    python verify_modbus.py --host <dest-host> --port 502 --unit 1

**F6. Firewall on the destination.** Check current state first.

    sudo ufw status

On `yau-sls-poc-lin001` this returns **`Status: inactive`** — the host has no
firewall and inbound filtering is entirely the **Azure NSG**. The `ufw` commands
below were therefore **not applied** and are kept only for a destination host
that does run `ufw`.

Modbus, for CI Server — only needed if F5 times out:

    sudo ufw allow from <SCADA-subnet>/24 to any port 502 proto tcp

REST API, for the OpenPLC Editor. Restrict to the single Editor host, not the
subnet — 8443 is the control channel (upload, start/stop the program) and JWT auth
is the only thing in front of it:

    sudo ufw allow from 10.0.0.21 to any port 8443 proto tcp

**F7. REST API reachable from the Editor host (run on 10.0.0.21).**

    curl -k https://<dest-lan-ip>:8443/api/login -X POST \
      -H 'Content-Type: application/json' \
      -d '{"username":"admin","password":"<see 02-environment/secrets.local.md>"}'

`-k` is required — the runtime serves a self-signed cert, so the Editor will show
a trust warning on first connect too. A JWT in the response means the path is open.

---

## PART G — Afterwards

- **CI Server:** ✅ done — `yau-poc-cicore1` (`10.0.0.21`) polls `10.0.0.17:502`
  (`04-scada/` — the Modbus client config is the one place a literal IP is used).
- **Do not leave both PLCs answering on 502** with the same tags. ✅ done — the
  source `dev-ubuntu` runtime was stopped and that host is retired.
- **Clean up transfer files on both hosts:** `rm -rf ~/openplc-migration`.
  `openplc-vol.tar.gz` contains the JWT secret — do not leave it lying around, and
  never commit either file to git.

### Rollback

Do NOT delete the stack — it owns the other ~20 containers.

1. Portainer → **Stacks** → the stack → **Editor** → paste back the E1 backup
   (`C:\Temp\dest-stack-backup.yml`), replacing the whole file.
2. **Update the stack**, again with *Re-pull image* and *Prune services* OFF.
   This removes the `openplc-runtime` service and leaves the rest as they were.
3. Then on the destination host:

        docker rm -f openplc-runtime 2>/dev/null
        docker volume rm azureuser-openplc-runtime-data
        docker rmi openplc-runtime-migrated:v4.1.10

4. Firewall, if F6 was applied:

        sudo ufw delete allow from 10.0.0.21 to any port 8443 proto tcp

The destination is back to its prior state. The source host was never modified
beyond a stop/start.

---

## Appendix 1 — If a pre-flight check (C3/C4) found a collision

- **Container name taken** → change `container_name:` to `wrps-openplc-runtime`.
- **Volume name taken** → use a new name in D2/D3 and in the compose file, e.g.
  `wrps-openplc-data`. The container path `/var/run/runtime` must NOT change.
- **Image name taken** → retag after D1:
  `docker tag openplc-runtime-migrated:v4.1.10 wrps/openplc-runtime:v4.1.10`
- **Port 502 taken** → publish elsewhere, e.g. `"5502:502"`, and point CI Server's
  Modbus client at port 5502. The container-side `502` must not change.
- **Port 8443 taken** on the destination → publish elsewhere, e.g.
  `"<dest-lan-ip>:9443:8443"`, and point the OpenPLC Editor at port 9443. The
  container-side `8443` must not change. Add this to the C4 pre-flight:

        sudo ss -lntp | grep -w 8443

## Appendix 2 — Login credentials

The runtime's `admin` password and JWT secret travel inside `restapi.db` / `.env`,
so the same credentials work on the destination. Nothing to reconfigure.
See `02-environment/secrets.local.md`.