Phase 12: Audit trail #13

Merged
cliu merged 5 commits from docs/phase-12-audit-trail into main 2026-10-01 17:38:25 +10:00
4 changed files with 291 additions and 0 deletions
Showing only changes of commit cfc48a888d - Show all commits

View file

@ -0,0 +1,39 @@
---
name: audit-trail
description: >-
CI Server R1.03 AUDIT Service and audit trail. Use when the user wants to know who changed
what and when (item value written, limit or description changed, item / section / object /
report inserted or deleted, login, logout, failed login, alarm acknowledged or shelved), read
or export the audit trail (AUDIT_TRAIL dataset, storage group ADT, old value / new value, user,
source HMI / LOC-ENG-MOD / ENT-ENG-MOD / OPER-INTERFACE / ODBC / QUICKLOAD), make an audit
report, switch auditing of an event type on or off (event classifiers, AUDIT_EVENT_CLAS_DF,
STORAGE, "Audit trailing wanted" / AUDIT_INFO on an item), filter audit events (selection
groups, AUDIT_SELGRP_DF, expression sbid / sbgp / user / nval), print or log audit events
(ADTLOG event logger, adtsys.sup, E_LAYOUT, SEL_GROUP), check AUDIT Service (adtsto, adtcol,
adtdst, audit_start), or change how long the audit trail is kept (ADT lifetime). Source
IM 36K10L25-01EN.
---
# Audit trail (R1.03)
Background: `knowledge/manuals/IM36K10L25-01EN.md` (processes, classifiers, setup keywords,
installed findings), `IM36K10G10-01EN.md` ch. 9 (overview), `IM36K10C40-01EN.md` (storage groups),
`IM36K10P20-01EN.md` (RQL for audit reports).
Rules:
- Run `system-check` first; follow the user's chosen way of working.
- The audit trail is evidence: single records cannot be deleted, and nothing in these tasks
deletes them. Changing the `ADT` storage group (lifetime, stop, delete units) loses history —
only with the user's explicit approval (skill `history-management`).
- Switching a classifier's `STORAGE` off stops recording that event type: confirm with the user.
- Setup files in `tls\sup` (`adtsys.sup`, per-process files) need admin rights: hand them to
the user (skill `portal-setup`, `change-setup-file.md`).
- Example names come from `CI-Eng-Sample` (selection group `CIENG_ITEMS`, report `CIENG_AUDIT`,
logger setup file `audit/adtlogci.sup`).
| Task | File | Status |
|---|---|---|
| Check AUDIT Service: processes, storage group `ADT`, classifier and selection group counts | `tasks/audit-checks.md` | verified |
| Read the audit trail: export, RQL query, audit report, what gets recorded by which tool | `tasks/read-audit-trail.md` | verified |
| Choose what is stored (classifier `STORAGE`), selection groups, event logger `ADTLOG` | `tasks/configure-audit.md` | verified (operator-side item value events from docs, ISS-049) |
| Distributor process, extra collectors, new classifier entities in C | L25 summary §2, §5 | from docs |

View file

@ -0,0 +1,59 @@
---
task: Check AUDIT Service (processes, storage group ADT, classifiers, selection groups)
version: R1.03
source: IM 36K10L25-01EN §1.2, §2.1.1, §2.3.1, §4.1; practiced on the test system 2026-10-01
status: verified
---
# AUDIT Service checks
All read-only.
## 1. Processes
```
tasklist /FI "IMAGENAME eq adt*"
durwfp -p adtsto -t 0 (errorlevel 0 = running)
```
Default (`tls\com\audit_start.cmd`): `adtsto.exe` (storage, process `ADTSTO`) and `adtcol.exe`
(collector `ADTCOL01`). AUDIT Service starts after HISTORY Service and refuses to start when
`his` is not running; `ADTCOL01` exits if storage group `ADT` does not exist (first start:
`ADTSTO` creates it). `adtdst` (distributor), `adtlog` (event logger) and `adtcol02` are only in
commented lines. Stop: `audit_stop.cmd` (`durstp` on `adt*`, admin).
## 2. Storage group `ADT`
```
@FIELDS
NAME,LIFE_TIME,ROLLOVER_INT,START_TIME,STOP_TIME,TYPE
@SELECT
'NAME = "ADT"'
@HIS_GROUP_DF
```
`dssqld -e adt.qlo adt.qli`. History type 6 (audit). Manual default when `ADTSTO` creates it:
rollover 1 day, unit lifetime 2 days; the test system had **52 weeks** lifetime, rollover
1 day. While storage of `ADT` is stopped, new events are refused (logged via UMH). Unit
details: skill `history-management`, `history-checks.md`.
## 3. Classifiers and selection groups
```
dssqld -e ecl.qlo -d AUDIT_EVENT_CLAS_DF
dssqld -e sgp.qlo -d AUDIT_SELGRP_DF
```
R1.03.00 test system: 463 classifiers (`BASIC_SET` 1), 444 with `STORAGE` 1. Off by default:
everything from source `COMPUTE` (PROCESS Service) on `ITEM`/`SUB-ITEM` values, status,
quality, limits and deadband, and `DATASET.NULL-ATTR.*.COMPUTE`. No selection groups exist
until you create one (the file `tls\dat\adt_sgp.dat` is created empty by `ADTSTO`).
Files: `%TLS_DAT_PATH%\adt_ecl.*` (classifiers), `adt_sgp.*` (selection groups); audit records
are in the history files of group `ADT` (`%TLS_HIS_PATH%`).
## 4. Setup files
Only `tls\sup\adtsys.sup` is installed (`QUESIZE = 200`, the rest commented): it is the keyword
reference and template. A process reads a setup file named after itself (`adtsto.sup`,
`adtcol01.sup`, `adtlog.sup` …) if present. Keywords: `configure-audit.md` §4 and the L25 summary.

View file

@ -0,0 +1,108 @@
---
task: Choose what AUDIT Service stores (classifier STORAGE, AUDIT_INFO), define selection groups, run an event logger (ADTLOG)
version: R1.03
source: IM 36K10L25-01EN §2.4, §3, §4.2, App. 1; practiced on CI-Eng-Sample 2026-10-01
status: verified (operator-side ITEM.* value events from docs, ISS-049)
---
# Configure the audit trail
## 1. Event classifiers — what is stored
Each event type is a classifier `<SUBJECT>.<ATTRIBUTE>.<ACTION>.<SOURCE>` in
`AUDIT_EVENT_CLAS_DF` (`NAME`, `SUBJECT`, `ATTRIBUTE`, `AGITATION` = action, `SOURCE`,
`BASIC_SET`, `STORAGE`). Only `STORAGE` can be changed. Configuration and DSS value changes use
subject `DATASET` with attribute `NULL-ATTR`, e.g. `DATASET.NULL-ATTR.MODIFY.LOC-ENG-MOD`
(Engineering Module) or `….ENT-ENG-MOD` (`dssqld`); logins use `CI-SERVER.NULL-ATTR.LOGIN.<source>`.
List the classifiers (and which are off): `dssqld -e ecl.qlo -d AUDIT_EVENT_CLAS_DF`. Change:
```
@FIELDS
NAME,STORAGE
@AUDIT_EVENT_CLAS_DF
"<SUBJECT>.<ATTRIBUTE>.<ACTION>.<SOURCE>",0
```
`dssqld -m` (the change itself is audited: `AUDIT_EVENT_CLAS_DF` `STORAGE` `MODIFY`).
**Not immediate:** the processes that report events (e.g. `DSSSDA`) cache the storage flags and
re-read them every `ECL_REFRESH` seconds (default 1800). In the test a switched-off classifier was
still recorded right after the change (ISS-050). Confirm with the user before
switching anything off.
### Item option *Audit trailing wanted* (`AUDIT_INFO`)
`ITEM_DF` / `SUB_ITEM_DF` / `EXTERNAL_ALARM_DF` field `AUDIT_INFO` (default 0). It did **not**
change what the tested tools record: DSS and Engineering Module value writes are audited for all
items, `itmval -p` for none. It is meant for the `ITEM.*` / `SUB-ITEM.*` classifiers (value,
status, limits) raised by ITEM Service for operator and PROCESS writes; PROCESS (`COMPUTE`) also
needs `AUDIT = YES` in `tls\sup\opc.sup` (installed `NO`) and its classifiers are off by default.
Not verified (ISS-049).
## 2. Selection groups
A selection group is a named, compiled filter for an AUDIT destination (`ADTLOG`, the
chronological overview). Dataset `AUDIT_SELGRP_DF`: `NAME` (≤ 32), `DESCRIPTION`, `EXPRESSION`
(memo), `COMPILE` (1 = compile now), `LISTING` (compiler output), `VALID_EXPRESSION`.
Expression language (from the compiler's own listing):
- Words compared with a **string**: `aai1 acnd acnm actm mai1 nond nonm reas sacs sact satr sbgp
sbid term user` (same meaning as the `E_LAYOUT` units in §4: `sbgp` subject, `satr` attribute,
`sact` action, `sacs` source, `sbid` subject id, `user` …).
- Words compared with a constant: `nval oval` (new / old value).
- Operators `= != > < >= <=`, `and`, `or`, parentheses. **No `LIKE`**: match a name prefix with a
range, e.g. `sbid >= "<Section>." and sbid < "<Section>/"` (`/` sorts right after `.`).
Example (`examples/CI-Eng-Sample/19_audit_df.qli`):
```
@FIELDS
NAME,DESCRIPTION,EXPRESSION,COMPILE
@AUDIT_SELGRP_DF
"CIENG_ITEMS","Audit events on CI-Eng-Sample",@"
sbid >= "CI-Eng-Sample." and sbid < "CI-Eng-Sample/"
"@,1
```
`dssqld -i`. Write the memo **over several lines** (`@"` newline … newline `"@`): on one line
`@"<text>"@` is taken as a file name (`CRL-E-EINVAL` / `CRL-E-ENOENT`). Check: export the dataset
— `LISTING` ends with `ADTT-I-CMP_SUCC, Expression successfully compiled` and
`VALID_EXPRESSION` is 1. An error (e.g. `ADTT-E-UNKNOWN_TOK`) makes the `dssqld` line fail with
`ADTT-E-CMP_NO_SUCC` and leaves `VALID_EXPRESSION` 0. Remove with `dssqld -r` (`NAME` only).
## 3. Event logger `ADTLOG`
`adtlog -n <name>` connects to `ADTSTO` and prints each new event as text (stdout, or the
printer in `PRT_DEV`). It is not started by default (`audit_start.cmd` has it commented out;
`fmfprt.sup` assigns the printer function `AUDIT_TRAIL` to `ADTLOG`). A process reads
`tls\sup\<name>.sup`, so one setup file per logger instance.
Without a setup file the default layout is three lines per event (time with milliseconds,
subject, attribute, action, source / subject id, user / old and new value). Verified test:
`adtlog -n adtlogt` in a console, then a value write — the logger printed the `LOGIN`, the
`ITEM_VAL` change and the `LOGOUT`. A console logger runs until stopped: Ctrl+C, or `durstp -m -p <name>` (skill
`system-management`, `bus-utilities.md`).
Filtered logger: `examples/CI-Eng-Sample/audit/adtlogci.sup` (`SEL_GROUP = "CIENG_ITEMS"`, one-line
`E_LAYOUT`, `TIMEFMT`, `HEADER`, `HEADER_AFTER`). The user copies it to
`%TLS_ROOT_PATH%\tls\sup\` (admin); then `adtlog -n adtlogci` printed only the CI-Eng-Sample
changes (no logins), one line each under the header. If the selection group does not exist or
did not compile, the logger logs an error and exits. To run it permanently, add a
`start /B /NORMAL %TLS_ROOT_PATH%\tls\exe\adtlog -n <name>` line to `audit_start.cmd` (admin,
restart).
## 4. Setup keywords (`adtsys.sup` is the template)
`MAX_OPEN_UNITS` (20), `PRT_DEV`, `QUESIZE` (kB), `SEL_GROUP`; `COL_PROC`, `DST_PROC`, `HIS_PROC`,
`STO_PROC` = `<node>, <name>`; `NODE_ECL_FILE`, `NODE_SGP_FILE`; `E_LAYOUT = <unit>,<width>,…`
(`AAI1 ACND ACNM ACTM MAI1 NOND NONM NOTM NVAL NWLN OVAL REAS SACS SACT SATR SBGP SBID SPCE TERM
USER`), `E_NOSPACE`, `TIMEFMT` (`DD/MM/YY hh:mm:ss.ppp` default — milliseconds shown, as the
manual says); `ECL_REFRESH` (1800 s), `FLOWC` (30, 10), `HBEAT` (10), `TRYCON` (10); `FORM_FEED`,
`HEADER` (repeat for more lines), `HEADER_AFTER` (re-print after n events; the header is also
printed once at start). Which keyword applies to which process: L25 §4.2.2.
## 5. Retention
Audit records live in history storage group `ADT` (rollover 1 day; lifetime 52 weeks on the
test system). Longer/shorter retention = change `LIFE_TIME` of `ADT` (skill
`history-management`); archive like any storage group. Ask before shortening it.

View file

@ -0,0 +1,85 @@
---
task: Read the audit trail (export, RQL query, audit report) and know what each tool records
version: R1.03
source: IM 36K10L25-01EN §1.1, §2.1.2, §3.1, App. 1; IM 36K10G10-01EN ch. 9; practiced on CI-Eng-Sample 2026-10-01
status: verified
---
# Read the audit trail
Read-only. Dataset `AUDIT_TRAIL` (storage group `ADT`); every field is read-only except
`APPLICATION_SUB` (the "modifiable ASCII application info") and `PROFILE`.
## 1. Fields
| Field | Content |
|---|---|
| `NAME` | `<ACTION_TIME>/<ACTION_STAMP>`, e.g. `01-OCT-26 17:05:00.181/84` |
| `ACTION_TIME`, `NOTIFICATION_TIME`, `MODIFICATION_TIME` | When it happened / reached AUDIT Service / `APPLICATION_SUB` was last changed |
| `SUBJECT_GROUP` | Subject: the **dataset name** for configuration and DSS value changes (`ITEM_DF`, `ITEM_VAL`, `SECTION_DF`, `REPORT_DF`, `AUDIT_EVENT_CLAS_DF` …) or a classifier subject (`CI-SERVER` for logins, `ITEM`, `SUB-ITEM` …) |
| `SUBJECT_ATTRIBUTE` | Changed field (`HIGH_LIMIT`, `ITEM_VALUE`, `DESCRIPTION` …); empty for insert/delete/login |
| `SUBJECT_ACTION` | `INSERT`, `MODIFY`, `DELETE`, `LOGIN`, `LOGOUT`, `LOGIN-ATTEMPT`, `ACKNOWLEDGE`, `SHELVE`, `UN-SHELVE` |
| `SUBJECT_SOURCE` | Tool that made the change (table in §2) |
| `SUBJECT_IDENT` | Record name: item, section, object (`<object>:<attribute>` for attribute values) … |
| `USER_NAME`, `PROFILE`, `TERMINAL` | CI Server user and profile; terminal often empty |
| `ACTION_NODE`/`ACTION_PROCESS`, `NOTIFICATION_NODE`/`NOTIFICATION_PROCESS` | Node and process (`DSSSDA00`, `JHMISET1086`, `DSSQLD00875` …) |
| `OLD_VALUE`, `NEW_VALUE` | Text, max 64 characters (longer values are cut) |
| `REASON`, `APPLICATION_INFO`, `APPLICATION_SUB` | Free text from the notifying program (empty in the tests) |
## 2. What is recorded (verified on R1.03.00)
| Action | Recorded as |
|---|---|
| `dssqld` load / modify / remove (any dataset) | `<DATASET>`, field, `INSERT`/`MODIFY`/`DELETE`, source **`ENT-ENG-MOD`**, user `FULL AUTHORIZATION`, plus a `CI-SERVER` `LOGIN`/`LOGOUT` pair per `dssqld` run (not `QUICKLOAD`) |
| Writing a value through DSS (`dssqld -m` on `ITEM_VAL`) | `ITEM_VAL` `ITEM_VALUE` `MODIFY`, old and new value — for **every** item, whatever its *Audit trailing wanted* (`AUDIT_INFO`) |
| Engineering Module: change an item field | `ITEM_DF` `<FIELD>` `MODIFY`, source `LOC-ENG-MOD`, user `ADMIN` (the logged-in user) |
| Engineering Module: right-click item › *Update value* | `ITEM_VAL` `ITEM_VALUE_STR` `MODIFY` (old/new) and `ITEM_VAL` `STATUS` `MODIFY`, source `LOC-ENG-MOD` |
| Engineering Module / Operator / OPC UA server login | `CI-SERVER` `LOGIN` / `LOGOUT`, source `LOC-ENG-MOD`, `OPER-INTERFACE`, `VIA-OPC-UAS` |
| `itmval -p` (ITEM Service API) | **nothing**, even with `AUDIT_INFO` 1 |
The `ITEM.VALUE.MODIFY.*` style classifiers (subject `ITEM`) were not produced by any of these
tools; they are expected from operator displays and PROCESS Service on items with
`AUDIT_INFO` 1 (open: ISS-049). The `SUBJECT_SOURCE` of a user interface comes from its *audit
trail id* in `dssana.sup` `USER_INTERFACE` (Engineering Module = `LOC-ENG-MOD`).
## 3. Export
`dssqld -e` skips `AUDIT_TRAIL` in a full export (`NO-EXPORT` in `dssqld.sup`); name it:
```
dssqld -e audit.qlo -d AUDIT_TRAIL
```
or filter (dates as `dd-mm-yyyy hh:mm:ss`, local time):
```
@FIELDS
NAME,SUBJECT_GROUP,SUBJECT_ATTRIBUTE,SUBJECT_ACTION,SUBJECT_SOURCE,SUBJECT_IDENT,USER_NAME,OLD_VALUE,NEW_VALUE
@SELECT
'SUBJECT_IDENT = "<Section>.<Tag>" AND ACTION_TIME > "01-10-2026 17:00:00"'
@AUDIT_TRAIL
```
`dssqld -e out.csv audit.qli -c` (CSV). **Keep `NAME` in `@FIELDS`**: without the key field the
export silently returns 0 records (verified). The whole trail can be large (5 000 records after a
year of test work). Wildcards: `SUBJECT_IDENT LIKE "<Section>.*"` (`*`, not `%`; `=` with `*`
matches nothing).
## 4. RQL query and audit report
```
column t as 'DD-MM-YYYY hh:mm:ss' heading 'Time';
select action_time t, subject_group, subject_attribute, subject_action, subject_source,
subject_ident, user_name, old_value, new_value
from audit_trail
where action_time > @date - 3600 and subject_ident like '<Section>.%'
order by action_time;
```
Use aliases with `COLUMN … AS 'A12'` to keep lines short (the fields are 16–256 characters
wide). Try it in the interactive generator (skill `reporting`, `rql-interactive.md`). The sample
report `CIENG_AUDIT` (`examples/CI-Eng-Sample/19_audit_df.qli`) lists the last hour of events on
`CI-Eng-Sample.*`; generate it like any ad-hoc report (`define-reports.md` §2). Generated from
`dssqld` it is stored but not printed (`FSL-E-PRT_UNKUSER`, ISS-045).
The Engineering Module also has the system report `AUDIT_TRAIL` (IM 36K10C30 App. 1).