--- manual: IM 36K10C40-01EN title: Engineering Module - Historical edition: 3rd Edition, Sep. 2023 version: R1.03 status: Event/Item and Scan/Time storage groups verified by quick load (Aggregation loads, values not yet seen — ISS-040) on CI-Eng-Sample 2026-10-01; ithloahis loading rules verified; Engineering Module forms, archive and retrieve from docs --- # Engineering Module - Historical (R1.03) Item storage groups (*Configuration > Historisation > Item storage groups*), aggregation and archive in the Engineering Module, plus how history is read. History manager internals: `IM36K10L20-01EN.md`; storage processes: `IM36K10L35-01EN.md`. Section numbers refer to the manual; dataset fields were checked on the installed system. ## 1. Historical database (§1) | Storage group kind | Dataset | Created by | |---|---|---| | Item storage groups | `ITEM_HISTORY` | the engineer, one per collection/storage type and schedule | | Alarm (`ALH`) | `ALARM_CHRONO` | system | | Audit (`ADT`) | `AUDIT_TRAIL` | system | | System log (`SYS_LOG`) | `SYS_LOG` | system | | Reports | report storage | system | Items are only stored after they are added to an item storage group (`ITEM_HIS_DF`, name `:`). Displays read history through the trend, alarm overview and report components. ## 2. Item storage group (§2.1, dataset `HIS_GROUP_DF`) | Tab / field | `HIS_GROUP_DF` | Notes | |---|---|---| | Name | `NAME` | max 16 characters | | Type | `TYPE` | `Item` | | Node | `NODE` | cannot be changed later | | Exclude from archive | `EXCLUDE_ARCHIVE` | | | Activation / deactivation time | `START_TIME`, `STOP_TIME` | `STOP_TIME` in the future = stop then (used by `remove.cmd`, ISS-005) | | First rollover, interval | `FIRST_ROLLOVER`, `ROLLOVER_INT` (`"1 hours"`, `"1 days"`, `"1 weeks"`) | | | Correct for daylight saving | `CORRECT_DAYLIGHT` | data is stored in UTC | | Life time, warning time | `LIFE_TIME`, `WARN_TIME` | units older than the life time are deleted at rollover; warning if not archived | | Collection/storage type | `COL_STOR_TYPE` | `Scan/Time`, `Event/Event`, `Event/Item`, `Direct/Event`, `Direct/Item`, `Aggregation` | | Priority | `PRIORITY` | which group answers when an item is in several groups (highest wins; also source for aggregation) | | Data compression | `DATA_COMP` | doubles stored as singles (15 → 7 digits) | | Store quality code | `STORE_QUALITY` | | | Scan interval, averaging | `SCAN_INTERVAL` (s), `AVERAGING_METHOD` (None / Normal / Difference), `AVERAGING_INTERVAL` | Scan/Time only | | Samples per record | `SAMPLES_RECORD` (default 100) | Event/Item, Direct/* | | Fill old record | `FILL_OLD` | Direct/* only | | Aggregation periods | `AGG_PERIOD_30MIN`, `_HOUR` (always on), `_SHIFT`, `_DAY`, `_WEEK`, `_MONTH`, `_YEAR`; `AGG_SHIFT_START`, `AGG_SHIFT_LENGTH`, `AGG_DAY_START`, `AGG_WEEK_START`, `AGG_CORRECT_*_DST` | Aggregation only | | Completion notification item | `AGG_COMP_NOT_ITEM` | string item set to `",,"`; period 2 hour, 3 shift, 4 day, 5 week, 6 month, 7 year, 8 30 min | Collection types: **Scan** (fixed interval, no buffering), **Event** (on change; needed for buffering and for migrating history to a CI Core), **Direct** (written by tools/applications). Storage types: **Time** (one record per scan time, predictable size), **Item** (one record per item with many samples), **Event** (one record per event; slow for many samples of one item). Item in a group (`ITEM_HIS_DF`): `NAME`, `ITEM_NAME`, `GROUP_NAME`; event triggers `ON_VALUE_CHANGE`, `ON_STATUS_CHANGE`, `ON_QUALITY_CHANGE`, `ON_OPTION_CHANGE`, `ON_EACH_UPDATE`, `ON_FIRST_UPDATE`, `ON_PASS_PIT`; extra values `STORE_VALUE`, `STORE_*_LIMIT`, `STORE_DEADBAND`; aggregates `STORE_AGG_MAX`, `_MIN`, `_AVG`, `_INTG` (+ `AGG_INTG_PERIOD` second/minute/hour/day), `_STDV`, `_CNT` (Boolean: times True), `_DIFF_SUM` (+ `AGG_DIFF_SUM_RANGE` rollover count, `AGG_DIFF_SUM_PERC` leap detection; for counters, more accurate than the integral). Typical: Real/Integer max, min, average, standard deviation, integral; Boolean counter; Integer counter totals differential summation. Aggregation quality (with *Store quality code*): 1000 × share of source values that were not offline (other bad statuses do not count). Weekly/monthly/yearly are built from daily values. `ithsag.sup` (aggregation process): `run_interval` (manual: *Heartbeat*, spreads the work, 600 s), `buffer_size` 20, `event_timeout` 1 s, `gap_time` 2880 min, `max_sample_delay` 14400 min, `max_open_files` 10, `queue_size` (installed **500** KB, manual default 200), `dif_sum_noise`. ## 3. Verified behaviour (CI-Eng-Sample, 2026-10-01) - Loaded `CIENG_SCAN10S` (Scan/Time, 10 s) and `CIENG_AGG` (Aggregation, 30 min + hour) with `dssqld -i`, 0 errors. **They stored nothing until CI Server was restarted** (ISS-006); after the restart `TANK1.LEVEL` appeared every 10 s (`SAMPLE_TIME` 12:00:10, 12:00:20, …, status 9). - The first unit of a new Event/Item or Aggregation group is dated 2 days back (= `gap_time`); a Scan/Time group's first unit starts at creation. - Reading one group: add `GROUP_NAME = ""` to the `ITEM_HISTORY` filter. Without it the rows of all groups holding the item are returned. - `ithloahis` into an Event/Item group: samples dated inside the active unit or a unit closed by a rollover were stored (a few seconds later); samples older than the first unit the storage process opened after the restart were dropped **without any message** (also with `-l`). The next unit's initial value then came from the dropped sample (ISS-038). - Aggregation: `CIENG_AGG` (30 min + hour, LEVEL max/min/avg) loaded, units and extra `…zzz…` files created, but no aggregate appeared in `ITEM_HISTORY` between the 12:00 restart and 13:00 (level varied 40–70 from 12:06 to 12:11; `ithsag` running, no message) — ISS-040. `INTERPOLATION = "Hour"` is a valid filter value; `"30 minutes"` is refused. - Full sample remove/reload with all three groups: 0 errors. ## 4. Archive (§2.2) 1. Define the device in `has.sup` (tape, or a CD/DVD/disk path) and restart. 2. *Historisation > Archive > Initialize storage medium* (label, e.g. the first date) — writes an ID file. 3. *Archive > Automatic archive*: daily time; unarchived data is written then. Big archives need a scratch folder (`TEMPORARY-PATH` in `has.sup`). 4. *Archive > Retrieve from archive*: period (whole archive runs only), group, keep-free disk space, life time of the restored data; mount the requested media. Restored data is used by reports and trends until its life time ends. (Known issue: alarm attribute databases are not restored, IM 36K01A50.) ## 5. Reading history (§4) - Classes (`DSS_OPEN_ALL("ITEM_HISTORY", "r", r)`, `DSS_SET_FILTER`, `DSS_READ_NEXT`, `DSS_GET_FLOAT(…,"ITEM_VALUE",…)`), reports (`SELECT … FROM ITEM_HISTORY WHERE ITEM_NAME = … and INTERPOLATION = 'Hour' and ATTRIBUTE = 'Maximum' …`), ODBC (only remote way), C / .NET API, `dssqld`, and `ithloahis` for writing. - Aggregates are selected with `INTERPOLATION` (`Hour`, `Day`, `Month`, `Year`, …) and `ATTRIBUTE` (`Maximum`, `Minimum`, `Average`, `Integration`, `Standard deviation`, `Counter`, `Differential summation`). - `ithloahis -f [-v] [-l] [-i] [-s
] [-t GMT|LST|LCT]`: SCAN or EVENT files (skill `system-management`, task `load-history.md`). ## 6. Troubleshooting (§5) `hisdbg` (3 unit info, 20 all groups, 21 group by name), `hisrep`, `hisrecover`, `isfrename`; archive: the `has.sup` status item and `hasdbg` (11 media, 8 overview, 10 delete a medium). Backup example (App. 1.4): `xcopy /E` of `%TLS_ROOT_PATH%\tls` and `%TLS_ALLUSERS_PATH%` into a dated folder (the manual does not say; stopping CI Server first gives a consistent copy).