cliu
83af63ab53
Full manual diffs (C50, R30, C60, L20, C40, L35, R10, R20, J10, L30, L50, P30, G10, C10) and read-only checks on the shared R1.05 test server where the permission system allowed them. Skills and summaries moved to R1.05. Issues ISS-092..129 (gaps 104-107, 114-115, 130-131 unused); ISS-074 closed (manual and software agree), ISS-091 corrected by ISS-119. version-differences: unlisted manual changes and installed-only changes. r105-verification: V5-V9 and new not-tested rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
148 lines
10 KiB
Markdown
148 lines
10 KiB
Markdown
---
|
||
manual: IM 36K10C40-01EN
|
||
title: Engineering Module - Historical
|
||
edition: 5th Edition, Feb. 2026 (R1.05; first summarized from the 3rd Edition, Sep. 2023, R1.03)
|
||
version: R1.05 (first written for R1.03)
|
||
status: R1.03 - Event/Item, Scan/Time and Aggregation storage groups verified by quick load on CI-Eng-Sample 2026-10-01; ithloahis loading rules verified; Engineering Module forms, archive and retrieve from docs. R1.05 - full text diff against R1.03 (2026-10-08): only §4.2 changed; the history setup files and dataset definitions (HIS_GROUP_DF, ITEM_HIS_DF, ITEM_HISTORY) were compared with snapshots of the R1.05 test server taken earlier on 2026-10-08 and are identical apart from instance lines; storage groups not re-tested on R1.05 (no history group in the reduced sample load on the shared server)
|
||
---
|
||
|
||
# Engineering Module - Historical (R1.05)
|
||
|
||
**R1.05 changes** (full text diff against the R1.03 edition, 2026-10-08): only §4.2 *Access from
|
||
applications* changed (R1.04): the list of interfaces is now ODBC, .NET API, RESTful API,
|
||
pyTOOLS, `dssqld`, `ithloahis`; the C API is no longer listed, and remote access is now ODBC
|
||
**and** the RESTful API (was "ODBC only"). See §5 below. Everything else is unchanged apart
|
||
from the contact page and a corrected manual title in a cross-reference.
|
||
|
||
Item storage groups (*Configuration > Historisation > Item storage groups*), aggregation and
|
||
archive in the Engineering Module, plus how history is read. History manager internals:
|
||
`IM36K10L20-01EN.md`; storage processes: `IM36K10L35-01EN.md`. Section numbers refer to the
|
||
manual; dataset fields were checked on the installed system.
|
||
|
||
## 1. Historical database (§1)
|
||
|
||
| Storage group kind | Dataset | Created by |
|
||
|---|---|---|
|
||
| Item storage groups | `ITEM_HISTORY` | the engineer, one per collection/storage type and schedule |
|
||
| Alarm (`ALH`) | `ALARM_CHRONO` | system |
|
||
| Audit (`ADT`) | `AUDIT_TRAIL` | system |
|
||
| System log (`SYS_LOG`) | `SYS_LOG` | system |
|
||
| Reports | report storage | system |
|
||
|
||
Items are only stored after they are added to an item storage group (`ITEM_HIS_DF`, name
|
||
`<group>:<item>`). Displays read history through the trend, alarm overview and report
|
||
components.
|
||
|
||
## 2. Item storage group (§2.1, dataset `HIS_GROUP_DF`)
|
||
|
||
| Tab / field | `HIS_GROUP_DF` | Notes |
|
||
|---|---|---|
|
||
| Name | `NAME` | max 16 characters |
|
||
| Type | `TYPE` | `Item` |
|
||
| Node | `NODE` | cannot be changed later |
|
||
| Exclude from archive | `EXCLUDE_ARCHIVE` | |
|
||
| Activation / deactivation time | `START_TIME`, `STOP_TIME` | `STOP_TIME` in the future = stop then (used by `remove.cmd`, ISS-005) |
|
||
| First rollover, interval | `FIRST_ROLLOVER`, `ROLLOVER_INT` (`"1 hours"`, `"1 days"`, `"1 weeks"`) | |
|
||
| Correct for daylight saving | `CORRECT_DAYLIGHT` | data is stored in UTC |
|
||
| Life time, warning time | `LIFE_TIME`, `WARN_TIME` | units older than the life time are deleted at rollover; warning if not archived |
|
||
| Collection/storage type | `COL_STOR_TYPE` | `Scan/Time`, `Event/Event`, `Event/Item`, `Direct/Event`, `Direct/Item`, `Aggregation` |
|
||
| Priority | `PRIORITY` | which group answers when an item is in several groups (highest wins; also source for aggregation) |
|
||
| Data compression | `DATA_COMP` | doubles stored as singles (15 → 7 digits) |
|
||
| Store quality code | `STORE_QUALITY` | |
|
||
| Scan interval, averaging | `SCAN_INTERVAL` (s), `AVERAGING_METHOD` (None / Normal / Difference), `AVERAGING_INTERVAL` | Scan/Time only |
|
||
| Samples per record | `SAMPLES_RECORD` (default 100) | Event/Item, Direct/* |
|
||
| Fill old record | `FILL_OLD` | Direct/* only |
|
||
| Aggregation periods | `AGG_PERIOD_30MIN`, `_HOUR` (always on), `_SHIFT`, `_DAY`, `_WEEK`, `_MONTH`, `_YEAR`; `AGG_SHIFT_START`, `AGG_SHIFT_LENGTH`, `AGG_DAY_START`, `AGG_WEEK_START`, `AGG_CORRECT_*_DST` | Aggregation only |
|
||
| Completion notification item | `AGG_COMP_NOT_ITEM` | string item set to `"<GMT seconds>,<period>,<group>"`; period 2 hour, 3 shift, 4 day, 5 week, 6 month, 7 year, 8 30 min |
|
||
|
||
Collection types: **Scan** (fixed interval, no buffering), **Event** (on change; needed for
|
||
buffering and for migrating history to a CI Core), **Direct** (written by tools/applications).
|
||
Storage types: **Time** (one record per scan time, predictable size), **Item** (one record per
|
||
item with many samples), **Event** (one record per event; slow for many samples of one item).
|
||
|
||
Item in a group (`ITEM_HIS_DF`): `NAME`, `ITEM_NAME`, `GROUP_NAME`; event triggers
|
||
`ON_VALUE_CHANGE`, `ON_STATUS_CHANGE`, `ON_QUALITY_CHANGE`, `ON_OPTION_CHANGE`, `ON_EACH_UPDATE`,
|
||
`ON_FIRST_UPDATE`, `ON_PASS_PIT`; extra values `STORE_VALUE`, `STORE_*_LIMIT`, `STORE_DEADBAND`;
|
||
aggregates `STORE_AGG_MAX`, `_MIN`, `_AVG`, `_INTG` (+ `AGG_INTG_PERIOD` second/minute/hour/day),
|
||
`_STDV`, `_CNT` (Boolean: times True), `_DIFF_SUM` (+ `AGG_DIFF_SUM_RANGE` rollover count,
|
||
`AGG_DIFF_SUM_PERC` leap detection; for counters, more accurate than the integral).
|
||
Typical: Real/Integer max, min, average, standard deviation, integral; Boolean counter;
|
||
Integer counter totals differential summation.
|
||
|
||
Aggregation quality (with *Store quality code*): 1000 × share of source values that were not
|
||
offline (other bad statuses do not count). Weekly/monthly/yearly are built from daily values.
|
||
|
||
`ithsag.sup` (aggregation process): `run_interval` (manual: *Heartbeat*, spreads the work,
|
||
600 s), `buffer_size` 20, `event_timeout` 1 s, `gap_time` 2880 min, `max_sample_delay` 14400
|
||
min, `max_open_files` 10, `queue_size` (installed **500** KB, manual default 200), `dif_sum_noise`.
|
||
Same values in the R1.05 `ithsag.sup` (compared 2026-10-08).
|
||
|
||
## 3. Verified behaviour (R1.03, CI-Eng-Sample, 2026-10-01)
|
||
|
||
- Loaded `CIENG_SCAN10S` (Scan/Time, 10 s) and `CIENG_AGG` (Aggregation, 30 min + hour) with
|
||
`dssqld -i`, 0 errors. **They stored nothing until CI Server was restarted** (ISS-006); after
|
||
the restart `TANK1.LEVEL` appeared every 10 s (`SAMPLE_TIME` 12:00:10, 12:00:20, …, status 9).
|
||
- The first unit of a new Event/Item or Aggregation group is dated 2 days back (= `gap_time`);
|
||
a Scan/Time group's first unit starts at creation.
|
||
- Reading one group: add `GROUP_NAME = "<group>"` to the `ITEM_HISTORY` filter. Without it the
|
||
rows of all groups holding the item are returned.
|
||
- `ithloahis` into an Event/Item group: samples dated inside the active unit or a unit closed by a
|
||
rollover were stored (a few seconds later); samples older than the first unit the storage
|
||
process opened after the restart were dropped **without any message** (also with `-l`). The
|
||
next unit's initial value then came from the dropped sample (ISS-038).
|
||
|
||
- Aggregation (`CIENG_AGG`, 30 min + hour, `TANK1.LEVEL` max/min/avg): the aggregation ran
|
||
(`%TLS_LOG_PATH%\ithsag.log`: *Aggregation Completed*, period type 8 = 30 min, 2 = hour) but stored
|
||
nothing while the source item was in two groups of equal priority 0. After giving
|
||
`CIENG_SCAN10S` priority 10 the 13:30–14:00 period gave max 77 / min 33 / average 52.7,
|
||
quality 1000 (ISS-040). Filter values: `INTERPOLATION = "30Minutes"` (not "30 minutes"),
|
||
`"Hour"`; `ATTRIBUTE = "Maximum"` etc. (`ithdbg` lists *Difference summation*).
|
||
- Full sample remove/reload with all three groups: 0 errors.
|
||
|
||
## 4. Archive (§2.2)
|
||
|
||
1. Define the device in `has.sup` (tape, or a CD/DVD/disk path) and restart.
|
||
2. *Historisation > Archive > Initialize storage medium* (label, e.g. the first date) — writes an
|
||
ID file.
|
||
3. *Archive > Automatic archive*: daily time; unarchived data is written then. Big archives
|
||
need a scratch folder (`TEMPORARY-PATH` in `has.sup`).
|
||
4. *Archive > Retrieve from archive*: period (whole archive runs only), group, keep-free disk
|
||
space, life time of the restored data; mount the requested media. Restored data is used by
|
||
reports and trends until its life time ends. (Known issue: alarm attribute databases are not
|
||
restored, IM 36K01A50.)
|
||
|
||
## 5. Reading history (§4; §4.2 changed in R1.04)
|
||
|
||
- Classes (`DSS_OPEN_ALL("ITEM_HISTORY", "r", r)`, `DSS_SET_FILTER`, `DSS_READ_NEXT`,
|
||
`DSS_GET_FLOAT(…,"ITEM_VALUE",…)`), reports (`SELECT … FROM ITEM_HISTORY WHERE ITEM_NAME = …
|
||
and INTERPOLATION = 'Hour' and ATTRIBUTE = 'Maximum' …`), and `ithloahis` for writing.
|
||
- Applications (§4.2, R1.05 edition; from docs):
|
||
|
||
| Interface | Kind | Reaches | Remote (no CI Server software on the client) | More |
|
||
|---|---|---|---|---|
|
||
| ODBC | interface for any application | datasets (`ITEM_HISTORY`, `ALARM_CHRONO`) | yes | skill `data-access` (L45 summary §1) |
|
||
| .NET API | interface for .NET applications | datasets | no | — |
|
||
| RESTful API (new in R1.04) | HTTP (GET, PUT …) for web applications | datasets | yes | skill `data-access` (L45 summary §9: `GetHistory`, `GetHistoryMultiTag`) |
|
||
| pyTOOLS (new in R1.04) | Python library | datasets | no | IM 36K10P40 §3.13 (no skill yet) |
|
||
| `dssqld` | tool | datasets | no | skill `system-management` |
|
||
| `ithloahis` | tool | items (writes history) | no | skill `system-management`, task `load-history.md` |
|
||
|
||
The R1.03 edition also listed a C API (local only); the R1.05 edition no longer does.
|
||
pyTOOLS (IM 36K10P40 §3.13, from docs) reads history directly from ITEM Service instead of
|
||
through DSS (faster): `dss.getHisGroupItems(...)` lists a group's items,
|
||
`dss.getItemHistory(...)` returns samples for an interval, with optional attribute and
|
||
aggregation period. Its names differ from the `ITEM_HISTORY` filter values (e.g. `AGGR_MAX`,
|
||
`30_MINUTES` instead of `Maximum`, `30Minutes`); not tested.
|
||
|
||
- Aggregates are selected with `INTERPOLATION` (`Hour`, `Day`, `Month`, `Year`, …) and
|
||
`ATTRIBUTE` (`Maximum`, `Minimum`, `Average`, `Integration`, `Standard deviation`,
|
||
`Counter`, `Differential summation`).
|
||
- `ithloahis -f <csv> [-v] [-l] [-i] [-s <section>] [-t GMT|LST|LCT]`: SCAN or EVENT files
|
||
(skill `system-management`, task `load-history.md`).
|
||
|
||
## 6. Troubleshooting (§5; unchanged in R1.05)
|
||
|
||
`hisdbg` (3 unit info, 20 all groups, 21 group by name), `hisrep`, `hisrecover`, `isfrename`;
|
||
archive: the `has.sup` status item and `hasdbg` (11 media, 8 overview, 10 delete a medium).
|
||
Backup example (App. 1.4): `xcopy /E` of `%TLS_ROOT_PATH%\tls` and `%TLS_ALLUSERS_PATH%` into
|
||
a dated folder (the manual does not say; stopping CI Server first gives a consistent copy).
|