cliu
bf7d88a027
Full manual diffs (L15, C20, E30, L25, J40, C10 §2, L10, P10, P11, J20, C30, P20, R40, E10) and read-only checks on the shared R1.05 test server. Skills and summaries moved to R1.05; new skill pytools (IM 36K10P40) from docs (no Python on the test server). Issues ISS-132..181; corrections to version-differences (E10, E30, P11, dss_ath_grp_u0104); C10 summary lists the 3 new authorization actions; r105-verification V10-V17. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
203 lines
12 KiB
Markdown
203 lines
12 KiB
Markdown
---
|
|
manual: IM 36K10P40-01EN
|
|
title: pyTOOLS Programmer's Guide
|
|
edition: 2nd Edition, Feb. 2026 (R1.05); new in R1.04 (1st Edition, Jan. 2025). Revision table: 2nd edition "layouts updated" only (manual new, no diff against R1.03)
|
|
version: R1.05
|
|
status: from docs, plus read-only file checks on the R1.05 test server (2026-10-09) - `dss.pyd`, `DSSSDAPY`, `dss.sup`, `dssana.sup` checked; no Python on the R1.05 test server, so no function was run there (not tested, ISS-178). R1.03 `dss.pyd` loaded on the development machine with Python 3.12 (`version()`, `busGetDirs()` only; CI Server not running)
|
|
---
|
|
|
|
# pyTOOLS Programmer's Guide (R1.05)
|
|
|
|
pyTOOLS is a **Python extension module (`dss`)** for scripts that read and change CI Server
|
|
data through DSS (Data Set Services), exchange BUS Service messages and read item history.
|
|
The module is the file `%TLS_ROOT_PATH%\tls\exe\dss.pyd`. Section numbers refer to the manual.
|
|
Practical use: skill `pytools`.
|
|
|
|
## 1. Overview and operating environment (§1)
|
|
|
|
- Runs on **CI Core and CI Portal**, not on CI View.
|
|
- Always uses the **local** DSS: a script cannot reach another CI Core by node number (only the
|
|
item history functions and BUS messages take an optional node number, §3.11, §3.13).
|
|
- Supported Python: **64-bit Python 3.12** (IM 36K01C10-01EN §1.3.3, coexisting software list;
|
|
summary `IM36K01C10-01EN.md`). Python is not part of the CI Server installation.
|
|
|
|
## 2. Setup (§2)
|
|
|
|
1. Install 64-bit Python from python.org on the Core / Portal (run the installer as
|
|
administrator).
|
|
2. Add the Windows environment variable `PYTHONPATH` = the `tls\exe` folder of CI Server. The
|
|
manual gives the default `C:\Program Files (x86)\Yokogawa\CIServer\tls\exe`; use
|
|
`%TLS_ROOT_PATH%\tls\exe` of the actual installation (on the R1.05 test server the install
|
|
root is `C:\Yokogawa\CIServer`, ISS-177).
|
|
3. In Python: `import dss`.
|
|
|
|
**Installed (checked 2026-10-09):** the module links `python3.dll` (stable ABI), `buslib64.dll`
|
|
and `databaselib64.dll` (both in `tls\exe` next to the module), so it is not tied to one Python
|
|
3.x minor version; Yokogawa only lists 3.12. `dss.pyd` already shipped with R1.03 (module 3.6,
|
|
r10200M, Aug. 2023, 124,928 bytes; no dedicated DSS server, no *Python Interface* entry in
|
|
`dssana.sup`); R1.05 has module **3.6.1** (r11609M, Jun. 2026, 137,728 bytes) with session-based
|
|
logon, the `PY_DSS_NOT_LOGGED_IN` exception and the `aggr_period` history argument (ISS-174).
|
|
|
|
## 3. DSS interface (§3)
|
|
|
|
### 3.1 Function groups (§3.1)
|
|
|
|
Connect/logon, DSS properties, open/close datasets, read, modify, indexing, filtering, locking,
|
|
event notification, BUS messages, date/time, item history, miscellaneous. All functions take the
|
|
**connection handle** first (except `version()`, `busGetDirs()`, `busSetName()`, `busSnap()`).
|
|
|
|
### 3.2 Connecting (§3.2)
|
|
|
|
| Function | Purpose |
|
|
|---|---|
|
|
| `dss.connect([user], [password])` | Attach to BUS Service and DSS; returns the connection handle (a dict with the DSS connection id and optionally user/password). With a user, a password is required and a logon is done at once |
|
|
| `dss.logon(conn, user, password)` | Log on (or switch to another user); a failed logon keeps the old rights |
|
|
| `dss.logout(conn)` | Log out |
|
|
| `dss.close(conn)` | Disconnect and free all DSS resources (also logs out) |
|
|
|
|
- One connection per script; a second `connect()` raises `PY_DSS_ALREADY_CONN`.
|
|
- Without a logon DSS gives **no data** (default); a logon without user/password is refused.
|
|
- The user name is used in the **audit trail**; pyTOOLS operations appear with source
|
|
`PYTOOLS`. Installed R1.05: `PYTOOLS` audit classifiers exist for login / logout / login
|
|
attempt, dataset insert / modify / delete and item value / status / quality changes (skill
|
|
`audit-trail`, `audit-checks.md`, ISS-141).
|
|
- Server side: pyTOOLS clients have their **own DSS data access server**, so they do not compete
|
|
with other DSS clients (§3.2 figure). Installed R1.05: `database_start.cmd` starts
|
|
`dsssda -n dsssdapy`; `dss.sup` routes `DSSSDA_SRV = -1, "PYDSS*", "DSSSDAPY"` (ISS-117).
|
|
- Login method: `dssana.sup` *User Interfaces* tab has `"Python Interface", "PYDSS"` (audit id 18);
|
|
*Login control* has `<local>` and `<remote>` → `Manual CI Server` (verified read-only on the
|
|
R1.05 test server 2026-10-09; R1.03 `dssana.sup` has neither line). J10 §2.9.1: pyTOOLS supports
|
|
only *Manual CI Server* (summary `IM36K10J10-01EN.md`); what happens with another method is not
|
|
documented (ISS-179). A pyTOOLS client that is not in the *User Interfaces* list cannot connect
|
|
(`dssana.sup` help).
|
|
|
|
### 3.3 DSS properties (§3.3)
|
|
|
|
`getAvailableDatasets(conn)` → list of dataset names; `getFieldNames(conn, dataset)` → list of
|
|
field names; `getFieldProperties(conn, dataset, field)` → dict `name`, `type`, `length`.
|
|
(Installed module also has `getDatasetProperties(conn, dataset)`, not in the manual, ISS-176.)
|
|
|
|
### 3.4 Open / close (§3.4)
|
|
|
|
`ds = dss.openDataset(conn, dataset, fields, mode)` — `fields` is the list of fields of interest
|
|
(only these are read and can be set); `mode` any of `r`, `u`, `d`, `i`, plus `t` = handle all
|
|
time fields as UTC (default local civil time). Open the same dataset several times if needed.
|
|
`dss.closeDataset(conn, ds)` frees it.
|
|
|
|
### 3.5 Reading (§3.5)
|
|
|
|
| Function | Returns |
|
|
|---|---|
|
|
| `readNext(conn, ds)` | Next record (dict field → value) in the order of the active index, honouring an active filter; raises `DSS-E-EOS` at the end |
|
|
| `readEqual(conn, ds, key)` | The record whose active index equals `key` (string, or dict for a composed index); first match on a non-unique index; filters ignored; the cursor moves past it |
|
|
| `readRecords(conn, ds)` | List of all records (the manual's "Returns: None" is wrong, ISS-175) |
|
|
| `rewindDataset(conn, ds)` | Cursor back to the first record |
|
|
|
|
### 3.6 Modifying (§3.6)
|
|
|
|
`insertRecord` / `insertRecords` (dict or list of dicts; fields not opened get defaults),
|
|
`updateRecord` / `updateRecords` (must contain the current index field), `deleteRecord(conn, ds,
|
|
key)`. The dataset must be opened with the matching mode letters (`i`, `u`, `d`). Changes go
|
|
through DSS like any other client (validation, audit trail).
|
|
|
|
### 3.7 Indexing (§3.7)
|
|
|
|
`getIndex(conn, dataset)` → index names (e.g. `STATUS_DF`: `NAME`, `STATUS_NUMBER`);
|
|
`setIndex(conn, ds, index_name)` changes the active index and thus the read order and the key
|
|
used by `readEqual`. Default: the primary index.
|
|
|
|
### 3.8 Filtering (§3.8)
|
|
|
|
`createFilter(conn, ds, expr)` → `activateFilter` → reads → `deactivateFilter` → `deleteFilter`
|
|
(an active filter cannot be deleted; one filter per open dataset). Expression: either a quoted
|
|
wildcard string (`"*.TANK1.*"`, applied to the current index field; `*` any string, `?` one
|
|
character), or comparisons `<field> <op> <constant>` with `= != < <= > >= LIKE`, joined by `AND` /
|
|
`OR`, brackets allowed. String constants are case-sensitive and in double quotes (so wrap the
|
|
whole expression in single quotes in Python); field names and operators are not; spaces around
|
|
operators are required; maximum 252 characters.
|
|
|
|
### 3.9 Locking (§3.9)
|
|
|
|
`lockRecords(conn, ds, [keys])` / `unlockRecords(...)` — other CI Server processes cannot change
|
|
or delete a locked record; locking an already locked record raises an exception.
|
|
|
|
### 3.10 Event notification (§3.10)
|
|
|
|
- Dataset level: `datasetEventNotify(conn, ds, mode)` with `i`, `u`, `d`; stop with
|
|
`datasetEventStop(conn, ds)`.
|
|
- Record level: `recordEventNotify(conn, ds, key, mode)` with `u`, `d`; stop with
|
|
`recordEventStop(conn, ds, key)`.
|
|
- Poll with `eventRead(conn)` → list of dicts `ds_name`, `event` (`i`/`u`/`d`), `id` (running
|
|
number), `record` (record after the change), `set_hnd`. Empty list when nothing happened.
|
|
- One queue per client; DSS **clears the queue when it overflows** (size from the DSS setup
|
|
file — `EVENT_Q_SLOTS` in `dss.sup`, default 100, ISS-181), so poll often enough.
|
|
- Overlapping requests give one event; a record can only have one request — stop it before
|
|
changing the mode (`PY_DSS_EVT_NOTIFY_DEF`).
|
|
|
|
### 3.11 BUS Service messages (§3.11)
|
|
|
|
- The script attaches to the BUS as `PYDSS_<number>` (new each run). To be addressable, call
|
|
`dss.busSetName('<NAME>')` **before** `connect()`, every time. (Installed `dss.sup` routes only
|
|
`PYDSS*` clients to `DSSSDAPY`; a script with its own name probably uses a general `DSSSDA`,
|
|
not checked, ISS-180.)
|
|
- `busSend(conn, {'process':…, 'node':…, 'code':…, 'message':…})` (node and code optional;
|
|
message always a string); `busSendReceive` waits up to 60 s for the reply (dict with
|
|
`message`); `busReceive(conn[, 'SYNC_WAIT'])` → dict (`process`, `node`, `reply`, `sys_type`,
|
|
`code`, `message`) or `None`; `busReply(conn, msg)` must answer within 60 s, passing `reply` and
|
|
`sys_type` back unchanged.
|
|
- `durstp -p <name>` stops a named pyTOOLS script: the script gets `dss.error` with code
|
|
`PY_DSS_STOP_REQUEST` and should close and exit.
|
|
- `busSnap('GENERAL'|'QUEUE'|'NODE'|'LOCK')` takes a snapshot, then `busSnap('FIRST')` /
|
|
`busSnap('NEXT')` return the entries (`None` at the end). Local node only (on a Portal you see
|
|
the Portal).
|
|
|
|
### 3.12 Date and time (§3.12)
|
|
|
|
- DSS time = seconds since 1970-01-01 as a float with milliseconds.
|
|
- `dateString(conn, t[, fmt])` → text; tokens `YYYY YY MMM MM M DDD DD D hh mm ss p…pppp`;
|
|
presets `ANSI`, `JULIAN`, `EUROPE`, `GENERAL`; default `DD-MMM-YY hh:mm:ss`.
|
|
- `dateDSS(conn, 'dd-mmm-yy hh:mm:ss.ppp')` → DSS time (missing date or time = now);
|
|
`dateRelDSS(conn, '[-]days,[+|-]hh:mm…')` → relative to now (e.g. `-1:00` one hour ago,
|
|
`-1,08:00` yesterday 08:00).
|
|
- `dateConvert(conn, t, 'GMT_TO_LCT' | 'GMT_TO_LST' | 'LCT_TO_GMT' | 'LST_TO_GMT' | 'LCT_TO_LST' |
|
|
'LST_TO_LCT')`. LST = standard time, LCT = with daylight saving.
|
|
- Interval fields (e.g. `REPORT_DF.LIFE_TIME`) are `<count>.<unit>`: `.0` s, `.1` min, `.2` h,
|
|
`.3` days, `.4` weeks, `.5` months, `.6` years (same as `dssutl`, skill `system-management`).
|
|
|
|
### 3.13 Item history (§3.13)
|
|
|
|
Two functions bypass DSS and ask ITEM Service directly (faster than reading `ITEM_HISTORY`):
|
|
|
|
- `getHisGroupItems(conn, group[, node])` → item names in a storage group.
|
|
- `getItemHistory(conn, item, group, start, end, max_sample[, node=, interval=, attribute=,
|
|
aggr_period=])` → list of tuples `(time, value, status, option_bits, quality)`; the list ends
|
|
with `None` when the period is complete, otherwise call again with the last time as start.
|
|
`interval` = interpolation step in **ms**; `attribute` = `VALUE` (default), `DEADBAND`,
|
|
limits, `SCALE_LOW/HIGH`, or for aggregation groups `AGGR_MAX`, `AGGR_MIN`, `AGGR_AVERAGE`,
|
|
`AGGR_INTG`, `AGGR_COUNT`, `AGGR_STDV`, `AGGR_DFMS`; `aggr_period` = `30_MINUTES`, `HOUR`,
|
|
`SHIFT`, `DAY`, `WEEK`, `MONTH`, `YEAR` (only periods configured in the group). The manual's
|
|
parameter list omits `item` (ISS-175).
|
|
|
|
### 3.14 Miscellaneous (§3.14)
|
|
|
|
`getErrorStack(conn)` (UMH stack as `[code, text]` pairs), `version()`, `busGetDirs()` (dict of
|
|
`TLS_*` folders; installed also returns `TLS_WAP`, `TLS_LOG`), `readSetup(conn, 'dur')` (setup
|
|
file without `.sup` as list of `[keyword, args…]`), `logMessage(conn, text)` (max 200 characters
|
|
to the CI Server message log), `umhLog(conn)`.
|
|
|
|
## 4. Exceptions (§4)
|
|
|
|
One exception class, `dss.error`, with `args = (code, message)`:
|
|
- **CI Server errors**: the last UMH message, e.g. `DSS-E-EOS` (end of dataset — the normal way
|
|
to end a `readNext` loop), `VIS-W-DUPLICATE` (insert of an existing key).
|
|
- **Module errors** `PY_DSS_*`: `ALREADY_CONN`, `DS_IS_CLOSED`, `EVT_NOTIFY_DEF`, `FLT_DEF`,
|
|
`FLT_NOT_ACTIV`, `FLT_NOT_FOUND`, `FLT_TOO_LONG` (manual misprints `FLY`), `MISG_IDX`,
|
|
`NO_CHAN_OPEN`, `NO_EVT_DEF`, `NOT_CONN`, `NOT_LOGGED_IN`, `UNK_ERR`, `UNK_FLD`, `UNK_FLD_TYPE`;
|
|
the installed module has more (ISS-176).
|
|
|
|
## Notes on the manual
|
|
|
|
The examples are carried over from an old Python 2 version (`101L` long integers, an old module
|
|
version string, a `FastTools` install path) and contain typos (e.g. `readEqual(con, dss, …)`
|
|
instead of the dataset handle, `msg`/`mesg` mixed up). They use `dss.connect('admin','')`; on a
|
|
real system use a CI Server user with only the rights the script needs. Details: ISS-175.
|