From 547e5606cdefb42434a4f8d258fac87e2c78297e Mon Sep 17 00:00:00 2001 From: cliu Date: Fri, 2 Oct 2026 14:47:33 +1000 Subject: [PATCH 1/2] Add RDBMS data exposure task to the data-access skill (tested in R1.05 only) Co-Authored-By: Claude Opus 5.5 --- .claude/skills/data-access/SKILL.md | 10 +- .../data-access/tasks/rdbms-exposure.md | 263 ++++++++++++++++++ 2 files changed, 271 insertions(+), 2 deletions(-) create mode 100644 .claude/skills/data-access/tasks/rdbms-exposure.md diff --git a/.claude/skills/data-access/SKILL.md b/.claude/skills/data-access/SKILL.md index a28342d..c502820 100644 --- a/.claude/skills/data-access/SKILL.md +++ b/.claude/skills/data-access/SKILL.md @@ -8,10 +8,13 @@ description: >- security policy, trusting client certificates, UaExpert), check why an OPC connection fails (BadCertificateUntrusted, BadSecurityChecksFailed, item stays NOT INIT), configure DCOM or DCOM tunnelling for OPC Classic, use ODBC / SimbaClient / SQL on datasets, or check the ODBC server - (odbsrv, port 1583, SSL certificate). Sources IM 36K10L45-01EN, IM 36K10E20-01EN. + (odbsrv, port 1583, SSL certificate), or copy item values and timestamps into Microsoft SQL + Server with RDBMS data exposure (dbc.sup, DEM_CONNECTION, DBCSND, DBCDEM, DATA_EXPOSURE_DF, + data exposure aggregation interval AGG_INTERVAL, ITEM_VALUES, SQL login, DSS-E-FLT_SYN_ERR). + Sources IM 36K10L45-01EN, IM 36K10E20-01EN, IM 36K10G10-01EN ch. 10. --- -# Data access: OPC and ODBC (R1.03) +# Data access: OPC, ODBC and RDBMS data exposure (R1.03; RDBMS exposure tested in R1.05 only) Background: `knowledge/manuals/IM36K10L45-01EN.md` (ODBC, OPC DA/A&E/UA client and server, setup files), `IM36K10E20-01EN.md` (Engineering Module forms, certificates, PubSub). Equipment @@ -28,6 +31,8 @@ Rules: node's) DA/A&E server: deadlock (manual §5.3). The OPC **UA** loopback is fine. - Changing `opxsys.sup`, `opxuas.sup`, `odbsrv.sup` or `ServerConfig.xml` needs admin rights and a restart: hand it to the user (skill `portal-setup`, task `change-setup-file.md`). +- The same applies to `dbc.sup` (RDBMS data exposure). SQL Server changes (mixed mode, logins, + jobs) are on another system: only with the user's and the SQL Server owner's agreement. | Task | File | Status | |---|---|---| @@ -37,3 +42,4 @@ Rules: | OPC DA / A&E client and server, DCOM, DCOM tunnelling | L45 summary §3–§5 | from docs (DA loopback not allowed) | | ODBC with SimbaClient | L45 summary §1 | from docs; ODBC server not running on the test system (ISS-008) | | OPC UA alarms (external alarms), methods, PubSub | L45 §6, E20 §2–§4 | from docs | +| RDBMS data exposure to SQL Server: SQL login, `dbc.sup` `DEM_CONNECTION`, `DATA_EXPOSURE_DF` (filter syntax), item aggregation interval, checks, table behaviour, SQL-side retention | `tasks/rdbms-exposure.md` | tested in R1.05 only (`ITEM_VALUES`); `ITEM_HISTORY` / `ITEM_INPUTS` not tested | diff --git a/.claude/skills/data-access/tasks/rdbms-exposure.md b/.claude/skills/data-access/tasks/rdbms-exposure.md new file mode 100644 index 0000000..d24817f --- /dev/null +++ b/.claude/skills/data-access/tasks/rdbms-exposure.md @@ -0,0 +1,263 @@ +--- +task: RDBMS data exposure — copy item values into Microsoft SQL Server (dbc.sup, DATA_EXPOSURE_DF, item aggregation interval) +version: R1.05 +source: IM 36K10G10-01EN ch. 10 (§10.1, §10.2); IM 36K10C10-01EN §3.3 (Item management › Data exposure aggregation); installed tls\sup\dbc.sup, tls\src\data_exposure_df.dss, tls\src\item_df.dss; practiced on a user system 2026-10-02 +status: tested in R1.05 only +--- + +# RDBMS data exposure to SQL Server + +**Tested in R1.05 only** (CI Server R1.05.00 r11609M, user system, SQL Server 2022 Developer +Edition on a separate domain-joined host). Not re-tested on R1.03 or on CI-Eng-Sample (user +decision 2026-10-02). On the R1.03 development machine the files `dbc.sup`, +`data_exposure_df.dss` and `item_df.dss` were checked read-only and match what is described +here (`dbc.sup` is identical in R1.03 and R1.05). + +**Changes live configuration on two systems** (SQL Server and CI Server): follow the user's +chosen way of working. SQL Server changes on a shared server need the owner's agreement. + +## How it works + +- `DBCSND` (sender) subscribes to the exposed CI Server data and sends it to `DBCDEM` + (receiver, the *data exposure module*), which writes it to SQL Server through ODBC. + `DEM_NODE = 0` in `dbc.sup` = receiver on the local node. `dbcdem.exe` is 64-bit. +- Each exposure is a record in dataset **`DATA_EXPOSURE_DF`** (Engineering Module: *Reporting › + Data exposures*). CI Server creates the SQL table itself when the first record is exposed. +- For source **`ITEM_VALUES`**, rows are produced only for items whose **`ITEM_DF.AGG_INTERVAL`** + (item form › *Item management* › data exposure aggregation; 60 s–7 days, 0 = off, default 0) + is set. Each interval adds rows with the value, min, max, average and sample count. +- Other sources per the manual: `ITEM_HISTORY` (needs a filter; see ISS-058), `ITEM_INPUTS` + (values written in SQL go back to CI Server), any other dataset on change. Only + `ITEM_VALUES` was tested. +- Manual warning (not tested): **deleting a data exposure drops its target table**, even if + another exposure uses the same table. + +## 1. SQL Server: read-only checks + +In SSMS on the SQL host: + +```sql +SELECT SERVERPROPERTY('IsIntegratedSecurityOnly') AS WindowsOnly; -- 1 = Windows only, 0 = mixed +EXEC xp_readerrorlog 0, 1, N'Server is listening on'; -- TCP port(s) +SELECT name, is_disabled FROM sys.server_principals WHERE name = 'sa'; +``` + +From the CI Server: + +```powershell +Test-NetConnection -Port 1433 +``` + +Pass: `TcpTestSucceeded : True`. If TCP 1433 already listens and the test passes, no TCP/IP or +firewall change is needed. + +## 2. Choose the authentication + +```powershell +(Get-CimInstance Win32_Service -Filter "Name='StartCIServer'").StartName +``` + +The CI Service normally runs as the **local** account `.\CIX_PROCESS`, also on a domain-joined +CI Server. A SQL Server on another host cannot authenticate a local account, so Windows +authentication does not work there: use a **SQL login** (ISS-059). The manual's advice to add +`CIX_PROCESS` to SQL Server as a Windows user only applies when that account is known to the +SQL host (domain account, or SQL Server on the same machine). + +## 3. SQL Server: mixed mode (only if `WindowsOnly = 1`) + +Server › *Properties* › *Security* › **SQL Server and Windows Authentication mode**, then restart +the SQL Server service. The restart drops all connections on that server: agree a time with the +owner. Make sure `sa` is disabled or has a strong password first. Check: `WindowsOnly` = 0. + +## 4. SQL Server: login with db_owner on one database + +Create it at **server** level (*Security › Logins › New Login*), not under *Database › Security › +Users* (that only maps an existing login). Do not call it `db_owner` (that is a role). + +```sql +CREATE LOGIN WITH PASSWORD = '', DEFAULT_DATABASE = , CHECK_POLICY = ON, CHECK_EXPIRATION = OFF; +USE ; +CREATE USER FOR LOGIN ; +ALTER ROLE db_owner ADD MEMBER ; +``` + +`db_owner` is needed because CI Server creates (and may drop or recreate) its tables. Keep the +database for CI Server only. + +## 5. CI Server: test the ODBC login + +64-bit PowerShell on the CI Server (prompts for the password, reads only): + +```powershell +$pw = Read-Host "password" -AsSecureString; $plain = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($pw)); $c = New-Object System.Data.Odbc.OdbcConnection("Driver={SQL Server};Server=;Database=;Uid=;Pwd=$plain"); $c.Open(); $cmd = $c.CreateCommand(); $cmd.CommandText = "SELECT SUSER_NAME(), DB_NAME()"; $r = $cmd.ExecuteReader(); [void]$r.Read(); "{0} | {1}" -f $r[0], $r[1]; $c.Close() +``` + +Pass: ` | `. The legacy `{SQL Server}` driver worked with SQL Server 2022. + +## 6. CI Server: `DEM_CONNECTION` in `dbc.sup` + +Admin rights; back up the file first. Change **only** the active `DEM_CONNECTION=` line near the +end (not the commented `! DEM_CONNECTION` in the `!DEFAULT` block): + +``` +DEM_CONNECTION="Driver={SQL Server};Server=;Database=;Uid=;Pwd=" +``` + +- The password is typed as is (no quotes or brackets). A `;` in it needs `Pwd={…}` (a `}` inside + is doubled); a `"` cannot be used. +- The password is plain text in the file. A system DSN (manual §10.2) only avoids that with + Windows authentication, which step 2 rules out for a remote SQL Server. Check who can read + `tls\sup` (ISS-061). +- Other `dbc.sup` keywords (queues, batches, retries, `ITH_GRAN_SEC`, `SND_BURST_RATE`, + `SND_AUTO_RECREATE`, `SND_NEED_SYNC`) were left at their defaults. + +Restart CI Server (elevated, Edit/Engineering Module closed): `Restart-Service StartCIServer`. + +## 7. Check the DBC processes + +```powershell +Get-Process dbcsnd, dbcdem -ErrorAction SilentlyContinue +``` + +```powershell +$m = Select-String -Path "$env:TLS_LOG_PATH\umhlog.log" -Pattern 'From: DBCSND'; $n = $m[-1].LineNumber; $a = Get-Content "$env:TLS_LOG_PATH\umhlog.log"; $a[($n-1)..($n+8)] +``` + +(Use the same with `From: DBCDEM`. `-Tail 300` can miss the block on a busy log.) + +| Message | Meaning | +|---|---| +| `DBCDEM`: `DBC-I-DB_INI_OK, Database connection initialisation SUCCEEDED` | SQL connection works | +| `DBCDEM`: `FSL-F-SU_EOF, End of file on setup file` | Harmless, at every start | +| `DBCSND`: `DBC-I-DEM_CON, Connection established to data exposure module` | Sender running | +| `DBCSND`: `DSS-E-FLT_SYN_ERR` / `DSSF-E-CMP_NO_SUCC` / `DBC-E-DSS_SUB` / `DBC-E-SUB_DAT` / `DBC-E-DAT_HND`, then `DUR-W-REM_PROC, Process DBCSND exited unexpectedly` | A `RECORDS` filter does not compile (step 8). **One bad record stops all exposures.** DBCSND is not restarted automatically: fix, then restart CI Server | + +## 8. Data exposure records + +List existing ones (read-only): + +```powershell +dssqld -e dexp.qli -d DATA_EXPOSURE_DF -c -l dexp.lst +``` + +Fields (`tls\src\data_exposure_df.dss`): `NAME`, `DESCRIPTION`, `EXPOSED_DATA` (source dataset, +e.g. `ITEM_VALUES`), `REMOTE_TABLE`, `ENABLED`, `UPDATE_PERIOD`, `RECORDS` (filter), +`EXPOSED_FIELD_1`…`32` with `REMOTE_COL_1`…`32`, `REF_NAME/SOURCE/VALUE_1`…`8`, `ALL_FIELDS`. + +Tested `ITEM_VALUES` record (`ENABLED` 1, `UPDATE_PERIOD` 0, `ALL_FIELDS` 0): + +| Exposed field | Remote column | +|---|---| +| `NAME` | `NAME` | +| `VALUE` | `VALUE` | +| `VALUE.MIN` | `VALUE_MIN` | +| `VALUE.MAX` | `VALUE_MAX` | +| `VALUE.AVG` | `VALUE_AVG` | +| `VALUE.COUNT` | `VALUE_COUNT` | +| `TIMESTAMP` | `TIMESTAMP` | + +**Filter syntax:** write the field name **without quotes**: `NAME LIKE "
*"`. The manual's +example `"NAME" like "ABC*"` (quoted field) fails with `DSS-E-FLT_SYN_ERR` (ISS-057). In a quick +load file quotes inside a string are doubled. Write `.qli` files as ASCII +(`Set-Content -Encoding ascii`). + +Change a field (here: fix the filter): + +``` +@FIELDS +NAME,RECORDS + +@DATA_EXPOSURE_DF +"","NAME LIKE ""
*""" +``` + +`dssqld -m .qli -l .qlo` → 1 modified, 0 errors. Disable instead of deleting with +`NAME,ENABLED` / `"",0`. Delete with `dssqld -r` and only the key +(`@FIELDS NAME`, `""`) → 1 removed — this drops the target table (manual). + +## 9. Aggregation interval on the items + +Read for a whole section (read-only; RQL `%` wildcard; verified also on R1.03): + +``` +set @next 0; +set @page_size 0; +select name, agg_interval from item_df where name like '
.%'; +quit; +``` + +`cmd /c "rptgen -s rptgen.rpt < agg.rql"` (skill `reporting`, task `rql-interactive.md`). + +Set it with `dssqld -m`: + +``` +@FIELDS +NAME,AGG_INTERVAL + +@ITEM_DF +"
.","1 Minutes" +``` + +`0` switches it off. The change took effect **without a CI Server restart**; rows appeared +within a few minutes. + +## 10. Check the SQL table + +```sql +USE ; +SELECT name, create_date FROM sys.tables WHERE name = ''; +SELECT COUNT(*) AS row_count, MIN([TIMESTAMP]) AS oldest FROM dbo.
; +SELECT TOP 20 NAME, VALUE, VALUE_MIN, VALUE_MAX, VALUE_AVG, VALUE_COUNT, + DATEADD(second, [TIMESTAMP], '1970-01-01') AS local_time +FROM dbo.
ORDER BY [TIMESTAMP] DESC; +``` + +Pass: the row count grows by about one row per item per interval. + +## Behaviour observed (R1.05) + +- Table created by CI Server: `NAME` varchar, `VALUE`/`VALUE_MIN`/`VALUE_MAX`/`VALUE_AVG` + float, `VALUE_COUNT` int, `TIMESTAMP` int; clustered key (`NAME`, `TIMESTAMP`). Rows are + inserted (the table is a history), not updated. +- `TIMESTAMP` = seconds since 1970 in CI Server **local time** (like RQL dates). Readable: + `DATEADD(second, [TIMESTAMP], '1970-01-01')`. +- `VALUE_COUNT` = updates in the interval (an analog item updated every second ≈ 55 per minute; + a steady item 1). +- Interval boundaries follow the start of exposure (e.g. :23 s), not the clock minute. +- In one snapshot, steady items had no row for the latest interval while changing items did + (ISS-060). Until that is clear, read a steady item's value at time T as its last row at or + before T. +- Rows **survive a CI Server restart** with `SND_NEED_SYNC = YES` (oldest timestamp unchanged, + count kept growing). +- Existing history is not back-filled: data starts when exposure starts. +- Volume: about one row per item per interval (49 items at 1 minute ≈ 70,000 rows a day). + Nothing deletes old rows. + +## Optional: retention on the SQL side + +Not part of CI Server. A SQL Server Agent job (not available on SQL Express) that deletes old +rows in small batches so CI Server's inserts are not blocked; owner `sa` works even with `sa` +disabled: + +```sql +USE msdb; +EXEC dbo.sp_add_job @job_name = N'', @owner_login_name = N'sa'; +EXEC dbo.sp_add_jobstep @job_name = N'', @step_name = N'Delete old rows', + @subsystem = N'TSQL', @database_name = N'', @command = N' +DECLARE @cutoff int = DATEDIFF(second, ''1970-01-01'', DATEADD(day, -, GETDATE())); +DECLARE @n int = 1; +WHILE @n > 0 +BEGIN + DELETE TOP (5000) FROM dbo.
WHERE [TIMESTAMP] < @cutoff; + SET @n = @@ROWCOUNT; + IF @n > 0 WAITFOR DELAY ''00:00:01''; +END'; +EXEC dbo.sp_add_schedule @schedule_name = N' daily 03:00', @freq_type = 4, @freq_interval = 1, @active_start_time = 030000; +EXEC dbo.sp_attach_schedule @job_name = N'', @schedule_name = N' daily 03:00'; +EXEC dbo.sp_add_jobserver @job_name = N'', @server_name = N'(local)'; +``` + +Test: `EXEC msdb.dbo.sp_start_job @job_name = N'';` then `msdb.dbo.sysjobhistory` +(`run_status` 1 = succeeded). Change the retention later with `sp_update_jobstep` (and +`sp_update_job @new_name` if the name contains the period). `GETDATE()` uses the SQL host's +clock; a different time zone only shifts the cut-off by hours. -- 2.45.3 From 28f5b7f7ee64d7baf7fd417603b7018c48ee239d Mon Sep 17 00:00:00 2001 From: cliu Date: Fri, 2 Oct 2026 14:47:33 +1000 Subject: [PATCH 2/2] Record RDBMS exposure in manual summaries, progress and issues ISS-057..061 Co-Authored-By: Claude Opus 5.5 --- .../tasks/setup-check.md | 7 +++++++ knowledge/issues.md | 5 +++++ knowledge/manuals/IM36K10C10-01EN.md | 4 +++- knowledge/manuals/IM36K10G10-01EN.md | 7 ++++++- knowledge/manuals/IM36K10L45-01EN.md | 21 +++++++++++++++++++ knowledge/progress.md | 3 ++- 6 files changed, 44 insertions(+), 3 deletions(-) diff --git a/.claude/skills/installation-security/tasks/setup-check.md b/.claude/skills/installation-security/tasks/setup-check.md index 2b7eb19..268de40 100644 --- a/.claude/skills/installation-security/tasks/setup-check.md +++ b/.claude/skills/installation-security/tasks/setup-check.md @@ -35,6 +35,13 @@ Run from the repo root, preferably elevated. Changes nothing. One line per check The first lines show the version and build (e.g. `R1.03.00 … (r10200M)`). - The web ports actually used are `SECURE_PORT_NUMBER` / `NON_SECURE_PORT_NUMBER` in `%TLS_ROOT_PATH%\tls\sup\jwuiisg.sup` (8443 / 8080 by default). +- Folder rights (not in the script): setup files can hold passwords (e.g. `dbc.sup`). Check + who can read and change them: + `foreach ($r in (Get-Acl "$env:TLS_ROOT_PATH\tls\sup").Access) { "{0} : {1}" -f $r.IdentityReference, $r.FileSystemRights }`. + Under Program Files, Users can read; an install outside Program Files (e.g. + `C:\Yokogawa\CIServer`, seen on an R1.05 system) can inherit *Authenticated Users: Modify*, + so any signed-in user can change setup files (ISS-061). Fixing it is a site decision (IT + Security Tool). ## Test system result (2026-09-30) diff --git a/knowledge/issues.md b/knowledge/issues.md index d93e924..655f4bc 100644 --- a/knowledge/issues.md +++ b/knowledge/issues.md @@ -76,3 +76,8 @@ development system (see "Test system" in `progress.md`); they do not exist on ot | ISS-054 | 2026-10-01 | AD (manual vs installed) | IM 36K10J40 vs the installed `dssana.sup`: the *Active Directory* tab has keywords the manual does not describe — `AD_SERVER`, `AD_SERVER_BACKUP` (host[:port], 389/636), `AD_SECURE` (LDAPS; installed `NO`, help says default yes), `AD_ACCESS_USER`/`AD_ACCESS_PASSWORD` (blank = the CI Server service account). The manual's *Profile priority* is `USER_DF.MERGE_PRIORITY`; *Directory group* is `DIRECTORY_GROUP` (68 characters). The SSO keytab path is given as `%TLS_WAP_PATH%\webapps\auth\WEB-INF` (§2.5.1) and `%TLS_ALLUSERS_PATH%\wap\webapps\auth\WEB-INF` (§3.4.3) — the same folder on the test system. | closed | Installed software is the reference; documented in the J40 summary §5 and `authorization/tasks/active-directory.md`. Not testable without a domain. Closed 2026-10-01 (confirmed by user). | `knowledge/manuals/IM36K10J40-01EN.md`; `.claude/skills/authorization/tasks/active-directory.md` | | ISS-055 | 2026-10-02 | GRAPHICS (Micro Trend) | A Micro Trend written into `CIENG_TrendLab` (`itemName[0..1]` = `TANK1.LEVEL`/`INFLOW`, `numberOfPens` 2, `realTime`, `fillWithHistory`) drew both pens on the desktop and in the browser the first time (about 13:33), then stayed **blank** on every later open on both clients: with and without the 48 spare trend pens, and with explicit `timeRange` 600 / `sampleCount` 60. No console error; CI Server running. Code: `MicroTrend.js` only starts when all properties are set (`startTrendIfAllPropertiesAreSet`; it also compares `scaleMax[i]` with `scaleMin[1]`). | open | Later 2026-10-02 the clean sample file (no explicit `timeRange`/`sampleCount`, 48 spare trend pens) drew both pens on the desktop and in the browser after a fresh login; after a *View > Reload* in the browser only LEVEL showed fully, INFLOW only a short piece: the behaviour is intermittent. **Cause 1 found 2026-10-02**: `setNumberOfPens()` calls `updatePenSource()`, which replaces every pen source with an empty one, so fixed `itemName[n]` values are lost when *Number of pens* is applied after them during load (order varies). Fix in the sample: connect `microTrend.itemName[n]` to the item with attribute `ItemName` (as the factory/test displays do); both pens then drew on 3 of 3 opens. **Remaining**: on 1 of 3 opens only the live part drew (no history fill); every property set calls `startTrend()` again, so history requests are repeated and can cancel each other (web-client code, not fixable in the display). The Operator Interface (desktop) showed the same: both pens live, no history fill on the next open, so the fill problem is not web-only. After a CI Server restart (14:30): web first open of a copy of the display drew both pens with history; the next opens (another copy, and the same display again) showed only the live part; the desktop drew history on one display and only the live part on the custom-range copy. No pattern found yet. Earlier note: still to try: build the Micro Trend in the Edit Module (compare the XML), connect `itemName[n]` instead of a fixed value (as in factory/test displays), a display with only the Micro Trend, a CI Server restart. | `.claude/skills/graphics/tasks/trend.md` §6; `examples/CI-Eng-Sample/displays/CIENG_TrendLab.xml` | | ISS-056 | 2026-10-02 | GRAPHICS (trend, manual vs installed) | (1) IM 36K10C50 Table 2.27-28 lists *Appearance - Show in Legend* as supported in the HTML display, but the web legend listed pens with `visibleLegend` false (the desktop hides them). (2) The legend position (`PositionSouthWest`) was not applied in the browser (not supported per the table). (3) The manual says a trend has up to 20 pens; the component property sheet has 20 slots, but the web code reserves 50 and `maximumPens` 50 works on the desktop. (4) Desktop: pen `title`/`titlePlacement`, pen limit values `lowLimit`… and chart `backgroundPaint` had no visible effect; time-axis minor ticks/grid showed nothing at a 10-minute range. | closed | Documented in `trend.md`; installed software is the reference. Close when the user confirms. Closed 2026-10-02 (confirmed by user). | `.claude/skills/graphics/tasks/trend.md` | +| ISS-057 | 2026-10-02 | DATA EXPOSURE (filter, manual vs installed) | IM 36K10G10 §10.1 gives the filter example `"NAME" like "ABC*"` (quoted field name). On R1.05 a `DATA_EXPOSURE_DF` record with `RECORDS` = `"NAME" LIKE "
*"` made `DBCSND` log `DSS-E-FLT_SYN_ERR` (+ `DSSF-E-CMP_NO_SUCC`, `DBC-E-DSS_SUB`, `DBC-E-SUB_DAT`, `DBC-E-DAT_HND`) and exit (`DUR-W-REM_PROC`), stopping **all** exposures. Tested on the user's R1.05 system (AID.WRPS items, exposure `WRPS-ITEM-VAL`). | closed | Closed 2026-10-02 (confirmed by user; user decision: installed software is the reference): the unquoted form `NAME LIKE "
*"` compiles and data flows. Tested in R1.05 only. | `.claude/skills/data-access/tasks/rdbms-exposure.md` (done); `knowledge/manuals/IM36K10G10-01EN.md` (done); `knowledge/manuals/IM36K10L45-01EN.md` §8 (done) | +| ISS-058 | 2026-10-02 | DATA EXPOSURE (ITEM_HISTORY, manual vs installed) | IM 36K10G10 ch. 10 lists `ITEM_HISTORY` as an exposure source (filter required). The installed `dbc.sup` (R1.03 and R1.05, identical) has the change-log line `PCL Aug-19 e12150 Refactor to support MS-SQL/ODBC, remove ITM history support`. An `ITEM_HISTORY` record (all fields, quoted filter, ISS-057) on the user's R1.05 system never created its table; it was disabled and then deleted, so whether ITEM_HISTORY works with a correct filter is **not tested**. | open | Treat ITEM_HISTORY exposure as unsupported and use `ITEM_VALUES` with `AGG_INTERVAL`. To close: on an R1.05 system, create an `ITEM_HISTORY` record with an unquoted filter and check whether `DBCSND` stays up and the table is created. | `.claude/skills/data-access/tasks/rdbms-exposure.md`; `knowledge/manuals/IM36K10G10-01EN.md`; `knowledge/manuals/IM36K10L45-01EN.md` §8 | +| ISS-059 | 2026-10-02 | DATA EXPOSURE (authentication) | IM 36K10G10 §10.2 says to add the `CIX_PROCESS` account to SQL Server as a Windows user (db_owner) in a domain. The CI Service runs as the **local** `.\CIX_PROCESS` (also on the user's domain-joined R1.05 CI Server), which a SQL Server on another host cannot authenticate. | closed | Closed 2026-10-02 (confirmed by user): use a SQL login with db_owner on the target database; SQL Server set to mixed mode (needs a SQL Server restart). Windows authentication only if the SQL host knows the account (not tested). Tested in R1.05 only. | `.claude/skills/data-access/tasks/rdbms-exposure.md` (done); `knowledge/manuals/IM36K10G10-01EN.md` (done) | +| ISS-060 | 2026-10-02 | DATA EXPOSURE (rows of steady items) | IM 36K10G10 ch. 10 says `ITEM_VALUES` is exposed once per interval, with the aggregation time as timestamp when the item did not change. In one snapshot on R1.05 (1-minute interval), changing items had rows at 02:13:23 while steady items (`VALUE_COUNT` 1) were still at 02:12:24. Whether steady items get a row every interval (maybe written later) or only after an update is not known. | open | Count rows per item over one hour (`SELECT NAME, COUNT(*) … WHERE [TIMESTAMP] > … GROUP BY NAME`) and compare steady with changing items. Until then the task says: take a steady item's last row at or before T. | `.claude/skills/data-access/tasks/rdbms-exposure.md` | +| ISS-061 | 2026-10-02 | INSTALLATION (folder permissions) | On the user's R1.05 system installed in `C:\Yokogawa\CIServer` (outside Program Files) `tls\sup` and `dbc.sup` inherit **Authenticated Users: Modify** (plus Users: Read): any signed-in user can read the SQL password in `dbc.sup` and change setup files. On the R1.03 development machine (Program Files) `tls\sup` gives Users: ReadAndExecute (read still possible, no write). | closed | Closed 2026-10-02 (user decision: shared internal dev/demo server, left as is). Option if needed, not applied or tested: `icacls /inheritance:r /grant:r "SYSTEM:(F)" "Administrators:(F)" "CIX_PROCESS:(R)"`; folder rights in general are set by the IT Security Tool. | `.claude/skills/installation-security/tasks/setup-check.md` (done); `.claude/skills/data-access/tasks/rdbms-exposure.md` (done) | diff --git a/knowledge/manuals/IM36K10C10-01EN.md b/knowledge/manuals/IM36K10C10-01EN.md index c779c93..9fd7f2e 100644 --- a/knowledge/manuals/IM36K10C10-01EN.md +++ b/knowledge/manuals/IM36K10C10-01EN.md @@ -141,7 +141,9 @@ recommended; tag and type cannot be changed (delete and recreate). immediately* (keep value/status/quality over a warm restart instead of NOT INIT), process areas, comment 1/2, map position (WGS 84 latitude/longitude). - *Item management*: blocked, OPC visible (+ read/write; also needs the section visible), - alarm inhibit, parent flags, data exposure aggregation interval (60 s–7 days, 0 = off). + alarm inhibit, parent flags, data exposure aggregation interval (60 s–7 days, 0 = off; + dataset field `ITEM_DF.AGG_INTERVAL`, e.g. `"1 Minutes"` by quick load; only for RDBMS + exposure of `ITEM_VALUES`, task `data-access/rdbms-exposure.md`, tested in R1.05 only). - *Limits and filtering*: HH/H/L/LL, deadband (values may be rounded), scale high/low, limit clamping (HH-LL, H-L, scale, none), out-of-range support, trend axis limits, PIT values. String items: max length, default 80. The manual says max 100; the installed software diff --git a/knowledge/manuals/IM36K10G10-01EN.md b/knowledge/manuals/IM36K10G10-01EN.md index 552d2cb..b256d14 100644 --- a/knowledge/manuals/IM36K10G10-01EN.md +++ b/knowledge/manuals/IM36K10G10-01EN.md @@ -95,7 +95,12 @@ skill `audit-trail`. **RDBMS exposure (ch. 10)**: copies CI Server data (`ITEM_VALUES` aggregated per interval, `ITEM_HISTORY` with a mandatory filter, `ITEM_INPUTS` for writes back, or any dataset on change) into SQL tables. Configure the connection string or system DSN in `dbc.sup`; with -Windows authentication add `CIX_PROCESS` to SQL Server (db_owner). +Windows authentication add `CIX_PROCESS` to SQL Server (db_owner). Deleting an exposure drops +its target table. **Tested in R1.05 only** for `ITEM_VALUES` (skill `data-access`, task +`rdbms-exposure.md`): the manual's filter example with a quoted field name (`"NAME" like +"ABC*"`) fails (ISS-057); the installed `dbc.sup` change log says ITEM_HISTORY support was +removed (ISS-058); with the local `.\CIX_PROCESS` a remote SQL Server needs a SQL login +(ISS-059). **Other**: Item search (wildcards, copy/drag to Excel or Trend); Excel Add-in reports; history storage groups and archiving (phase 9). diff --git a/knowledge/manuals/IM36K10L45-01EN.md b/knowledge/manuals/IM36K10L45-01EN.md index e670704..06e048a 100644 --- a/knowledge/manuals/IM36K10L45-01EN.md +++ b/knowledge/manuals/IM36K10L45-01EN.md @@ -180,3 +180,24 @@ and 2.0, custom + automation interfaces), A&E client (`opxaec`), A&E server (`op suggested test client. - Discovery: LDS on 4840 only when several servers are on a host; GDS for central certificate management (concepts only). + +## 8. RDBMS data exposure (not in this manual) + +IM 36K10L45 does not describe RDBMS data exposure; it is in the Tutorial (IM 36K10G10 ch. 10) +and the item form (IM 36K10C10 §3.3). Summary here because it is an ACCESS-type interface; +**tested in R1.05 only** (user system, 2026-10-02; R1.03 files checked read-only and identical). +Procedure: skill `data-access`, task `rdbms-exposure.md`. + +- Processes `DBCSND` (sender, subscribes to DSS data) and `DBCDEM` (receiver, writes to MS SQL + Server via ODBC); setup file `dbc.sup` (sender and receiver tabs; connection string + `DEM_CONNECTION`, default `Driver={SQL Server};Server=localhost\SQLEXPRESS;…`). +- Exposures are records in `DATA_EXPOSURE_DF` (source dataset, remote table, filter `RECORDS`, + up to 32 field → column pairs or all fields). The target table is created by CI Server. +- `ITEM_VALUES` exposure is driven by the per-item aggregation interval `ITEM_DF.AGG_INTERVAL` + (60 s–7 days, 0 = off): value, min, max, average, count and a local-time `TIMESTAMP` (seconds + since 1970) per interval. +- Filter field names must not be quoted (`NAME LIKE "
*"`); the manual's quoted example + fails and stops all exposures (ISS-057). `ITEM_HISTORY` exposure: manual vs installed change + log disagree (ISS-058). +- The CI Service account `.\CIX_PROCESS` is local, so a remote SQL Server needs a SQL login and + mixed mode (ISS-059). diff --git a/knowledge/progress.md b/knowledge/progress.md index 1fe8984..5e2a82b 100644 --- a/knowledge/progress.md +++ b/knowledge/progress.md @@ -109,7 +109,7 @@ now come before classes and objects (phase 6), and the other interfaces follow a | C10 | Integrator time unit: integration granule `itm_int_secs` in `itm.sup` ([task](../.claude/skills/item-engineering/tasks/sub-items.md)) | passed | passed | 60 → `RUN_SECONDS` +1 per minute; restored to 1 (file identical to backup), +1 per second again ([ISS-010](issues.md)). Elevated restarts run by the user. | | G10 | Displays, symbols, Operator Interface test | display done (phase 10); symbols not practiced | display added | Display `CIENG_Tank` built in the Edit Module and checked in the HTML display. | | L41/C10 | Modbus TCP line, station, scan type, points, connected items by quick load ([task](../.claude/skills/equipment/tasks/modbus-station.md)) | passed | passed | Loopback to CI Server's own Modbus slave (`eqpmdcmem` + `eqpmdcslvtcp` on 5020, started by hand). Line load created and started `EQPCIENG`; removal stopped it and deleted its files. Write/read-back, off-scan/on-scan checked; full remove/reload from sample files. Protocol must be Open Modbus/TCP ([ISS-024](issues.md)). Mirror start files copied to `tls\com` by the user; after a restart everything came back under the CI Service and the setpoint was re-sent on connect. | -| G10 | Alarms, reports, audit, RDBMS, history | alarms (phase 8), history (phase 9), reports (phase 11), audit (phase 12) done; RDBMS not practiced | alarms, history, reports, audit added | Covered in their own phases (7–12). | +| G10 | Alarms, reports, audit, RDBMS, history | alarms (phase 8), history (phase 9), reports (phase 11), audit (phase 12) done; RDBMS passed on a user R1.05 system (see the RDBMS row below) | alarms, history, reports, audit added; RDBMS n/a (tested in R1.05 only) | Covered in their own phases (7–12); RDBMS added 2026-10-02. | | R10/R20 | Read a setup file: active settings and keyword help ([task](../.claude/skills/portal-setup/tasks/read-setup-file.md)) | passed | n/a | Read-only. Compared with the Setup File Editor GUI for `wfla_display.sup`, `wflgdi.sup` and all 9 tabs of `jwuiisg.sup`: fields and values identical. Manual differences logged as ISS-012 to ISS-017 (installed software taken as the reference). | | R10 | Check web access for CI View: ports, Remote Connect page, `DEPLOY` folder ([task](../.claude/skills/portal-setup/tasks/check-web-access.md)) | passed | n/a | 8080/8443 listening, page 200, six `DEPLOY` folders. | | R10/R20 | Change a setup file safely ([task](../.claude/skills/portal-setup/tasks/change-setup-file.md)) | passed | n/a | `jwuiisg.sup` HTTP port 8080 → 8081 (text edit, elevated restart): 8081 served the page, 8080 refused; restored, hash identical to backup. Editor: *OK* writes at once, *Reset Field* restores the default (ISS-016). | @@ -117,6 +117,7 @@ now come before classes and objects (phase 6), and the other interfaces follow a | R10 | Login method / login control | not practiced | n/a | Moved to `dssana.sup` (ISS-017); phase 5 (C10) and 13 (J40). | | L45/E20 | OPC UA client → own OPC UA server loopback by quick load; certificate exchange; node IDs ([task](../.claude/skills/data-access/tasks/opcua-client.md)) | passed | passed | Station `CIENG_UA` (`opc.tcp://localhost:34493`, Basic256Sha256/SignAndEncrypt): `UA.LEVEL_COPY` follows `TANK1.LEVEL` (55, 62.5, 71, 33, 44.4), `UA.SRV_STATE` 0. Certificates trusted by the user (client side, then server side). Entity namespace `urn:Yokogawa:CI_SERVER_OPC_UA_SERVER` found by browsing `OPCUAC_TAG_DF` (ISS-032). Full remove/reload clean, reconnects without a new exchange. User configured a point in the Engineering Module: browse stores the item object, value node must be typed (ISS-033); test item/point removed afterwards. | | L45 | ACCESS read-only checks: processes, ports, OPC UA server config/PKI, ODBC server ([task](../.claude/skills/data-access/tasks/access-checks.md)) | passed | n/a | ISS-008 cause found: `odbsrv` cannot load its SSL certificate (folder `tls\pki\odbsrv` missing) and exits. | +| G10/C10 | RDBMS data exposure to SQL Server: SQL login, `dbc.sup` `DEM_CONNECTION`, `DATA_EXPOSURE_DF`, `ITEM_DF.AGG_INTERVAL`, SQL-side retention job ([task](../.claude/skills/data-access/tasks/rdbms-exposure.md)) | passed (R1.05 only) | n/a — tested in R1.05 only (user decision) | 2026-10-02 on the user's R1.05.00 system (not the R1.03 dev machine): 49 `AID.WRPS.*` items at 1 minute into a remote SQL Server 2022; rows survive a CI Server restart; 60-day purge job. Quoted filter field stopped DBCSND ([ISS-057](issues.md)); ITEM_HISTORY not tested ([ISS-058](issues.md)); SQL login needed ([ISS-059](issues.md)); steady-item rows ([ISS-060](issues.md)); folder rights ([ISS-061](issues.md)). R1.03 `dbc.sup` and dataset definitions checked read-only (identical). | | L30 | BUS utilities: `durwfp`, setup files (`dur.sup`, `durm_1_udp.sup`, `time_table.sup`, `fmfprt.sup`) ([task](../.claude/skills/system-management/tasks/bus-utilities.md)) | passed (read-only) | n/a | `umhman`, `umhtestprog`, `umhmsg` fail (ISS-030). Unlicensed run-time limit hit and documented (ISS-031). | | L30 | MDUR node links, encryption, logical addresses, printer devices | not practiced | n/a | Single node, no printer devices. | | L55 | CI Exchange for CENTUM VP / Prodigy | not practiced | n/a | Needs CENTUM VP (RGS, Vnet/IP) or Prodigy data. | -- 2.45.3