--- task: RDBMS data exposure — copy item values into Microsoft SQL Server (dbc.sup, DATA_EXPOSURE_DF, item aggregation interval) version: R1.05 source: IM 36K10G10-01EN ch. 10 (§10.1, §10.2); IM 36K10C10-01EN §3.3 (Item management › Data exposure aggregation); installed tls\sup\dbc.sup, tls\src\data_exposure_df.dss, tls\src\item_df.dss; practiced on a user system 2026-10-02 status: tested in R1.05 only --- # RDBMS data exposure to SQL Server **Tested in R1.05 only** (CI Server R1.05.00 r11609M, user system, SQL Server 2022 Developer Edition on a separate domain-joined host). Not re-tested on R1.03 or on CI-Eng-Sample (user decision 2026-10-02). On the R1.03 development machine the files `dbc.sup`, `data_exposure_df.dss` and `item_df.dss` were checked read-only and match what is described here (`dbc.sup` is identical in R1.03 and R1.05 apart from the site's `DEM_CONNECTION` line; `data_exposure_df.dss` and `item_df.dss` identical; re-checked read-only on the R1.05 test server 2026-10-08). **Changes live configuration on two systems** (SQL Server and CI Server): follow the user's chosen way of working. SQL Server changes on a shared server need the owner's agreement. ## How it works - `DBCSND` (sender) subscribes to the exposed CI Server data and sends it to `DBCDEM` (receiver, the *data exposure module*), which writes it to SQL Server through ODBC. `DEM_NODE = 0` in `dbc.sup` = receiver on the local node. `dbcdem.exe` is 64-bit. - Each exposure is a record in dataset **`DATA_EXPOSURE_DF`** (Engineering Module: *Reporting › Data exposures*). CI Server creates the SQL table itself when the first record is exposed. - For source **`ITEM_VALUES`**, rows are produced only for items whose **`ITEM_DF.AGG_INTERVAL`** (item form › *Item management* › data exposure aggregation; 60 s–7 days, 0 = off, default 0) is set. Each interval adds rows with the value, min, max, average and sample count. - Other sources per the manual: `ITEM_HISTORY` (needs a filter; see ISS-058), `ITEM_INPUTS` (values written in SQL go back to CI Server), any other dataset on change. Only `ITEM_VALUES` was tested. - Manual warning (not tested): **deleting a data exposure drops its target table**, even if another exposure uses the same table. ## 1. SQL Server: read-only checks In SSMS on the SQL host: ```sql SELECT SERVERPROPERTY('IsIntegratedSecurityOnly') AS WindowsOnly; -- 1 = Windows only, 0 = mixed EXEC xp_readerrorlog 0, 1, N'Server is listening on'; -- TCP port(s) SELECT name, is_disabled FROM sys.server_principals WHERE name = 'sa'; ``` From the CI Server: ```powershell Test-NetConnection -Port 1433 ``` Pass: `TcpTestSucceeded : True`. If TCP 1433 already listens and the test passes, no TCP/IP or firewall change is needed. ## 2. Choose the authentication ```powershell (Get-CimInstance Win32_Service -Filter "Name='StartCIServer'").StartName ``` The CI Service normally runs as the **local** account `.\CIX_PROCESS`, also on a domain-joined CI Server. A SQL Server on another host cannot authenticate a local account, so Windows authentication does not work there: use a **SQL login** (ISS-059). The manual's advice to add `CIX_PROCESS` to SQL Server as a Windows user only applies when that account is known to the SQL host (domain account, or SQL Server on the same machine). ## 3. SQL Server: mixed mode (only if `WindowsOnly = 1`) Server › *Properties* › *Security* › **SQL Server and Windows Authentication mode**, then restart the SQL Server service. The restart drops all connections on that server: agree a time with the owner. Make sure `sa` is disabled or has a strong password first. Check: `WindowsOnly` = 0. ## 4. SQL Server: login with db_owner on one database Create it at **server** level (*Security › Logins › New Login*), not under *Database › Security › Users* (that only maps an existing login). Do not call it `db_owner` (that is a role). ```sql CREATE LOGIN WITH PASSWORD = '', DEFAULT_DATABASE = , CHECK_POLICY = ON, CHECK_EXPIRATION = OFF; USE ; CREATE USER FOR LOGIN ; ALTER ROLE db_owner ADD MEMBER ; ``` `db_owner` is needed because CI Server creates (and may drop or recreate) its tables. Keep the database for CI Server only. ## 5. CI Server: test the ODBC login 64-bit PowerShell on the CI Server (prompts for the password, reads only): ```powershell $pw = Read-Host "password" -AsSecureString; $plain = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($pw)); $c = New-Object System.Data.Odbc.OdbcConnection("Driver={SQL Server};Server=;Database=;Uid=;Pwd=$plain"); $c.Open(); $cmd = $c.CreateCommand(); $cmd.CommandText = "SELECT SUSER_NAME(), DB_NAME()"; $r = $cmd.ExecuteReader(); [void]$r.Read(); "{0} | {1}" -f $r[0], $r[1]; $c.Close() ``` Pass: ` | `. The legacy `{SQL Server}` driver worked with SQL Server 2022. ## 6. CI Server: `DEM_CONNECTION` in `dbc.sup` Admin rights; back up the file first. Change **only** the active `DEM_CONNECTION=` line near the end (not the commented `! DEM_CONNECTION` in the `!DEFAULT` block): ``` DEM_CONNECTION="Driver={SQL Server};Server=;Database=;Uid=;Pwd=" ``` - The password is typed as is (no quotes or brackets). A `;` in it needs `Pwd={…}` (a `}` inside is doubled); a `"` cannot be used. - The password is plain text in the file. A system DSN (manual §10.2) only avoids that with Windows authentication, which step 2 rules out for a remote SQL Server. Check who can read `tls\sup` (ISS-061). - Other `dbc.sup` keywords (queues, batches, retries, `ITH_GRAN_SEC`, `SND_BURST_RATE`, `SND_AUTO_RECREATE`, `SND_NEED_SYNC`) were left at their defaults. Restart CI Server (elevated, Edit/Engineering Module closed): `Restart-Service StartCIServer`. ## 7. Check the DBC processes ```powershell Get-Process dbcsnd, dbcdem -ErrorAction SilentlyContinue ``` ```powershell $m = Select-String -Path "$env:TLS_LOG_PATH\umhlog.log" -Pattern 'From: DBCSND'; $n = $m[-1].LineNumber; $a = Get-Content "$env:TLS_LOG_PATH\umhlog.log"; $a[($n-1)..($n+8)] ``` (Use the same with `From: DBCDEM`. `-Tail 300` can miss the block on a busy log.) | Message | Meaning | |---|---| | `DBCDEM`: `DBC-I-DB_INI_OK, Database connection initialisation SUCCEEDED` | SQL connection works | | `DBCDEM`: `FSL-F-SU_EOF, End of file on setup file` | Harmless, at every start | | `DBCSND`: `DBC-I-DEM_CON, Connection established to data exposure module` | Sender running | | `DBCSND`: `DSS-E-FLT_SYN_ERR` / `DSSF-E-CMP_NO_SUCC` / `DBC-E-DSS_SUB` / `DBC-E-SUB_DAT` / `DBC-E-DAT_HND`, then `DUR-W-REM_PROC, Process DBCSND exited unexpectedly` | A `RECORDS` filter does not compile (step 8). **One bad record stops all exposures.** DBCSND is not restarted automatically: fix, then restart CI Server | ## 8. Data exposure records List existing ones (read-only): ```powershell dssqld -e dexp.qli -d DATA_EXPOSURE_DF -c -l dexp.lst ``` Fields (`tls\src\data_exposure_df.dss`): `NAME`, `DESCRIPTION`, `EXPOSED_DATA` (source dataset, e.g. `ITEM_VALUES`), `REMOTE_TABLE`, `ENABLED`, `UPDATE_PERIOD`, `RECORDS` (filter), `EXPOSED_FIELD_1`…`32` with `REMOTE_COL_1`…`32`, `REF_NAME/SOURCE/VALUE_1`…`8`, `ALL_FIELDS`. Tested `ITEM_VALUES` record (`ENABLED` 1, `UPDATE_PERIOD` 0, `ALL_FIELDS` 0): | Exposed field | Remote column | |---|---| | `NAME` | `NAME` | | `VALUE` | `VALUE` | | `VALUE.MIN` | `VALUE_MIN` | | `VALUE.MAX` | `VALUE_MAX` | | `VALUE.AVG` | `VALUE_AVG` | | `VALUE.COUNT` | `VALUE_COUNT` | | `TIMESTAMP` | `TIMESTAMP` | **Filter syntax:** write the field name **without quotes**: `NAME LIKE "
*"`. The manual's example `"NAME" like "ABC*"` (quoted field) fails with `DSS-E-FLT_SYN_ERR` (ISS-057). In a quick load file quotes inside a string are doubled. Write `.qli` files as ASCII (`Set-Content -Encoding ascii`). Change a field (here: fix the filter): ``` @FIELDS NAME,RECORDS @DATA_EXPOSURE_DF "","NAME LIKE ""
*""" ``` `dssqld -m .qli -l .qlo` → 1 modified, 0 errors. Disable instead of deleting with `NAME,ENABLED` / `"",0`. Delete with `dssqld -r` and only the key (`@FIELDS NAME`, `""`) → 1 removed — this drops the target table (manual). ## 9. Aggregation interval on the items Read for a whole section (read-only; RQL `%` wildcard; verified also on R1.03): ``` set @next 0; set @page_size 0; select name, agg_interval from item_df where name like '
.%'; quit; ``` `cmd /c "rptgen -s rptgen.rpt < agg.rql"` (skill `reporting`, task `rql-interactive.md`). Set it with `dssqld -m`: ``` @FIELDS NAME,AGG_INTERVAL @ITEM_DF "
.","1 Minutes" ``` `0` switches it off. The change took effect **without a CI Server restart**; rows appeared within a few minutes. ## 10. Check the SQL table ```sql USE ; SELECT name, create_date FROM sys.tables WHERE name = ''; SELECT COUNT(*) AS row_count, MIN([TIMESTAMP]) AS oldest FROM dbo.
; SELECT TOP 20 NAME, VALUE, VALUE_MIN, VALUE_MAX, VALUE_AVG, VALUE_COUNT, DATEADD(second, [TIMESTAMP], '1970-01-01') AS local_time FROM dbo.
ORDER BY [TIMESTAMP] DESC; ``` Pass: the row count grows by about one row per item per interval. ## Behaviour observed (R1.05) - Table created by CI Server: `NAME` varchar, `VALUE`/`VALUE_MIN`/`VALUE_MAX`/`VALUE_AVG` float, `VALUE_COUNT` int, `TIMESTAMP` int; clustered key (`NAME`, `TIMESTAMP`). Rows are inserted (the table is a history), not updated. - `TIMESTAMP` = seconds since 1970 in CI Server **local time** (like RQL dates). Readable: `DATEADD(second, [TIMESTAMP], '1970-01-01')`. - `VALUE_COUNT` = updates in the interval (an analog item updated every second ≈ 55 per minute; a steady item 1). - Interval boundaries follow the start of exposure (e.g. :23 s), not the clock minute. - In one snapshot, steady items had no row for the latest interval while changing items did (ISS-060). Until that is clear, read a steady item's value at time T as its last row at or before T. - Rows **survive a CI Server restart** with `SND_NEED_SYNC = YES` (oldest timestamp unchanged, count kept growing). - Existing history is not back-filled: data starts when exposure starts. - Volume: about one row per item per interval (49 items at 1 minute ≈ 70,000 rows a day). Nothing deletes old rows. ## Optional: retention on the SQL side Not part of CI Server. A SQL Server Agent job (not available on SQL Express) that deletes old rows in small batches so CI Server's inserts are not blocked; owner `sa` works even with `sa` disabled: ```sql USE msdb; EXEC dbo.sp_add_job @job_name = N'', @owner_login_name = N'sa'; EXEC dbo.sp_add_jobstep @job_name = N'', @step_name = N'Delete old rows', @subsystem = N'TSQL', @database_name = N'', @command = N' DECLARE @cutoff int = DATEDIFF(second, ''1970-01-01'', DATEADD(day, -, GETDATE())); DECLARE @n int = 1; WHILE @n > 0 BEGIN DELETE TOP (5000) FROM dbo.
WHERE [TIMESTAMP] < @cutoff; SET @n = @@ROWCOUNT; IF @n > 0 WAITFOR DELAY ''00:00:01''; END'; EXEC dbo.sp_add_schedule @schedule_name = N' daily 03:00', @freq_type = 4, @freq_interval = 1, @active_start_time = 030000; EXEC dbo.sp_attach_schedule @job_name = N'', @schedule_name = N' daily 03:00'; EXEC dbo.sp_add_jobserver @job_name = N'', @server_name = N'(local)'; ``` Test: `EXEC msdb.dbo.sp_start_job @job_name = N'';` then `msdb.dbo.sysjobhistory` (`run_status` 1 = succeeded). Change the retention later with `sp_update_jobstep` (and `sp_update_job @new_name` if the name contains the period). `GETDATE()` uses the SQL host's clock; a different time zone only shifts the cut-off by hours.