SLT replication monitoring

SLT replication monitoring

This monitor is dedicated to the monitoring of SAP LT Replication Server (SLT).
It provides a monitoring equivalent to the replication figures of transaction LTRC.
It discovers the replication configurations defined on the system, then reports for each
replicated table how many changed records are still waiting to be replicated, and how far
behind the table has fallen.

Note: the connector must point to the SLT server, that is the system where transaction LTRC opens. On a system without SLT, the monitor stops with the error IUUC_REPL_HDR does not exist on this system - this is not an SLT server. Some replication source systems do hold that table without running SLT: there, the configurations are found but every one of them reports a collection error, because the SLT functions cannot be called.

Prerequisites

The RFC user needs the following on the SLT server:

S_RFC for function group IUUC_REPL_API, which holds both function modules used by this monitor.
S_RFC for function group SDTX, used to read the configuration table.
DMIS display authority. The function modules check it before doing anything and refuse the call without it, even when S_RFC is granted.
Display authorization on table IUUC_REPL_HDR.

Note: counting unprocessed records makes an onward RFC call from the SLT server to each source system. The user therefore also needs authorization in the source systems, not only on the SLT server. A missing authorization there is reported as an authorization error against the configuration concerned.

Configuration hints

Use the surveillance table to adjust the monitoring settings:
- Thresholds
- Severity
- Filters

Configurations are discovered automatically on every run, so a configuration added in SLT is picked up without changing this monitor.
Only configurations selected by at least one active surveillance line are read. The others cost nothing, which matters because counting unprocessed records makes SLT call each source system.

Latency window

Defines how far back the monitor looks when reading latency, in minutes.
Each table is judged on its most recent measurement inside the window, so a long window does not keep older peaks in the figures. The window only limits how far back SLT is asked for data.
MEDIAN is the exception: SLT only records a median on its hourly figures, so the monitor uses the most recent hourly figure that carries one. With a long window, that median can be several hours old, and an old peak can then stay in the figures. Use at least 60 minutes for MEDIAN, and keep the window close to that if the median should reflect the last hour.
Default is 60 minutes.
This setting has no effect on the count of unprocessed records, which is always the current value.

Per-occurrence alarm

This option allows to tune the volume of alarms that will be generated:
- If enabled, you will receive one alarm per table over a threshold, with details.
- If disabled, you will only get one alarm per rule and configuration, listing the tables that breached. The first 20 tables are named and the others are counted, so the message stays readable in integrations; the metadata still carries every table.

Send metadata

If enabled, this parameter will send the collected tables as metadata to the configured plugins/integrations
It can usually be retrieved as logs or tabular data within the corresponding integrations
All tables of the configurations selected by an active surveillance line are sent, not only the tables matching a line's table filter
Configurations for which SLT refuses to count unprocessed records are sent with the tables latency was returned for, marked with the reason in unsupportedReason
When a figure could not be collected, the row carries the reason in collectionError. A configuration that could not be collected at all is sent as a single row without a table
The median latency comes from SLT's hourly figures, so its interval is sent separately in medianIntervalEnd; latencyIntervalEnd is the interval of the minimum and maximum

Surveillance table

Parameter	Description
Active	Use this field to activate or deactivate a line of configuration.
Configuration	A filter to match only a given replication configuration. It is matched against the schema name and the description, so either can be used. An exclusion such as `!TEST` excludes the configuration when either of them contains the text.
Table	A filter to match only a given replicated table.
Max unprocessed records	The threshold for the number of records waiting to be replicated. An alarm is sent when the count reaches this value. Leave at `0` to disable this check.
Only if not draining	Restricts the above check to backlogs that are not being absorbed. See the note below.
Latency figure	Which latency value the latency threshold applies to: MIN, MAX or MEDIAN. See the mapping below.
Max latency	The threshold for the latency, in seconds. An alarm is sent when the selected figure is above this value. Leave at `0` to disable this check.
Alarm if master job stopped	If checked, an alarm is sent when the master job of the configuration is not running.
Severity	The severity of the alarm.
enable Alarm	If checked, this line of surveillance will be used for alarm generation.
enable QOS	If checked, this line of surveillance will be used for metric generation.

Note: with Only if not draining enabled, an alarm is only raised if the backlog has not shrunk by more than 5% since the previous run. A backlog shrinking faster than that means the replication is catching up, and stays quiet. A backlog that is stable, growing, or only moving up and down a little alarms, because a backlog frozen at a high value is not being processed — and a steady alarm is kept open instead of closing on every small dip. The 5% is measured between two runs, so it depends on the schedule: with a run every minute, a backlog must shrink by more than 5% every minute to count as draining, which is much stricter than 5% every 15 minutes. Choose the schedule with that in mind. The very first run only records a starting point and raises no alarm on this check. If a table cannot be counted on a run, for example because the SLT server cannot reach the source system, its last known count is kept, so the comparison resumes as soon as counting works again.

Note: a configuration whose migration is finished usually keeps its entry in SLT with a stopped master job. A line with Configuration * and Alarm if master job stopped checked would then report it as halted on every run, and still read it from SLT. Exclude such configurations with the Configuration filter; an excluded configuration is neither read nor alarmed. !OLD_MIGRATION excludes every configuration whose name or description contains OLD_MIGRATION. To exclude several unrelated ones, join them with +: !ECC_2019+!HR_TEST. Do not use a comma for this: !ECC_2019,!HR_TEST only excludes the first one.

Note: an absolute threshold alone is rarely usable on its own. A high volume table can sit permanently above any reasonable value while replicating perfectly, and a low volume table may never reach it even when stuck. This is why Only if not draining is enabled by default.

Latency figures

SLT records latency per table and per measurement interval. The monitor uses the most recent interval available within the latency window.
When SLT splits that interval into several portions, they are combined: MAX is the highest of the portions and MIN the lowest, so one lagging portion is never hidden by a fast one.

Latency figure	Description
MIN	The lowest latency measured during the interval.
MAX	The highest latency measured during the interval. Catches the worst moment.
MEDIAN	The median latency of the interval. Ignores isolated peaks.

Note: SLT only records a median on its hourly figures. When MEDIAN is selected, the monitor uses the most recent interval that actually carries one. If no interval carries a median, the table reports no latency rather than a latency of zero, and the latency check does not alarm for that table.

Note: latency is only recorded for tables in delta replication. Tables in initial load, or stopped, have no latency figure at all. Such a table is reported without a latency, and is never reported as being up to date. A latency threshold on a table that is not in delta replication can therefore never be reached.

Note: when SLT cannot read a table's logging table, the table is reported without a count of unprocessed records, never with a count of zero, and the monitor log shows a warning. The Max unprocessed records check cannot alarm on that table until its logging table can be read again, and a backlog alarm already open for it closes: a logging table can stay unreadable for good, and an alarm held open for it would never close.

Note: the backlog comparison of Only if not draining is kept only by scheduled runs. Running the monitor with the Test button does not change the count the next scheduled run compares against.

Unsupported configurations

Some replication scenarios are not supported by the SLT programming interface used to count unprocessed records. The call is refused by SAP with an unsupported sender system type, even when the configuration is running normally.
For such a configuration, the unprocessed records are reported as not monitored, and the monitor log shows the reason. The Max unprocessed records check can never alarm on it.
This is deliberately not reported as healthy and does not raise an alarm by itself. It is not a fault, but it must remain visible: a check silently skipped would leave a permanently green monitor over a replication that nothing is watching.
Everything else is still checked. Latency is collected through a different interface: when SLT returns it, the latency threshold, the latency metric and the metadata apply as for any configuration. A stopped master job and collection errors still raise their alarms.
In the metadata, the tables of such a configuration show Unprocessed:not supported and carry the reason in unsupportedReason.

Alarm content

The alarm message contains the configuration, the table concerned, the value that breached and the threshold it breached.
For a backlog alarm, the count recorded at the previous run is appended when one is available, so the trend can be read from the alarm itself.
A stopped master job raises its own alarm on the configuration, independently of any table threshold. It is raised once per configuration, even when several surveillance lines select it, at the highest severity among those lines that have Alarm if master job stopped checked.
A configuration that could not be collected, for example when the SLT server cannot reach the source system, raises an alarm carrying the reason, also once per configuration at the highest severity of the lines selecting it.
While a configuration cannot be collected, its open backlog and latency alarms are kept open unchanged, instead of closing and reopening once collection works again. They close only when a run actually finds the problem gone, or when the configuration is no longer monitored.
In integrations that show the metric of an alarm, a backlog alarm is reported under SLT_UNPROCESSED_RECORDS and a latency alarm under SLT_LATENCY. An alarm covering both a backlog and a latency breach is reported under SLT_UNPROCESSED_RECORDS. Master job and collection alarms carry no metric.

Examples

Example 1

Active	Configuration	Table	Max unprocessed records	Only if not draining	Latency figure	Max latency	Alarm if master job stopped	Severity	enable Alarm	enable QOS
true	*	*	10000	true	MAX	300	true	MAJOR	true	true

Effect: Sends a MAJOR alarm for each table holding at least 10000 unprocessed records that is not being absorbed, and for each table whose worst latency exceeds 5 minutes. Also alarms if a master job is stopped. Sends the record counts and the maximum latency as metrics.

Example 2

Active	Configuration	Table	Max unprocessed records	Only if not draining	Latency figure	Max latency	Alarm if master job stopped	Severity	enable Alarm	enable QOS
true	ECC_PRD_CFIN_S4P	BKPF	0	true	MEDIAN	60	false	WARNING	true	true

Effect: Sends a WARNING alarm when the median latency of table BKPF in the Central Finance configuration exceeds 1 minute. The backlog check is disabled, and a stopped master job does not alarm on this rule.

Generated metrics

metricId	metricUnit	metricTarget	metricDescription
SLT_UNPROCESSED_RECORDS	Records	[CONFIGURATION][TABLE]	Sends the number of records waiting to be replicated, per configuration and table.
SLT_LATENCY	Seconds	[CONFIGURATION][TABLE][STATISTIC]	Sends the latency of the table, per configuration and table. The STATISTIC target records which figure was sent, so a series is never ambiguous.

Note: only the latency figure selected on the surveillance line is sent, not all three. A replication configuration can carry several hundred tables, and sending the minimum, maximum and median for each would triple an already large number of series.

Note: when several surveillance lines match the same table, its metrics are still sent once per run. Only lines that select different latency figures add a series, one per figure.

Note: metrics are sent from the first run even when no threshold is configured. This is intentional, so the normal range of a system can be observed before choosing thresholds.

Note: a table with no value for a metric is not sent, rather than being sent as zero. This applies to the latency of tables that are not in delta replication.

Table of Contents