Skip to main content
GIFRÖST / CDC / Notifications

Track every snapshot from start to finish.

Notifications report the state of initial and ad hoc snapshots. They show when an operation starts, which table is being processed, when a table scan finishes, and whether the complete snapshot completed, was skipped, or was aborted.

PurposeSnapshot progress and final result.
ChannelsSINK, LOG, and JMX.
SnapshotsInitial and incremental.
CorrelationOperation and signal identifiers.

What Notifications are​

Each notification describes one event in a snapshot lifecycle. A sequence of notifications lets an operator follow the operation and confirm its final status. Notifications are the primary way to verify that an operation requested through Signals was actually performed by the connector.

Initial SnapshotConnector startup

Reports a snapshot started during connector initialization, or SKIPPED when the connector determines that a snapshot is not required.

Incremental SnapshotAd hoc operation

Reports an incremental snapshot requested with execute-snapshot, including table progress and the final result.

Configure Notifications​

Configure notification settings under Connectors → Source → Signal → Notification.

Required fields
Enabled notification channels names (notification.enabled.channels) — select sink, log, or jmx. One notification channel can be configured for a connector.
Notification topic name (notification.sink.topic.name) — required for the SINK channel.
Automatic topic creation

In GIFRÖST, the notification topic is created automatically when the connector is created or updated.

JSON configuration​

The selected values are saved in the connector configuration.

SINK channel example
{
"notification.enabled.channels": "sink",
"notification.sink.topic.name": "example-notifications"
}

Notification channels​

A channel determines where Debezium sends a notification. The payload format stays the same, but the way an operator reads it depends on the selected channel.

SINKKafka topic and UI

Sends every snapshot state change to notification.sink.topic.name. Records are available under Topics and in the Notifications tab.

LOGKafka Connect worker logs

Writes notifications to the logs of the worker running the connector. It does not create a separate Kafka topic.

JMXConnector JMX bean

Exposes notifications through the connector's JMX bean for external monitoring tools.

Notification format​

Every Debezium notification contains five primary fields.

FieldMeaning
idUnique notification identifier. For an incremental snapshot, it corresponds to the identifier sent with the execute-snapshot signal.
aggregate_typeOperation category, for example Initial Snapshot or Incremental Snapshot.
typeCurrent event or state, such as STARTED, IN_PROGRESS, TABLE_SCAN_COMPLETED, COMPLETED, ABORTED, or SKIPPED.
additional_dataState-specific details such as the connector name, current table, processed key, or number of scanned rows.
timestampNotification creation time in milliseconds since the UNIX epoch.
Identifier limitation

When several incremental snapshots run at the same time, id might not identify the originating signal unambiguously. Because of a Debezium limitation, notifications for several active snapshots can receive the identifier of the most recently sent signal.

Notification topic​

The SINK channel writes notifications to the topic configured in notification.sink.topic.name.

Step 01Configure SINK

Select sink and enter the notification topic name.

Step 02Open Topics

After creating or updating the connector, open <Connector> → Topics.

Step 03Consume records

Choose partitions, offset, record limit, and read range. Each record represents one notification.

Notification types and statuses​

Initial Snapshot​

An initial snapshot can start when the connector is launched. The connector evaluates snapshot.mode and stored offsets to decide whether a snapshot is required.

StatusMeaning
STARTEDThe initial snapshot has started.
IN_PROGRESSThe connector is scanning a table. additional_data can include data_collections and current_collection_in_progress.
TABLE_SCAN_COMPLETEDOne table scan has finished. The payload contains scanned_collection, total_rows_scanned, and status.
COMPLETEDThe complete initial snapshot finished successfully.
ABORTEDThe initial snapshot was interrupted.
SKIPPEDThe connector determined that a snapshot was not required, for example after a restart with snapshot.mode=when_needed and valid offsets.

Incremental Snapshot​

Incremental Snapshot notifications describe an ad hoc snapshot started with execute-snapshot. Notifications from one execution share the same id.

StatusMeaning
STARTEDThe incremental snapshot has started.
IN_PROGRESSOne selected table is being read. The event can be emitted repeatedly while subsequent chunks are processed.
TABLE_SCAN_COMPLETEDOne selected table has finished scanning. A separate notification is emitted for every table.
COMPLETEDThe complete incremental snapshot finished for all selected tables.
ABORTEDThe operation was interrupted, for example after a valid stop-snapshot signal.

Table scan result​

For an incremental snapshot, TABLE_SCAN_COMPLETED includes additional_data.status.

ValueMeaning
SUCCEEDEDThe table was scanned successfully.
EMPTYThe table contains no records.
NO_PRIMARY_KEYThe snapshot cannot finish because the table has no primary key.
SKIPPEDThe table type cannot be processed; review the connector logs.
SQL_EXCEPTIONAn SQL exception occurred while the snapshot query was running.
UNKNOWN_SCHEMAThe connector could not find the selected table schema.

Status-specific additional data​

Values in angle brackets below indicate data types or placeholders.

STARTED​

{
"connector_name": "<name>",
"data_collections": "<tables>"
}
  • connector_name — connector identifier included in the notification.
  • data_collections — table or set of tables included in the snapshot.

IN_PROGRESS​

{
"current_collection_in_progress": "<table>",
"connector_name": "<name>",
"maximum_key": "<key>",
"last_processed_key": "<key>",
"data_collections": "<tables>"
}
  • current_collection_in_progress — table currently being processed.
  • maximum_key — greatest key value determined for the current table.
  • last_processed_key — last key processed when the notification was created.
  • data_collections — tables that remain in the operation scope.

TABLE_SCAN_COMPLETED​

{
"scanned_collection": "<table>",
"connector_name": "<name>",
"total_rows_scanned": "<number>",
"data_collections": "<tables>",
"status": "<status>"
}
  • scanned_collection — table whose scan has finished.
  • total_rows_scanned — number of records read from that table.
  • data_collections — tables related to the operation.
  • status — table scan result, for example SUCCEEDED.

COMPLETED​

{
"connector_name": "<name>"
}

ABORTED​

{
"connector_name": "<name>",
"data_collections": "<tables>"
}

SKIPPED​

{
"connector_name": "<name>"
}

Snapshots with multiple tables​

When one signal covers several tables, all notifications in the execution share the same operation identifier. Each table produces a separate TABLE_SCAN_COMPLETED, followed by one COMPLETED for the entire snapshot. scanned_collection identifies the table whose scan finished.

Dynamic data collection list

data_collections changes while an incremental snapshot is running. The first notification contains every selected table; later notifications omit tables that have already finished processing.

Notifications tab in GIFRÖST​

The connector dashboard's Notifications tab presents records read from the SINK channel. The active channel and notification topic name are displayed in the upper-right corner.

Filters and controls
All — initial and incremental snapshot notifications.
Initial — only Initial Snapshot notifications.
Incremental — only Incremental Snapshot notifications.
Search — filters records; the refresh control enables dynamic updates.

Each row contains a timestamp, operation identifier, snapshot type, and state. Consecutive rows with the same id show subsequent stages of one operation.

Grouped and Ungrouped views

The Grouped view is available in the current version. The Ungrouped view is currently empty.

Relate Notifications to Signals​

Expected lifecycle
execute-snapshot — expect STARTED → IN_PROGRESS → TABLE_SCAN_COMPLETED → COMPLETED.
stop-snapshot — a successfully stopped incremental snapshot produces ABORTED.
Table topic verification — incremental snapshot records have op = r and source.snapshot = incremental.

Technical references​