Supported databases
Gifröst uses Kafka Connect connectors to read change events from source systems and move them into the platform pipeline. The supported database version range depends on the selected Debezium/Kafka Connect plug-in version. The platform can be configured with the connector version required by the target database version and deployment environment.
The range is based on the compatibility of Debezium and Kafka Connect connectors used by the platform and on the required database drivers.
Source systems
Source databases
Source databases are systems from which Gifröst captures data changes through Kafka Connect source connectors. The list below shows database versions supported by the appropriate Debezium connector versions.
IBM Db2
Debezium Db2 connector
11.5
Informix
Debezium Informix connector
12, 14MongoDB
Debezium MongoDB connector
3.2 - 8.0
MySQL
Debezium MySQL connector
5.7 - 9.1
Oracle
Debezium Oracle connector
11g - 23ai
PostgreSQL
Debezium PostgreSQL connector
9.6 - 17
SQL Server
Debezium SQL Server connector
2017, 2019, 2022| Database | Source connector | Supported database versions | Notes |
|---|---|---|---|
| IBM Db2 | io.debezium.connector.db2.Db2Connector | 11.5 | Uses the Debezium Db2 source connector. |
| Informix | io.debezium.connector.informix.InformixConnector | 12, 14 | Uses Informix JDBC driver 4.50.11 in the Debezium 3.1 tested set. |
| MongoDB | io.debezium.connector.mongodb.MongoDbConnector | 3.2, 3.4, 3.6, 4.0, 4.2, 4.4, 5.0, 6.0, 7.0, 8.0 | Captures changes from MongoDB replica sets or sharded deployments configured for Debezium. |
| MySQL | io.debezium.connector.mysql.MySqlConnector | 5.7, 8.0.x, 8.4.x, 9.0, 9.1 | Uses MySQL binary log based CDC. |
| Oracle | io.debezium.connector.oracle.OracleConnector | 11g, 12c, 19c, 21c, 23ai | Oracle CDC is supported from Oracle 11g; 23ai requires LogMiner mode. |
| PostgreSQL | io.debezium.connector.postgresql.PostgresConnector | 9.6, 10, 11, 12, 13, 14, 15, 16, 17 | Uses logical decoding with pgoutput or decoderbufs. |
| SQL Server | io.debezium.connector.sqlserver.SqlServerConnector | 2017, 2019, 2022 | Uses SQL Server CDC capabilities. |
The connector list is flexible. Additional standard, non-standard, or custom connectors can be added on request when they are available as Kafka Connect plug-ins and can be deployed to the Kafka Connect worker.
Target databases
Target systems
Target databases
Target databases are systems to which Gifröst writes data from Kafka topics by using sink connectors. The main connector for relational databases is JDBC Sink, which uses the database dialect and the appropriate JDBC driver.
| Target database | Sink connector | Connector version | Supported database versions | Notes |
|---|---|---|---|---|
| PostgreSQL | io.confluent.connect.jdbc.JdbcSinkConnector / io.debezium.connector.jdbc.JdbcSinkConnector | Confluent 10.8.4, Debezium 3.4.0.Final | 9.6, 10, 11, 12, 13, 14, 15, 16, 17, 18 | Requires a compatible PostgreSQL JDBC driver. Confluent JDBC Sink includes the PostgreSQL JDBC driver and supports PostgreSqlDatabaseDialect. |
| Oracle | io.confluent.connect.jdbc.JdbcSinkConnector / io.debezium.connector.jdbc.JdbcSinkConnector | Confluent 10.8.4, Debezium 3.4.0.Final | 11g, 12c, 19c, 21c, 23ai | JDBC Sink requires the correct Oracle JDBC driver, for example ojdbc8 or ojdbc10. |
| MySQL | io.confluent.connect.jdbc.JdbcSinkConnector / io.debezium.connector.jdbc.JdbcSinkConnector | Confluent 10.8.4, Debezium 3.4.0.Final | 5.7, 8.0.x, 8.4.x, 9.0, 9.1 | Requires a compatible MySQL Connector/J driver. Confluent JDBC Sink supports MySqlDatabaseDialect. |
| Microsoft SQL Server | io.confluent.connect.jdbc.JdbcSinkConnector / io.debezium.connector.jdbc.JdbcSinkConnector | Confluent 10.8.4, Debezium 3.4.0.Final | 2017, 2019, 2022 | Confluent JDBC Sink includes SQL Server drivers and supports SqlServerDatabaseDialect. |
| IBM Db2 | io.confluent.connect.jdbc.JdbcSinkConnector / io.debezium.connector.jdbc.JdbcSinkConnector | Confluent 10.8.4, Debezium 3.4.0.Final | 11.5 | Requires the Db2 JDBC driver, for example db2jcc4.jar. |
| SAP HANA | io.confluent.connect.jdbc.JdbcSinkConnector | Confluent 10.8.4 | According to the SAP HANA JDBC driver in use | Confluent JDBC Sink supports SapHanaDatabaseDialect, but Confluent does not publish a closed SAP HANA engine-version list. |
| SQLite | io.confluent.connect.jdbc.JdbcSinkConnector | Confluent 10.8.4 | According to the bundled SQLite JDBC 4.0 driver | Recommended mainly for testing and development. |
| Apache Derby | io.confluent.connect.jdbc.JdbcSinkConnector | Confluent 10.8.4 | According to the Derby JDBC driver in use | Supported through DerbyDatabaseDialect; Confluent does not publish a closed Derby version list. |
| Sybase | io.confluent.connect.jdbc.JdbcSinkConnector | Confluent 10.8.4 | According to the Sybase JDBC driver in use | Supported through SybaseDatabaseDialect; Confluent does not publish a closed Sybase version list. |
| Vertica | io.confluent.connect.jdbc.JdbcSinkConnector | Confluent 10.8.4 | According to the Vertica JDBC driver in use | Supported through VerticaDatabaseDialect; Confluent does not publish a closed Vertica version list. |
| MongoDB | io.debezium.connector.mongodb.MongoDbSinkConnector | Debezium 3.1.3.Final | 6.0, 7.0, 8.0 | Debezium MongoDB Sink writes CDC events from Kafka topics to a target MongoDB database. |
JDBC connectors can also support other JDBC-compatible databases when the correct JDBC 4.0 driver is available and installed on every Kafka Connect worker. For some JDBC dialects, the connector vendor does not publish a closed database-engine version list; compatibility then comes from the JDBC driver in use and deployment testing. The list can be extended on request with standard, non-standard, or custom Kafka Connect connectors.
The HDFS and AWS S3 connectors visible in Connector Manager are supported target systems for files and objects, but they are not relational databases. They can be documented separately as target storage systems.