0

we are trying to upgrade debezium-connector-sqlserver-1.0.0. to debezium-connector-sqlserver-1.9.5..After upgrade ,the debezium connector status api always shows running status ,even if connector has exception.the server has adopt open jdk 8.

[2022-08-26 11:20:10,668] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host testserver, port 1433 has failed. Error: "testserver. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:237)
        at com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:288)
        at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2466)
        at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:676)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2957)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2628)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2471)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1470)
        at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:915)
        at io.debezium.jdbc.JdbcConnection.lambda$patternBasedFactory$1(JdbcConnection.java:243)
        at io.debezium.jdbc.JdbcConnection$ConnectionFactoryDecorator.connect(JdbcConnection.java:128)
        at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:882)
        at io.debezium.connector.sqlserver.SqlServerConnection.connection(SqlServerConnection.java:221)
        at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:877)
        at io.debezium.jdbc.JdbcConnection.queryAndMap(JdbcConnection.java:630)
        at io.debezium.jdbc.JdbcConnection.queryAndMap(JdbcConnection.java:504)
        at io.debezium.connector.sqlserver.SqlServerConnection.getMaxTransactionLsn(SqlServerConnection.java:274)
        at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.getToLsn(SqlServerStreamingChangeEventSource.java:439)
        at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.executeIteration(SqlServerStreamingChangeEventSource.java:153)
        at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.executeIteration(SqlServerStreamingChangeEventSource.java:59)
        at io.debezium.connector.sqlserver.SqlServerChangeEventSourceCoordinator.executeChangeEventSources(SqlServerChangeEventSourceCoordinator.java:98)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
[2022-08-26 11:20:10,716] WARN Going to restart connector after 10 sec. after a retriable exception (io.debezium.connector.common.BaseSourceTask)

But the connector status api is showing running status. {"name":"sql-connector-4","connector":{"state":"RUNNING","worker_id":"192.168.115.98:8083"},"tasks":[{"id":0,"state":"RUNNING","worker_id":"192.168.115.98:8083"}],"type":"source"}

my connector configuration:

{
"name": "sql-connector-4",
"config": {
"name": "sql-connector-4",
"connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",
"database.hostname": "testserver",
"database.port": "1433",
"database.user": "sa",
"database.password": "test@123",
"database.server.id": "219",
"database.server.name": "testserver",
"database.dbname": "kafkaTesting",
"table.whitelist": "dbo.testtable",
"database.history.kafka.bootstrap.servers": "serverip:9092",
"database.history.kafka.topic": "dbhistory.demo",
"include.schema.changes": "true",
"errors.retry.timeout":1,
"errors.retry.delay.max.ms":1000
} }
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Shala
  • 31
  • 4
  • as mention in documentation, i had configured the retry timeout in the connector config..but still it is restring after every 10 seconds,whereas i had set the retry delay as only 1 seconds."errors.retry.timeout":1, "errors.retry.delay.max.ms":1000 – Shala Aug 26 '22 at 06:08
  • is there any configuration available to limit the no of retry? – Shala Aug 26 '22 at 06:09
  • No, I don't think you can limit the retries. The error probably is caught and retired because it assumes it's only a temporary network error (databases under high load may not accept some requests). But your error doesn't seem JDK/Debezium version related. Make sure your database values are correct. If you think it's a bug, open a ticket in Debezium JIRA – OneCricketeer Aug 26 '22 at 14:21
  • according to kafka connect documentation,we can limit the no of retries.https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html – Shala Aug 26 '22 at 16:00
  • i have configured the properties errors.retry.timeout,errors.retry.delay.max.ms,errors.tolerance .but still it keeps on retrying .https://cwiki.apache.org/confluence/display/KAFKA/KIP-298%3A+Error+Handling+in+Connect – Shala Aug 26 '22 at 16:03
  • You're using Debezium, which is a source, not a sink connector, therefore those documents dont apply – OneCricketeer Aug 26 '22 at 17:19
  • the same properties are applicable for source configuration as well.. https://docs.confluent.io/platform/current/installation/configuration/connect/source-connect-configs.html – Shala Aug 29 '22 at 04:46
  • i guess the problem is with the kafka connect version.is it possible to upgrade the kafka connect alone? currently we are using kafka_2.13-2.7.0 – Shala Aug 29 '22 at 10:40
  • Sure, you should run connect cluster separately from the broker anyway – OneCricketeer Aug 30 '22 at 02:45

0 Answers0