1

i have found a lot of responses to this question, but none works for me.. the error from the kafka-connects log is this:

Caused by: java.sql.SQLException: No suitable driver found for jdbc:postgresql://postgres-dest:5432/shipment_db?user=postgres&password=postgres

This is the compose where kafka connect is runned:

version: '3.3'
services:
    postgres:
        container_name: postgres
        ports:
            - '5432:5432'
        environment:
            - POSTGRES_USER=postgres
            - POSTGRES_PASSWORD=postgres
            - POSTGRES_DB=shipment_db
            - PGPASSWORD=password
        image: 'debezium/postgres:13'

    postgres-dest:
        container_name: postgres-dest
        ports:
            - '5433:5432'
        environment:
            - POSTGRES_USER=postgres
            - POSTGRES_PASSWORD=postgres
            - POSTGRES_DB=shipment_db
            - PGPASSWORD=password
        image: 'debezium/postgres:13'

    zookeeper:
        container_name: zookeeper
        ports:
            - '2181:2181'
            - '2888:2888'
            - '3888:3888'
        image: 'debezium/zookeeper:1.7'

    kafka:
        container_name: kafka
        ports:
            - '9092:9092'
        links:
            - 'zookeeper:zookeeper'
        image: 'debezium/kafka:1.7'
        environment:
            ZOOKEEPER_CONNECT: zookeeper:2181

    connect:
        image: debezium/connect:1.7
        hostname: connect
        container_name: connect
        ports:
            - 8083:8083
        environment:
            BOOTSTRAP_SERVERS: kafka:9092
            GROUP_ID: 1
            CONFIG_STORAGE_TOPIC: my_connect_configs
            OFFSET_STORAGE_TOPIC: my_connect_offsets
            STATUS_STORAGE_TOPIC: my_connect_statuses
            CONNECT_BOOTSTRAP_SERVERS: kafka:9092
            CONNECT_GROUP_ID: connect-cluster-A
            CONNECT_PLUGIN_PATH: /kafka/connect,/kafka/data
            #EXTERNAL_LIBS_DIR: /kafka/external_libs,/kafka/data
            #CLASSPATH: /kafka/connect
            #KAFKA_CONNECT_PLUGINS_DIR: /kafka/connect, /kafka/data
        volumes:
            - type: bind
              source: ./plugins
              target: /kafka/data
        depends_on:
            - zookeeper
            - kafka
            - postgres
        links:
            - zookeeper
            - kafka
            - postgres
            - postgres-dest

the CONNECT_PLUGIN_PATH (plugin.dir) is setted to /kafka/data and into are inserted the plugin for jdbc with the postgresql driver in the same folder.

When i load the kafka-connect's container i found in the logs:

2021-12-23 15:44:43,524 INFO || Loading plugin from: /kafka/data/postgresql-42.2.19.jar

then i suppose that driver is loaded correctly. The connector is downloaded from here jdbc connector

The configuration of the connector is the follow:

curl -H 'Content-Type: application/json' --data '
{
  "name": "jdbc-sink",
  "config": {
    "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
    "tasks.max": "1",
    "topics": "accounts",
    "connection.url": "jdbc:postgresql://postgres-dest:5432/shipment_db?user=postgres&password=postgres",
    "auto.create": "true",
    "insert.mode": "upsert",
    "pk.fields": "id",
    "pk.mode": "record_value" 
  }
}
}' http://localhost:8083/connectors

I have tryed many solution:

  • setup the CLASSPATH with /kafka/data ( the driver + plugins dir )
  • setup the environment variable KAFKA_CONNECT_PLUGINS_DIR to "/kafka/connect, /kafka/data"

but none of this solutions works.. I think that the problem are in the connection between kafka-connect and postgres-dest container but i don't undestand where the problem is, given that the connection-url of connector seems writed correctly.

Thanks in advance for any response.

Al3
  • 183
  • 1
  • 3
  • 11

0 Answers0