0

I have following docker-compose.yml configuration for my Debezium connector:

    version: '3.1'

    services:

    db:
        image: mysql
        network_mode: host
        environment:
        MYSQL_ROOT_PASSWORD: 123456
        volumes:
        - ./mysql.cnf:/etc/mysql/conf.d/mysql.cnf
        ports:
        - 3306:3306
        container_name: mydb

    zookeeper:
        image: confluentinc/cp-zookeeper
        network_mode: host
        ports:
        - "2181:2181"
        environment:
        ZOOKEEPER_CLIENT_PORT: 2181
        container_name: myzookeeper

    kafka:
        image: confluentinc/cp-kafka
        network_mode: host
        depends_on:
        - zookeeper
        - db
        ports:
        - "9092:9092"
        environment:
        KAFKA_ZOOKEEPER_CONNECT: localhost:2181
        KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
        KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
        KAFKA_BROKER_ID: 1
        KAFKA_MIN_INSYNC_REPLICAS: 1
        container_name: mykafka

    connector:
        image: debezium/connect:0.10
        network_mode: host
        ports:
        - "8083:8083"
        environment:
        GROUP_ID: 1
        CONFIG_STORAGE_TOPIC: my_connect_configs
        OFFSET_STORAGE_TOPIC: my_connect_offsets
        BOOTSTRAP_SERVERS: localhost:9092
        HOST_NAME: localhost
        depends_on:
        - zookeeper
        - db
        - kafka
        container_name: myconnector

When I want to create a connector I receive the following error:

    curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name": "mystore-connector", "config": { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "tasks.max": "1", "database.hostname": "localhost", "database.port": "3306", "database.user": "morteza", "database.password": "morteza_password", "database.server.id": "223344", "database.server.name": "dbserver1", "database.whitelist": "mystore", "database.history.kafka.bootstrap.servers": "localhost:9092", "database.history.kafka.topic": "dbhistory.mystore" } }'
    HTTP/1.1 400 Bad Request
    Date: Fri, 08 Mar 2019 09:48:34 GMT
    Content-Type: application/json
    Content-Length: 255
    Server: Jetty(9.4.12.v20180830)

    {"error_code":400,"message":"Connector configuration is invalid and contains the following 1 error(s):\nUnable to connect: Public Key Retrieval is not allowed\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"}

And Debezium connector shows this error:

    The connection password is empty   [io.debezium.connector.mysql.MySqlConnector]
    myconnector  | 2019-03-08 09:38:26,755 INFO   ||  Failed testing connection for jdbc:mysql://localhost:3306/?useInformationSchema=true&nullCatalogMeansCurrent=false&useSSL=false&useUnicode=true&characterEncoding=UTF-8&characterSetResults=UTF-8&zeroDateTimeBehavior=CONVERT_TO_NULL with user 'morteza'   [io.debezium.connector.mysql.MySqlConnector]

I have set the value of the field database.password as you see, but I receive the error for the password of the database.

Seyed Morteza Mousavi
  • 6,855
  • 8
  • 43
  • 69
  • `Public Key Retrieval is not allowed` is the error returned from the create `POST`. Some of the suggestions in [this thread](https://stackoverflow.com/questions/50379839/connection-java-mysql-public-key-retrieval-is-not-allowed) might be worth trying. One of the suggestions is that it's just an invalid username/password combination – Robin Moffatt Mar 08 '19 at 10:22
  • Here's an example of [Debezium with MySQL](https://github.com/confluentinc/demo-scene/blob/master/ksql-workshop/scripts/create-mysql-source.sh) and [docker-compose](https://github.com/confluentinc/demo-scene/blob/master/ksql-workshop/docker-compose.yml), if it's useful to try against your config. – Robin Moffatt Mar 08 '19 at 10:24
  • Robin, I received the same error even with root and pass 123456. – Seyed Morteza Mousavi Mar 08 '19 at 10:59
  • Setting `localhost` will not work for Docker compose services for Kafka talking to Zookeeper, and Connect to Kafka. `network_mode: host` is mostly a hack. – OneCricketeer Mar 08 '19 at 16:59
  • @cricket_007 I could telnet MySQL server within Connect container (I installed telnet inside the container). It can reach to the database. Also, Connect config and offset topics is created in Kafka. Somehow debezium connector does not receive my password. They already discussed the problem in https://groups.google.com/forum/#!topic/debezium/Ml14RfEDxL8. They say they've solved the problem in version 0.9 but I used version 0.10 and it does not work. – Seyed Morteza Mousavi Mar 08 '19 at 18:52
  • My point is that you should really be using the Docker container names, and **not** `localhost` at all. It's just a bad configuration, basically, you are trying to make the Debezium Connect instance connect **to itself**, not your database. – OneCricketeer Mar 08 '19 at 20:28
  • 1
    I think the problem lies in this message `Public Key Retrieval is not allowed`. IIRC there were compatibility problems between JDBC driver and MySQL driver versions in the past when SSL was somewho used. Please see for example https://community.atlassian.com/t5/Confluence-questions/MySQL-Public-Key-Retrieval-is-not-allowed/qaq-p/778956 – Jiri Pechanec Mar 12 '19 at 13:53
  • Thank you @cricket_007. I will update my docker-compose.yml file to not use docker `network_mode: host`. – Seyed Morteza Mousavi Mar 14 '19 at 14:08
  • 1
    @JiriPechanec thank you. I changed mysql version to 5.7 be setting `image: mysql:5.7` and it works fine! I wonder why it is not suupoerted. I found it in https://debezium.io/blog/2018/12/19/debezium-0-9-0-beta2-released/ that debezium 0.9 will support mysql 8. By the way, could you post your answer, so other users can use it for similar situation? – Seyed Morteza Mousavi Mar 14 '19 at 14:11

2 Answers2

1

As Jiri Pechanec suggested, I changed image: mysql to image: mysql:5.7 in docker-compose.yml file and now it works fine.

Seyed Morteza Mousavi
  • 6,855
  • 8
  • 43
  • 69
1

This error (Public Key Retrieval is not allowed) can come about in Debezium 0.9 when the user in the source MySQL database has not been granted the required privileges. In Debezium 0.8 the error was Unable to connect: Communications link failure.

I wrote a quick blog about this here because I encountered this issue and the error messages are kinda non-obvious :)

tl;dr: ALTER USER 'debezium'@'%' IDENTIFIED WITH mysql_native_password BY 'dbz';

Robin Moffatt
  • 30,382
  • 3
  • 65
  • 92