0

Am new to debezium, am trying to connect my kafka connector to my already existing MySQL database on my WAMP server locally.

I started zookeeper and kafka according to the tutorial documentation and then I started kafka connector. In my post man I sent the following JSON to my kafka connector but I keep getting a bad request response.

Start Zookepper

docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:0.7

Start Kafka

docker run -it --rm --name kafka -p 9092:9092 --link zookeeper:zookeeper debezium/kafka:0.7

Start Kafka Connector

docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e OFFSET_STORAGE_TOPIC=my_connect_offsets --link zookeeper:zookeeper --link kafka:kafka debezium/connect:0.7

Using my Post Man I sent the following

{ "name": "shopcentra-connector", "config": { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "tasks.max": "1", "database.hostname": "127.0.0.1", "database.port": "3306", "database.user": "root", "database.password": "root", "database.server.id": "5444", "database.server.name": "shopcentra", "database.whitelist": "shopcentra", "database.history.kafka.bootstrap.servers": "kafka:9092", "database.history.kafka.topic": "dbhistory.shopcentra", "include.schema.changes": "true" } }

My Bad Response

{
"error_code": 400,
"message": "Connector configuration is invalid and contains the following 1 error(s):\nUnable to connect: Communications link failure\n\nThe last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"
}
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
DaviesTobi alex
  • 610
  • 1
  • 9
  • 34
  • `database.hostname": "127.0.0.1"`... This implies your database is running within the connect container, which is not the case. You need to use your external IP address for the host – OneCricketeer Jun 21 '18 at 14:22
  • @cricket_007 but I have wamp running on the localhost, does that mean I can not connect to my wamp localhost? – DaviesTobi alex Jun 21 '18 at 14:39
  • Your container has its own network stack. Localhost is itself, not your wamp environment. For more information https://stackoverflow.com/questions/40746453/how-to-connect-to-docker-host-from-container-on-windows-10-docker-for-windows and https://stackoverflow.com/a/43541681/2308683 – OneCricketeer Jun 21 '18 at 14:42
  • The alternative is to migrate your WAMP database into a Mysql container (or run all of Apache, PHP, and Mysql as containers) – OneCricketeer Jun 21 '18 at 14:43
  • @cricket_007 okay how do I migrate my wamp database into a mysql container? and Would changes I make in that container reflect after the container is closed? – DaviesTobi alex Jun 21 '18 at 14:48
  • See the documentation section: Where to store data - https://hub.docker.com/_/mysql/ ... You can find the existing database data, then, in theory, you would use a volume mount to that directory. Or you can use `mysqldump` to export/import – OneCricketeer Jun 21 '18 at 15:48
  • is there a way to connect to the mysql server running on the machine outside the docker network of the kafka container from the kafka container ? @OneCricketeer – Sithija Piyuman Thewa Hettige Sep 07 '22 at 13:41
  • @SithijaPiyumanThewaHettige Please create your own post – OneCricketeer Sep 07 '22 at 14:36

1 Answers1

0

If you’re using Docker, you can use a pre-configured MySQL image debezium/example-mysql. Alternatively make sure that you configure the instance with a user granted the appropriate permissions:

GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT  ON *.* TO 'debezium' IDENTIFIED BY 'dbz';

and then per database too:

`GRANT ALL PRIVILEGES ON demo.* TO 'debezium'@'%';`
Anderson Couto
  • 9
  • 3