Kafka broker pod cannot connect to zookeeper

Question

I am running a zookeeper locally using the cli in WSL:

zookeeper-server-start.sh config/zookeeper.properties

and I have created a Kafka broker pod using Kubernetes StatefullSet resource type using a .yaml file which is as follows:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kafka
spec:
  serviceName: kafka
  replicas: 1
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
      - name: kafka
        image: debezium/kafka
        ports:
        - containerPort: 9092
          name: kafka
        env:
        - name: KAFKA_BROKER_ID
          value: '0'
        - name: ZOOKEEPER_CONNECT
          value: 'host.docker.internal:2181'

The pod gets stuck in crashloopbackoff stage and the logs are as follows:

2023-02-21 07:14:29,741 - INFO  [main-SendThread(host.docker.internal:2181):ClientCnxn$SendThread@1181] - Opening socket  connection to server host.docker.internal/192.168.65.2:2181.
2023-02-21 07:14:29,747 - WARN  [main-SendThread(host.docker.internal:2181):ClientCnxn$SendThread@1300] - Session 0x0 fo r sever host.docker.internal/192.168.65.2:2181, Closing socket connection. Attempting reconnect except it is a SessionEx piredException.
java.net.ConnectException: Connection refused
        at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
2023-02-21 07:14:29,949 - INFO  [main:Logging@66] - [ZooKeeperClient Kafka server] Closing.

I have tried changing .yaml configuration file but no success.

Please edit your question title - all-caps is generally regarded as equivalent to shouting, and therefore rude. — Jon Skeet, Feb 21 '23 at 07:23

score 0 · Accepted Answer · answered Feb 21 '23 at 14:30

0

Only Docker Desktop configures host.docker.internal network address for containers it manages. Kubernetes pods likely will not have access to it

answered Feb 21 '23 at 14:30

OneCricketeer

179,855
19
132
245

Yes but docker initializes the "host.docker.internal" ip-address in '/etc/hosts' file....I am not sure why K8s pods doesn't have its access. Also just to make everything clear: -> I am using docker desktop to get docker and K8s support – Prakhar Feb 22 '23 at 07:32
You can `kubectl exec` into any pod and look at `/etc/hosts` file. But test with something other than Zookeeper/Kafka, which have more particular network requirements. https://stackoverflow.com/a/66782901/2308683 Or, as mentioned above, use Strimzi to run both Zookeeper (which is no longer needed for Kafka, anyway) and the brokers in k8s rather than trying to do a split-installation of Zookeeper and Kafka – OneCricketeer Feb 22 '23 at 20:46

Kafka broker pod cannot connect to zookeeper

1 Answers1