0

I have setup the Kafka cluster on my single node vanilla Kubernetes cluster using this link: Below are my mainfest yaml files:

00-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: "kafka"
  labels:
    name: "kafka"

01-zookeeper.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: zookeeper-service
  name: zookeeper-service
  namespace: kafka
spec:
  type: NodePort
  ports:
    - name: zookeeper-port
      port: 2181
      nodePort: 30181
      targetPort: 2181
  selector:
    app: zookeeper
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: zookeeper
  name: zookeeper
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zookeeper
  template:
    metadata:
      labels:
        app: zookeeper
    spec:
      containers:
        - image: wurstmeister/zookeeper
          imagePullPolicy: IfNotPresent
          name: zookeeper
          ports:
            - containerPort: 2181

02-kafka.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: kafka-broker
  name: kafka-service
  namespace: kafka
spec:
  ports:
  - port: 9092
  selector:
    app: kafka-broker
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: kafka-broker
  name: kafka-broker
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-broker
  template:
    metadata:
      labels:
        app: kafka-broker
    spec:
      hostname: kafka-broker
      containers:
      - env:
        - name: KAFKA_BROKER_ID
          value: "1"
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: Node_port_IP_of_zookeeper_service:2181
        - name: KAFKA_LISTENERS
          value: PLAINTEXT://0.0.0.0:9092
        - name: KAFKA_ADVERTISED_LISTENERS
          value: PLAINTEXT://Node_port_IP_of_kafka_broker_service:9092
        image: wurstmeister/kafka
        imagePullPolicy: IfNotPresent
        name: kafka-broker
        ports:
        - containerPort: 9092

I have used below command to setup NodePort for kafka service:

kubectl expose deployment kafka-broker --type=NodePort --port=9092 -n kafka

Problem statement: I am able to connect to kafka cluster if I am inside the server in which I have hosted my kubernetes cluster using below command:

./kafka-topics.sh --describe --topic test --bootstrap-server My_server_IP:NodePort

Below is the output: Output But, if I am executing this command from anywhere outside the cluster using the same binary build of kafka, I am not able to connect. Below is the output: Output

telnet server_public_ip 9092 is working fine even from outside the cluster.

Please help me in accessing this cluster from the outside world. Thank you in advance.

Akshay Seth
  • 57
  • 1
  • 1
  • 8
  • Do not use random Medium blogs to deploy Kafka (that link has several configuration flaws)... Use an actually tested and supported product like https://strimzi.io ; for understanding what you're doing wrong, see this blog on the basics https://strimzi.io/blog/2019/04/17/accessing-kafka-part-1/ – OneCricketeer Apr 03 '23 at 18:06

0 Answers0