I have setup the Kafka cluster on my single node vanilla Kubernetes cluster using this link: Below are my mainfest yaml files:
00-namespace.yaml
apiVersion: v1 kind: Namespace metadata: name: "kafka" labels: name: "kafka"
01-zookeeper.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: zookeeper-service
name: zookeeper-service
namespace: kafka
spec:
type: NodePort
ports:
- name: zookeeper-port
port: 2181
nodePort: 30181
targetPort: 2181
selector:
app: zookeeper
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: zookeeper
name: zookeeper
namespace: kafka
spec:
replicas: 1
selector:
matchLabels:
app: zookeeper
template:
metadata:
labels:
app: zookeeper
spec:
containers:
- image: wurstmeister/zookeeper
imagePullPolicy: IfNotPresent
name: zookeeper
ports:
- containerPort: 2181
02-kafka.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: kafka-broker
name: kafka-service
namespace: kafka
spec:
ports:
- port: 9092
selector:
app: kafka-broker
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: kafka-broker
name: kafka-broker
namespace: kafka
spec:
replicas: 1
selector:
matchLabels:
app: kafka-broker
template:
metadata:
labels:
app: kafka-broker
spec:
hostname: kafka-broker
containers:
- env:
- name: KAFKA_BROKER_ID
value: "1"
- name: KAFKA_ZOOKEEPER_CONNECT
value: Node_port_IP_of_zookeeper_service:2181
- name: KAFKA_LISTENERS
value: PLAINTEXT://0.0.0.0:9092
- name: KAFKA_ADVERTISED_LISTENERS
value: PLAINTEXT://Node_port_IP_of_kafka_broker_service:9092
image: wurstmeister/kafka
imagePullPolicy: IfNotPresent
name: kafka-broker
ports:
- containerPort: 9092
I have used below command to setup NodePort for kafka service:
kubectl expose deployment kafka-broker --type=NodePort --port=9092 -n kafka
Problem statement: I am able to connect to kafka cluster if I am inside the server in which I have hosted my kubernetes cluster using below command:
./kafka-topics.sh --describe --topic test --bootstrap-server My_server_IP:NodePort
Below is the output:
But, if I am executing this command from anywhere outside the cluster using the same binary build of kafka, I am not able to connect. Below is the output:
telnet server_public_ip 9092
is working fine even from outside the cluster.
Please help me in accessing this cluster from the outside world. Thank you in advance.