I am using AWS MSK for Kafka. I have provisioned it via terraform. Here are my properties ::
I have 3 broker nodes.
auto.create.topics.enable=true
default.replication.factor=3
min.insync.replicas=1
num.io.threads=8
num.network.threads=5
num.partitions=3
num.replica.fetchers=2
replica.lag.time.max.ms=30000
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
unclean.leader.election.enable=true
zookeeper.session.timeout.ms=18000
I am getting this error below in consumer.
{"level":"error","msg":"Error while fetching kafka message:[16] Not Coordinator For Group: the broker returns this error code if it receives an offset fetch or commit request for a group that it is not a coordinator for","time":"2022-07-19T11:30:02Z"}
{"level":"info","msg":"No new kafka message(my_topic_name) in last 10s(currentTime: 2022-07-19 11:30:11.97251011 +0000 UTC), err:context deadline exceeded","time":"2022-07-19T11:30:11Z"}
This topic's description ::
min.insync.replicas 1
message.format.version 2.6-IV0
max.message.bytes 15728640
unclean.leader.election.enable true
PARTITION OLDEST_OFFSET NEWEST_OFFSET EMPTY LEADER REPLICAS IN_SYNC_REPLICAS
0 0 637545 false b-2.kafkaurl.amazonaws.com:9092 1,2,3 1,2,3
1 0 637953 false b-1.kafkaurl.amazonaws.com:9092 1,2,3 1,2,3
2 0 637774 false b-3.kafkaurl.amazonaws.com:9092 1,2,3 1,2,3
I found The group coordinator is not available-Kafka but it wont help me as I have 3 nodes and therefore, the default value of offsets.topic.replication.factor should work fine for me.
