1

I was going through AWS 's service MSK which is Managed Service for Kafka. I wanted to check how many messages have not been consumed by the consumer after I send them from a producer.

Is the only way to do it via enabling the Monitoring options AWS provides?

Because I tried the general Kafka commands and other Stackoverflow answers related to Kafka, they have not worked for MSK. And why can't we do this? When I hit the --describe option with my zookeeper string in my instnace, there is no LAG column there either.

sameerawsnoob
  • 105
  • 1
  • 10

1 Answers1

1

Execute below command from local Kafka/bin directory

bin/kafka-consumer-groups.sh --bootstrap-server kafka-bootstrap-ip:9092 --describe --group my-group

  1. Update kafka-bootstrap-ip with actual kafka cluster ip address.

  2. Update my-group with actual group name

kus
  • 446
  • 3
  • 7
  • Any other way to check the messages inside the MSK? So what is the LAG actually here? – sameerawsnoob Apr 29 '22 at 06:46
  • 1
    This command give you the LAG, you may want to refer the documentaion https://kafka.apache.org/documentation/#basic_ops_consumer_lag And this doesnt matter if kafka hosted by AWS(MSK) or on premises deployment. This can be use in both the cases. – kus Apr 29 '22 at 14:42
  • 1
    you can also use AWS Cloudwatch that may give you more user friendly. This require some additional configuration https://docs.aws.amazon.com/msk/latest/developerguide/metrics-details.html , https://docs.aws.amazon.com/msk/latest/developerguide/cloudwatch-metrics.html – kus Apr 29 '22 at 14:49