0

I already kept auto.offset.reset = earliest

Here my scenario is different;

I have 1 zookeeper and 3 brokers.Now i stopped consumer and started posting messages (lets say 4 messages, later i stopped Kafka server (1 zookeeper and 3 brokers), and now i made consumer up. After some time i made up Kafka server(1 zookeeper and 3 brokers).

Since my 4 messages are not processed with consumer it should processes now,but which is not happening.

Simply Ged
  • 8,250
  • 11
  • 32
  • 40
Rafi
  • 19
  • 1
  • 5

1 Answers1

0

This is normal behaviour. If no broker is available, then you cannot produce messages and so messages are lost. In this case, your producer should receive an error when attempting to produce messages: then he must handle this case and keep not posted messages until at least one Kafka broker is available.

Zebra8844
  • 141
  • 5
  • I made all the brokers are up only consumer is down, and now I posted messages.since my brokers are up producer is able to produce the message to the Kafka brokers.but all these messages are not consumed by consumer coz it's in down. Now message which I posted should be in Kafka broker. Later I stopped Kaka brokers ,and then after some time restarted both Kafka brokers and consumer, ideally consumer has to consume message which I posted earlier. But it was not happening – Rafi Dec 06 '19 at 18:58
  • We need to know more about your configuration. Can you post your server.properties file of your brokers and also the topics configuration (using for instance kafka-topics --describe). Also, the contents of the kafka-logs (or whatever directory configured in server.properties) directory will help. In what language your consumer is written ? Java, Python, ... ? Can you provide source file ? – Zebra8844 Dec 07 '19 at 08:44
  • We have created producer and consumer in dotnet core (c# Language) as a services, and we are using dockers to run the services and kada server (1 zookeeper,3 brokers) – Rafi Dec 07 '19 at 13:59