14

I am using kafka-python for accessing Kafka. I try to create a Kafka Producer:

kafka_producer = KafkaProducer(bootstrap_servers=['kafka:9092'])

but this fails with exception kafka.errors.NoBrokersAvailable: NoBrokersAvailable.

I've found out I need to add api_version parameter to the KafkaProducer:

kafka_producer = KafkaProducer(bootstrap_servers=['kafka:9092'],
                               api_version=(0, 10, 1))

This command works.

The question is: how to determine value of api_version?

kafka-broker-api-versions.sh --bootstrap-server localhost:9092 gives me something, but I am not sure if there's a number I can use. I tried random values like api_version=(20, 2, 1) and it also worked.

Michal Špondr
  • 1,337
  • 2
  • 21
  • 44
  • 2
    Ideally, it's the version of Kafka you've installed, or the lowest version you're targeting. `0.10.2` is a safe default – OneCricketeer Jul 18 '19 at 00:54
  • I am more interested in *why* is it a safe default. Source? – Michal Špondr Jul 18 '19 at 07:35
  • 1
    That's that point at which *any version* of newer clients should work - https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix – OneCricketeer Jul 18 '19 at 14:57
  • @cricket_007 Thanks, that's useful. I assume I can set any version to `api_version` parameter in KafkaProducer constructor, if it does not exist the class selects something on its own. – Michal Špondr Jul 22 '19 at 10:03

2 Answers2

3

This was solved in the discussion in the comments already:

Ideally, it's the version of Kafka you've installed, or the lowest version you're targeting. 0.10.2 is a safe default

That's that point at which any version of newer clients should work - cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix

Dennis Jaheruddin
  • 21,208
  • 8
  • 66
  • 122
-2

Let kafka-python guess your Kafka API version:

from kafka import KafkaProducer  # pip3 install kafka-python

k = KafkaProducer(bootstrap_servers="my-kafka.host.internal")
print(k.config['api_version'])
mr.wolle
  • 1,148
  • 2
  • 13
  • 21