2

I am a newbie to Kafka and PyKafka.I know that a producer and a consumer are made in PyKafka via the below code.

from pykafka import KafkaClient
client = KafkaClient("localhost:9092")

topic = client.topics["topicname"]
producer = topic.get_producer()
consumer = topic.get_simple_consumer()

I want to know what is KafkaClient, and how it is helping in creating producer and consumer.

I have read we can create cluster and broker also using client.cluster and client.broker, but I can't understand the use of client here.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Sayantan Kundu
  • 51
  • 4

2 Answers2

0

To make terms simpler, replace Kafka with "server".

You interact with servers with clients.

To interact with Kafka, in particular, you send messages to topics via producers, and get messages with consumers.

I don't know this library off-hand, but .broker and .cluster aren't actually "making a Kafka broker / cluster", only establishing a connection to an existing one, from which you can issue later commands.

You need the client. on those function calls because the client is a wrapper around both

To know why it is structured in this way, you'd have to ask the developers themselves

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
0

pykafka.KafkaClient is the root object of the PyKafka API, providing an interface to Kafka brokers as well as the ability to instantiate consumer and producer instances. The KafkaClient can be thought of as a representation of the totality of one Python process' interaction with a given Kafka cluster. There is no direct comparison between KafkaClient and any of the concepts mentioned in the official Kafka documentation.

It's totally possible in theory to design a python Kafka client library that doesn't have a "client" class like KafkaClient. We decided not to since in our opinion a single root class provides a cleaner, more learnable interface than a bag of various classes.

Emmett Butler
  • 5,969
  • 2
  • 29
  • 47