0

I am having issues consuming from a given offset using python, in Java I had implemented a KafkaConsumer and used

consumer.seek(new TopicPartition(topic, 0), 3)

to consume from my third offset in partition 0.

But when using Python I am using an AvroConsumer, which has a seek method but only takes the partition as a parameter and not an offset to consume from. it seems to consume then from the beginning of the topic. Is there a way with AvroConsumer to consume from a certain offset? Thanks

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
AnonymousAlias
  • 1,149
  • 2
  • 27
  • 68

1 Answers1

1

It appears that you provide the offset as a parameter when creating the topic partition.

I don't have an example of your code to model this off of so I'll just write it like this.

confluent_kafka.TopicPartition(topic,
                               partition,
                               offset)

consumer.seek(tp)

This is based on the integration test in the library for this code. I have highlighted the relevant test and linked below

Integration Test For Confluent Kafka Python

Zergleb
  • 2,212
  • 15
  • 24
  • Correct thank you! Any idea where I can look at the AvroConsumer class I couldn't find it when I was looking – AnonymousAlias Nov 30 '18 at 09:49
  • Feel free to check out my other question https://stackoverflow.com/questions/53558646/avroconsumer-consume-from-timestamp-using-offsets-for-times :) – AnonymousAlias Nov 30 '18 at 13:46