0

I am trying to find a suitable Python library to do stream processing with streams Kafka topics, Kafka streams. Specifically, I am looking for libraries that support the following operations.

  • KStream-to-Kstream
  • KTable-to-Ktable
  • KTable-to-KTable Foreign-Key Join
  • KStream-to-KTable

as listed here https://kafka.apache.org/24/documentation/streams/developer-guide/dsl-api.html#joining. Can anyone point me in the right direction?

I have looked into the confluent_kafka library and kafka_python library but apart from supporting simple consumption of Kafka topics, they don't look like they have out-of-the-box support for more complex operations.

Eagle1992
  • 3
  • 1

1 Answers1

0

You could check out Faust or bytewax. I think either of those would allow you to do complex operations in Python like windowing, aggregations and joins in some sense.

Unfortunately, I don’t think Faust isn’t being actively maintained by Robinhood anymore.

zander_
  • 31
  • 2