How can I get kafka schema registry in Pyspark?

Question

I am looking at the relevant library for PySpark to get the schema registry from Kafka and decode the data. Does anyone know what is the code/library convert from scala to pyspark in scala-code?

Answer 1

You can use requests package to send requests to schema-registry restAPI and get the schema of your topic and also if you are listening to some specific topics you can cache the schema of them on spark and use them

Answer 2

Pyspark can import and use any JVM Spark class. Any Scala other Java examples you find, therefore should just work

Running custom Java class in PySpark