'KafkaDStream' object has no attribute '_get_object_id'

Asked Oct 10 '18 at 10:01

Active Oct 10 '18 at 11:16

Viewed 321 times

I'm try to connect Kafak Topic which use Avro to Spark Stream. I try bellow code:

schema_registry_client = CachedSchemaRegistryClient(url='schema_registry_url')
serializer = MessageSerializer(schema_registry_client)

def decoder(s):
    decoded_message = serializer .decode_message(s)
    return decoded_message


kvs = KafkaUtils.createDirectStream(ssc, topic, {'bootstrap.servers': brokers}, valueDecoder=decoder)

However i got bellow error when i ran this:

'Kafka DStream' object has no attribute '_get_object_id'

How can we overcome this?

edited Oct 10 '18 at 11:16

asked Oct 10 '18 at 10:01

GihanDB

Could try a different way to read Avro https://github.com/AbsaOSS/ABRiS – OneCricketeer Oct 10 '18 at 12:57
@cricket_007, Thank for your suggestion, But ABRiS only support for Scala isn't it? – GihanDB Oct 11 '18 at 05:52
It's all Java code... You're already importing `CachedSchemaRegistryClient`, which is also a Java class. – OneCricketeer Oct 11 '18 at 05:53
Sorry my bad, Sins examples are written in Scala, I was confused. Sorry again – GihanDB Oct 11 '18 at 05:58
Actually, nevermind, I didn't realize you're importing confluent-kafka python code... See https://stackoverflow.com/questions/36023860/how-to-use-a-scala-class-inside-pyspark – OneCricketeer Oct 11 '18 at 06:00

'KafkaDStream' object has no attribute '_get_object_id'

0 Answers0