ClassCastException on the same class deserializing from kafka stream and Avro

Question

I have generated my Avro Java classes from Avro schema with avro-maven-plugin. I serialize my avro class to a byte array and I write it into a kafka topic.

Then I have a kafka stream that tries to manipulate avro data to do something. During the deserialization process I get a ClassCastExcetion from the same class. I read that this problem is generated due to a different ClassLoader that Avro uses on fallback (a new instance of ClassLoader).

There is a method to force Avro to use the caller's ClassLoader or something similar?

KafkaStream properties

this.props = new Properties();
        this.props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
        this.props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        this.props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
        this.props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.ByteArray().getClass());

I'm using a String key and the serialized avro's byte array, then I need to manually deserialize the avro's payload. I use avro's decoder to deserialize like this:

AvroPayload stp = AvroPayload.fromByteBuffer(ByteBuffer.wrap(bytes));

or even like this:

AvroPayload stp = AvroPayload.getDecoder().decode(ByteBuffer.wrap(bytes));

With the first version, debugging I can see that if I remain in the avro's generated class context the byte array is correctly deserialized into the AvroPayload class. Returning that new instance perhaps throws a ClassCastException

can you post the properties that you are using while creating the Kafka Stream? — BogdanSucaciu, Jul 04 '19 at 15:17
can you post how you are actually serializing and deserializing messages? — BogdanSucaciu, Jul 04 '19 at 17:13
@firegloves are you using spring-devtools? Funny enough it was the issue in my case, dev-tools was loading a different class loader so the cast couldn't happen. I fixed mine by removing dev-tools (tried the accepted answer in the link and it didn't work).: https://stackoverflow.com/questions/46848557/same-class-not-assignable-classloader-for-same-class-different — Bruno Gasparotto, Jan 14 '20 at 09:12
unfortunately i'm using quarkus, but thank you for the reply — firegloves, Jan 14 '20 at 10:16

score 2 · Accepted Answer · answered Jul 05 '19 at 16:32

The only solution I found is to put as suggested the avro classes into an external jar and then import it.

It's not a good solution because it requires a lot of configuration to keep coupled your avro schemas and your generated classes, but it's the only I found.

I configured a maven project that generates the avro's classes jar into a directory, without artifact version in its name, so I can always import the latest version without having to change the pom.

If someone will discover another solution, please post it

ClassCastException on the same class deserializing from kafka stream and Avro

1 Answers1