1

I have Java POJO as this

class LibraryEvent {
   String name ;
   int id ;
   Book book;
 
}

class Book{
  String name;
  String author ;
}

How to covert it to avro schema and then to avro record programmatically ? Is there a way to do this by having avro schema and class autogenerated in out folder using annotations? I'm trying to avoid generating schema as string and then filling avro record explicitly.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
avocado
  • 39
  • 1
  • 3

1 Answers1

0

avoid generating schema as string

You could use SchemaBuilder to avoid that problem and give you a type-safe way to programmatically define a schema.

If there is not a specific reason to start with Java (which can be done with @AvroEncode annotation or ReflectData), it is far easier to start with an IDL file

A direct translation of those POJO classes would look like this

protocol EventProtocol {

  record Book {
    union {null, string} name;
    union {null, string} author;
  }  
 
  record LibraryEvent {
    union {null, string} name;
    int id;
    union {null, Book} book;
  }
  
}

Then the Maven plugin mentioned in the docs would create those two classes with the AVSC schema embedded within each class as a static field.

You still need to "fill record explicitly" since this just creates the classes, doesn't call any setter methods

If you are still against being schema-first, and if you are using Kafka with the Confluent Avro Serializers, they have a section on using reflection

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Two queries : - 1. If I go ahead with IDL method , how can I convert idl->avsc->autogenerated pojos at application startup to improve performance . I want to avoid conversion of idl->POJO for each kafka message. All the avro idl are kept in a single package .2. How can I convert kafka message to generic record without using genericRecord.put(field , value) for each fields and value in message? – avocado Aug 20 '21 at 10:03
  • 1) Did you read the link provided and see the maven plugin definition? When you compile the project, you'll get POJO classes, and like I said, AVSC are included within those, should you need them (which you really don't for producing to Kafka). IDL conversion is only done once at project compile time, not at runtime per event. They can be moved to different packages with the namespace annotation. 2) The maven plugin creates specific record subclasses, not GenericRecord, so you would use POJO setter methods – OneCricketeer Aug 20 '21 at 12:15