0

As of now am creating avro message from avsc schema file. Using below code snippet

static byte[] fromJasonToAvro(String json, String schemastr) throws Exception {

        InputStream input = new ByteArrayInputStream(json.getBytes());
        DataInputStream din = new DataInputStream(input);

        Schema schema = Schema.parse(schemastr);

        Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);

        DatumReader<Object> reader = new GenericDatumReader<Object>(schema);
        Object datum = reader.read(null, decoder);

        GenericDatumWriter<Object> w = new GenericDatumWriter<Object>(schema);
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();

        Encoder e = EncoderFactory.get().binaryEncoder(outputStream, null);

        w.write(datum, e);
        e.flush();

        return outputStream.toByteArray();
    }


    public static void main(String[] args) throws Exception {

        StringBuilder sb = new StringBuilder();
        StringBuilder jsb = new StringBuilder();

        ClassLoader classloader = Thread.currentThread().getContextClassLoader();
        InputStream is = classloader.getResourceAsStream("RsvpAvroSchema.avsc");
        InputStream js = classloader.getResourceAsStream("JsonMessage.dat");
        
        InputStreamReader isr = new InputStreamReader(is, StandardCharsets.UTF_8);
        InputStreamReader jisr = new InputStreamReader(js, StandardCharsets.UTF_8);
        BufferedReader br = new BufferedReader(isr);
        BufferedReader jbr = new BufferedReader(jisr);
        br.lines().forEach(line -> sb.append(line));
        jbr.lines().forEach(line -> jsb.append(line));

        System.out.println(sb);
        System.out.println(jsb);
        
        System.out.println(new String(fromJasonToAvro(jsb.toString(), sb.toString()), StandardCharsets.UTF_8));

But I've created avro classes (data structure) too from avsc using maven plugin. But now not sure how to use that main class of avro message data structure with string json message to produce avro message ?

Can anyone share how to do it ?

Update:

How to create Avro object from string Json ? Already have avro classes available in my project.

2nd update

public class AvroInstance {

    static DecoderFactory DEFAULT_FACTORY = new DecoderFactory();
    static DatumReader<Object> reader = new GenericDatumReader<Object>(RSVP.SCHEMA$);
    static Object rsvpOB;

    public Object avroInstance(String JsonString) {

        try {
            rsvpOB = reader.read(null, DEFAULT_FACTORY.jsonDecoder(RSVP.SCHEMA$, JsonString));
        } catch (IOException e) {
            e.printStackTrace();
        }
        return rsvpOB;

    }
Tim
  • 69
  • 7
  • Where are you sending the Avro message to? A Kafka topic? If so, configure the Producer to use the KafkaAvroSerializer - see https://docs.confluent.io/platform/current/schema-registry/serdes-develop/serdes-avro.html – Kevin Hooke May 16 '21 at 16:34
  • @Kevin I believe the intention is to not use a Registry https://stackoverflow.com/questions/67518900/produce-kafka-avro-messages-without-schema-registry-url – OneCricketeer May 17 '21 at 12:39

1 Answers1

0

You can replace your AVSC file reader with the static SCHEMA string stored in the generated class to get the same string...

You could also just use libraries that already do this

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • I've updated my code the way you had suggest, please check in 2nd update in question AvroInstance. Does it looks fine to you ? – Tim May 17 '21 at 14:55
  • Seems okay, by why are you returning an `Object` instead of `RSVP` instance? – OneCricketeer May 17 '21 at 15:02
  • have modified to RSVP. But tell me one thing what does from_avro returns ? – Tim May 22 '21 at 14:17
  • I'm not sure what you're referring to. That's not part of my answer or your question... But that's a Spark method that would return a Dataframe – OneCricketeer May 22 '21 at 18:32