SerDe using Apache AVRO in JAVA

Question

I am trying to serialize and deserialize a list of JSON objects

Below is my JSON file

[
    {
        "id": "01",
        "Status": "Open",
        "siteId": "01",
        "siteName": "M1"
    },
    {
        "id": "02",
        "Status": "Open",
        "siteId": "02",
        "siteName": "M2"
    },
    {
        "id": "03",
        "Status": "Open",
        "siteId": "03",
        "siteName": "M3"
    }
]

code :

public static void main(String args[]) throws IOException {
        //creating schema
        Schema schema = ReflectData.get().getSchema(document.class);
        System.out.println("schema created: "+ schema.toString());

        //get JSON list
        List<document> list = getJsonList();


        File fileR = new File("<<path>>/jsonlist.avro");

        //Serialize objects to file
        DatumWriter<document> writerR = new ReflectDatumWriter(document.class);  // Serialize objects to in-memory binary data
        DataFileWriter<document> outR = new DataFileWriter(writerR).create(schema, fileR);    // Write binary data to file
        for(int i = 0 ;i<list.size();i++)
        {
            outR.append((document) list.get(i));
        }

        outR.close();
        System.out.println("Serialize objects to file...\n");

        //Deserialize objects from file
        DatumReader<document> readerR = new ReflectDatumReader(document.class);
        DataFileReader<document> inR = new DataFileReader(fileR, readerR);
        System.out.println("Deserialize objects from file...");
        for(document doc : inR) {
            System.out.println(doc.toString());
        }
        inR.close();

}
private static List<document> getJsonList() throws IOException {
        final ObjectMapper objectMapper = new ObjectMapper();
        String fileName = "jsonList.json";
        ClassLoader classLoader = <<className>>.class.getClassLoader();
        File file = new File(classLoader.getResource(fileName).getFile());
        System.out.println("File Found : " + file.exists());

        //list of json objects
        List<document> list = objectMapper.readValue(file,new TypeReference<List<document>>(){});
        returnlist;
    }

When I am trying to run the above code the file is getting serialized to check the deseralization the output is coming as

document@3246fb96
document@2e222612

I am not sure if the code for serialization is correct or where I am going wrong in the deserialization code

Referred : https://liyanxu.blog/2018/02/07/apache-avro-examples/

Please suggest!!

I'll have a look in a few hours when I get free, in case no one else can help before that — Andy Valerio, Mar 20 '21 at 07:57
Hi! I'm missing two things to be able to run it: - The class "caseDocument", how does it look? Is it the same as "document"? - The method "getJsonList()" is missing also, could you copy/paste it in the question? Thanks a lot and speak later — Andy Valerio, Mar 20 '21 at 09:05
Hi, Yes caseDocument is the same as document and getJsonList method added. Please Have a look. Thanks! — StudentL, Mar 20 '21 at 13:32
You should define your Avro schema, then generate the Document class rather than depend on reflection — OneCricketeer, Mar 23 '21 at 01:18

score 0 · Accepted Answer · answered Mar 20 '21 at 15:14

0

Your code actually works, what you missed is to add a toString() method to your Document class.

Add something like this:

@Override
public String toString() {
    return String.format("Document ID: {%s}, status: {%s}, site ID: {%s}, site name: {%s}", id, Status, siteId, siteName);
}

to the document class (which should really be Document with capital letter). And it will print out what you want, for example something like this given the example:

Deserialize objects from file...
Document ID: {01}, status: {Open}, site ID: {01}, site name: {M1}
Document ID: {02}, status: {Open}, site ID: {02}, site name: {M2}
Document ID: {03}, status: {Open}, site ID: {03}, site name: {M3}

Alternatively, you could use the Lombok annotation @ToString. You can look into it here if you are interested.

answered Mar 20 '21 at 15:14

Andy Valerio

851
6
13

Additional explanation, just in case you are interested. What you see in your output, (something like document@3246fb96) is actually a reference to an instance of the class document. You see, in Java only the primitive types (int, byte, short, long, etc...) will print an actual value. For all the other classes, included whatever class you define, you have to implement the method toString() to define what the string representation of that class should be; if you don't do that, the standard output will be a memory reference to your object (that is, a meaningless sequence of characters). – Andy Valerio Mar 20 '21 at 15:21
Hi, It's working!!! Really Thanks a lot!!Means alot – StudentL Mar 21 '21 at 05:43

SerDe using Apache AVRO in JAVA

1 Answers1

Linked