Process json files in spring batch

Question

I have a zip file containing multiple json files. I have unzipped them then got POJO object from json using below code:

reader = new BufferedReader(new FileReader(file));
Gson gson = new GsonBuilder().create();
Element[] people = gson.fromJson(reader, Element[].class);

but I need to process these json files one by one using spring batch. Can someone help me how I can achieve this in spring batch and I want to read json file using chunk of 1000 My json object is very complex. Example:

{
    "students": {
      "subelements": {
          "dep": {
            "data": [
              "XYZ"
            ]
          }
        }
      }
}

What represents an item in this file? Which fragment corresponds to the `Element` class? — Mahmoud Ben Hassine, Sep 17 '18 at 13:06
Element is main class which holds id, type and list of properties — R.Henderson, Sep 17 '18 at 15:48

Mahmoud Ben Hassine · Accepted Answer · 2018-09-18T06:17:39.470

1

Your data structure is not one of the types you could handle with Spring Batch out-of-the-box. See more details here: https://stackoverflow.com/a/51933062/5019386.

So I think in your case, you would need to create a custom item reader to parse a specific fragment of your input file.

edited Sep 18 '18 at 06:17

answered Sep 17 '18 at 19:45

Mahmoud Ben Hassine

28,519
3
32
50

Can I read just list of items(elements array) using spring batch? and ignore all other json data during reading – R.Henderson Sep 18 '18 at 05:45
No that's not possible. Your file needs to be of the form `[{object1}, {object2}, {object3}]` to be able to use the `JsonItemReader`. – Mahmoud Ben Hassine Sep 18 '18 at 06:21
so then best is to use json stream api ? – R.Henderson Sep 18 '18 at 10:50
Exactly, your custom item reader should use a json streaming api. – Mahmoud Ben Hassine Sep 18 '18 at 11:35

Process json files in spring batch

1 Answers1