Spring Batch: How to setup a FlatFileItemReader to read a json file?

Question

My approach so far:

@Bean
FlatFileItemReader<Blub> flatFileItemReader() {
    FlatFileItemReader<Blub> reader = new FlatFileItemReader<>();

    reader.setResource(new FileSystemResource("test.json"));

    JsonLineMapper lineMapper = new JsonLineMapper();

    reader.setLineMapper(lineMapper);

    return reader;
}

The challenge is: reader.setLineMapper() cannot use the JsonLineMapper. How to use the JsonLineMapper properly?

score 6 · Answer 1 · answered Aug 20 '18 at 10:53

create a class BlubJsonLineMapper

public class BlubJsonLineMapper implements LineMapper<Blub> {

    private ObjectMapper mapper = new ObjectMapper();


    /**
     * Interpret the line as a Json object and create a Blub Entity from it.
     * 
     * @see LineMapper#mapLine(String, int)
     */
    @Override
    public Blub mapLine(String line, int lineNumber) throws Exception {
        return mapper.readValue(line, Blub.class);
    }

}

then you can set in the FlatFileItemReader

@Bean
FlatFileItemReader<Blub> flatFileItemReader() {
    FlatFileItemReader<Blub> reader = new FlatFileItemReader<>();

    reader.setResource(new FileSystemResource("test.json"));

    BlubJsonLineMapper lineMapper = new BlubJsonLineMapper();

    reader.setLineMapper(lineMapper);

    return reader;
}

Ty for your answer! Is there a way to use the JsonLineMapper and not to implement an own LineMapper? — yN., Aug 20 '18 at 10:58
JsonLineMapper is convert the everyline json to *Map*, but your requirement is to convert to **Blub**, you can use a anonymous class that implement the LineMapper in your flatFileItemReader() {....} — clevertension, Aug 20 '18 at 11:01

Mahmoud Ben Hassine · Accepted Answer · 2020-03-20T09:42:48.910

How to setup a FlatFileItemReader to read a json file?

It depends on the format of your json file:

1. Each line is a json object (known as NDJson)

For example:

{object1}
{object2}

then you have two options:

1.1 Use the JsonLineMapper which returns a Map<String, Object>. In this case, your reader should also return Map<String, Object> and you can use an item processor to transform items from Map<String, Object> to Blub (BTW, transforming data from one type to another is a typical use case for an item processor)
1.2 Use a custom implementation of LineMapper<Blub> based on Jackson or Gson or any other library (as shown in the answer by @clevertension)

2. Lines are wrapped in a json array

For example:

[
 {object1},
 {object2}
]

then you can use the new JsonItemReader that we introduced in version 4.1.0.M1 (See example in the blog post here: https://spring.io/blog/2018/05/31/spring-batch-4-1-0-m1-released#add-a-new-json-item-reader).

There are similar questions to this one, I'm adding them here for reference:

I'm fiddling with format 1. which option would you suggest best? I'm complete newbie to Spring batch and I've been shaving my head over how to do this right. Doesn't seem like there is one complete example on the internet for it save for this and a couple other stack answers. Is every batch coder reading JSON is pro already? :O — Rohan Kumar, Jun 22 '19 at 19:39
I've so far been able to code this: ```public class DbsnpJsonItemReader extends FlatFileItemReader> { public DbsnpJsonItemReader(File file) { Resource resource = new GzipLazyResource(file); setResource(resource); setLineMapper(new JsonLineMapper()); } ``` — Rohan Kumar, Jun 22 '19 at 19:41

score 2 · Answer 3 · edited Dec 14 '18 at 07:25

2

I have build a small demo for Json. If you need any more than it, let me know I can build another example for you

https://github.com/bigzidane/spring-batch-jsonListItem-reader

edited Dec 14 '18 at 07:25

Community

1
1

answered Aug 20 '18 at 16:06

Nghia Do

2,588
2
17
31

Spring Batch: How to setup a FlatFileItemReader to read a json file?

3 Answers3

1. Each line is a json object (known as NDJson)

2. Lines are wrapped in a json array

Linked