Use Univocity Parsers with Spring Batch

Question

I am trying o use Univocity Parsers within Spring Batch. The problem I am facing is how to integrate them.

Spring Batch Chunk Steps follow the flow for each row of the given file:

I need to use Univocity inside an ItemReader. It executes the read() method for each row of the input file (ie. CSV File). The only thing I did was using a BeanListProcessor to read and convert items directly to my Java Object returning a List of the parsed Beans, but I do not want to load all the records at once, to avoid OutOfMemory exceptions. I did not find anything else that could help me.

I have tried using this answer as an example, but could not figure out anything to return one item at a time.

@Override
public Address read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
    CsvParserSettings parserSettings = new CsvParserSettings();
    //settings
    CsvRoutines routines = new CsvRoutines(parserSettings);
    for (Address address : routines.iterate(Address.class, input, "UTF-8")) {
        /*
         *here I need to return only the current object, 
         *the return of this method will be passed as an argument to a processor
         *the next time this method is called it has to return the next one
         *Could not figure out how to control what is the current.
        */
        return ???:
    }
    return ???;
}

How can I use Univocity inside my ItemReader reading one row at a time, still using a BeanProcessor to parse my rows automatically to my Java Object?

Jeronimo Backes · Answer 1 · 2019-08-31T06:04:15.647

Hi I'm the author of the lib. routines.iterate(Address.class, input, "UTF-8") will return an Iterable<Address>. You can get an Iterator<Address> from that and keep this Iterator<Address> in memory.

Every time you need to read the next Address, simply call iterator.next().

I believe your method should be written as:

private Iterator<Address> iterator;

@Override
public Address read() throws Exception, UnexpectedInputException,   ParseException, NonTransientResourceException {
    if(iterator == null){
        CsvParserSettings parserSettings = new CsvParserSettings();
        //settings
        CsvRoutines routines = new CsvRoutines(parserSettings);
        iterator = routines.iterate(Address.class, input, "UTF-8")).iterator();
    } 
    if(iterator.hasNext()){
        return iterator.next();
    } else {
        iterator = null;
    }
    return null;
}

Hope this helps.

Thank you for the answer! But how can I add that to a Reader? As the method `read()` executes **returning** each row at a time for the Processor (as the image in question)? If I user the code you have posted, it will iterate over all te objects before returning, or it will return always the 1st. — henrique romao, Aug 30 '19 at 10:21

Use Univocity Parsers with Spring Batch

1 Answers1