13

Our processor returns a List<?> (effectively passing a List<List<?>>) to our ItemWriter.

Now, we observed that the JdbcBatchItemWriter is not programmed to handle item instanceof List. We also observed to process item instanceof List; we need to write a custom ItemSqlParameterSourceProvider.

But the sad part is that it returns SqlParameterSource which can handle only one item and again not capable of handling a List.

So, can someone help us understand how to handle list of lists in the JdbcBatchItemWriter?

Dean Clark
  • 3,770
  • 1
  • 11
  • 26
chatanyday
  • 145
  • 1
  • 1
  • 5

2 Answers2

28

Typically, the design pattern is:

Reader -> reads something, returns ReadItem
Processor -> ingests ReadItem, returns ProcessedItem
Writer -> ingests List<ProcessedItem>

If your processor is returning List<Object>, then you need your Writer to expect List<List<Object>>.

You could do this by wrapping your JdbcBatchItemWriter as a delegate in an ItemWriter that looks something like this:

public class ListUnpackingItemWriter<T> implements ItemWriter<List<T>>, ItemStream, InitializingBean {

    private ItemWriter<T> delegate;

    @Override
    public void write(final List<? extends List<T>> lists) throws Exception {
        final List<T> consolidatedList = new ArrayList<>();
        for (final List<T> list : lists) {
            consolidatedList.addAll(list);
        }
        delegate.write(consolidatedList);
    }

    @Override
    public void afterPropertiesSet() {
        Assert.notNull(delegate, "You must set a delegate!");
    }

    @Override
    public void open(ExecutionContext executionContext) {
        if (delegate instanceof ItemStream) {
            ((ItemStream) delegate).open(executionContext);
        }
    }

    @Override
    public void update(ExecutionContext executionContext) {
        if (delegate instanceof ItemStream) {
            ((ItemStream) delegate).update(executionContext);
        }
    }

    @Override
    public void close() {
        if (delegate instanceof ItemStream) {
            ((ItemStream) delegate).close();
        }
    }

    public void setDelegate(ItemWriter<T> delegate) {
        this.delegate = delegate;
    }

}
Dean Clark
  • 3,770
  • 1
  • 11
  • 26
  • Excellent solution suggested. We were planning to do something more complex by implementing http://forum.spring.io/forum/spring-projects/batch/122284-how-to-write-a-list-of-items-with-a-itemwriter; but your solution is generic enough. In latter we would require a lot of job to be done. Testing it was also going to get cumbersome. – chatanyday Jun 17 '16 at 07:51
  • Very good formatting of the code, do you have a template available for this? – Serge Aug 01 '16 at 18:27
  • It may be more efficient to built a list and execute a writer once. – Serge Aug 01 '16 at 18:47
  • 1
    @Dean Clark : I'm getting null pointer exception at delegate.write(consolidatedList); – yuva ツ Apr 24 '17 at 06:02
  • @yuva Are you sure you've set a delegate? This is a bare-bones version of the class. In reality, it should also implement `InitializingBean` and have an assertion that a delegate writer has been provided. – Dean Clark Apr 25 '17 at 11:17
  • I recommend creating a new question with your current setup. – Dean Clark Apr 26 '17 at 14:55
  • I ask how to set Resource File? I am tried this example but I don't khnow how to set the output file – Majdi Taleb Feb 21 '20 at 23:37
  • You'd need to set it in your delegate `ItemWriter`. This writer can work for a file writer, DB writer, MQ writer, etc. So you may want to use a `FlatFileItemWriter` as your delegate, and can set your output resource there. – Dean Clark Apr 24 '20 at 15:19
  • Is there any sample code snippet on how to create a delete for `FlatFileItemWriter` ? – Paramesh Korrakuti Jun 16 '20 at 21:21
1
public class ListUnpackingItemWriter<T> implements FlatFileItemWriter<List<T>>, ItemStream, InitializingBean {

    @Override
    public void afterPropertiesSet() {
        setLineAggregator(item -> String.join("\n", item.stream().map(T::toString).collect(Collectors.toList())));
    }

}

Just added a custom line aggregator to the above solution, this helps in writing the content to a file by using FlatFileItemWriter<List<T>>. You can replace T with actual class name to avoid compilation error while calling toString() method.

Paramesh Korrakuti
  • 1,997
  • 4
  • 27
  • 39