1

In my particular scenario, I have to persist huge entities read from files to the database but my commit-interval is configured with value = 1. Also I'm not quite sure if the delegate processor in the CompositeItemProcessor used are idempotent. So My question is if there is a way of preventing the wasted time on reprocessing and avoid many unused queries to the db. I also want to use a SkipListener to log the errors in reading/processing/writing into an specific table and suspect such configuration (no reprocessing) is not compliant with that.

I am using spring batch 2.1.9.

Thanks in advance.

____________________________UPDATE 07-05-2016 ____________________________

After a few days of research I realized there is a conceptual debate between some users and spring developers.

I know now that this behavior woks as designed on skipped exceptions thrown during writing phase as said by @MichaelMinella in the comment for the answer in this 2014 post:

The ItemWriter#write method receives a list of items. Without going through them one at a time, there is no way for us to determine which in the list threw the exception in the writer.

So is no way yet to the framework discover the failed item without scanning for the whole list (even if the chunk size is 1). The Inner behavior of the framework to do this uses the FaultTolerantChunkProcessor and the RetryTemplate is described in this 2013 post more discussion on the matter can be found on this spring batch forum 2012 post.

Community
  • 1
  • 1
guilhermerama
  • 750
  • 9
  • 21

1 Answers1

0

If your chunk size is equal to one the item's shouldn't be processed multiple times. When a skip occurs, spring batch removes the skipped item in the chunk and reprocesses it. When the chunk only contains one element there should be no elements processed mulitple times.

joshiste
  • 2,638
  • 1
  • 14
  • 19
  • I expected this behavior too @joshiste, but in practice the framework always reprocesses the the list cached on the reading and processes mini-chunks for each item to know what item as responsible for the error, even if the this list contains only one element. What I want to know is how to override this behavior or configure the batch to skip this expensive retry. Maybe this isn't possible if I want to use an SkipCheckListener to register errors on processing too. – guilhermerama Jun 23 '16 at 10:58