Keeping totals from each Spring Batch thread

Question

I am implementing a Spring batch application that does some database updates. There is no input/output files here, just straight database updates.

The program is laid out as such:

1. Preparation tasklet calls a DB procedure to get all job parameters and work that needs to to be done
2. the work gets sent to the Partitioner where the work is divided into threads
3. Invocation tasklet starts and calls on each thread to call a different procedure that does the actual updates on the database
4. After all threads are done, an "end" procedure is called to close out any DB logging

My issue is between step 3 and 4. After each thread is done it collects the Procedure output parameters one of which is how much work it was actually updated. I want to be able to keep a running total of all threads and how much work it was done by all. This info will then be passed into step 4 which will include those totals into the DB logging tables.

Currently I'm adding any output from procedure in step 3 into a Map<String, String> resultMap and I can see each thread's total numbers. My issue is how to keep a total of ALL threads to be passed into step 4.

I tried declaring a couple of variables at the top of the Invocation Tasklet which will be used to keep a running total. The problem is their values are reset after each thread even though I'm using += when assigning the values from the Map

public class SpringBatchInvocationTasklet  implements Tasklet, StepExecutionListener {

   int totalWork = 0;
   int totalWorkProcessed = 0;


    @Override
   public RepeatStatus execute(StepContribution arg0, ChunkContext chunkContext) throws Exception {
      Map<String, String> resultMap = SpringBatchDAO.process(workList, dbLogId, commitVal, userName, threadNo);

      totalWork += Integer.parseInt(resultMap.get(SpringBatchConstants.TOTAL_WORK));
      totalWorkProcessed += Integer.parseInt(resultMap.get(SpringBatchConstants.TOTAL_WORK_PROCESSED));

    }
}

To summarize, I need to be able to get a running total of all threads for what work they all did and pass it into the end process.

Thank you for any help on this!

Answer 1

According to your setup, your tasklet will be shared by multiple threads, so it needs to be thread safe.

The problem is their values are reset after each thread even though I'm using += when assigning the values from the Map

The += operator is not thread safe, as this is not an atomic operation. You need to use synchronization/locks around your code or use a thread-safe Map implementation (like ConcurrentHashMap) and AtomicInteger ( see incrementAndGet method) instead of int in for your totalWork and totalWorkProcessed variables.

The following question/answers might help: Is the += operator thread-safe in Java?

Hope this helps.