0

I have a spring batch job which is processing data similar to below. I have a requirement to aggregate the values for each key, and then use that summed value in a final processor. I can't control the order of the data received.

Key - Value
A - 1
A - 5
A - 2
B - 2
B - 2
B - 2
C - 1
A - 2
..
..
..
B - 4 
X - 10

I have setup a pre-processor, which takes each row and checks the StepExecutionContext for an existing aggregated value for the key. It creates a new one or amends an existing one if not. At the end of this step all key values must have been aggregated.

My second processor then checks for distinct key names, and it attempts to read the aggregated value from the StepExecutionContext. Of course the aggregated values are not present in the second processor's StepExecutionContext.

In another thread I've read that I should use a ExecutionContextPromotionListener to promote the StepExecutionContext from my first processor to the JobExecutionContext which would make them available to the second process. I know the ExecutionContextPromotionListener takes a list of key names to promote, but in my case I don't know the list of keys to promote until the end of the first processor. Can anybody suggest a workaround?

Peter Wippermann
  • 4,125
  • 5
  • 35
  • 48
emeraldjava
  • 10,894
  • 26
  • 97
  • 170
  • 2
    Don't put each piece of data into the StepExecutionContext independently. Create a map, insert that into the StepExecutionContext and put your data's key/values in there. That way, you will promote the key of the map to the JobExecutionContext. – Michael Minella May 01 '14 at 14:13
  • I understood there is limit to the size of data that can be promoted from the StepContext to the JobContext. My map could get quiet large and hit this limit quickly, so i thought it was better to put a lot of small key values. – emeraldjava May 01 '14 at 15:51
  • There is no limit in the context itself. The database has a character limit, but using individual keys won't get you around that limit (in fact it may make it worse). – Michael Minella May 01 '14 at 18:27

0 Answers0