1

I have a job in which: The first step is a TaskletStep which retrieves some records(approx. 150-200) from a database table into a list.

The second step retrieves data from some other table and requires the list of records retrieved in the previous step for processing.

I came across three ways to do this: 1)putting the list retrieved in first step in StepExecutionContext and then promoting it to JobExecutionContext to share data between steps.

2)using spring's caching concept i.e. using @cacheable

3)programmatically putting the list in the ApplicationContext

What is the best way to achieve this(it would be better if it can be explained with an example), keeping in mind two main concerns: if the volume of data retrieved in the first step increases and performance

user2971387
  • 109
  • 1
  • 3
  • 11
  • Also, the data that will be retrieved in the second step can be huge, i.e. in millions. And processing of each record in this second step requires the data retrieved from the first step. – user2971387 Nov 09 '13 at 08:03

1 Answers1

2

Remember that objects in step context are stored into database,so you must be sure that objects are serializable and are really a few. If you are sure, put objects in your jobExecutionContext (as solution 1.) or use a bean holder (Passing data to future step); this type of approach is valid ONLY if data in first step is SMALL.
Else, you can process data in step2 without data retrivial in step1, but easly manage a cache of step1 data while processing data in step2; in this way you don't need step1, don't need to store step1 data to database, but step1 data lookup while processing millions record in step2 doesn't impact in terms of time processing.

I hope I was clear, English is not my language

Luca Basso Ricci
  • 17,829
  • 2
  • 47
  • 69