0

I'm fairly new to Spring Batch and I would like to know if the following is possible. I am creating a batch job that requires some initial data queried from a database. So would be something like the following be possible. Below is an item reader the first step in the job execution. This initial step will query all FOOs:

@Bean
public JdbcCursorItemReader fooItemReader() {
    return
        new JdbcCursorItemReaderBuilder()
            .dataSource(dataSource)
                .sql(QueryConstants.ALL_FOOS)
    .build();
}

@Bean(name = "step-one")
public Step stepOne() {
    return stepBuilderFactory
        .get("step-one")
            .reader(fooItemReader())
           ...
    .build();
}

The above cursor reader and step would ideally pull all of the "FOO"s from a a database. Given the above, I would like to know if it is possible to take the data from step-one and pass it to a step-two to select all "BAR"s for the given "FOO" from the list of "FOO"s from step-one. Something like below:

@Bean
public JdbcCursorItemReader barsForFooItemReader() {
    //some how for each foo in the list of FOOs from step-one retrieve it's BARs
   
    return
        new JdbcCursorItemReaderBuilder()
            .dataSource(dataSource)
                .sql(QueryConstants.BARS_FOR_FOO)
    .build();
}

@Bean(name = "step-two")
public Step stepOne() {
    return stepBuilderFactory
        .get("step-one")
            .reader(barsForFooItemReader())
           ...
    .build();
}

Is something like this possible with Spring Batch? Is Spring Batch "right" for something like this? Any help would be most appreciated!!

UPDATE:

I looked into the suggested posted from the comments and they are not exactly what I'm looking for and leaves some unanswered questions. So I'm going to to attempt to clarify what I am trying to accomplish with Spring Batch by adding an example (below) of how I would implement the above without Spring Batch and using JdbcTemplate.

@Repository
public class FooBarRepository {

    private final NamedParameterJdbcTemplate namedParameterJdbcTemplate;

    @Autowired
    public FooRepository(NamedParameterJdbcTemplate namedParameterJdbcTemplate) {
        this.namedParameterJdbcTemplate = namedParameterJdbcTemplate;
    }

    /**
     * Get all FOOs from database
     * @return
     */
    public List<String> getAllFoos() {
        String query = "SELECT * FROM FOO;";

        try {
            return namedParameterJdbcTemplate.query(query, (resultSet, i) -> resultSet.getString("FOO_NAME"))';'
        }
        catch (DataAccessException ex) {
            //... log exception
            return Collections.emptyList();
        }
    }

    /**
     * Get all Bars for a given foo name
     * @param fooName
     * @return
     */
    public List<String> getBarsForFoo(String fooName) {
        String query = "SELECT * FROM BAR CROSS JOIN FOO WHERE FOO.NAME = :fooName";
        
        try {
            return namedParameterJdbcTemplate.query(query, new MapSqlParameterSource().addValue("fooName", fooName), (resultSet, i) -> resultSet.getString("BarFoo"));
        }
        catch (DataAccessException ex) {
            //... log exception
            return Collections.emptyList();
        }
    }
}

Above is a simple dao repo. There's a query for retrieving all FOOs and another for retrieving all Bars for a given fooName. Below is how I would would use both methods.

@Component
public class FooBarProcessor {

    private final FooRepository fooRepository;

    @Autowired
    public FooBarProcessor(FooRepository fooRepository) {
        this.fooRepository = fooRepository;
    }

    public void processFooBars() {
        List<String> foos = fooRepository.getAllFoos();
        Map<String, List<String>> foobars = new HashMap<>();
        
        foos.forEach(foo -> {
            List<String> bars = fooRepository.getBarsForFoo(foo);
            foobars.put(foo, bars);
        });
    }
}

In the above, I get a list of all the FOOs and then for each FOO I retrieve it's BARs. (I hope the above makes sense logically)

I want to accomplish something similar with Spring Batch as I expect the data set to be quite large.

Joseph Freeman
  • 1,644
  • 4
  • 24
  • 43
  • Does this answer your question? [How can we share data between the different steps of a Job in Spring Batch?](https://stackoverflow.com/questions/2292667/how-can-we-share-data-between-the-different-steps-of-a-job-in-spring-batch) – Mahmoud Ben Hassine Jun 23 '20 at 07:56
  • This might help as well: https://stackoverflow.com/questions/8117060/storing-in-jobexecutioncontext-from-tasklet-and-accessing-in-another-tasklet/8121102#8121102 – Mahmoud Ben Hassine Jun 23 '20 at 07:56
  • I'll check both of those post out and see if they help. Thank you! – Joseph Freeman Jun 23 '20 at 13:56

0 Answers0