Spring Batch - Read query from file and execute it on database

Question

I know I can simply read the file straight from step1, a moment before setting the sql query into the reader, but I want to keep the process of reading the query separate from database reading.

Here is my job configuration.

@Configuration
public class BatchConfiguration {

    [...]

    @Bean
    @StepScope
    public JdbcCursorItemReader<Map<String, Object>> dynamicSqlItemReader() {
        JdbcCursorItemReader<Map<String, Object>> jir = new JdbcCursorItemReader<>();
        jir.setSql((String) contextHolder.getContext().get("fileContent"));
        jir.setDataSource(dataSource);
        jir.setRowMapper(new ColumnMapRowMapper());
        return jir;
    }

    private FlatFileItemReader<String> flatFileItemReader() {
        [...]
    }

    private ItemWriter<? super String> sysoItemWriter() {
        return (ItemWriter<String>) list -> {
            for (String element : list) {
                System.out.println(element);
            }
            contextHolder.getContext().put("fileContent", list.get(0));
        };
    }

    @Bean
    public ItemWriter<Map<String, Object>> customerItemWriter() {
        return list -> {
            for (Map<String, Object> stringObjectMap : list) {
                System.out.println(stringObjectMap);
            }
        };
    }

    @Bean
    public Step step0() {
        return stepBuilderFactory.get("step0")
                .<String, String>chunk(1)
                .reader(flatFileItemReader())
                .writer(sysoItemWriter())
                .build();
    }

    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1")
                .<Map<String, Object>, Map<String, Object>>chunk(10)
                .reader(dynamicSqlItemReader())
                .writer(customerItemWriter())
                .build();
    }


    @Bean
    public Job job() throws Exception {
        return jobBuilderFactory.get("job")
                .incrementer(new RunIdIncrementer())
                .start(step0())
                .next(step1())
                .build();
    }
}

This throws a java.lang.IllegalArgumentException: The SQL query must be provided because the contextHolder.getContext().get("fileContent") is still null at time of setting the query.

I don't know a priori: path to sql query is going to be a batch parameter. But basically they are `select`. — Angelo Tricarico, Mar 15 '18 at 18:29

score 0 · Answer 1 · answered Mar 15 '18 at 11:31

0

Before step1, you could write a tasklet for building the query and putting it into context, so that it stays separate and also it becomes available to step1. See more about tasklet here: Tasklet to delete a table in spring batch

answered Mar 15 '18 at 11:31

gargkshitiz

2,130
17
19

I also tried using a tasklet, but the problem is still there. `@Bean public Step step0() { return stepBuilderFactory.get("step0") .tasklet((stepContribution, chunkContext) -> { File f = new File("D:\\query.sql"); contextHolder.getContext().put("fileContent", FileUtils.readFileToString(f, "UTF-8")); return RepeatStatus.FINISHED; }) .build(); }` – Angelo Tricarico Mar 15 '18 at 13:15

score 0 · Answer 2 · answered Mar 19 '18 at 05:07

You are not using your created contextHolder properly that's why the value there is null.

Make sure you are putting your data in flatFileItemReader() in contextHolder in directly as a map because when you are getting value, you are using contextholder.getContext(). Since it's an simple map,not a ApplicationContext, the method you are using does not exist.

Spring Batch - Read query from file and execute it on database

2 Answers2