Spring Boot + Spring Batch + HSQLDB -> Configure HSQLDB for JobRepository

Question

I am new to Spring Boot. I'm trying to write an application that takes advantage of Spring Batch to write data from one database to another. I'm reading from Mongo and writing to SQL Server. I am hosting my JobRepository in-memory and have coded it to use HSQLDB.

I am having some issues pointing Spring.Batch to the proper Data Source so I'm focusing on ensuring that I'm writing my jobs to the HSQLDB now. To do that, I've essentially commented out the SQL DB for now.

I have the following BatchConfiguration class:

@Configuration
@EnableBatchProcessing
public class BatchConfigurer extends DefaultBatchConfigurer {

    @Autowired
    @Qualifier("batchDataSource")
    public DataSource batchDataSource;

    @Bean
    public JdbcTemplate jdbcTemplate(DataSource dataSource){
        return new JdbcTemplate(dataSource);
    }


    @Bean
    public JdbcTransactionManager batchTransactionManager() {
        final JdbcTransactionManager transactionManager = new JdbcTransactionManager();
        transactionManager.setDataSource(batchDataSource);
        return transactionManager;
    }


    @Bean
    public JobRepository jobRepositoryBean() throws Exception {
        JobRepositoryFactoryBean fb = new JobRepositoryFactoryBean();
        fb.setDatabaseType("HSQL");
        fb.setDataSource(batchDataSource);
        fb.setTransactionManager(batchTransactionManager());
        return fb.getObject();
    }


    @Override
    @Bean
    protected JobLauncher createJobLauncher() throws Exception {
        SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
        jobLauncher.setJobRepository(this.jobRepositoryBean());
        jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
        jobLauncher.afterPropertiesSet();
        return jobLauncher;
    }

}

When I load this class I'm seeing the following output in the log:

2021-06-24 17:18:42.412 WARN 31940 --- [ main] o.s.b.c.c.a.DefaultBatchConfigurer : No transaction manager was provided, using a DataSourceTransactionManager

2021-06-24 17:18:42.419 INFO 31940 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: HSQL

2021-06-24 17:18:43.154 WARN 31940 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning

I'm trying to understand why the database type is inferred. It looks like I'm setting it already. Furthermore, when I uncomment the other Data Source the inferred type is SQLSERVER, which is wrong. Am I incorrectly initializing and setting the configuration for this data type?

I figured I'd try to get this working well before tackling the multiple data source situation again.

Mahmoud Ben Hassine · Accepted Answer · 2021-06-25T19:42:37.927

0

Just declaring a TransactionManager and a JobRepository as beans is not sufficient and will not be taken into account by Spring Batch. You need to provide a BatchConfigurer for that. Since you made your configuration class extend DefaultBatchConfigurer, you can override getTransactionManager and getJobRepository, something like:

@Configuration
@EnableBatchProcessing
public class BatchConfigurer extends DefaultBatchConfigurer {

    @Autowired
    @Qualifier("batchDataSource") // ensure this is the HSQLDB here
    public DataSource batchDataSource;


    @Override
    public JdbcTransactionManager getTransactionManager() {
        final JdbcTransactionManager transactionManager = new JdbcTransactionManager();
        transactionManager.setDataSource(batchDataSource);
        return transactionManager;
    }


    @Override
    public JobRepository getJobRepository() throws Exception {
        JobRepositoryFactoryBean fb = new JobRepositoryFactoryBean();
        fb.setDatabaseType("HSQL");
        fb.setDataSource(batchDataSource);
        fb.setTransactionManager(getTransactionManager());
        return fb.getObject();
    }


    @Override
    public JobLauncher getJobLauncher() throws Exception {
        SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
        jobLauncher.setJobRepository(this.jobRepositoryBean());
        jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
        jobLauncher.afterPropertiesSet();
        return jobLauncher;
    }

}

This is explained in the reference documentation here: Java Config.

edited Jun 25 '21 at 19:42

answered Jun 25 '21 at 12:22

Mahmoud Ben Hassine

28,519
3
32
50

Got it. Thanks for the clarification. One issue I'm having with your suggested code is: java: getJobRepository() in com.test.config.BatchConfigurer cannot override getJobRepository() in org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer overridden method does not throw java.lang.Exception – Corez Jun 25 '21 at 18:43
Yes, you got the idea, so you can adapt the code as needed. I updated the answer accordingly. – Mahmoud Ben Hassine Jun 25 '21 at 19:43
well...that doesn't fix the issue - it's actually in the getJobRepository method. I'm trying to make it so that someone could use this code fragment in the future. I ended up using overriding createJobLauncher() - would that do the same thing? – Corez Jun 25 '21 at 20:01
Since you did not provide a minimal complete example, I did not compile that code, so please adapt it accordingly. The way it works is that if you want to provide a custom job repository (based on your HSQLDB), you need to do it through a `BatchConfigurer#getJobRepository`. I will let you check exception handling. – Mahmoud Ben Hassine Jun 25 '21 at 20:06
Okay - what I really want to do is to just have Batch use a particular data source. That's my main goal. I have two and it keeps picking up the wrong one. – Corez Jun 25 '21 at 20:07
But you said you want to use Im-memory HSQL and this is what I see from the logs: `using meta data indicating: HSQL` since you autowired `batchDataSource` and set it on the job repository. Isn't this what you want? You might also need to override the setter of the datasource and set the `batchDataSource` in it. If this does not solve your issue, please provide a [minimal example](https://stackoverflow.com/help/minimal-reproducible-example) that reproduces the issue to be able to help you in an efficient way. This might help as well: https://stackoverflow.com/questions/25540502 – Mahmoud Ben Hassine Jun 26 '21 at 05:50

Spring Boot + Spring Batch + HSQLDB -> Configure HSQLDB for JobRepository

1 Answers1