spring batch junit with multiple datasources. spring data jpa unable to save data in-memory database

Question

My scenario is that batch job should read data from 1 DB (H2) and dump data into another DB (postgres).Hence I have configured multiple datasources

I have written batch job using spring batch 4.2 and Spring boot 2.2.5.RELEASE. Now I need to write testcase for end to end testing.

I have used custom Batch Configurer using ResourcelessTransactionManager to avoid saving of batch table meta data in in-memory and actual database and it's working. However,I am unable to save business data in H2 in-memory database in junit by calling readerRepository.save(randomReader); and same is working if I replace H2 in-memory database with postgres testconatiner in junit.

Sample junit of repository test

@Slf4j
@SpringBootTest
@Import(LiquibaseConfigReader.class)
class ReaderRepositoryTest {

    @Autowired
    private ReaderRepository readerRepository;

    @BeforeEach
    void setUp() {
        readerRepository.deleteAll();
    }

    @Test
    void shouldFetchAllRecords() {
        var randomReader = RandomDataGenerator.randomReader();
        assertThat(readerRepository.findAll()).hasSize(0);
        readerRepository.save(randomReader);
        assertThat(readerRepository.findAll()).hasSize(1);
    }
}

I am unable to understand why I am able to save data in postgres test container but not in in-memory H2 database while running Junit Test.

To save batch table meta-data in in-memory map, i tried below solutions and now batch table meta data is not being saved anywhere in DB.However, I started facing spring data jpa issue in junit as mentioned above.

Java Spring Batch using embedded database for metadata and a second database for other data

Spring boot + spring batch without DataSource

https://github.com/riversoforion/spring-batch-inmem/blob/master/src/main/java/com/example/batch/CustomBatchConfigurer.java

Is it because of ResourcelessTransactionManager? If yes, how could I avoid saving batch table metadata in any database including in-memory.

I have scrolled through multiple questions given here but none is able to save business data in h2 database.

Answer 1

Hard to answer without your source code. As far as I can understand, you want to have one DB to read data from (H2) and one to write data into (postgres).

And you don't want Spring batch to write its meta-data in both databases.

Then you write, that you can't understand why you can't write to H2 - I assume, you only mean the spring batch meta-data here.

So a possible solution may be to alter the BatchConfigurer:

...
public class CustomBatchConfigurer extends DefaultBatchConfigurer {

    @Autowired
    @Qualifier("repoDataSource")
    private DataSource dataSource;

    @Override
    protected JobRepository createJobRepository() throws Exception {
        JobRepositoryFactoryBean factoryBean = new JobRepositoryFactoryBean();
        factoryBean.setDatabaseType(DatabaseType.POSTGRES.getProductName());
        factoryBean.setTablePrefix("TMP_");
        factoryBean.setIsolationLevelForCreate("ISOLATION_REPEATABLE_READ");
        factoryBean.setDataSource(this.dataSource);
        factoryBean.afterPropertiesSet();
        return factoryBean.getObject();
    }

Here you autowire in the DataSource repoDataSource, then the createJobRepository() is overwritten, which the DefaultBatchConfigurer uses to create the JobRepository. Instead of default BATCH_ it uses a table prefix TMP_ and instead of default ISOLATION_SERIALIZED transaction isolation level is set to ISOLATION_REPEATABLE_READ.

Spring doesn't call any of the create-methods in DefaultBatchConfigurer directly as bean definitions, so InitializingBean afterPropertiesSet() and FactoryBean getObject() have to be called explicitely.

If you really don't want to have Spring Batch do a big part of its job and help batch management with its meta-data, your question is a possible duplicate of Spring-Batch without persisting metadata to database? .