Is there a way to skip persisting metadata for Spring Batch only for particular jobs?

Question

There are a couple of examples of how we could use Spring Batch without persisting metadata to database. Here are some examples and related questions regarding the matter:

• Spring-Batch without persisting metadata to database?

• How to avoid Spring batch persistence of metadata in DB

• Spring Batch - How to prevent batch from storing transactions in DB

• Spring boot + spring batch without DataSource

However I have a slightly different use case: I have some jobs that run every hour or so, of which I would like to persist the metadata into my database (e.g. creating reports, running some tests, both of which might be slightly heavy in processing). I have some other types of jobs that run every minute or so (e.g. unlocking user accounts which are locked due to repeated wrong entry of password, etc.) which do not involve much of processing but a simple sql query.

Here's the question in two parts:

1. Is there a way to keep metadata of the first type of jobs (e.g. reports processing) in the database while not using database persistence at all for second type of jobs (e.g. unlocking user accounts)?

2. Or, would even using Spring Batch for second type of jobs be overkill/not needed at all? Would a method with @Scheduled annotation be enough?

Answer 1

Here's how I achieved what I'm looking for, in regards to the suggestions by Mahmoud Ben Hassine (Removed @EnableBatchProcessing, for some unrelated issue - see here):

I have two configuration classes:

@Configuration
public class SpringBatchConfiguration extends DefaultBatchConfigurer {

    @Inject public SpringBatchConfiguration(DataSource dataSource) {
        super(dataSource);
    }

    @Bean(name = "persistentJobLauncher")
    public JobLauncher jobLauncher() throws Exception {
        return super.createJobLauncher();
    }

    @Bean
    @Primary
    public StepBuilderFactory stepBuilderFactory() {
        return new StepBuilderFactory(super.getJobRepository(), super.getTransactionManager());
    }

    @Bean
    @Primary
    public JobBuilderFactory jobBuilderFactory(){
        return new JobBuilderFactory(super.getJobRepository());
    }

    @Bean
    public JobExplorer jobExplorer() {
        return super.getJobExplorer();
    }

    @Bean
    public JobRepository jobRepository() {
        return super.getJobRepository();
    }

    @Bean
    public ListableJobLocator jobLocator() {
        return new MapJobRegistry();
    }
}

and the in-memory one:

@Configuration
public class SpringInMemoryBatchConfiguration extends DefaultBatchConfigurer {

    @Inject public SpringInMemoryBatchConfiguration() {
    }

    @Bean(name = "inMemoryJobLauncher")
    public JobLauncher inMemoryJobLauncher() throws Exception {
        return super.createJobLauncher();
    }

    @Bean(name = "inMemoryStepBuilderFactory")
    public StepBuilderFactory stepBuilderFactory() {
        return new StepBuilderFactory(super.getJobRepository(), super.getTransactionManager());
    }

    @Bean(name = "inMemoryJobBuilderFactory")
    public JobBuilderFactory inMemoryJobBuilderFactory(){
        return new JobBuilderFactory(super.getJobRepository());
    }
}

and when I want to start a "persistent" job, I use @Qualifier(value = "persistentJobLauncher") JobLauncher launcher and to start an "in-memory" one: @Qualifier(value = "inMemoryJobLauncher") JobLauncher launcher.

Answer 2

1. Is there a way to keep metadata of the first type of jobs (e.g. reports processing) in the database while not using database persistence at all for second type of jobs (e.g. unlocking user accounts)?

This is a matter of configuration. You can use Spring profiles for the datasource. The idea is to define a persistent datasource and an in-memory one. Then configure all your jobs as usual with a datasource and run them with the right profile at runtime. Here is a quick example:

import javax.sql.DataSource;

import com.zaxxer.hikari.HikariDataSource;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType;

@Configuration
@EnableBatchProcessing
public class JobsConfiguration {

    @Bean
    public Job job1() {
        return null;
    }

    @Bean
    public Job job2() {
        return null;
    }

    @Bean
    @Profile("in-memory")
    public DataSource embeddedDataSource() {
        return new EmbeddedDatabaseBuilder()
                .setType(EmbeddedDatabaseType.HSQL)
                .addScript("/org/springframework/batch/core/schema-hsqldb.sql")
                .build();
    }

    @Bean
    @Profile("persistent")
    public DataSource persistentDataSource() {
        HikariDataSource hikariDataSource = new HikariDataSource();
        // TODO configure datasource
        return hikariDataSource;
    }
}

If you run your app with -Dspring.profiles.active=in-memory, there will be only the embedded datasource bean in your application context, which will be used by the job repository automatically configured by @EnableBatchProcessing.

2. Or, would even using Spring Batch for second type of jobs be overkill/not needed at all? Simply a method with @Scheduled annotation would be enough?

which do not involve much of processing but a simple sql query: If it's a simple sql query and you don't need to persist meta-data, then it is probably better to use a method annotated with @Scheduled that uses a jdbcTemplate to run the query.