How to run Spring Batch Jobs in certain order (Spring Boot)?

Question

I'm developing with Spring Batch using Spring Boot.

I'm with the minimal configuration provided by Spring Boot and defined some Jobs (no XML configuration at all). But when I run the application,

SpringApplication.run(App.class, args);

the jobs are sequentially executed in some arbitrary order.

I'm defining the jobs this way in @Configuration annotated classes, Spring do the rest:

@Bean
public Job requestTickets() {
    return jobBuilderFactory.get(Config.JOB_REQUEST_TICKETS)
            .start(stepRequestTickets())
            .build();
}

How can I instruct the framework to run the jobs in a certain order?

EDIT: Could this warning give a hint? (Maybe has nothing to be)

2016-12-29 17:45:33.320  WARN 3528 --- [main] o.s.b.c.c.a.DefaultBatchConfigurer: No datasource was provided...using a Map based JobRepository

So you have one job with multiple steps or multiple jobs? Seems like, you have only one job. Do you mean **steps** when you say **jobs**? Show code for `stepRequestTickets()`. — Sabir Khan, Dec 29 '16 at 08:02
I have multiple JOBS. I want to run jobs sequentially in a fixed order, but can't find the way to order jobs. This is just an snippet of how I define a Job. — Gerard Bosch, Dec 29 '16 at 16:40

Sabir Khan · Accepted Answer · 2017-01-03T07:35:03.117

1.You first disable automatic job start by specifying spring.batch.job.enabled=false in application.properties

2.In your main class, do - ApplicationContext ctx = SpringApplication.run(SpringBatchMain.class, args); assuming your main class is named - SpringBatchMain.java.

This will initialize context without starting any jobs.

3.Once context is initialized, either you can do - JobLauncher jobLauncher = (JobLauncher) ctx.getBean("jobLauncher"); or do Autowired for this JobLauncher bean in main class and launch specific jobs sequentially in specific sequential order by invoking , jobLauncher.run(job, jobParameters).

You can get specific job instances from context initialized at step # 2.

You can always use any ordered collection to put your jobs there and launch jobs by iterating over that collection.

4.This above technique works as long as your JobLauncher is configured to be synchronous i.e. main thread waits for jobLauncher.run() call to complete and that is default behavior of jobLauncher.

If you have defined your jobLauncher to use AsyncTaskExecutor then jobs will be started in parallel and sequential ordering will not be maintained.

Hope it helps !!

EDIT:

I was experimenting with @Order annotation as pointed by Stephane Nicoll and it seems to help only in creating an Ordered collection of jobs and that you can iterate and launch jobs in that order.

This below component gives me jobs in Order specified ,

@Component
public class MyJobs {
    @Autowired
    private List<Job> jobs;

    public List<Job> getJobs() {
        return jobs;
    }
}

and I can do , MyJobs myJobs = (MyJobs) ctx.getBean("myJobs"); in main class provided bean is defined,

@Bean
    public MyJobs myJobs() {
        return new MyJobs();
    }

I can iterate over myJobs and launch jobs in that order as specified by @Order annotation.

It worked out! But an extra Q: Where do you place your `@Bean`code? If I place it in `main` method it works fine. But if I place it inside `@Component` I get a Spring error: `BeanDefinitionStoreException: Invalid bean definition with name 'myJobs' defined in class path resource [.../MyJobs.class]: factory-bean reference points back to the same bean definition` — Gerard Bosch, Jan 03 '17 at 11:46
I found out that the `@Bean` definition is not necessary. I guess `@Component` already creates the required bean. — Gerard Bosch, Jan 03 '17 at 11:49
I mean, you need to go with Order if you have too many jobs and its difficult to manage. For one or two jobs, you can simply go with manual launch of job without using Order annotation. — Sabir Khan, Jan 03 '17 at 12:01
I manage it with `@Order` and iterating over a list of jobs to run jobs. But I didn't need the `@Bean myJobs()` code to be declared. I could make `ctx.getBean(MyJobs.class).getJobs()` without that @Bean declared, just with the `@Component` part. Thx to make it work :) — Gerard Bosch, Jan 03 '17 at 15:55

score 3 · Answer 2 · edited Jan 02 '17 at 10:56

3

Order them.

@Bean
@Order(42)
public Job requestTickets() {
    return jobBuilderFactory.get(Config.JOB_REQUEST_TICKETS)
            .start(stepRequestTickets())
            .build();
}

See the javadoc of @Order for more details.

edited Jan 02 '17 at 10:56

Sabir Khan

9,826
7
45
98

answered Dec 28 '16 at 14:58

Stephane Nicoll

31,977
9
97
89

`@Order` didn't worked out. I inserted the annotation exactly as you suggest with `@Order(1)`, `@Order(2)`, `@Order(3)` to my 3 jobs, but they are executed like before. – Gerard Bosch Dec 29 '16 at 16:46
Then you're not using the job launcher. Or you are using a very old Spring Boot version. – Stephane Nicoll Dec 30 '16 at 08:56
@StephaneNicoll: What do you mean by **Then you're not using the job launcher**. `jobLauncher.run(job, jobParameters)` runs specific jobs so its like manual ordering. I tested @Order too and its not working as described in your answer. I am using Spring Boot 1.4.0. Not sure if OP has any update too? – Sabir Khan Jan 03 '17 at 05:39
For me whatever `Job` bean is defined first in `@Configuration` class, that is executed first irrespective of `Order` specified, – Sabir Khan Jan 03 '17 at 05:43

score 3 · Answer 3 · answered Sep 19 '18 at 13:46

Here is an illustration of the solution.

This is so weird, it looks like we're hacking the process.

spring.batch.job.enabled=false

@SpringBootApplication
@EnableBatchProcessing
public class MyApplication {

    public static void main(String[] args)
            throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {

        ConfigurableApplicationContext ctx = SpringApplication.run(MyApplication.class, args);
        JobLauncher jobLauncher = (JobLauncher) ctx.getBean("jobLauncher");
        Job job1= (Job) ctx.getBean("job1");
        Job job2= (Job) ctx.getBean("job2");
        jobLauncher.run(job1,new JobParameters());
        jobLauncher.run(job2,new JobParameters());
    }

}

score 1 · Answer 4 · answered Dec 30 '16 at 09:00

I don't have enough rep to comment. But have you tried just to manually launch your jobs in the order you want?

You need to set spring.batch.job.enabled=false in your application.properties, so that your jobs are not run automatically.

Then just use a launcher to launch your jobs in the order you want.

@RunWith(SpringRunner.class)
@SpringBootTest(classes = { TestConfiguration.class, TestDataSourceConfiguration.class, TestBatchConfig.class })
public class JobOrderTest {

    @Autowired
    JobLauncher jobLauncher;

    @Mock
    Job firstJob;

    @Mock
    Job secondJob;

    @Mock
    Job thirdJob;

    @Mock
    JobParametersValidator jobParametersValidator;

    @Test
    public void jobInOrderTest() throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException {

        when(firstJob.getName()).thenReturn(UUID.randomUUID().toString());
        when(secondJob.getName()).thenReturn(UUID.randomUUID().toString());
        when(thirdJob.getName()).thenReturn(UUID.randomUUID().toString());
        when(firstJob.getJobParametersValidator()).thenReturn(jobParametersValidator);
        when(secondJob.getJobParametersValidator()).thenReturn(jobParametersValidator);
        when(thirdJob.getJobParametersValidator()).thenReturn(jobParametersValidator);

        jobLauncher.run(firstJob, new JobParameters());
        jobLauncher.run(secondJob, new JobParameters());
        jobLauncher.run(thirdJob, new JobParameters());
    }

}

Here is the output

2016-12-30 09:48:36.457  INFO 144860 --- [cTaskExecutor-1] o.s.b.c.l.support.SimpleJobLauncher      : Job: [firstJob] launched with the following parameters: ...
2016-12-30 09:48:36.457  INFO 144860 --- [cTaskExecutor-1] o.s.b.c.l.support.SimpleJobLauncher      : Job: [firstJob] completed with the following parameters: ...
2016-12-30 09:48:36.478  INFO 144860 --- [cTaskExecutor-2] o.s.b.c.l.support.SimpleJobLauncher      : Job: [secondJob] launched with the following parameters: ...
2016-12-30 09:48:36.478  INFO 144860 --- [cTaskExecutor-2] o.s.b.c.l.support.SimpleJobLauncher      : Job: [secondJob] completed with the following parameters: ...
2016-12-30 09:48:36.508  INFO 144860 --- [cTaskExecutor-3] o.s.b.c.l.support.SimpleJobLauncher      : Job: [thirdJob] launched with the following parameters: ...
2016-12-30 09:48:36.508  INFO 144860 --- [cTaskExecutor-3] o.s.b.c.l.support.SimpleJobLauncher      : Job: [thirdJob] completed with the following parameters: ...

score -1 · Answer 5 · answered Dec 28 '16 at 15:55

if your one job is dependent on the second and so on, then do something like this.

@Configuration
@EnableBatchProcessing
@Import(DataSourceConfiguration.class)
public class AppConfig {

    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Bean
    public Job job(@Qualifier("step1") Step step1, @Qualifier("step2") Step step2) {
        return jobs.get("myJob").start(step1).next(step2).build();
    }

    @Bean
    protected Step step1(ItemReader<Person> reader, ItemProcessor<Person, Person> processor, ItemWriter<Person> writer) {
        return steps.get("step1")
            .<Person, Person> chunk(10)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .build();
    }

    @Bean
    protected Step step2(Tasklet tasklet) {
        return steps.get("step2")
            .tasklet(tasklet)
            .build();
    }
}

I think your are talking about ordering steps, but what I need is to run JOBS in a certain order, as `Job_n` depends on `Job_n-1`. — Gerard Bosch, Dec 29 '16 at 16:42

How to run Spring Batch Jobs in certain order (Spring Boot)?

5 Answers5

Linked