1

I'm using a spring batch to read a CSV file and write it to the DB, using the controller trigger. On starting the application, before I hit from the browser url, I see the print statements from my reader, on the startup. Although it doesn't print it for my processor or writer, which are in separate classes which I have autowired. Is it because the reader is a bean?

I see the print statements from my FlatFileItemReader in the log on the application startup. But the print statements for my processor and writer only show up in the console when I hit the controller url. I've tried adding spring.batch.job.enabled=false in the application.properties file, but it doesnt stop the execution of the reader bean. How can I prevent auto execution of the reader bean in the SpringBatchConfig class:

SpringBatchConfig class:

@Configuration
@EnableBatchProcessing
public class SpringBatchConfig {
    
    @Autowired
    private JobBuilderFactory jobBuilderFactory;
    
    @Autowired
    private StepBuilderFactory stepBuilderFactory;
    
    @Autowired
    private DataSource dataSource;
    
    @Autowired
    private DBWriter writer1;
    
    @Autowired
    private Processor processor1;
    
    //Step 1 - CSV to DB
    @Bean
    public FlatFileItemReader<User> itemReader() {

        FlatFileItemReader<User> flatFileItemReader = new FlatFileItemReader<>();
        flatFileItemReader.setResource(new FileSystemResource("src/main/resources/users.csv"));
        flatFileItemReader.setName("CSV-Reader");
        flatFileItemReader.setLinesToSkip(1);
        flatFileItemReader.setLineMapper(lineMapper());
        System.out.println("inside file reader 1 !!!!!");
        return flatFileItemReader;
    }

    @Bean
    public LineMapper<User> lineMapper() {

        DefaultLineMapper<User> defaultLineMapper = new DefaultLineMapper<>();
        DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();

        lineTokenizer.setDelimiter(",");
        lineTokenizer.setStrict(false);
        lineTokenizer.setNames(new String[]{"id", "name", "dept", "salary"});

        BeanWrapperFieldSetMapper<User> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
        fieldSetMapper.setTargetType(User.class);

        defaultLineMapper.setLineTokenizer(lineTokenizer);
        defaultLineMapper.setFieldSetMapper(fieldSetMapper);

        return defaultLineMapper;
    }

    @Bean
    public Step step1() throws Exception{   // Step 1 - Read CSV and Write to DB
        return stepBuilderFactory.get("step1")
                .<User,User>chunk(100)
                .reader(itemReader())
                .processor(processor1)
                .writer(writer1)
                .build();
    }

   @Bean
    public Job job() throws Exception{
        return this.jobBuilderFactory.get("BATCH JOB")
                .incrementer(new RunIdIncrementer())
                .start(step1())
                .build();
    }

DBWriter class:

@Component
public class DBWriter implements ItemWriter<User> {

    @Autowired
    private UserRepository userRepository;

    @Override
    public void write(List<? extends User> users) throws Exception {
    System.out.println("Inside DB Writer");
        System.out.println("Data Saved for Users: " + users);
            userRepository.save(users);
    }
}

Processor class:

@Component
public class Processor implements ItemProcessor<User, User> {

    private static final Map<String, String> DEPT_NAMES =
            new HashMap<>();

    public Processor() {
        DEPT_NAMES.put("001", "Technology");
        DEPT_NAMES.put("002", "Operations");
        DEPT_NAMES.put("003", "Accounts");
    }

    @Override
    public User process(User user) throws Exception {
        String deptCode = user.getDept();
        String dept = DEPT_NAMES.get(deptCode);
        user.setDept(dept);
        user.setTime(new Date());
        System.out.println(String.format("Converted from [%s] to [%s]", deptCode, dept));
        return user;
    }
}

Controller Class:

@RestController
@RequestMapping("/load")
public class LoadController {

    @Autowired
    JobLauncher jobLauncher;

    @Autowired
    Job job;

    @GetMapping("/users")
    public BatchStatus load() throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {


        Map<String, JobParameter> maps = new HashMap<>();
        maps.put("time", new JobParameter(System.currentTimeMillis()));
        JobParameters parameters = new JobParameters(maps);
        JobExecution jobExecution = jobLauncher.run(job, parameters);

        System.out.println("JobExecution: " + jobExecution.getStatus());

        System.out.println("Batch is Running...");
        while (jobExecution.isRunning()) {
            System.out.println("...");
        }

        return jobExecution.getStatus();
    }
}
CtrlAltElite
  • 487
  • 8
  • 32

1 Answers1

3

The spring.batch.job.enabled=false property is used to prevent running jobs at application startup.

The method that creates the reader will be still be called at configuration time, so it's normal that you see the print statement. But that does not mean the reader was called inside a running job.

Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50
  • Fair. Also, the same happens with the logger statements. The ones in the reader wont print when I hit the url. The ones in processor and writer do. Is there a way I can have them all print on triggering through the url? – CtrlAltElite Sep 30 '20 at 12:01
  • Also I am seeing those statements before the startup before the ```Tomcat started at port 8080``` and ```Started ApplicationName in 8 seconds``` logs – CtrlAltElite Sep 30 '20 at 12:10
  • 1
    Print statements in processor/writer are in the code that will be executed when the job is actually launched when you hit the url (ie the `process` and `write` methods). The print statement that you put in the reader bean definition method will only be printed at configuration time when creating the singleton bean `itemReader`. If you remove `@Component` on processor/writer and create bean definition methods for them with print statements in those bean definition methods you will see them at configuration time as well just like the reader. – Mahmoud Ben Hassine Sep 30 '20 at 12:42
  • Noted. Thanks. I just saw your video "High performance Batch processing". Big fan of your work. Also, can you have a look at my other question? https://stackoverflow.com/questions/64094079/spring-batch-invalid-object-name-batch-job-instance – CtrlAltElite Oct 01 '20 at 14:48
  • Thank you! I will try to help on the other question. – Mahmoud Ben Hassine Oct 01 '20 at 14:55
  • Also, if I annotate the reader bean with @StepScope, the print and log statements, arent printed at the configuration time but while execution – CtrlAltElite Oct 05 '20 at 10:59
  • 1
    `@StepScope` means the bean will only be created at runtime when the step is started. This is used for [late binding](https://docs.spring.io/spring-batch/docs/4.2.x/reference/html/step.html#late-binding) of job parameters and attributes from the execution context which are not known at configuration time. – Mahmoud Ben Hassine Oct 05 '20 at 12:22