1

I'm processing multiple input files with multi-format lines using ClassifierCompositeItemProcessor. But when using StepBuilderFactory stream to write the files, I'm unable to pass the Resource filename dynamically. Filename should be the respective input file name. Any help would be much appreciated.

Input File 1 (data-111111-12323.txt)

1#9999999#00001#2#RecordType1
2#00002#June#Statement#2020#9#RecordType2
3#7777777#RecordType3

Input File 2 (data-22222-23244.txt)

1#435435#00002#2#RecordType1
2#345435#July#Statement#2021#9#RecordType2
3#645456#RecordType3

Expected output file 1 (data-111111-12323.txt)

1#9999999#00001#2#RecordType1#mobilenumber1
2#00002#June#Statement#2020#9#RecordType2#mobilenumber2
3#7777777#RecordType3#mobilenumber3

Expected output file 2 (data-22222-23244.txt)

1#9999999#00001#2#RecordType1#mobilenumber1
2#00002#June#Statement#2020#9#RecordType2#mobilenumber2
3#7777777#RecordType3#mobilenumber3

Step

        public Step partitionStep() throws Exception {
            ItemReader reader = context.getBean(FlatFileItemReader.class);
            ClassifierCompositeItemWriter writer = context.getBean(ClassifierCompositeItemWriter.class);
            return stepBuilderFactory.get("statementProcessingStep.slave").<String, String>chunk(12).reader(reader).processor(processor()).writer(writer)
                    .stream(recordType0FlatFileItemWriter())
                    .stream(recordType1FlatFileItemWriter())
                    .build();
                }

Processor

@Bean
    @StepScope
    public ItemProcessor processor() {
            ClassifierCompositeItemProcessor<? extends RecordType, ? extends RecordType> processor = new ClassifierCompositeItemProcessor<>();
SubclassClassifier classifier = new SubclassClassifier();
        Map typeMap = new HashMap();
        typeMap.put(RecordType0.class, recordType0Processor);
        typeMap.put(RecordType1.class, recordType1Processor);
classifier.setTypeMap(typeMap);
        processor.setClassifier(classifier);
return processor;
}

Writer

@Bean
    public FlatFileItemWriter<RecordType1> recordType1FlatFileItemWriter() throws Exception{
        FlatFileItemWriter<RecordType1> writer = new FlatFileItemWriter<>();
        writer.setResource( new FileSystemResource("record1.txt")); //This filename should be dynamic
        writer.setAppendAllowed(true);
        writer.setLineAggregator(new DelimitedLineAggregator<RecordType1>() {{
            setDelimiter("#");
            setFieldExtractor(new BeanWrapperFieldExtractor<RecordType1>() {
                {
                    setNames(new String[] { "RecordType", "ID1", "ID2", "ID3"});
                }
            });
        }});
        return  writer;
    }
pulikuttie
  • 35
  • 5
  • 1
    Are "Input File 1" and "Input File 1" passed as job parameters? If yes, you can create a step-scoped writer and use late-binding to inject the file from job parameters at runtime, see https://docs.spring.io/spring-batch/docs/4.3.x/reference/html/step.html#late-binding – Mahmoud Ben Hassine Jan 20 '21 at 12:39
  • Genius! Just adding a @StepScope worked! Thank you tons!!! – pulikuttie Jan 21 '21 at 01:44
  • @MahmoudBenHassine I had to change the implementation to synchronous batch processing as I had to implement `PeekableItemReader`. So couldn't have Partitioning. In this scenario, `StepScope` didn't work. Since the files are read from a folder, I used `ResourceAware` to set the current resource in Reader. But I'm unable to retrieve the resource name in `ItemWriter`. I've asked a separate question. Could you please have a look? [link](https://stackoverflow.com/questions/66624001/spring-batch-resourceaware-get-resource-name-in-classifiercompositeitemwriter) – pulikuttie Mar 15 '21 at 00:37

1 Answers1

1

You can make your item reader/writer step-scoped and inject values from job parameters or step/job execution context using late-binding. For example:

@StepScope
@Bean
public FlatFileItemReader flatFileItemReader(@Value("#{jobParameters['input.file.name']}") String name) {
   return new FlatFileItemReaderBuilder<Foo>()
        .name("flatFileItemReader")
        .resource(new FileSystemResource(name))
        .build();
}

You can find more details in the Late Binding of Job and Step Attributes section of the reference documentation.

Mahmoud Ben Hassine
  • 28,519
  • 3
  • 32
  • 50