3

I have a question about the order of processing of csv files with Spring Batch, I have several csv files to process, when I launch the spring batch I do not know in what order spring processes these files.

I would like to know how Spring chose the first file to treat ? and where can I find this setting ?

Is it possible to define a sort ? for example, processing files based on a date written on the file name? and how can I customize the choice of the first files to process?

Thank you all,

Mohcine
  • 57
  • 10

2 Answers2

3

Spring batch's MultiResourceItemReader uses Comparator<Resource> to preserve ordering. If we don't provide comparator, then the default ordering will be based on file name. If you want to give your custom sorting, you can write your own comparator logic like following(sort based on last modified time). 

public class FileModifiedComparator implements Comparator<FileSystemResource>{

        @Override
        public int compare(FileSystemResource file1, FileSystemResource file2) {
       //comparing based on last modified time
            return Long.compare(file1.lastModified(),file2.lastModified());
        }
 }

You can modify the comparator to check modify your sort logic such as file name, created etc. for eg: return file1.getFilename().compareTo(file2.getFilename()); or return Long.compare(file1.contentLength(),file2.contentLength()); and in the MultiResourceItemReader bean, set this comparator.

<bean id="fileModifiedComparator" class="FileModifiedComparator"/> 

<bean id="multiResourceReader"
    class=" org.springframework.batch.item.file.MultiResourceItemReader">
    <property name="resources" value="file:inputs/input*.csv" />
    <property name="delegate" ref="flatFileItemReader" />
    <property name="comparator" ref="fileModifiedComparator" />
</bean>
Binu
  • 754
  • 6
  • 15
0
@Bean
public MultiResourceItemReader<Employee> criminalBackgroundReader() {
    MultiResourceItemReader<Employee> resourceItemReader = new MultiResourceItemReader<Employee>();
    ClassLoader cl = this.getClass().getClassLoader();
    ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(cl);

    Resource[] resources = null;
    try {
        resources = resolver.getResources("file:" + EmployeeInputFilePath);
    } catch (IOException e) {
        log.error("===== Error occurding reading input file directory: ", e);
        return null;
    }


    resourceItemReader.setResources(resources);
    resourceItemReader.setDelegate(flatFileRpReader());
    resourceItemReader.setComparator(new FileComparator());
    resourceItemReader.setStrict(false);
    return resourceItemReader;
}


public class FileComparator implements Comparator<Resource>{
    @Override
    public int compare(Resource file1, Resource file2) {
    //comparing based on File Name compare it to Enum-InputFileType
    return InputFileType.fromString(StringUtils.substringBefore(file1.getFilename(), ".")) .compareTo(InputFileType.fromString(StringUtils.substringBefore(file1.getFilename(),"."));
}
}
KiranV
  • 1