How to read flat file header and body separately in Spring Batch

Question

i'm doing a simple batch job with Spring Batch and Spring Boot.

I need to read a flat file, separate the header data (first line) from the body data (rest of lines) for individual business logic processing and then write everything into a single file.

As you can see, the header has 5 params that have to be mapped to one class, and the body has 12 which have to be mapped to a different one.

I first thought of using FlatFileItemReader and skip the header. Then use the skippedLinesCallback to handle that line, but i couldn't figure out how to do it.

I'm new to Spring Batch and Java Config. If someone can help me writing a solution for my problem i would really aprecciate it!

I leave here the input file:

01.01.2017|SUBDCOBR|12:21:23|01/12/2016|31/12/2016
01.01.2017|12345678231234|0002342434|BORGIA RUBEN|27-32548987-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,01
01.01.2017|12345673201234|2342434|ALVAREZ ESTHER|27-32533987-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,02
01.01.2017|12345673201234|0002342434|LOPEZ LUCRECIA|27-32553387-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,12
01.01.2017|12345672301234|0002342434|SILVA JESUS|27-32558657-9|NC|A|2062-
00010443|142,12|30/08/2017|142,12

Cheers!

EDIT 1: This would be my first attepmt . My "body" POJO is called DetalleFacturacion and my "header" POJO is CabeceraFacturacion. The reader I thought to do it with DetalleFacturacion pojo, so i can skip the header and treat it later... however i'm not sure how to assign header's data into CabeceraFacturacion.

public FlatFileItemReader<DetalleFacturacion> readerDetalleFacturacion(){

    FlatFileItemReader<DetalleFacturacion> reader = new FlatFileItemReader<>();

    reader.setLinesToSkip(1);
    reader.setResource(new ClassPathResource("/inputFiles/GLEO-MN170100-PROCESO01-SUBDFACT-000001.txt"));

    DefaultLineMapper<DetalleFacturacion> detalleLineMapper = new DefaultLineMapper<>();

    DelimitedLineTokenizer tokenizerDet = new DelimitedLineTokenizer("|");
    tokenizerDet.setNames(new String[] {"fechaEmision", "tipoDocumento", "letra", "nroComprobante",  
                                "nroCliente", "razonSocial", "cuit", "montoNetoGP", "montoNetoG3",
                                "montoExento", "impuestos", "montoTotal"});

    LineCallbackHandler skippedLineCallback = new LineCallbackHandler() {

        @Override
        public void handleLine(String line) {

            String[] headerSeparado = line.split("|");

            String printDate = headerSeparado[0];
            String reportIdentifier = headerSeparado[1];
            String tituloReporte = headerSeparado[2];
            String fechaDesde = headerSeparado[3];
            String fechaHasta = headerSeparado[4];

            CabeceraFacturacion cabeceraFacturacion = new CabeceraFacturacion();
            cabeceraFacturacion.setPrintDate(printDate);
            cabeceraFacturacion.setReportIdentifier(reportIdentifier);
            cabeceraFacturacion.setTituloReporte(tituloReporte);
            cabeceraFacturacion.setFechaDesde(fechaDesde);
            cabeceraFacturacion.setFechaHasta(fechaHasta);

        }
    };

    reader.setSkippedLinesCallback(skippedLineCallback);

    detalleLineMapper.setLineTokenizer(tokenizerDet);
    detalleLineMapper.setFieldSetMapper(new DetalleFieldSetMapper());
    detalleLineMapper.afterPropertiesSet();
    reader.setLineMapper(detalleLineMapper);

    // Test to check if it is saving correctly data in CabeceraFacturacion
    CabeceraFacturacion cabeceraFacturacion = new CabeceraFacturacion();
    System.out.println("Print Date:"+cabeceraFacturacion.getPrintDate());
    System.out.println("Report Identif: 
    "+cabeceraFacturacion.getReportIdentifier());

    return reader;

}

Niraj Sonawane · Accepted Answer · 2018-02-27T12:50:47.350

2

You are correct . You need to use skippedLinesCallback to handle skip lines.

You need to implement LineCallbackHandler interface and add you processing in handleLine method.

LineCallbackHandler Interface passes the raw line content of the lines in the file to be skipped. If linesToSkip is set to 2, then this interface is called twice.

This is how you can define Reader for the same.

Java Config - Spring Batch 4

@Bean
    public FlatFileItemReader<POJO> myReader() {

        return FlatFileItemReader<pojo>().
                .setResource(new FileSystemResource("resources/players.csv"));
                .name("myReader")
                .delimited()
                .delimiter(",")
                .names("pro1,pro2,pro3")
                .targetType(POJO.class)
                .skippedLinesCallback(skippedLinesCallback)             
                .build();

    }

edited Feb 27 '18 at 12:50

answered Feb 27 '18 at 05:57

Niraj Sonawane

10,225
10
75
104

Thank you so much Niraj for your answer. This is what i tried, but i still don't know how to store the data of the header in one pojo and the body in another one. Because when i define the FlatFileItemReader i do it with just one pojo inside the <> . – Fede Lopez Feb 27 '18 at 11:58
if you set the linesToSkip to 1 then for that line (Header line) LineCallbackHandler.handleLine method will be get called. Batch Framework will pass that raw line content to that method. then u need to parse that line as per you need. – Niraj Sonawane Feb 27 '18 at 12:40
Perfect. I understand up to that point. I updated the code once more. I read the line with line mapper and tokenizer. Is this the way to read it? Because in the test i did to see if it was saving correctly the data, gave me null for both params. – Fede Lopez Feb 27 '18 at 13:06
I get it. it´s just a raw String line. Then i can use line.split("|") and save the params with set . Thank you very much Niraj! – Fede Lopez Feb 27 '18 at 13:14
You are welcome... could you please accept the answers ?.this will also help others who might face same issue.thanks – Niraj Sonawane Feb 27 '18 at 16:56
Niraj, would you please take a look to my other question? is the continuation of this same project. https://stackoverflow.com/questions/49055709/how-to-return-indefinetely-to-previous-step-when-condition-of-a-following-step-i – Fede Lopez Mar 02 '18 at 20:46

How to read flat file header and body separately in Spring Batch

1 Answers1