Camel bindy marshal to file creates multiple header row

Question

I have the following camel route:

from(inputDirectory)
  .unmarshal(jaxb)
  .process(jaxb2CSVDataProcessor)
  .split(body()) //because there is a list of CSVRecords
  .marshal(bindyCsvDataFormat)
  .to(outputDirectory); //appending to existing file using "?autoCreate=true&fileExist=Append"

for my CSV model class I am using annotations:

@CsvRecord(separator = ",", generateHeaderColumns = true)
...

and for properties

@DataField(pos = 0)
...

My problem is that the headers are appended every time a new csv record is appended.

Is there a non-dirty way to control this? Am I missing anything here?

is the headers appended for every record or per file consumed? — dimitrisli, Dec 15 '16 at 00:18
That is expected because you call the marshal in the splitter. Try without the splitter. — Claus Ibsen, Dec 15 '16 at 15:57
Well I have to use a splitter as each input file will be producing multiple CSV Records I am afraid.. — theo, Dec 15 '16 at 16:15
@ClausIbsen you were right, bindy will handle the list of CsvRecords when I remove the splitter. The problem still remains though.. Since I am appending to the same file (to endpoint), everytime a input a file (from endpoint) bindy ends up recreating the headers.. — theo, Dec 16 '16 at 09:05
You then need to skip the first line if the file exists, you would need to add some logic yourself to figure this out — Claus Ibsen, Dec 16 '16 at 09:57

score 1 · Answer 1 · answered Mar 20 '19 at 09:03

I made a work around which is working quite nicely, creating the header by querying the columnames of the @DataField annotation. This is happening once the first time the file is written. I wrote down the whole solution here:

How to generate a Flat file with header and footer using Camel Bindy

score 0 · Accepted Answer · answered Dec 16 '16 at 15:18

0

I ended up adding a processor that checks if the csv file exists just before the "to" clause. In there I do a manipulation of the byte array and remove the headers.

answered Dec 16 '16 at 15:18

theo

915
1
9
28

Stephen Lawson · Answer 3 · 2021-06-02T07:28:12.123

Hope this helps anyone else. I needed to do something similar where after my first split message I wanted to supress the header output. Here is a complete class (the 'FieldUtils' is part of the apache commons lib)

package com.routes;

import java.io.OutputStream;

import org.apache.camel.Exchange;
import org.apache.camel.dataformat.bindy.BindyAbstractFactory;
import org.apache.camel.dataformat.bindy.BindyCsvFactory;
import org.apache.camel.dataformat.bindy.BindyFactory;
import org.apache.camel.dataformat.bindy.FormatFactory;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;
import org.apache.commons.lang3.reflect.FieldUtils;

public class StreamingBindyCsvDataFormat extends BindyCsvDataFormat {

    public StreamingBindyCsvDataFormat(Class<?> type) {
        super(type);
    }

    @Override
    public void marshal(Exchange exchange, Object body, OutputStream outputStream) throws Exception {
        final StreamingBindyModelFactory factory = (StreamingBindyModelFactory) super.getFactory();
        final int splitIndex = exchange.getProperty(Exchange.SPLIT_INDEX, -1, int.class);
        final boolean splitComplete = exchange.getProperty(Exchange.SPLIT_COMPLETE, false, boolean.class);

        super.marshal(exchange, body, outputStream);

        if (splitIndex == 0) {
        factory.setGenerateHeaderColumnNames(false); // turn off header generate after first exchange
        } else if(splitComplete) {
        factory.setGenerateHeaderColumnNames(true); // turn on header generate when split complete
        }
    }

    @Override
    protected BindyAbstractFactory createModelFactory(FormatFactory formatFactory) throws Exception {
        BindyCsvFactory bindyCsvFactory = new StreamingBindyModelFactory(getClassType());
        bindyCsvFactory.setFormatFactory(formatFactory);
        return bindyCsvFactory;
    }

    public class StreamingBindyModelFactory extends BindyCsvFactory implements BindyFactory {

        public StreamingBindyModelFactory(Class<?> type) throws Exception {
            super(type);
        }

        public void setGenerateHeaderColumnNames(boolean generateHeaderColumnNames) throws IllegalAccessException {
            FieldUtils.writeField(this, "generateHeaderColumnNames", generateHeaderColumnNames, true);
        }

    }

}

Camel bindy marshal to file creates multiple header row

3 Answers3