2

I'm using camel with bindy (2.16.0) to parse a csv file. The file contains a header and a footer. Both are used as metadata to describe common data for all other records. (Customer defined so I cant change the CSV Format)

Im using Bindy to parse the data for me. The issue that I'm having is that for CSVRecord you cannot exclude the footer. I'm able to skip the header but the parsing of the data fails because it cannot parse the footer since the data format is different.

Is there a way to exclude the last line/footer from CSVRecord bindy or maybe have camel read and remove the last line in another way?

improvdev
  • 21
  • 3

2 Answers2

1

No this is currently not supported in bindy. Is the footer in your case a single line only? Or does it have any special leading marker to indicate its a footer?

We could maybe improve bindy to support skipping footer. So maybe a footer by default is just the last line. But just wonder if people may have multiple lines as footer?

You would need to manually remove that last line yourself before parsing it with bindy. If the file is not big, and you can have it in memory you can use a Camel processor / bean and remove the last line from the message body.

Claus Ibsen
  • 56,060
  • 7
  • 50
  • 65
  • 1
    It does have the first column as a descriptor of what the data is aka F for footer, H for header and D for data. For my case the footer is only 1 line. The footer only states how many records are in the entire data set. I would assume that you would only need one line for footer. I think if bindy could parse the header information and the footer information and give access to that information, that would be very helpful. It seems that is how fixedLengthRecord works. Although, maybe having a footer and header is out of scope of how csv files should be constructed. – improvdev Oct 30 '15 at 20:22
  • Yeah there is many different ways this has been done. So maybe bindy needs a spi plugin where you can configure what is a header, data, and footer. So you can do isHeader(line) => line.startsWith("H,") ... and so on – Claus Ibsen Oct 31 '15 at 08:06
0

You can consider an alternate solution:

  • Take the csv from customer
  • preprocess csv through a cleanup step: parse header/fooder and get it out of the way. But collect the common parameters so you can enrich other objects.
  • Parse the rest of the "clean csv" through bindy, and then enrich with header/footer common
Espresso
  • 5,378
  • 4
  • 35
  • 66