0

I have an SSIS package that loops through hundreds of Apache error log files and writes the contents to a centralized database.

I've noticed that occasionally the log files will contain a malformed row, where it's simply missing a delimiter between two columns and therefore SSIS interprets it as having bad metadata (missing a column) and the package fails.

The current work around is to skip the file entirely, but I'd really like to simply skip the malformed row as there are often thousands of useful rows and just one bad one. Setting OnError and OnTruncation to redirect doesn't help because it's a metadata error that causes the data source component to fail.

Any ideas?

James G
  • 118
  • 11
  • 1
    Check the following answer, it may helps: https://stackoverflow.com/questions/48056021/how-to-skip-a-bad-row-in-ssis-flat-file-source/48067039#48067039 – Hadi Dec 05 '19 at 18:49
  • I had considered this and started to explore, but the SSIS task runs RBAR and is just too slow. Each file has roughly 5k rows and there are a couple hundred of them per 5 minute cycle. Still, I'll upvote because I can see this being a viable solution for a lot of people. – James G Dec 05 '19 at 19:31

0 Answers0