Azure webjob with Linq To CSV doesn't work in production

Question

We are importing a .csv file with around 80.000 rows. We would like to do it on a webjob because it will be repeated in the future. We are using LinqToCsv to read in the .csv and process the information. When running locally in our Console app everything seems to be working fine. But once we deploy to the actual Azure WebJob we get an exception from LinqToCsv.

There were 1 or more exceptions while reading data using type "xxxx". Reading file "xxx.csv".

It appears it can't parse some rows in the file correctly but it does work locally. When shortening the .CSV file to 10 rows everything seems to work in production as well. So I assume nothing is wrong with the actually LinqToCSV class setup. These are the settings btw:

 CsvFileDescription inputFileDescription = new CsvFileDescription
        {
            SeparatorChar = ';',
            FirstLineHasColumnNames = true,
            IgnoreUnknownColumns = true,
            EnforceCsvColumnAttribute = true
        };

Could someone explain this strange behaviour and how to possibility solve it without testing the .CSV row by row?

Do you have any invalid characters somewhere in the dataset? — MikeDub, Mar 20 '16 at 23:06
Can't really test that since it's working perfectly fine locally. Can't really check every row :s it's 80.000 rows long. — Kevin Cloet, Mar 20 '16 at 23:21
Sorry I meant, either do a search(ctrl + F) in the worksheet through excel or write a quick local function that checks for a given set of characters that could be invalid, I only bring this up as one of our production apps has an issue with " characters and csv uploads. Alternatively have you tried it with a different type of azure offering (ie. web app)? — MikeDub, Mar 21 '16 at 00:04
It‘s probably a problem of encodings. It reads the file in one encoding locally, while reads the file in another encoding on the server. Save your file as UTF-8, and try again. — Jack Zeng, Mar 21 '16 at 01:45
Problem solved. The problem was decimal parsing because globalization issues — Kevin Cloet, Mar 21 '16 at 08:33

score 0 · Accepted Answer · answered Mar 21 '16 at 08:32

0

The problem was globalization. Locally my decimals were comma separated and production was a dot separation. Adding the following to the CsvFileDescription fixed my issue:

FileCultureName = "nl"

answered Mar 21 '16 at 08:32

Kevin Cloet

2,956
1
19
36

Azure webjob with Linq To CSV doesn't work in production

1 Answers1