3

I was working in a project with CsvHelper in Visual Studio 2015. It worked fine. Then I switched to Visual Studio 2013 to do some test and CsvHelper worked very slow. When in VS 2015 takes 2 seconds at maximum to read a large file in VS2013 takes more than 5 minutes.

I test the same project opened in VS2013 and VS2015. In VS2013 runs slow, in VS2015 the opposite. The same project.

So, VS2013 must be doing something to cause the CsvHelper slow speed. Any ideas?

EDIT: To clarify, I run every test in debug mode.

I add my actual reading function:

    internal void ReadInCSVPoint3DNew(string absolutePath)
    {
        CultureInfo Culture = new CultureInfo("en-US");
        List<Point3D> result = new List<Point3D>();
        string value;
        using (TextReader fileReader = File.OpenText(absolutePath))
        {
            var csv = new CsvReader(fileReader);
            csv.Configuration.HasHeaderRecord = false; // The head is not in good format, so, setting it true doesn't work.
            csv.Read(); // Skip Head
            while (csv.Read())
            {
                string[] Strings = new String[13];
                for (int i = 0; csv.TryGetField<string>(i, out value); i++)
                {
                    Strings[i] = value;
                }

                Point3D point = new Point3D()
                {
                    PointX = (decimal)float.Parse(Strings[0], Culture),
                    PointY = (decimal)float.Parse(Strings[1], Culture),
                    PointZ = (decimal)float.Parse(Strings[2], Culture),
                    X = (decimal)float.Parse(Strings[3], Culture),
                    Y = (decimal)float.Parse(Strings[4], Culture),
                    Z = (decimal)float.Parse(Strings[5], Culture),
                    Intensity = (int)float.Parse(Strings[6], Culture),
                    LaserIndex = (int)float.Parse(Strings[7], Culture),
                    Azimuth = (int)float.Parse(Strings[8], Culture),
                    Distance = (decimal)float.Parse(Strings[9], Culture),
                    AdjustTime = (long)float.Parse(Strings[10], Culture),
                    TimeStamp = (long)float.Parse(Strings[11], Culture),
                    VerticalAngle = (int)float.Parse(Strings[12], Culture)
                };
                result.Add(point);
            }
        }
        this.Data = result;
    }
Ángel Javier Mena Espinosa
  • 275
  • 2
  • 19
  • 2
    Is it the same in debug mode vs release? Is the framework version the same? If it isn't the debugger slowing things down, then I doubt Visual Studio itself will have anything to do with the problem. – ProgrammingLlama Jun 07 '18 at 07:56
  • 1
    The same project with the same properties in the same mode. – Ángel Javier Mena Espinosa Jun 07 '18 at 07:59
  • 1
    Well any [mcve] ? Any bench that narrow down the issue to a block of code ? Because for now it's not even enought information to describe the issue – Drag and Drop Jun 07 '18 at 08:05
  • 1
    OK, but is it in debug or release? Point being, the debugger in the two versions could be different, so can you compare the speed in release mode? – ProgrammingLlama Jun 07 '18 at 08:06
  • @ajmena how can you be sure? There are compiler, debugger, Framework, NuGet *and* performance differences between an old version like 2013 and the old but not-yet-unsupported VS 2015. Why did you downgrade in the first place? Why aren't you using the *current* version? – Panagiotis Kanavos Jun 07 '18 at 08:39
  • I tested in debug mode. Now I tested in release too and same result. 1-2 seconds in VS2015 vs 5-6 minutes in VS2013. – Ángel Javier Mena Espinosa Jun 07 '18 at 08:41
  • 2
    @ajmena besides, instead of assuming that the IDE is at fault, *profile* your application. "Same result" with completely different IDEs, packages etc doesn't mean anything. – Panagiotis Kanavos Jun 07 '18 at 08:41
  • 1
    Besides, the code has a few bugs that will affect performance in older *garbage collectors*. You are constructing a new empty string array and throwing it away in every line. That's 1 orphan object to collect on every line. You are reading cell values as strings instead of eg decimals in a loop and *then* parsing them. That's more waste - CsvHelper can handle the reading and parsing itself without temporary strings. Bad parsing and casting too `(decimal)float.Parse` why? – Panagiotis Kanavos Jun 07 '18 at 08:45
  • 1
    Finally, you can't benchmark code by timing a single or even multiple executions. Most likely you are measuring the GC performance of an older framework version. Use a program like BenchmarkDotNet that measures *real* execution times, allocations, GC times etc and makes enough measurements to ensure the results are accurate. – Panagiotis Kanavos Jun 07 '18 at 08:48
  • The project in VS2015 is a class library for external application of different customers. One of these application is in VS2013. I didn't think it would be conflict. The speed problem arise testing the dll. – Ángel Javier Mena Espinosa Jun 07 '18 at 08:49
  • I use "(decimal)float.Parse" because if I use only "decimal.Parse" with values like "2.979207395260004e-15" crashes. I need a decimal; otherwise further float loses precision in some operations that I do. If is another way I would like to know. – Ángel Javier Mena Espinosa Jun 07 '18 at 09:08
  • So do a binary search and narrow down what part of your code is taking that long by commenting out some lines. – MineR Jun 07 '18 at 09:17
  • 1
    @ajmena `decimal.Parse("2.979207395260004e-15",System.Globalization.NumberStyles.Float)` as [shown here](https://stackoverflow.com/questions/3879463/parse-a-number-from-exponential-notation). BTW applications are *NOT* "in VS 2013". They target specific frameworks. You can use the same project with multiple IDEs although that doesn't answer the question `Why VS 2013?`. – Panagiotis Kanavos Jun 07 '18 at 09:29
  • @ajmena `The speed problem arise testing the dll.` so you measured the *test runner's performance* instead of the actual code? Which testing framework, runner and *versions* did you use? How did you measure performance? You still haven't explained what you did except saying "it's the same" which is clearly not the case – Panagiotis Kanavos Jun 07 '18 at 09:30
  • 1
    @ajmena of course the obvious solution is to open that one last project in Visual Studio 2015 or better yet, 2017 and be done with it. The dlls will work the same in every IDE. It's not the IDE's version that determines assembly compatibility – Panagiotis Kanavos Jun 07 '18 at 09:32
  • You were right about the "garbage collectors". I removed the array string and know runs at same speed in VS2013 and VS2015. Thank you very much. – Ángel Javier Mena Espinosa Jun 07 '18 at 09:46
  • BTW decimal.Parse("2.979207395260004e-15",System.Globalization.NumberStyles.Float) doesn't work. Input string was not in correct format. – Ángel Javier Mena Espinosa Jun 07 '18 at 09:54
  • System.Globalization.NumberStyles.Float works, sorry. In my case I need to add the culture. – Ángel Javier Mena Espinosa Jun 07 '18 at 10:02

1 Answers1

1

The problem was in the code. Panagiotis Kanavos pointed out:

Blockquote Besides, the code has a few bugs that will affect performance in older garbage collectors. You are constructing a new empty string array and throwing it away in every line. That's 1 orphan object to collect on every line.

I removed the string array from the code and now works at same speed in VS2013 and VS2015. I changed the parses too. The code now looks like this:

    internal void ReadInCSVPoint3DNew(string absolutePath)
    {
        CultureInfo Culture = new CultureInfo("en-US");
        List<Point3D> result = new List<Point3D>();
        using (TextReader fileReader = File.OpenText(absolutePath))
        {
            var csv = new CsvReader(fileReader);
            csv.Configuration.HasHeaderRecord = false; // The head is not in good format, so, setting it true doesn't work.
            csv.Read(); // Skip Head
            while (csv.Read())
            {
                Point3D point = new Point3D()
                {
                    PointX = decimal.Parse(csv.GetField(0), System.Globalization.NumberStyles.Float, Culture),
                    PointY = decimal.Parse(csv.GetField(1), System.Globalization.NumberStyles.Float, Culture),
                    PointZ = decimal.Parse(csv.GetField(2), System.Globalization.NumberStyles.Float, Culture),
                    X = decimal.Parse(csv.GetField(3), System.Globalization.NumberStyles.Float, Culture),
                    Y = decimal.Parse(csv.GetField(4), System.Globalization.NumberStyles.Float, Culture),
                    Z = decimal.Parse(csv.GetField(5), System.Globalization.NumberStyles.Float, Culture),
                    Intensity = int.Parse(csv.GetField(6), System.Globalization.NumberStyles.Float, Culture),
                    LaserIndex = int.Parse(csv.GetField(7), System.Globalization.NumberStyles.Float, Culture),
                    Azimuth = int.Parse(csv.GetField(8), System.Globalization.NumberStyles.Float, Culture),
                    Distance = decimal.Parse(csv.GetField(9), System.Globalization.NumberStyles.Float, Culture),
                    AdjustTime = long.Parse(csv.GetField(10), System.Globalization.NumberStyles.Float, Culture),
                    TimeStamp = long.Parse(csv.GetField(11), System.Globalization.NumberStyles.Float, Culture),
                    VerticalAngle = int.Parse(csv.GetField(12), System.Globalization.NumberStyles.Float, Culture)
                };
                result.Add(point);
            }
        }
        this.Data = result;
    }

Thanks Panagiotis Kanavos for the great help.

Ángel Javier Mena Espinosa
  • 275
  • 2
  • 19