Parsing text file separated by comma

Question

I am developing an application to read a text file and plot a graph. the text file will be something like

#Pattern Name Item1, Item2, Item3, gx1, gy1, gz1 
115, 80, 64, 30.752, 27.587, 15.806 
195, 151, 130, 108.983, 102.517, 66.353 
94, 123, 156, 43.217, 50.874, 93.700 
88, 108, 65, 26.158, 37.980, 17.288 
130, 129, 177, 68.096, 66.289, 127.182 
100, 190, 171, 71.604, 119.764, 122.349 
.........................................
........................................
#Pattern Name2 Item1, Item2, Item3, gx1, gy1, gz1 
115, 80, 64, 30.752, 27.587, 15.806 
195, 151, 130, 108.983, 102.517, 66.353 
94, 123, 156, 43.217, 50.874, 93.700 
88, 108, 65, 26.158, 37.980, 17.288 
130, 129, 177, 68.096, 66.289, 127.182 
100, 190, 171, 71.604, 119.764, 122.349

etc.

I need the value gx1,gy1 . I planned to read them by counting spaces and commas (3 each). But Does it sounds in appropriate? Is there any optimized or more good logic for that?

One option is the built in `TextFieldParser` class http://msdn.microsoft.com/en-us/library/microsoft.visualbasic.fileio.textfieldparser.aspx — Alex K., Apr 25 '13 at 09:48
This appears twice in your example file: #Pattern Name2 Item1, Item2, Item3, gx1, gy1, gz1 — Ash, Apr 25 '13 at 09:49
ya they are freequently appear. But we need to take care it separately. — Zigma, Apr 25 '13 at 09:51

score 2 · Answer 1 · answered Apr 25 '13 at 09:43

2

How about using Linq?

var allparts = File.ReadLines(filename)
                .Where(line => !line.StartsWith("#"))
                .Select(line => line.Split(','))
                .Select(parts => new {gx1=parts[3], gy1=parts[4]} ) 
                .ToList();

answered Apr 25 '13 at 09:43

I4V

34,891
6
67
79

3

Bad performance. Read line by line. When data comes in CSV and it's not a student sample then CSV file may have huge sizes – Regfor Apr 25 '13 at 09:45
3

@Regfor `ReadLines` already does it line by line. It is not `File.ReadAllLines` – I4V Apr 25 '13 at 09:47
@I4V so var allparts will be unique for every line and I want 2 values separately. I will check it soon and comment.. Thanks – Zigma Apr 25 '13 at 09:50
@Zigma `allparts` will be a List of an anonymous type which has properties `gx1` and `gy1` (as `allparts[i].gx1`) – I4V Apr 25 '13 at 10:23

score 1 · Answer 2 · answered Apr 25 '13 at 09:44

1

What you have is a simple CSV file. I would recommend either:

1) Using a CSV parsing library (for example http://www.filehelpers.com/)
2) Parsing this yourself on a line by line basis:

String[] lines = File.ReadAllLines(...);
foreach(String line in lines)
{
   String parts[] = line.Split(",", StringSplitOptions.IgnoreEmpty);
   double gx1 = Double.Parse(parts[5]);
   double gx2 = Double.Parse(parts[6]);
}

You will however need to add in some validation / line skipping for your special lines

answered Apr 25 '13 at 09:44

Ian

33,605
26
118
198

@Regfor: Bad performance compare to what? I know it will have an impact on memory, but at some point every line has to be read, all at once or one by one... – Martin Mulder Apr 25 '13 at 09:59
CSV File may be huge. When huge CSV file comes - memory consumption is huge, performance is slow comparing to line be line reading. – Regfor Apr 25 '13 at 10:07

Tim Schmelter · Accepted Answer · 2013-04-26T10:20:53.287

You should use an existing csv-parser like this.

However, assuming that the name of the pattern is unique and you want to acess it later:

var gxDict = new Dictionary<string, List<Tuple<double, double>>>();
List<Tuple<double, double>> currentGxList = null;
foreach (string line in File.ReadLines(filePath))
{
    if (line.StartsWith("#Pattern"))
    {
        string[] headers = line.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
        string patternField = headers.First();
        string patterName = string.Join(" ", patternField.Split().Skip(1).Take(2));
        List<Tuple<double, double>> gxList = null;
        if (gxDict.TryGetValue(patterName, out gxList))
            currentGxList = gxList;
        else
        {
            currentGxList = new List<Tuple<double, double>>();
            gxDict.Add(patterName, currentGxList);
        }
    }
    else
    {
        if (currentGxList != null)
        {
            string[] values = line.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
            double gx1;
            double gy1;
            string gx1Str = values.ElementAtOrDefault(3);
            string gy1Str = values.ElementAtOrDefault(4);
            if (double.TryParse(gx1Str, out gx1) && double.TryParse(gx1Str, out gy1))
            {
                currentGxList.Add(Tuple.Create(gx1, gy1));
            }
        }
    }
}

// now you can access it in this way:

List<Tuple<double, double>> nameItem1Vals = gxDict["Name Item1"];
foreach (var xy in nameItem1Vals)
    Console.WriteLine("gx1: {0} gy1: {1}", xy.Item1, xy.Item2);

// or in a loop:

foreach (var kv in gxDict)
{
    string pattern = kv.Key;
    Console.WriteLine("pattern: {0}", pattern);
    foreach (var xy in nameItem1Vals)
        Console.WriteLine("gx1: {0} gy1: {1}", xy.Item1, xy.Item2);
}

Note that i avoid Linq when it comes to IO where you need event handling on file level and when i try to parse input which could also cause exceptions.

Edit : `if (int.TryParse(gx1Str, out gx1) && int.TryParse(gy1Str, out gy1))` . Also values are double in type as you can see in question — Zigma, Apr 26 '13 at 04:27
I cant acces the values. I need them to pass it to [DataBindXY](http://msdn.microsoft.com/en-IN/library/system.web.ui.datavisualization.charting.datapointcollection.databindxy.aspx) method as x , y. — Zigma, Apr 26 '13 at 10:12
@Zigma: because i've changed the value of the dictionary later from a tuple to a List. Have a look at the edited version now. — Tim Schmelter, Apr 26 '13 at 10:21
This is working even if with some minor modifications. Avoided `Linq`because not so aware about that. — Zigma, Apr 26 '13 at 10:59

score 0 · Answer 4 · answered Apr 25 '13 at 09:45

0

What about reading each line and then line.Split(", ")? Sou you can have:

var items = line.Split(", ");
var gx1 = items[3];
var gy1 = items[4];
var gz1 = items[5];

answered Apr 25 '13 at 09:45

Szellszi

123
1
1
6

Parsing text file separated by comma

4 Answers4

Linked