Writing list to CSV file

Question

I have a List of List of int called NN which i would like to write to a csv file like this:

List<List<int>> NN = new List<List<int>>();

The NN list:

1,2,3,4,5,6
2,5,6,3,1,0
0,9,2,6,7,8

And the output csv file should look like this:

1,2,0
2,5,9
3,6,2
4,3,6
5,1,7
6,0,8

What is the best way to achieve that?

If there is a better representation you would recommend instead of the nested list i'll be glad to know.

(The purpose is that each list of int is the weights between the last and next layer in the neural network).

So basically `NN` is a list of column data, right? – Yacoub Massad May 28 '16 at 12:30 — Yacoub Massad, May 28 '16 at 12:30
Nested `for` loops would a simple way. – Andrew Morton May 28 '16 at 12:34 — Andrew Morton, May 28 '16 at 12:34

score 9 · Accepted Answer · answered May 28 '16 at 12:36

Here is how you can do it:

List<List<int>> NN = new List <List<int>>
{
    new List<int> {1,2,3,4,5,6},
    new List<int> {2,5,6,3,1,0},
    new List<int> {0,9,2,6,7,8}
};

//Get expected number of rows
var numberOfRows = NN[0].Count; 

var rows =
    Enumerable.Range(0, numberOfRows) //For each row
    .Select(row => NN.Select(list => list[row]).ToList()) //Get row data from all columns
    .ToList();

StringBuilder sb = new StringBuilder();

foreach (var row in rows)
{
    sb.AppendLine(string.Join(",", row));
}

var result = sb.ToString();

Regis Portalez · Answer 2 · 2016-05-28T13:33:35.400

What you want to achieve is basically a matrix transpose and then write the data to a file.

What is the most efficient way to transpose a matrix is a complex question, and really depends on your architecture.

If you're not really concerned about super optimizing that for your processor (or accelerator), I would go for a simple nested for loop, accumulating data in intermediate memory representation:

string[] lines = new string[NN[0].Count]; // assume all lines have equal length
for(int i = 0; i < NN.Count; ++i) {
  for(int j = 0; j < NN[i].Count; ++j) {
     lines[j] += NN[i][j] + ((i==NN.Count - 1) ? "" : ",");
  }
}
File.WriteAllLines("path.csv", lines);

As first optimization pass, I wouldn't recommend using a list of lists, since accessing elements will be quite intensive. A bidimensional array would make a better job.

int[,] NN = new int[3,6] {{1, 2, 3, 4, 5, 6 }, {2, 5, 6, 3, 1, 0}, {0, 9, 2, 6, 7, 8}};

            string[] lines = new string[NN.GetLength(1)];
            for (int i = 0; i < NN.GetLength(0); ++i)
            {
                for (int j = 0; j < NN.GetLength(1); ++j)
                {
                    lines[j] += NN[i,j] + ((i == NN.GetLength(0) - 1) ? "" : ",");
                }
            }
            File.WriteAllLines("path.csv", lines);

Here is a performance test, for 500x500 elements (without counting the write to file):

To improve this solution I would first make the transpose in memory (without writing anything to a file or strings), and then perform a join(,) and write to a file (as a single byte array).

If you want to further optimize, think there is always room for it :)

For example, on x86, depending on the instruction set you have, you can read this article. On a CUDA enabled device, you can read that.

Anyway, a good solution will always involve aligned memory, sub-blocking, and close-to-metal written code (intrinsics or assembly).

*"a good solution will always involve aligned memory, sub-blocking, and close-to-metal written code (intrinsics or assembly)."* None of which you're going to get in C#, which makes the whole top half of this answer a bit irrelevant. — Cody Gray - on strike, May 28 '16 at 12:56
aligning memory is possible in C# (pinvoke virtualalloc), sub-blocking is just a matter of indexing, and Pinvoking sub routines is also possible (for example to transpose subblocks) — Regis Portalez, May 28 '16 at 13:02

Writing list to CSV file

2 Answers2