-2

I have a c# program that reads in arbitrary CSV files. Columns can be of type int, float or double.

I am currently loading this data into a List is there any way to perform basic arithmetic operations between columns. I.e. add two columns. If the columns are of different types I would like to follow standard type promotion.

Is there an easy way to achieve this, should columns be encapsulated within an object?

Here is a code sample that shows the behaviour I am after. I do need to keep the separate types, due to memory and res

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;

namespace Example
{
    class Program
    {
        static void Main(string[] args)
        {
            List<IList> data = new List<IList>();

            data.Add(Enumerable.Range(0, 1000).ToList()); // Add a list of ints
            data.Add(Enumerable.Range(0, 1000).Select(v=>(float)v).ToList()); // Add a list of floats
            data.Add(Enumerable.Range(0, 1000).Select(v => (double)v).ToList()); // Add a list of doubles

            data.Add(GenerateColumn(data[0], data[1], Operation.Add));
            data.Add(GenerateColumn(data[1], data[2], Operation.Divide));
        }

        // This is what I would do if the lists were all the same type
        static IList GenerateColumn(IList colA, IList colB,Operation operation)
        {
            List<double> result = null;

            switch (operation)
            {
                case Operation.Add:
                    result = colA.Zip(colB, (a, b) => a + b).ToList();
                    break;
                case Operation.Subtract:
                    result = colA.Zip(colB, (a, b) => a - b).ToList();
                    break;
                case Operation.Multiply:
                    result = colA.Zip(colB, (a, b) => a * b).ToList();
                    break;
                case Operation.Divide:
                    result = colA.Zip(colB, (a, b) => a / b).ToList();
                    break;
            }
            return result;
        }

        public enum Operation
        {
            Add,
            Subtract,
            Multiply,
            Divide
        }
    }
}
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Hugoagogo
  • 1,598
  • 16
  • 34
  • 1
    What happened when you tried it? Can you show the code and describe specifically what error you're getting? – Rufus L Jan 29 '19 at 22:25
  • I was previously using a single data type (stored in a List>) but with some of the files I need to process ran into memory limitations (already loading 15GB of data), as well as precision limits . To solve this I will predominantly be using Integers & floats. – Hugoagogo Jan 29 '19 at 23:20
  • Use `dynamic` like Alexei suggested. – mjwills Jan 29 '19 at 23:35
  • `but with some of the files I need to process ran into memory limitations` Perhaps consider using `yield return` or `IEnumerable` or streams or the like so that you aren't having the entire contents of the file in RAM at one time? – mjwills Jan 30 '19 at 03:26
  • I did think about using IEnumerable but I need to do some analysis before passing it off to a plotting library. – Hugoagogo Jan 30 '19 at 03:32

2 Answers2

0

There is no support for generic arithmetic operations so you have to pick something else that fits your goal.

Just making all values double is one option and ignore non-numeric fields. This will simplify conversion rules (to none).

Another way out is to use dynamic for all values (C# Adding two Generic Values). It will give you following exact promotion rules and ability to do math operations on any values (as long as values are actually of correct type and not just strings). That's probably what you really looking for if you are looking at making rules compiled (when col[0] = col[1] + col[2] * col[3] is part of program's code).

If you planning to parse expressions for columns you may as well add support for promoting values to broader numeric types as part of the parser.

Alexei Levenkov
  • 98,904
  • 14
  • 127
  • 179
-1

Based on Alexei's Answer this is the solution I came up with based on using dynamic.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;

namespace Example
{
    class Program
    {
        static void Main(string[] args)
        {
            List<List<dynamic>> data = new List<List<dynamic>>();

            data.Add(new List<dynamic>());
            data[0].AddRange(Enumerable.Range(0, 1000).Cast<dynamic>()); // Add a list of ints
            data.Add(new List<dynamic>());
            data[1].AddRange(Enumerable.Range(0, 1000).Select(v=>(float)v).Cast<dynamic>()); // Add a list of ints
            data.Add(new List<dynamic>());
            data[2].AddRange(Enumerable.Range(0, 1000).Select(v => (double)v).Cast<dynamic>()); // Add a list of ints

            data.Add(GenerateColumn(data[0], data[0], Operation.Add)); // Result should be int column
            data.Add(GenerateColumn(data[0], data[1], Operation.Subtract)); // Result should be float column
            data.Add(GenerateColumn(data[1], data[2], Operation.Divide)); // Result should be double column

            foreach (List<dynamic> lst in data)
            {
                Debug.WriteLine((Type)lst[0].GetType());
            }
        }

        // This is what I would do if the lists were all the same type
        static List<dynamic> GenerateColumn(List<dynamic> colA, List<dynamic> colB,Operation operation)
        {
            List<dynamic> result = null;

            switch (operation)
            {
                case Operation.Add:
                    result = colA.Zip(colB, (a, b) => a + b).ToList();
                    break;
                case Operation.Subtract:
                    result = colA.Zip(colB, (a, b) => a - b).ToList();
                    break;
                case Operation.Multiply:
                    result = colA.Zip(colB, (a, b) => a * b).ToList();
                    break;
                case Operation.Divide:
                    result = colA.Zip(colB, (a, b) => a / b).ToList();
                    break;
            }
            return result;
        }

        public enum Operation
        {
            Add,
            Subtract,
            Multiply,
            Divide
        }
    }
}

Output is

System.Int32
System.Single
System.Double
System.Int32
System.Single
System.Double
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Hugoagogo
  • 1,598
  • 16
  • 34