11

I'm playing around with a very simple program to take an array of doubles and return the standard deviation. This part worked but I wanted to make the code more reusable. I would like to make it so the method can accept a parameter of any type that could be considered numeric and return the standard deviation instead of hardcoding a double type (like I initially did in this program). How does one go about this and what is the proper term for it?

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;


    namespace ConsoleApplication5
    {
        class Program
        {
            static void Main(string[] args)
            {
                double[] avg = { 3.4, 55.6, 10.0, 4.5, 2, 2 };
                double x = avg.Average();
                //first round of testing
                Console.WriteLine("The average of the first array is below ");
                Console.WriteLine(x);
                Console.WriteLine("below should be the standard deviation!");
                Console.WriteLine(CalculateStandardDeviation(avg));
                Console.ReadLine();
                int[] intAvg = { 4, 3, 5, 6, 2 };
                double secondAvg = intAvg.Average();
                Console.WriteLine("The average of the second array is below ");
                Console.WriteLine(secondAvg);
                //this is where the error is happening
                //CalculateStandardDeviation(secondAvg);

            }
            //this is where I tried to make the query more reusable
            public static double CalculateStandardDeviation(IEnumerable<double> values)
            {

                double avg = values.Average();
                double sum = 0;
                foreach (double d in values)
                {
                    sum += Math.Pow((d - avg), 2);

                }

                return Math.Pow(sum / (values.Count() - 1),.5);

            }
        }
    }
DaveShaw
  • 52,123
  • 16
  • 112
  • 141
wootscootinboogie
  • 8,461
  • 33
  • 112
  • 197
  • 2
    You could create a method for each type you would consider a valid "input" and let overloading handle it - all the other types then might just cast to `double` and call the version that takes `IEnumerable`. You might be able to do it with generics (see [this](http://stackoverflow.com/questions/3329576/generic-constraint-to-match-numeric-types) for example), but there's really no good way to generically constrain to only numeric types. – mellamokb Apr 16 '13 at 19:23
  • I think the fundamental issue is your method is requesting _double_ type, but you're feeding it a collection of _int_. You would need to convert it or create an overload that takes integers and converts it to a `double` set instead. – Chris Sinclair Apr 16 '13 at 19:23
  • I would like for the CalculateStandardDeviation method to be able to accept a parameter of any numeric type. The program works if I change the IEnumerable type to double, or int..but not both. I'd like to be able to use a float, int, double, money, whatever else type of array and have the method work instead of hardcoding the return type. – wootscootinboogie Apr 16 '13 at 19:23
  • A quick Google search for "c# generic numeric" found a really novel approach here: http://stackoverflow.com/questions/3329576/generic-constraint-to-match-numeric-types – David Apr 16 '13 at 19:23
  • 2
    @wootscootinboogie You don't have that kind of constraint in C#, there's no base `Numeric` type. That's why e.g. `Sum` or `Average` have all these overloads... – Patryk Ćwiek Apr 16 '13 at 19:24
  • @Trustme-I'maDoctor good to know that it wasn't possible before I began to bang my head against the wall :) – wootscootinboogie Apr 16 '13 at 19:25
  • 1
    @mellamokb So I just need to overload the method with all possible numeric types that I want to include? – wootscootinboogie Apr 16 '13 at 19:26
  • 1
    @wootscootinboogie: That would be one simple way to handle it, yes. However, they can all share a common implementation. You don't need to (and shouldn't) copy/paste and duplicate the code into every method. – mellamokb Apr 16 '13 at 19:26
  • 1
    @wootscootinboogie Just be careful of the `int` overload version; you may not want to use integer division/averages (e.g., `5 / 2 = 2`) – Chris Sinclair Apr 16 '13 at 19:27

6 Answers6

7

You could use something like this:

public static decimal CalculateStandardDeviation<T>(IEnumerable<T> values)
{
    IEnumerable<decimal> decimalValues = values.Select(v => Convert.ToDecimal(v));

    decimal result = 0;

    // calculate standard deviation on decimalValues

    return result;
}

It will throw an exception if values contains values that can't be converted to a decimal, but will work if the values are of an appropriate type, and I think that makes perfect sense.

JLRishe
  • 99,490
  • 19
  • 131
  • 169
  • I'm unfamiliar with some of this syntax (noob). Could you explain the => operator? – wootscootinboogie Apr 16 '13 at 19:40
  • 3
    It's a [lambda expression](http://msdn.microsoft.com/en-us/library/bb397687.aspx). When used in `IEnumerable.Select()`, it essentially means "take (what's on the left of `=>`) and give me (what's on the right of `=>`)". – JLRishe Apr 16 '13 at 19:43
  • @JLRishe: I think you could replace `Select(v => Convert.ToDecimal(v))` with `Select(Convert.ToDecimal)`. Now Wootscootinboogie doesn't need to think about lambda expressions :P – Brian Apr 16 '13 at 20:13
  • 1
    @Brian That's a great suggestion and that idea hadn't crossed my mind, but unfortunately it doesn't work. It produces a compile error: "No overload for 'System.Convert.ToDecimal(object)' matches delegate 'System.Func'". You could do `values.Cast().Select(Convert.ToDecimal)`, but I think that's more convoluted than it's worth. – JLRishe Apr 16 '13 at 20:26
5

Unfortunately, there is no base class for all numbers. You can do this with a generic run-time checking method, or a compile-time safe set of overloads.

Generic Method:

public static T CalculateStandardDeviation(IEnumerable<T> values)
{
    var valueArray = values.Select(Convert.ToDecimal).ToArray();

    //...

    return (T)standardDeviation;
}

The problem with using a single generic method is that you can't put a type constraint on the type parameter that would restrict it to only numeric types. You would have to resort to failing at run-time. There would be nothing to stop you from calling the method with an array of strings, or objects, or Colors, or HttpWebRequests, etc. and unless you do in fact know how to calculate the standard deviation of a color, you should probably stick to individual overrides for a particular numeric type:

I would recommend using the decimal type as your main implementation, and then casting everything to it.

Type-Specific Overloads:

public static decimal CalculateStandardDeviation(IEnumerable<decimal> values)
{
    //...
}

public static double CalculateStandardDeviation(IEnumerable<double> values)
{
    return (double)CalculateStandardDeviation(values.Select(Convert.ToDecimal));
}

public static int CalculateStandardDeviation(IEnumerable<int> values)
{
    return (int)CalculateStandardDeviation(values.Select(Convert.ToDecimal));
}

// etc...
Dan
  • 9,717
  • 4
  • 47
  • 65
  • `values.Cast()` will fail. You can't type-cast between the value types. You'd have to change it to something like `values.Select(v => (decimal)v)` so the compiler can apply the conversion operator. – Chris Sinclair Apr 16 '13 at 19:34
  • 1
    @Chris In .Net 4.5 it works. I don't know if that's always been the case. I just tried it now. – Dan Apr 16 '13 at 19:36
  • @Dan It will compile, but I think when _executed_ at runtime (and in this case, when _iterated_ since it's deferred) it will throw a runtime exception. I don't _think_ this has changed in 4.5 (I don't see how it could as it performs a cast on generic type `T` and has no knowledge of the explicit/implicit conversion operators) EDIT: I will try to double-check though (perhaps new overloads were added) – Chris Sinclair Apr 16 '13 at 19:38
  • @Chris I didn't just compile it. If you call Cast on an integer array, or a double array, it successfully converts each value to a decimal and gives you an array containing them. Try it. – Dan Apr 16 '13 at 19:41
  • 2
    Standard deviation of colors: `Color.FromArgb(255 - CalculateStandardDeviation(colors.Select(c=>c.A)), CalculateStandardDeviation(colors.Select(c=>c.R)), CalculateStandardDeviation(colors.Select(c=>c.G)), CalculateStandardDeviation(colors.Select(c=>c.B)))`. Similar colors will be close to black and opaque. – Tim S. Apr 16 '13 at 19:43
  • 2
    @Tim Nice! Now we can add an overload for Colors! That's going to be super useful to many, I'm sure! :) – Dan Apr 16 '13 at 19:44
  • @Dan I just ran in a WinRT (4.5) app `IEnumerable values = new int[]{1, 2, 3, 4, 5}; IEnumerable decimals = values.Cast(); foreach(var d in decimals) { Debug.WriteLine(d.ToString()); }` and got a `System.InvalidCastException` on its first iteration. – Chris Sinclair Apr 16 '13 at 19:55
  • I really dislike the repetitiveness of this approach. If you choose to take this approach for more than a small number of functions, I would recommend making use of some form of code templating. Maybe [T4 Text Templates](http://msdn.microsoft.com/en-us/library/dd820620.aspx). – Brian Apr 16 '13 at 19:56
  • 1
    @TimS. Did you find a way to calculate the standard deviation of an `HttpWebRequest` as well? – Nick Freeman Apr 16 '13 at 19:57
  • 1
    @Chris You are right. I never iterated the result. So silly of me. I will update the code. Nice catch. I'm sorry I doubted you. – Dan Apr 16 '13 at 19:59
  • @Brian You must hate the `Convert` class, then with it's 300 some-odd repetitive methods. You'd probably prefer a single `TTarget Convert(TSource value)` implementation. – Dan Apr 16 '13 at 20:01
  • 1
    @Dan No problem; it's not like I'm Jon Skeet or anything. – Chris Sinclair Apr 16 '13 at 20:01
  • @Dan: You misunderstand. I'm not refusing to accept providing hundreds of repetitive methods. I am merely saying not to write out those repetitive methods by hand. For example, making the fix that Chris just suggested is more error-prone if you do so by hand. Find/Replace tends to be risky, too. – Brian Apr 16 '13 at 20:09
  • 3
    @Brian agreed, but there are very few numeric types, so I don't believe that to be a huge issue. – Dan Apr 16 '13 at 20:17
2

Use C# Generics.

Your function signature will be:

public static T CalculateStandardDeviation(IEnumerable<T> values)

And you can use it like:

int stdDev = CalculateStandardDeviation([int-array]);
double stdDev = CalculateStandardDeviation([double-array]);

Please follow this link:

http://msdn.microsoft.com/en-us/library/ms379564%28VS.80%29.aspx

Edit:

To resolve the Average issue on the generic types, please take a look in this library:

How to Implement Generic Method to do Math calculations on different value types

Obs: Suggestion from Brian.

Community
  • 1
  • 1
gustavodidomenico
  • 4,640
  • 1
  • 34
  • 50
  • The method will probably want some constraints on it. I imagine there exist types for which a standard deviation can not be calculated. – David Apr 16 '13 at 19:24
  • 1
    Not sure on the generic usage; how would it handle averages, addition, division, etc? – Chris Sinclair Apr 16 '13 at 19:24
  • 1
    How do you do subtraction and `Average` on type `T`? – mellamokb Apr 16 '13 at 19:24
  • I don't see why this answer should be down voted. It may not be a complete answer, but it is certainly not an unclear or egregious answer. If you are down voting, could you explain why an update to the answer couldn't help? – Nick Freeman Apr 16 '13 at 19:31
  • This is on the right track. To add restraints, what you need to do is specify where T: integer, double, decimal...etc, that will limit the types the method/class will accept to only those that refer to numbers – Crwydryn Apr 16 '13 at 19:37
  • @mellamokb: The problem of using Generics with Operator Overloading is solvable via Jon Skeet's `Operator` class. See [Miscellaneous Utility Library](http://www.yoda.arachsys.com/csharp/miscutil/) – Brian Apr 16 '13 at 19:38
  • 1
    @NickFreeman: I downvoted the answer because it is incorrect. There is no way to implement the method with the given signature, and will not as it stands help the OP solve their question. I always remove a downvote if it is no longer warranted. – mellamokb Apr 16 '13 at 19:39
  • 1
    My point was to introduce the Generics concept and rise the user criativity forward a well designed solution. Thank you @NickFreeman, I agree with you. – gustavodidomenico Apr 16 '13 at 19:40
  • Now I think that the answer is complete, @mellamokb? – gustavodidomenico Apr 16 '13 at 19:46
  • 1
    @gustavodidomenico: I honestly think due diligence would be an example, working implementation since it is a little unclear how this would actually work. But I have removed my downvote. – mellamokb Apr 16 '13 at 19:48
  • 1
    This doesn't fully solve the problem, in that there is no way to contrain `T` to be numeric (or alternatively, to constrain `T` to have the requisite operators). However, if you're willing to accept the risk of runtime exception (which you should probably catch), it's cleaner than the alternative approach of taking an IEnumerable and either casting it or using `dynamic`. If you're not willing to accept the risk of runtime exceptions, [Dan's Answer](http://stackoverflow.com/a/16045464/18192) is probably better. – Brian Apr 16 '13 at 19:52
  • I agree with you @mellamokb. But you should first think that I am trying to help, and just after that you should think something else. That is the meaning of a community. Thank you for the removed vote. – gustavodidomenico Apr 16 '13 at 19:53
0

EDIT You should use JLRishe's answer, it's much more elegant than this.

You should probably start by adding generics to your method and use the type converter to transform your unknown input into doubles like so :

public static double CalculateStandardDeviation<TSource>(IEnumerable<TSource> inputs)
{
    var converter = TypeDescriptor.GetConverter(typeof (double));
    if (!converter.CanConvertFrom(typeof(TSource)))
        return 0;

    var values = new List<double>();
    foreach (var value in inputs)
    {
        values.Add((double) converter.ConvertFrom(value));
    }

    // Your logic here ...
    return ...;
}

I did not tested this snippet but you get the idea.

gxtaillon
  • 1,016
  • 1
  • 19
  • 33
0

Foreword: this answer builds on How to verify whether a type overloads/supports a certain operator? and http://www.codeproject.com/Articles/87438/TinyLisp-A-Language-and-Parser-to-See-LINQ-Express The second link shows how to compile and evaluate linq expressions.

In short you could forego static type safety and check for the ability of a type to support specific operations at runtime (first link), in case it does not you could throw an exception as the following sample demonstrates:

void Main()
{
    DoAdd<float>(5,6);
    DoAdd<int>(5,6);
    DoAdd<bool>(true,false);
}


// Define other methods and classes here
static void DoAdd<T>(T in1, T in2){
if(!HasAdd<T>()){throw new Exception("Unsupported Type!");}
 var c1 = Expression.Constant(in1, typeof(T));
 var c2 = Expression.Constant(in2, typeof(T));
 var expression=Expression.Add(c1, c2);
 Expression<Func<T>> lExpression = Expression.Lambda<Func<T>>(expression);
 Func<T> fExpression = lExpression.Compile();
 Console.WriteLine(fExpression());

}


static bool HasAdd<T>() {
    var c = Expression.Constant(default(T), typeof(T));
    try {
        Expression.Add(c, c); // Throws an exception if + is not defined
        return true;
    } catch {
        return false;
    }
}
Community
  • 1
  • 1
Andrea Scarcella
  • 3,233
  • 2
  • 22
  • 26
  • Reasons why I don't like this answer: 1) It has to throw an exception for you to know whether or not you can add, which comes with a performance overhead. 2) If there is no add you throw a general Exception, which forces the caller to catch a general exception. Meaning they have no knowledge of why it was thrown and can do nothing to correct it. 3) It does not fix the problem of having to wait until runtime to know whether or not you can call the method with a specific type. Compile time error would be preferred. – Nick Freeman Apr 16 '13 at 20:20
0

Passing an IEnumerable of Numeric Values as a parameter to method will be supported in C# 6.0

Will Yu
  • 546
  • 5
  • 12