up to 15 times faster calculation c++ vs C# is this legit?

Question

it's my first attempt to produce a native c++ code, then calling functions from c#, to test the overhead i was hearing about when interoping /invoking code. the test is simple math calculation, first a single run, and then 10K iterations in a loop.

class Program
{
    [DllImport("ExportTest.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
    public static extern void doWarmUp();
    [DllImport("ExportTest.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
    public static extern int doOne();
    [DllImport("ExportTest.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
    public static extern int doLongOne();

    static void Main(string[] args)
    {
        doWarmUp();
        Stopwatch sw = new Stopwatch();
        int test;

        sw.Reset();
        sw.Start();
        test = csdoOne();
        sw.Stop();
        Console.WriteLine("Res CsdoOne:{0} ", sw.ElapsedTicks);
        sw.Reset();
        sw.Start();
        test = doOne();
        sw.Stop();
        Console.WriteLine("Res C++ doOne :{0} ", sw.ElapsedTicks);


        sw.Reset();
        sw.Start();
        test = doLongOne();
        sw.Stop();
        Console.WriteLine("Res C++ doLongOne:{0}\tTics: {1} ", test, sw.ElapsedTicks);
        sw.Reset();
        sw.Start();
        test = csdoLongOne();
        sw.Stop();
        Console.WriteLine("Res C# doLongOne:{0}\tTics: {1} ", test, sw.ElapsedTicks);
        Console.Read();
    }
    static int csdoOne()
    {
        int res;
        res  = 5 * 4;
        return res;
    }
    static int csdoLongOne()
    {
        int r1, r2;
        r1 = 0; r2 = 0;
        for (int i = 0; i < 10500; i++)
        {

            r1 = (5 * 4);
            r2 = i * 2;
            r1 += r2;
        }
        return r2;
    }
}

if it is interesting to see the c++ code i will post it too, i did not do it cause it is actually the exact same code in both function the warm-up function is just void doWarmUp(){ return;}, as i have notes that only first call has any overhead.

the results as plotted are astonishing as i really like c#

Res Cs doOne:  4
Res C++ doOne: 17
Res c++ doLongOne: 21018        Tics: 30
Res C#  doLongOne: 21018        Tics: 446

it's only simple math... am i doing anything wrong ?

`doOne` is possibly a noop which is statically resolved to `20`. Also, are both DLLs compiled with optimizations? — Rotem, Oct 18 '15 at 12:59
Post disassembly. The way I compiled it I got `mov eax, 20998` for the C++ version, so the measurement would be almost 100% noise and overhead. — harold, Oct 18 '15 at 12:59
Not too sure about the accuracy of your measurements; although it is clear that certain actions are much faster in C++ (or any other language not having to account for something as big as the .NET Framework and all what it implies). On the other hand, there are lots of advantanges of C# over C++. Drawing generic conclusions (= 15 times faster calculation) from a so simplistic test (even by assuming that these results are right) is not too reasonable. — varocarbas, Oct 18 '15 at 13:04
Yes, you are doing some wrong. Profiling code that does nothing useful generates useless profiling results. The optimizer of a C++ compiler completely eliminates the loop since it is entirely unnecessary and simply returns 2 * 10499. The optimizer in the .NET jitter never removes loops, a design choice, it can only remove the useless r1 assignments. — Hans Passant, Oct 18 '15 at 13:05
@Rotem funny, as i am not sure on the C++ no optimisation done... yet. c# though is defaulted Optimize Code `v`. — Avia Afer, Oct 18 '15 at 13:09
@HansPassant as i sateted, it's only a simple test, as i am planning to assign an array of structs next test, see if populating 100k array with generated data through c++ would still be faster than on managed (localy) — LoneXcoder, Oct 18 '15 at 13:17
@HansPassant thanks for your comment, i will do some other tests that involves some more real work to be done . — LoneXcoder, Oct 18 '15 at 13:27
@HansPassant yet another test, about 5 times faster than managed alternative that might be further optimized http://stackoverflow.com/questions/33219095/faster-way-to-return-data-as-an-array-interoping-c?answertab=active#tab-top — LoneXcoder, Oct 20 '15 at 19:35
I would also recommend that you ngen your C# code - my understanding is that ngen can apply more aggressive optimizations than the JIT compiler can, and thus, you can achieve faster code. MSDN: https://msdn.microsoft.com/en-us/library/6t9t5wcf%28v=vs.110%29.aspx — antiduh, Oct 28 '15 at 21:46

score -1 · Answer 1 · answered Oct 28 '15 at 21:40

Your testing is flawed. You call very short method only once, there is almost nothing to measure. Fluctuations (variability) in measurements will be higher than timing itself. This invalidates your first method timing.

You should call doOne a billion times and divide the result by billion, on floats (not ints).

Not sure how to explain second method timing. It is possible that you encountered one of those rare moments where .Net is garbage collecting right in the middle of testing, adding a substantial amount of time. This can be easily checked. Run your test several times. If this was the case, most timings would be close to each other with a handful of outliers.

Also second method is also too short to be measured without a loop. 10K iterations is hardly measureable on a processor that runs billions of instructions per second.

up to 15 times faster calculation c++ vs C# is this legit?

1 Answers1