0

I wrote some code to test performance for arrays in both C#(.NET CLR) and Java(Java 8, Windows). For normal flat arrays, .NET showed to be a little faster than Java.

When I wrote some code for testing 2d arrays (using jagged arrays), I noticed a clear gap between Java and C#. Java version running more than 2x faster than C#!

Here's my C# code:

class ArrayTest
{
    public int [][] jagged;

    public ArrayTest(int width, int height)
    {
        Height = height;
        Width = width;
        Random rng = new Random();
        jagged = new int[height][];
        for (int i = 0; i < height; i++)
        {
            jagged[i] = new int[width];
            for (int j = 0; j < width; j++)
            {
                jagged[i][j] = rng.Next(1024);
            }
        }
    }

    public int Height { get; private set; }

    public int Width { get; private set; }

    public void DoMath(ArrayTest a)
    {
        for (int i = 0; i < Height; i++)
        {
            for (int j = 0; j < Width; j++)
            {
                jagged[i][j] *= a.jagged[i][j];
            }
        }
    }

}



class Program
{
    static void Main(string[] args)
    {
        const int loop = 500;
        int width = 800, height = 800;
        ArrayTest a1 = new ArrayTest(width, height),
            a2 = new ArrayTest(width, height);

        Stopwatch sw = new Stopwatch();
        sw.Start();
        for (int i = 0; i < loop; i++)
        {
            a1.DoMath(a2);
        }
        sw.Stop();

        Console.WriteLine("Time taken: " + sw.ElapsedMilliseconds);
        Console.ReadKey();
    }
}

In my computer, the measured part takes about 2200ms to run.

Here's the Java version:

    public class ArrayTest {
    private int width, height;
    private int[][] array;

    public ArrayTest(int width, int height) {
        this.width = width;
        this.height = height;
        array = new int[height][width];
        Random rng = new Random();
        for (int i = 0; i < height; i++) {

            for (int j = 0; j < width; j++) {
                array[i][j] = rng.nextInt(1024);
            }
        }
    }

    public int getWidth() {
        return width;
    }
    public void setWidth(int width) {
        this.width = width;
    }
    public int getHeight() {
        return height;
    }
    public void setHeight(int height) {
        this.height = height;
    }
    public int[][] getArray() {
        return array;
    }

    public void doMath(ArrayTest a) {
        for (int i = 0; i < height; i++) {
            for (int j = 0; j < width; j++) {
                array[i][j] *= a.array[i][j];
            }
        }
    }

}

    public class Main {

        public static void main(String[] args) {
            // TODO Auto-generated method stub
            final int loop = 500;
            int width = 800,
                    height = 800;
            ArrayTest a1 = new ArrayTest(width, height),
                    a2 = new ArrayTest(width, height);
            long start, end;


            start = java.lang.System.currentTimeMillis();
           for (int i = 0; i < loop; i++) {
               a1.doMath(a2);
           }
            end = java.lang.System.currentTimeMillis();
            System.out.println("Elapsed time: " + (end - start));
        }

    }

It takes about 930ms to run.

Until now, it is C# 2200ms vs Java 930ms.

However, when I change my C# method to look like this:

public void DoMath(ArrayTest a)
    {
        int[][] _jagged = this.jagged,
            _a = a.jagged;
        int[] __jagged, __a;
        for (int i = 0; i < _jagged.Length; i++)
        {
            __jagged = _jagged[i];
            __a = _a[i];
            for (int j = 0; j < __jagged.Length; j++)
            {
                __jagged[j] *= __a[j];
            }
        }
    }

Then my C# code becomes as fast as Java! (930ms) I took a really long time until I got to this code, and I'm still not sure why it is fastest than the first version (and way uglier).

So, here are my questions:

  1. Why is this last code so much more eficiente than the first one?
  2. Can I rewrite it so that it becomes even more efficient (or less ugly, but as efficient as this one)?

[edit]

I'm using .NET 4.5 and Java 8, both are console applications. The .NET version I'm running in release mode and without the VS debugger attached.

[edit 2]

Just to be clear: it is NOT a micro-benchmark or anything like that. I just want to make operations through 2d arrays faster than (or as fast as) Java, and I prefer to have a cleaner code.

Yan Paulo
  • 369
  • 1
  • 3
  • 12
  • Why are you limiting the C# to 1024 but not the Java one? This may introduce "some" overhead... its also not clear what your test conditions were or how you measured the time. – Ron Beyer Apr 30 '15 at 02:50
  • Why do you keep creating [micro-benchmarks](http://stackoverflow.com/questions/29933973/java-vs-net-performance)? – Elliott Frisch Apr 30 '15 at 02:52
  • I'm just interested in this specific functionality for both platforms. I specifically need to improve some code I am writing in both C# and Java. – Yan Paulo Apr 30 '15 at 02:54
  • 1
    Are you comparing Console C# vs Console Java? Because these languages can get very distinctly slow using crazy techs like WinRT or other stuff, also specifying what .NET and Java version could give the specialist guys somewhere to look at. So far from my usage C# always been absurdly faster than Java, when using the reflection specially.Just some thoughts. – Felype Apr 30 '15 at 02:56
  • Ron Beyer, I also limited the values in Java version to 1024. For measuring, I used Stopwatch class in C# and Java.lang.System.currentTimeMilis() for Java. – Yan Paulo Apr 30 '15 at 02:59
  • I'm no expert and no lang lawyer but I'd say `new int[height][width];` in java ought to be faster than new `int[height][];` and then declaring a whole new array and its dimensions in C#. Because, the processor will have to make some new arrangements into the memory, we're talking about primitive types not sparse lists or binary trees, arrays have to be side by side on memory, the processor will have to move the whole jagged[][] to somewhere else just to add a new array to it. Well that's my only guess i bet. would you please try `int[,] jagged = new int[height, width];` instead if u have time? – Felype Apr 30 '15 at 03:12
  • Nope! I read somewhere that what Java does internally is the same as we do with jagged arrays in C#. Had already tried with multidimensional array (int[,]) in C#, but the code becomes even more slower (jumps to about 2800ms). – Yan Paulo Apr 30 '15 at 03:20
  • Why did you open another question about this topic? I think it would be better if you add this to your question from yesterday. http://stackoverflow.com/questions/29933973/java-vs-net-performance – gdir Apr 30 '15 at 04:00
  • Sorry, it is because I didn't put the question very well in the another topic. And also, it is on hold. Answers can't be posted there anymore. – Yan Paulo Apr 30 '15 at 04:28

4 Answers4

1

Try making this very small change:

public void DoMath(ArrayTest a)
{
    int height = Height;
    int width = Width;

        for (int i = 0; i < height; i++)
        {
            for (int j = 0; j < width; j++)
            {
                jagged[i][j] *= a.jagged[i][j];
            }
        }


}

This almost tripled performance on my machine. Did you set the C# project to be run in Release mode or Debug mode? There is a big difference there.

Here are my results:

  • As you had it: 5555ms (running on my laptop, debug mode, debugger attached)
  • With the modifications above (same conditions as above): 1808ms
  • Running in release mode outside debugger (CTRL-F5): 907ms

So why does that small change make such a difference? Running in debug mode disables optimizations in C#, so the compiler is not optimizing calls to the Height and Width property. Every time you loop, it goes to the property, which then has to go to the backing field to get the value, and then return back.

Since you are using fields and not properties in Java, you are not having to make this round-trip each loop. Try replacing the Width and Height in the loop with getWidth and getHeight and see what it does.

Running in release mode with optimizations should make sure you are comparing apples to apples. Even without the modification above, just changing to release mode the C# code got 3 times faster.

Ron Beyer
  • 11,003
  • 1
  • 19
  • 37
  • I'm already running my code in release mode and outside the debugger. The change above really turns the execution a little faster (goes from **2200ms** to **2000ms**), but still way behind Java. – Yan Paulo Apr 30 '15 at 03:24
  • Outside the debugger an in release mode are two different things. – Ron Beyer Apr 30 '15 at 03:25
  • True! Forgot to mention release mode. – Yan Paulo Apr 30 '15 at 03:28
  • 1
    If you get 2000ms in release mode something else is going on, I got 5555 the way you wrote it, and with the minor change and release mode I was around 900ms, you should have similar performance gains. – Ron Beyer Apr 30 '15 at 03:29
  • It is because I'm running my code in an old Pentium Dual Core 4440. In Debug mode with debugger attached I get nothing less than **9800ms** ;) – Yan Paulo Apr 30 '15 at 03:32
1

I have made some tweaks on C# code for maximum (as far as I know) use of the technology here is the "doMath" tweaked method:

public void DoMath(ArrayTest a)
{
    //Stopwatch st = new Stopwatch();
    //st.Start();
    //for (int i = 0; i < Height; i++)
    Parallel.For(0, Height, (i) =>
    {
        for (int j = 0; j < Width; j++)
        {
            jagged[i][j] *= a.jagged[i][j];
        }
    });
    //st.Stop();
    //Console.WriteLine(st.ElapsedMilliseconds);
}

Even doing this optimization and Enabling Optimize Code on project settings, C# is still slower in comparison. What is funny is, using uniform 2D Array (int[,]) for this example is weirdly slower. I also changed Java to use nanoTime instead of currentTimeMillis, because as far as I know, currentTimeMillis can use some optimizations to "lie".

My results was:

Java:

378.3 / 379.4 / 374.7

C#:

924 / 932 / 913

This is on i7 processor at 2.2ghz, turbo boost probably disabled by Hyper-V Windows 8 bug.

What is happening here? Did you notice we're just doing all the same operation 500 times? Java's optimizations allows it to store an internal shortcut to reach the response faster. How and why can I tell? Because if we surround DoMath function in nanoTime checkers, we'll see that the very first DoMath runs take about 13ms to complete, the second takes about 7ms, then 4, then 2 and it reduces further and furter until the optimizations kick in and apply its shortcut to that whole procedure. Meanwhile our C# counterpart will be from the first to the last DoMath operation taking 2ms~ to complete.

As we know, Java's compiler also optimize by simplifying formulas at compile time, the association of these 2 optimizing features might cause this result to happen, by the other hand, C# will be indeed and in fact doing all the math step by step without using any shortcut.

** Edit ** If we further optimize C# in this way:

public void DoMath(ArrayTest a)
{
    Parallel.For(0, Height, (i) =>
    {
        int[] _jagged = jagged[i];
        int[] _a = a.jagged[i];
        for (int j = 0; j < _jagged.Length; j++)
        {
            _jagged[j] *= _a[j];
        }
    });
    //st.Stop();
}

We can achieve these results: C#:

343, 326, 317

Now as a C# user i find this amazing to know. I've learned one important thing here: myArray[i][j] is much slower than creating a new _myArray[] that points to myArray[i]. This kind of optimization led C# to run as fast as java optimizations, but required the use of Parallel Processing. The C# is still doing all the math over and over like good old C would, but each row in height is being processed by a different thread, these threads are limited and regulated by .NET runtime and they run concurrently taking great advantage on multiple processor cores, but probably not as much on single-core processors.

Felype
  • 3,087
  • 2
  • 25
  • 36
  • Can you test my last version of DoMath? Also, I made several different experiments with 2d arrays in C# in a new code and got to excelente results (one of those equal or superior to Java). Should I post it as new answer, so that you can take a look? If not, where should I post? I'm kinda new here, y'know. – Yan Paulo Apr 30 '15 at 15:00
  • Btw, very clever your idea of testing each execution of DoMath on Java. – Yan Paulo Apr 30 '15 at 15:02
  • Yes yes, I see, got very good results here, but I have to test one more thing. Edit: yes fine, excellent :) Updating soon. – Felype Apr 30 '15 at 15:07
  • Whoooooa! So this is what this Paralell.For() is a bout. Didn't know that, this is amazing! Now I will post my code. – Yan Paulo Apr 30 '15 at 15:33
  • About my new code: should I edit the topic, or should I post as an answer? – Yan Paulo Apr 30 '15 at 15:37
  • According to Stack Meta, you should mark as 'correct answer' one answer you find to be the 'correct answer'. – Felype Apr 30 '15 at 16:14
  • Yes! Your answer is the correct. Just didn't mark yet so that people won't think the topic is "closed". – Yan Paulo Apr 30 '15 at 17:05
1

I made these new tests and my new C# code is running more or less at the same speed as Java's.

Here is the code:

class ArrayTest
    {
        public int [][] jagged;

        public ArrayTest(int width, int height)
        {
            Height = height;
            Width = width;
            Random rng = new Random();
            jagged = new int[height][];
            for (int i = 0; i < height; i++)
            {
                jagged[i] = new int[width];
                for (int j = 0; j < width; j++)
                {
                    jagged[i][j] = rng.Next(1024);
                }
            }
        }

        public int Height { get; private set; }

        public int Width { get; private set; }

        public int this[int i, int j]
        {
            get
            {
                return jagged[i][j];
            }
            set
            {
                jagged[i][j] = value;
            }
        }

        public void DoMath(ArrayTest a)
        {
            int width = Width, height = Height;
            for (int i = 0; i < height; i++)
            {
                for (int j = 0; j < width; j++)
                {
                    //jagged[i][j] *= a.jagged[i][j];
                    jagged[i][j] = jagged[i][j] * a.jagged[i][j];
                }
            }
        }

        public void DoMathProp(ArrayTest a)
        {
            int width = Width, height = Height;
            for (int i = 0; i < height; i++)
            {
                for (int j = 0; j < width; j++)
                {
                    //this[i, j] *= a[i, j];
                    this[i, j] = this[i, j] * a[i, j];
                }
            }
        }

        public void DoMathFaster(ArrayTest a)
        {
            int width = Width, height = Height;
            int[][] jagged = this.jagged,
                _a = a.jagged;
            for (int i = 0; i < height; i++)
            {
                for (int j = 0; j < width; j++)
                {
                    //jagged[i][j] *= _a[i][j];
                    jagged[i][j] = jagged[i][j] * _a[i][j];
                }
            }
        }

        public void DoMathForeach(ArrayTest a)
        {
            int i = 0, j = 0;
            foreach (var line in a.jagged)
            {
                j = 0;
                foreach (var item in line)
                {
                    //jagged[i][j] *= item;
                    jagged[i][j] = jagged[i][j] * item;
                    j++;
                }
                i++;
            }
        }

        public void DoMathFastest(ArrayTest a)
        {
            int[][] _jagged = this.jagged,
                _a = a.jagged;
            int[] __jagged, __a;
            for (int i = 0; i < _jagged.Length; i++)
            {
                __jagged = _jagged[i];
                __a = _a[i];
                for (int j = 0; j < __jagged.Length; j++)
                {
                    //__jagged[j] *= __a[j];
                    __jagged[j] = __jagged[j] * __a[j];
                }
            }
        }

        public int Sum()
        {
            int sum = 15;
            int[][] jagged = this.jagged;
            for (int i = 0; i < jagged.Length; i++)
            {
                for (int j = 0; j < jagged[i].Length; j++)
                {
                    sum = sum + jagged[i][j];
                }
            }
            return sum;
        }

        public int SumProp()
        {
            int sum = 15;
            int width = Width, height = Height;
            for (int i = 0; i < height; i++)
            {
                for (int j = 0; j < width; j++)
                {
                    sum = sum + this[i, j];
                }
            }
            return sum;
        }

        public int SumForeach()
        {
            int sum = 15;
            foreach (var line in jagged)
            {
                foreach (var item in line)
                {
                    sum += item;
                }
            }
            return sum;
        }

        public int SumFast()
        {
            int sum = 15;
            int[][] jagged = this.jagged;
            for (int i = 0; i < jagged.Length; i++)
            {
                int[] _jagged = jagged[i];
                for (int j = 0; j < jagged[i].Length; j++)
                {
                    sum += _jagged[j];
                }
            }
            return sum;
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            const int loop = 1000;
            int width = 800, height = 800;
            ArrayTest a1 = new ArrayTest(width, height),
                a2 = new ArrayTest(width, height);

            Stopwatch sw = new Stopwatch();

            //DoMath
            sw.Start();
            for (int i = 0; i < loop; i++)
            {
                a1.DoMath(a2);
            }
            sw.Stop();

            Console.WriteLine("DoMath: " + sw.ElapsedMilliseconds);

            //DoMathProp
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.DoMathProp(a2);
            }
            sw.Stop();

            Console.WriteLine("DoMathProp: " + sw.ElapsedMilliseconds);

            //DoMathFaster
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.DoMathFaster(a2);
            }
            sw.Stop();

            Console.WriteLine("DoMathFaster: " + sw.ElapsedMilliseconds);

            //DoMathForeach
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.DoMathForeach(a2);
            }
            sw.Stop();

            Console.WriteLine("DoMathForeach: " + sw.ElapsedMilliseconds);

            //DoMathFastest
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.DoMathFastest(a2);
            }
            sw.Stop();

            Console.WriteLine("DoMathFastest: " + sw.ElapsedMilliseconds);

            //Sum
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.Sum();
            }
            sw.Stop();

            Console.WriteLine("Sum: " + sw.ElapsedMilliseconds);

            //SumProp
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.SumProp();
            }
            sw.Stop();

            Console.WriteLine("SumProp: " + sw.ElapsedMilliseconds);

            //SumForeach
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.SumForeach();
            }
            sw.Stop();

            Console.WriteLine("SumForeach: " + sw.ElapsedMilliseconds);

            //SumFast
            sw.Restart();
            for (int i = 0; i < loop; i++)
            {
                a1.SumFast();
            }
            sw.Stop();

            Console.WriteLine("SumFast: " + sw.ElapsedMilliseconds);

            Console.ReadKey();
        }
    }

I won't post the code for Java since it is quite similar to this one.

And here are the results for each implementation:

C#, using *= operator

Run #1:
DoMath: 3931
DoMathProp: 3011
DoMathFaster: 3358
DoMathForeach: 3102
DoMathFastest: 1770
Sum: 729
SumProp: 745
SumForeach: 709
SumFast: 753

Run #2:
DoMath: 3945
DoMathProp: 2978
DoMathFaster: 3311
DoMathForeach: 3102
DoMathFastest: 1764
Sum: 730
SumProp: 744
SumForeach: 708
SumFast: 753

Run #3:
DoMath: 4041
DoMathProp: 2976
DoMathFaster: 3314
DoMathForeach: 3092
DoMathFastest: 1777
Sum: 732
SumProp: 748
SumForeach: 714
SumFast: 751

C#, no *= operator:

Run #1:
DoMath: 3045
DoMathProp: 2977
DoMathFaster: 2681
DoMathForeach: 3394
DoMathFastest: 1802
Sum: 738
SumProp: 753
SumForeach: 716
SumFast: 755

Run #2:
DoMath: 2966
DoMathProp: 2953
DoMathFaster: 2631
DoMathForeach: 3382
DoMathFastest: 1747
Sum: 734
SumProp: 738
SumForeach: 703
SumFast: 755

Run #3:
DoMath: 2965
DoMathProp: 2959
DoMathFaster: 2642
DoMathForeach: 3383
DoMathFastest: 1773
Sum: 729
SumProp: 747
SumForeach: 707
SumFast: 750

Java:

Run #1:
doMath: 1728.550504
doMathProp: 1747.916189
doMathFaster: 1736.039763
doMathForeach: 1734.981001
doMathFastest: 1732.675896
sum: 626.478196
sumProp: 627.959249
sumForeach: 629.493964
sumFast: 649.372465

Run #2:
doMath: 1791.048725
doMathProp: 1888.965311
doMathFaster: 1826.43116
doMathForeach: 1806.1341
doMathFastest: 1833.68291
sum: 772.088533
sumProp: 697.429715
sumForeach: 660.004415
sumFast: 756.06105

Run #3:
doMath: 1819.834545
doMathProp: 1821.309067
doMathFaster: 1809.052345
doMathForeach: 1719.76124
doMathFastest: 1719.466335
sum: 616.398258
sumProp: 627.816464
sumForeach: 650.758328
sumFast: 642.416087

As you can see, C#'s fastest method (DoMathFastest) always gets very close to Java's one (DoMath). My guess is that it is because C#'s flat array access (already tested it) is quicker than Java's and the method ends executing more quickly. Even thought the operations in DoMathsFastest

__jagged = _jagged[i];
 __a = _a[i];

introduce some overhead, in the end it is compensated by faster access. In Java, however, it doesn't make any clear difference.

It's just my guess. I'm not really sure if this is what happens.

Thanks for the answers!

[edit]

Found out some interesting new information by testing. Found ou that the

a[i][j] *= b

operation is always slower than

a[i][j] = a[i][j] * b

operation, but this is valid only for members of a jagged array accessed by the full index (a[i][j]).

Also found that using an indexer (this[i,j]) is as fast as directly accessing the array members by (this.jagged[i][j]) in some cases, and in some other cases it is even faster! Quite amazing.

Edited the original post with new code and test results for comparison.

Yan Paulo
  • 369
  • 1
  • 3
  • 12
0

I changed DoMath method as follow:

    public void DoMath(ArrayTest a)
    {
        int[][] arr = a.jagged;
        for (int i = 0; i < Height; i++)
        {
            for (int j = 0; j < Width; j++)
            {
                jagged[i][j] *= arr[i][j];
            }
        }
    }

I only cached the a.jagged to a local variable(arr) and it takes 1400 ms instead of 1650 ms. I tested with java and it took 1120 ms. I used 10000x10000 array with 10 iterations.

I have some points about this performance problem that I think it can be useful:

1- When testing use large arrays to avoid caching memory in the CPU that may confuse us. For example 200 MB or larger. So using 10000x10000 array with 10 iterations is better.

2- I read in adsamcik's answer that in .NET Core 2.0 the performance of multidimensional array is almost twice time better than gagged array.

3- When the performance matter it is usual that we generate ugly code. For example using pointers(unsafe code in C#) in image processing is very common. And you can use pointer for this problem and it may improve performance a lot. But you can't use pointers in Java.

A response for your other question about why the performance improved when you used __jagged = _jagged[i]; is: because of caching of a memory location like one I used. It reduce memory reference from two times to one time.

Mostafa Vatanpour
  • 1,328
  • 13
  • 18