57

Edit: I apologize everybody. I used the term "jagged array" when I actually meant to say "multi-dimensional array" (as can be seen in my example below). I apologize for using the incorrect name. I actually found jagged arrays to be faster than multi-dimensional ones! I have added my measurements for jagged arrays.

I was trying to use a jagged multi-dimensional array today, when I noticed that it's performance is not as I would have expected. Using a single-dimensional array and manually calculating indices was much faster (almost two times) than using a 2D array. I wrote a test using 1024*1024 arrays (initialized to random values), for 1000 iterations, and I got the following results on my machine:

sum(double[], int): 2738 ms (100%)
sum(double[,]):     5019 ms (183%)
sum(double[][]):    2540 ms ( 93%)

This is my test code:

public static double sum(double[] d, int l1) {
    // assuming the array is rectangular
    double sum = 0;
    int l2 = d.Length / l1;
    for (int i = 0; i < l1; ++i)
        for (int j = 0; j < l2; ++j)
            sum += d[i * l2 + j];
    return sum;
}

public static double sum(double[,] d) {
    double sum = 0;
    int l1 = d.GetLength(0);
    int l2 = d.GetLength(1);
    for (int i = 0; i < l1; ++i)
        for (int j = 0; j < l2; ++j)
            sum += d[i, j];
    return sum;
}

public static double sum(double[][] d) {
    double sum = 0;
    for (int i = 0; i < d.Length; ++i)
        for (int j = 0; j < d[i].Length; ++j)
            sum += d[i][j];
    return sum;
}

public static void Main() {
    Random random = new Random();
    const int l1  = 1024, l2 = 1024;
    double[ ] d1  = new double[l1 * l2];
    double[,] d2  = new double[l1 , l2];
    double[][] d3 = new double[l1][];

    for (int i = 0; i < l1; ++i) {
        d3[i] = new double[l2];
        for (int j = 0; j < l2; ++j)
            d3[i][j] = d2[i, j] = d1[i * l2 + j] = random.NextDouble();
    }
    //
    const int iterations = 1000;
    TestTime(sum, d1, l1, iterations);
    TestTime(sum, d2, iterations);
    TestTime(sum, d3, iterations);
}

Further investigation showed that the IL for the second method is 23% larger than that of the first method. (Code size 68 vs 52.) This is mostly due to calls to System.Array::GetLength(int). The compiler also emits calls to Array::Get for the jagged multi-dimensional array, whereas it simply calls ldelem for the simple array.

So I am wondering, why is access through multi-dimensional arrays slower than normal arrays? I would have assumed the compiler (or JIT) would do something similar to what I did in my first method, but this was not actually the case.

Could you plese help me understand why this is happening the way it is?


Update: Following Henk Holterman's suggestion, here is the implementation of TestTime:

public static void TestTime<T, TR>(Func<T, TR> action, T obj,
                                   int iterations)
{
    Stopwatch stopwatch = Stopwatch.StartNew();
    for (int i = 0; i < iterations; ++i)
        action(obj);
    Console.WriteLine(action.Method.Name + " took " + stopwatch.Elapsed);
}

public static void TestTime<T1, T2, TR>(Func<T1, T2, TR> action, T1 obj1,
                                        T2 obj2, int iterations)
{
    Stopwatch stopwatch = Stopwatch.StartNew();
    for (int i = 0; i < iterations; ++i)
        action(obj1, obj2);
    Console.WriteLine(action.Method.Name + " took " + stopwatch.Elapsed);
}
Daniel A.A. Pelsmaeker
  • 47,471
  • 20
  • 111
  • 157
Hosam Aly
  • 41,555
  • 36
  • 141
  • 182
  • 1
    A while ago I found a blog post where a matrix inversion (or something like it) was to be optimized. The results proved that jagged arrays where a lot faster than multidimensional arrays. I cannot remember which blog it was though. – dalle Jan 22 '09 at 12:17
  • 1
    It was the B# .NET Blog: http://community.bartdesmet.net/blogs/bart/archive/2007/02/27/c-quiz-need-for-speed.aspx http://community.bartdesmet.net/blogs/bart/archive/2007/03/13/answers-to-c-quiz-need-for-speed.aspx – dalle Jan 22 '09 at 12:20
  • Is this built in release mode with optimizations? – yfeldblum Jan 22 '09 at 13:07
  • Yes @Justice, it is, and ran from the command line, with process priority set to realtime and thread priority set to highest. – Hosam Aly Jan 22 '09 at 13:22
  • Are you positive that this code is really being optimized by the JIT-compiler? – John Leidegren Feb 28 '09 at 14:22
  • @John, I run it in Release mode from console, so it should be optimized. – Hosam Aly Feb 28 '09 at 14:41
  • not sure if it changed over time and .NET Version but I ran the code currently and 2D Array is fastest depending on compiled mode -Debug compiled: `-[] sum took 00:00:04.9336682 -[,] sum took 00:00:06.6317560 -[][] sum took 00:00:07.3573180` and it changes when release compiled: `sum took 00:00:01.4856096 sum took 00:00:02.2553230 sum took 00:00:01.4376627` – Falco Alexander Oct 22 '15 at 09:46
  • FYI this issue is now logged against the coreclr github project/repository: https://github.com/dotnet/coreclr/issues/4059#issuecomment-208491798 – redcalx Apr 11 '16 at 19:30

10 Answers10

51

Single dimensional arrays with a lower bound of 0 are a different type to either multi-dimensional or non-0 lower bound arrays within IL (vector vs array IIRC). vector is simpler to work with - to get to element x, you just do pointer + size * x. For an array, you have to do pointer + size * (x-lower bound) for a single dimensional array, and yet more arithmetic for each dimension you add.

Basically the CLR is optimised for the vastly more common case.

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • I apologize. I was actually using a multi-dimensional array, but I used the wrong term. Sorry! – Hosam Aly Jan 22 '09 at 12:09
  • I'd been assuming multi-dimensional anyway, to be honest :) A jagged array is typically a "vector of vectors" in CLR terms, so I'm not surprised at it being faster than a multi-dimensional array. – Jon Skeet Jan 22 '09 at 12:28
  • 29
    I'm baffled by this, a multidimensional array should be faster than a jagged array. It's the CLRs fault if anything. – John Leidegren Feb 28 '09 at 14:23
  • 3
    A good compiler should be able to move all bound checking in front of the loop and generate basically the same code as d1 for d2. This just proves that the MS compiler is not very good (for arrays). – ILoveFortran Feb 28 '09 at 15:11
  • 2
    @ILoveFortran: The JIT compiler (where the checks are actually emitted or omitted) is heavily optimized for speed of execution - the target is to have the JIT compilation faster than a typical page fault. Even then, the x64 JIT compiler does exactly the optimalization you're talking about, and the new compiler (not production release yet), RyuJIT, manages to get even more optimizations in. Also, the fact is, that even the x86 compiler removes bound checking altogether if you use `for (i = 0; i < ar.Length; i++)`, because that guarantees the bounds check in the for cycle itself. – Luaan Dec 03 '13 at 13:24
  • Single dimensional arrays with a lower bound of 0 also partially implement IList, and therefore ICollection and IEnumerable, which is somewhat noteworthy to me. – Kyle Baran Jun 25 '14 at 12:12
  • Curious if this answer is still as accurate today as it was in '09? – BVernon Sep 02 '21 at 05:53
  • 1
    @BVernon: I strongly suspect it is - the performance of less-specialized arrays (i.e. ones with more flexible bounds and dimensions) may well have improved, but fundamentally it's still a more complicated scenario than an IL vector. – Jon Skeet Sep 02 '21 at 06:02
  • 1
    .NET 7 (scheduled to drop later this year) includes JIT improvements related to multi-dimensional arrays. See Stephen Toub's blog post on the .NET 7 performance improvements: https://devblogs.microsoft.com/dotnet/performance_improvements_in_net_7/ Example JIT improvement that he references: https://github.com/dotnet/runtime/pull/70271 – Ryan Thomas Sep 05 '22 at 00:22
12

Array bounds checking?

The single-dimension array has a length member that you access directly - when compiled this is just a memory read.

The multidimensional array requires a GetLength(int dimension) method call that processes the argument to get the relevant length for that dimension. That doesn't compile down to a memory read, so you get a method call, etc.

In addition that GetLength(int dimension) will do a bounds check on the parameter.

JeeBee
  • 17,476
  • 5
  • 50
  • 60
  • Hmm good thinking have you verified this in someway (debugged the code, used reflector etc)? – Sandeep Datta Jan 22 '09 at 12:01
  • 1
    I know in Java that a method call to a getter or setter actually optimises out the method call and directly accesses the value. I can't see why .NET would be different. Also there will be bounds checks on the argument to GetLength(int index). – JeeBee Jan 22 '09 at 12:04
  • I apologize. I was actually using a multi-dimensional array, but I used the wrong term. Sorry! – Hosam Aly Jan 22 '09 at 12:08
  • I was actually talking about multidimensional arrays thankfully! – JeeBee Jan 22 '09 at 12:09
5

Interestly, I ran the following code from above using VS2008 NET3.5SP1 Win32 on a Vista box, and in release/optimize the difference was barely measurable, while debug/noopt the multi-dim arrays were much slower. (I ran the three tests twice to reduce JIT affects on the second set.)

  Here are my numbers: 
    sum took 00:00:04.3356535
    sum took 00:00:04.1957663
    sum took 00:00:04.5523050
    sum took 00:00:04.0183060
    sum took 00:00:04.1785843 
    sum took 00:00:04.4933085

Look at the second set of three numbers. The difference is not enough for me to code everything in single dimension arrays.

Although I haven't posted them, in Debug/unoptimized the multidimension vs. single/jagged does make a huge difference.

Full program:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;

namespace single_dimension_vs_multidimension
{
    class Program
    {


        public static double sum(double[] d, int l1) {    // assuming the array is rectangular 
            double sum = 0; 
            int l2 = d.Length / l1; 
            for (int i = 0; i < l1; ++i)   
                for (int j = 0; j < l2; ++j)   
                    sum += d[i * l2 + j];   
            return sum;
        }

        public static double sum(double[,] d)
        {
            double sum = 0;  
            int l1 = d.GetLength(0);
            int l2 = d.GetLength(1);   
            for (int i = 0; i < l1; ++i)    
                for (int j = 0; j < l2; ++j)   
                    sum += d[i, j]; 
            return sum;
        }
        public static double sum(double[][] d)
        {
            double sum = 0;   
            for (int i = 0; i < d.Length; ++i) 
                for (int j = 0; j < d[i].Length; ++j) 
                    sum += d[i][j];
            return sum;
        }
        public static void TestTime<T, TR>(Func<T, TR> action, T obj, int iterations) 
        { 
            Stopwatch stopwatch = Stopwatch.StartNew();
            for (int i = 0; i < iterations; ++i)      
                action(obj);
            Console.WriteLine(action.Method.Name + " took " + stopwatch.Elapsed);
        }
        public static void TestTime<T1, T2, TR>(Func<T1, T2, TR> action, T1 obj1, T2 obj2, int iterations)
        {
            Stopwatch stopwatch = Stopwatch.StartNew(); 
            for (int i = 0; i < iterations; ++i)    
                action(obj1, obj2); 
            Console.WriteLine(action.Method.Name + " took " + stopwatch.Elapsed);
        }
        public static void Main() {   
            Random random = new Random(); 
            const int l1  = 1024, l2 = 1024; 
            double[ ] d1  = new double[l1 * l2]; 
            double[,] d2  = new double[l1 , l2];  
            double[][] d3 = new double[l1][];   
            for (int i = 0; i < l1; ++i)
            {
                d3[i] = new double[l2];   
                for (int j = 0; j < l2; ++j)  
                    d3[i][j] = d2[i, j] = d1[i * l2 + j] = random.NextDouble();
            }    
            const int iterations = 1000;
            TestTime<double[], int, double>(sum, d1, l1, iterations);
            TestTime<double[,], double>(sum, d2, iterations);

            TestTime<double[][], double>(sum, d3, iterations);
            TestTime<double[], int, double>(sum, d1, l1, iterations);
            TestTime<double[,], double>(sum, d2, iterations);
            TestTime<double[][], double>(sum, d3, iterations); 
        }

    }
}
Cameron
  • 2,903
  • 1
  • 30
  • 31
  • I extended your test program to 3 dimensional arrays (256*256*256), and got the following results (without debugger, release build, .Net 4.5, Intel Core 2 Duo @2.20GHz, 64-bit Win7): http://pastebin.com/SUtMSXkk Here's the program in case you are interested: http://pastebin.com/Uzh9jrAM This goes to show that the differences are very much non-negligible. – d3dave Nov 22 '13 at 15:59
  • Yes, it's same here. I suspect the OP had selected the "Release" configuration, and then run the program with F5, instead of from command line. Or perhaps it's because we are running 64 bit, or maybe it's because the JIT is now optimized for this case.. – Erti-Chris Eelmaa Oct 24 '14 at 15:38
  • Actually, see what happens if you change numbers a little higher, eg if it runs ~30 seconds. I got pretty significant results. Eg multi-dimensional array was 2x slower than jagged array. (jagged array run 30 seconds, multi-dimensional run 1:24 seconds) – Erti-Chris Eelmaa Oct 24 '14 at 18:55
3

Because a multidimensional array is just a syntactic sugar as it is really just a flat array with some index calculation magic. On the other hand, a jagged array is like, an array of arrays. With a two-dimensional array, accessing an element requires reading the memory just once, while with a two level jagged array, you need to read the memory twice.

EDIT: Apparently the original poster mixed up "jagged arrays" with "multi-dimensional arrays" so my reasoning doesn't exactly stand. For the real reason, check Jon Skeet's heavy artillery answer above.

Tamas Czinege
  • 118,853
  • 40
  • 150
  • 176
  • I apologize. I was actually using a multi-dimensional array, but I used the wrong term. Sorry! – Hosam Aly Jan 22 '09 at 12:07
  • 1
    @DrJokepu: It should be faster to use a multidimensional array than a jagged array, but actually it is the other way around. – dalle Jan 22 '09 at 12:16
  • 1
    This "index calculation magic" is the heart of my question. Shouldn't it be (at least) as fast as my first method? – Hosam Aly Jan 22 '09 at 12:16
3

Which is fastest depends your arrays sizes.

Image for easy read:

enter image description here

Console result:

// * Summary *

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18363.997 (1909/November2018Update/19H2)
Intel Core i7-6700HQ CPU 2.60GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.1.302
  [Host]        : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT
  .NET Core 3.1 : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT

Job=.NET Core 3.1  Runtime=.NET Core 3.1

|           Method |    D |            Mean |         Error |        StdDev |      Gen 0 |     Gen 1 |     Gen 2 |  Allocated |
|----------------- |----- |----------------:|--------------:|--------------:|-----------:|----------:|----------:|-----------:|
| 'double[D1][D2]' |   10 |        376.2 ns |       7.57 ns |      12.00 ns |     0.3643 |         - |         - |     1144 B |
| 'double[D1, D2]' |   10 |        325.5 ns |       3.71 ns |       3.47 ns |     0.2675 |         - |         - |      840 B |
| 'double[D1][D2]' |   50 |      4,821.4 ns |      44.71 ns |      37.34 ns |     6.8893 |         - |         - |    21624 B |
| 'double[D1, D2]' |   50 |      5,834.1 ns |      64.35 ns |      60.20 ns |     6.3629 |         - |         - |    20040 B |
| 'double[D1][D2]' |  100 |     19,124.4 ns |     230.39 ns |     454.77 ns |    26.2756 |    0.7019 |         - |    83224 B |
| 'double[D1, D2]' |  100 |     23,561.4 ns |     299.18 ns |     279.85 ns |    24.9939 |         - |         - |    80040 B |
| 'double[D1][D2]' |  500 |  1,248,458.7 ns |  11,241.19 ns |  10,515.01 ns |   322.2656 |  160.1563 |         - |  2016025 B |
| 'double[D1, D2]' |  500 |    966,940.8 ns |   5,694.46 ns |   5,326.60 ns |   303.7109 |  303.7109 |  303.7109 |  2000034 B |
| 'double[D1][D2]' | 1000 |  8,987,202.8 ns |  97,133.16 ns |  90,858.41 ns |  1421.8750 |  578.1250 |  265.6250 |  8032582 B |
| 'double[D1, D2]' | 1000 |  3,628,421.3 ns |  72,240.02 ns | 177,206.01 ns |   179.6875 |  179.6875 |  179.6875 |  8000036 B |
| 'double[D1][D2]' | 1500 | 26,496,994.4 ns | 380,625.25 ns | 356,037.09 ns |  3406.2500 | 1500.0000 |  531.2500 | 18048064 B |
| 'double[D1, D2]' | 1500 | 12,417,733.7 ns | 243,802.76 ns | 260,866.22 ns |   156.2500 |  156.2500 |  156.2500 | 18000038 B |
| 'double[D1][D2]' | 3000 | 86,943,097.4 ns | 485,339.32 ns | 405,280.31 ns | 12833.3333 | 7000.0000 | 1333.3333 | 72096325 B |
| 'double[D1, D2]' | 3000 | 57,969,405.9 ns | 393,463.61 ns | 368,046.11 ns |   222.2222 |  222.2222 |  222.2222 | 72000100 B |

// * Hints *
Outliers
  MultidimensionalArrayBenchmark.'double[D1][D2]': .NET Core 3.1 -> 1 outlier  was  removed (449.71 ns)
  MultidimensionalArrayBenchmark.'double[D1][D2]': .NET Core 3.1 -> 2 outliers were removed, 3 outliers were detected (4.75 us, 5.10 us, 5.28 us)
  MultidimensionalArrayBenchmark.'double[D1][D2]': .NET Core 3.1 -> 13 outliers were removed (21.27 us..30.62 us)
  MultidimensionalArrayBenchmark.'double[D1, D2]': .NET Core 3.1 -> 1 outlier  was  removed (4.19 ms)
  MultidimensionalArrayBenchmark.'double[D1, D2]': .NET Core 3.1 -> 3 outliers were removed, 4 outliers were detected (11.41 ms, 12.94 ms..13.61 ms)
  MultidimensionalArrayBenchmark.'double[D1][D2]': .NET Core 3.1 -> 2 outliers were removed (88.68 ms, 89.27 ms)

// * Legends *
  D         : Value of the 'D' parameter
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Gen 0     : GC Generation 0 collects per 1000 operations
  Gen 1     : GC Generation 1 collects per 1000 operations
  Gen 2     : GC Generation 2 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 ns      : 1 Nanosecond (0.000000001 sec)

Benchmark code:

[SimpleJob(BenchmarkDotNet.Jobs.RuntimeMoniker.NetCoreApp31)]
[MemoryDiagnoser]
public class MultidimensionalArrayBenchmark {
    [Params(10, 50, 100, 500, 1000, 1500, 3000)]
    public int D { get; set; }

    [Benchmark(Description = "double[D1][D2]")]
    public double[][] JaggedArray() {
        var array = new double[D][];
        for (int i = 0; i < array.Length; i++) {
            var subArray = new double[D];
            array[i] = subArray;

            for (int j = 0; j < subArray.Length; j++) {
                subArray[j] = j + i * 10;
            }
        }

        return array;
    }

    [Benchmark(Description = "double[D1, D2]")]
    public double[,] MultidimensionalArray() {
        var array = new double[D, D];
        for (int i = 0; i < D; i++) {
            for (int j = 0; j < D; j++) {
                array[i, j] = j + i * 10;
            }
        }

        return array;
    }
}

huang
  • 919
  • 11
  • 22
  • I ran my own quick tests (some other code) and Multi-Dim seemed to be faster in general. It would make sense multi-dim is faster and was supprised in 2002 when learning .net that Jagged was faster. Much more efficient math can be done on multi-dim though one would need a big chunk of contiguous memory. What seems to be even faster in .net core is a self made version using a single array "int outValue = MyArray[x * DIM1 * DIM2 + y* DIM2 + z]". – SunsetQuest Sep 12 '20 at 21:50
2

Jagged arrays are arrays of class references (other arrays) up until the leaf array which may be an array of a primitive type. Hence memory allocated for each of the other arrays can be all over the place.

Whereas a mutli-dimensional array has its memory allocated in one contigeous lump.

AnthonyWJones
  • 187,081
  • 35
  • 232
  • 306
  • I apologize. I was actually using a multi-dimensional array, but I used the wrong term. Sorry! – Hosam Aly Jan 22 '09 at 12:10
  • Interestingly, if you allocate the jagged array consecutively, it will be consecutive in memory too, even though it's a bunch of references (.NET managed heap does not look for free space, unlike malloc etc.). This means that unless you're doing something very wrong, you're still getting good cache use. – Luaan Dec 03 '13 at 13:28
1

I think it has got something to do for the fact that jagged arrays are actually arrays of arrays hence there are two levels of indirection to get to the actual data.

Sandeep Datta
  • 28,607
  • 15
  • 70
  • 90
  • I apologize. I was actually using a multi-dimensional array, but I used the wrong term. Sorry! – Hosam Aly Jan 22 '09 at 12:12
  • @Henk: And what's more surprising is the fact that (IMO) bounds checking for multi-dimentional arrays in a for loop with fixednum iterations can be optimized away due to the regular (=rectangular) nature of the array! I guess this optimization isn't being done for some obscure reason. – Sandeep Datta Feb 28 '09 at 16:33
1

I'm with everyone else here

I had a program with three dimension array, let me tell you that when I moved the array into two dimension, I saw a huge boost and then I moved to a one dimension array.

In the end, I think I saw over 500% performance boost in the execution time.

only drawback was the complexity added to find out where was what in the one dimensional array, versus the three one.

Fredou
  • 19,848
  • 10
  • 58
  • 113
0

I think multi-dimensional is slower, the runtime has to check two or more(three dimensional and up) bounds check.

Michael Buen
  • 38,643
  • 9
  • 94
  • 118
-1

Bounds checking. Your "j" variable could exceed l2 provided "i" was less than l1. This would not be legal in the second example

Damien_The_Unbeliever
  • 234,701
  • 27
  • 340
  • 448
  • I didn't downvote, but isn't bounds checking applied in both cases? – Dirk Vollmar Jan 22 '09 at 11:59
  • Bound checking is correct (or at least a relevant aspect), but the reason given is wrong (although I didn't downvote it), it's just that there is more bounds checking with the jagged array. GetLength(int) checks the parameter (>0, – JeeBee Jan 22 '09 at 12:02
  • My point was that in *the posted code*, where a multi-dimensional array was simulated using a singe-dimensional one, that invalid indexes code be used, provided that the arithmetic arrived at a final index value within the bounds – Damien_The_Unbeliever Jan 22 '09 at 15:42