C# array of objects, very large, looking for a better way

Question

Ok, so In one of my projects, im trying to remake the way it stores certain variables, i have a simple array of objects. The class that these objects refer to is:

class Blocks
{
    public byte type = Block.Empty;
    byte lastblock = Block.Zero;
}

I plan to add more to it, in the current class type is what the objects current value is, and lastblock is what the object used to be.

i create the array like this:

blocks = new Blocks[width * depth * length];

for (int i = 0; i < ((width * length) * depth); i++)
{
    blocks[i] = new Blocks();
}

The problem that I'm having, is that when i create an array that's very large (512,512,512 or 134217728 for those of you whom don't like math), the array gets huge, upwards of 3.5 gigs.

The old way that this array was created was a little simpler, but much harder to expand, it simple created an array of bytes representing the current block, and seems to only use 2 megs of ram when its actually loaded (which i dont get since 134217728 bytes should be around 134 megs... right?). It just baffles me that object references could generate that much more ram usage.

Am i doing something wrong, or should i just go back to the old way it was done? I would like the object references simply because it means that all my variables are in 1 array rather then in 4 separate arrays, which seems as though it would be better for the system.

EDIT:

After working through a few different ways of doing this, i found that changing

class Blocks

to

struct Blocks

Made a world of difference, Thank you community for that welcome tip for future use, unfortunately i didn't want to just add two bytes in a struct and call it done, that was where i stopped to test my design and wound up with the original problem. After adding anything else to the struct (anything else that's on my list at least, meaning either a player object reference or a player name string.) It causes an out-of-memory exception that means I'm not going to be able to use this system after all.

But the knowledge of how to do it will be quite helpful in the future. For that, thank you again.

Answer 1

Here's what I tried with a structure type instead of a class type:

public struct Block
{
    public byte _current;
    public byte _last;
}

public static void RunSnippet()
{
    Block[] blocks = new Block[512 * 512 * 512];

    for (int i = 0; i < ((512 * 512) * 512); i++)
    {
        blocks[i] = new Block();
    }
}

The snippet ran almost instantly and ate around 267 Mb of RAM.

So give struct a try if that's possible.

Answer 2

You can use List class to manage unlimited number of objects. Please take a look at the link that I provided. You can add infinite (well, theoretically) number of objects into a list.

Using lists, you can easily access any item by their index. It also has methods to search, sort and manipulate objects contained in it.

If you use list, your code will look somewhat like below -

List<Blocks> blocks = new List<Blocks>();

for (int i = 0; i < ((width * length) * depth); i++)  // The amount of items that you want to add
{
    Blocks b = new Blocks();
    blocks.Add(b);
}

You can access every item in this list as follows -

foreach(Blocks b in blocks)
{
    // You have the object, do whatever you want
}

You can find any particular index of an object contained in the list. See this method example.

So using a list, you will be able to easily manage a large number of objects in an uniform way.

To learn more, go here.

Answer 3

Read this: Object Overhead: The Hidden .NET Memory Allocation Cost.

The total cost of an object of yours is like 16 bytes (on a 32 bits system). (8 bytes for "header", 4 bytes for your fields, 4 for the reference) And 512 * 512 * 512 * 16 = 2.1gb :-)

But you are probably on a 64 bits system, so it's 16 + 8 + 8 = 28, so 4.29gb.

Answer 4

You should consider using "struct" instead of "class".

http://msdn.microsoft.com/en-us/library/ah19swz4(v=vs.71).aspx

"The struct type is suitable for representing lightweight objects such as Point, Rectangle, and Color. Although it is possible to represent a point as a class, a struct is more efficient in some scenarios. For example, if you declare an array of 1000 Point objects, you will allocate additional memory for referencing each object. In this case, the struct is less expensive."

Please publish your results if you try so.

Answer 5

When you create the array, you also instantiate a Blocks on each cell. Do you really need to do that?

blocks[i] = new Blocks();

When you don't instantiate a Blocks, you'd just have an array of empty references. In the access code you could check for null and return a default value. Something along these lines:

if(blocks[i,j] == null)  return new  Blocks();
else return blocks[i,j];

When writing also check if there is one, if not, create it first. That should save a lot of memory.

Using jagged arrays or nested lists should also help a lot.

Regards GJ

Answer 6

Structs are the way forward here, and would open up the possibility of optimising with unsafe code/pointer arithmetic

struct Block
{
    byte Current;
    byte Last;
}

Block[] blocks = new Block[512 * 512 * 512];

unsafe
{
    Block* currentBlock = &blocks;

    for (int i = 0; i < (512 * 512) * 512; i++)
    {
        currentBlock->Current = 0xff;
        currentBlock->Last = 0x00;

        currentBlock++;
    }
}

Of course someone will come along and say mutable structs are evil! (Only if you dont know how to use them)