VB.NET Memory Size of Dictionary?

Question

I have a class:

Class Test

    Friend myDict As Dictionary(Of Byte, Byte) = New Dictionary(Of Byte, Byte)

    Friend Test1 As Boolean
    Friend Test2 As Boolean
    Friend Test3 As Boolean
    Friend Test4 As Boolean

    Friend Test5 As Byte
    Friend Test6 As Byte

    Friend Test7 As Byte

    Friend Test8 As Boolean

   Public Sub New(smt As String)

    Test1 = True
    Test2 = True
    Test3 = True
    Test4 = True

    Test5 = 51
    Test6 = 58
    Test7 = 0

    Test8 = True

   ' ADDING 64 DICTIONARY ENTRIES HERE

    myDict.Add(11, 0)
    myDict.Add(21, 0)
    myDict.Add(31, 0)
    myDict.Add(41, 0)
    myDict.Add(51, 0)
    myDict.Add(61, 0)
    myDict.Add(71, 0)
    myDict.Add(81, 0)

    myDict.Add(12, 0)
    myDict.Add(22, 0)
    myDict.Add(32, 0)
    myDict.Add(42, 0)
    myDict.Add(52, 0)
    myDict.Add(62, 0)
    myDict.Add(72, 0)
    myDict.Add(82, 0)

    myDict.Add(13, Nothing)
    myDict.Add(23, Nothing)
    myDict.Add(33, Nothing)
    myDict.Add(43, Nothing)
    myDict.Add(53, Nothing)
    myDict.Add(63, Nothing)
    myDict.Add(73, Nothing)
    myDict.Add(83, Nothing)

    myDict.Add(14, Nothing)
    myDict.Add(24, Nothing)
    myDict.Add(34, Nothing)
    myDict.Add(44, Nothing)
    myDict.Add(54, Nothing)
    myDict.Add(64, Nothing)
    myDict.Add(74, Nothing)
    myDict.Add(84, Nothing)

    myDict.Add(15, Nothing)
    myDict.Add(25, Nothing)
    myDict.Add(35, Nothing)
    myDict.Add(45, Nothing)
    myDict.Add(55, Nothing)
    myDict.Add(65, Nothing)
    myDict.Add(75, Nothing)
    myDict.Add(85, Nothing)

    myDict.Add(16, Nothing)
    myDict.Add(26, Nothing)
    myDict.Add(36, Nothing)
    myDict.Add(46, Nothing)
    myDict.Add(56, Nothing)
    myDict.Add(66, Nothing)
    myDict.Add(76, Nothing)
    myDict.Add(86, Nothing)

    myDict.Add(17, 0)
    myDict.Add(27, 0)
    myDict.Add(37, 0)
    myDict.Add(47, 0)
    myDict.Add(57, 0)
    myDict.Add(67, 0)
    myDict.Add(77, 0)
    myDict.Add(87, 0)

    myDict.Add(18, 0)
    myDict.Add(28, 0)
    myDict.Add(38, 0)
    myDict.Add(48, 0)
    myDict.Add(58, 0)
    myDict.Add(68, 0)
    myDict.Add(78, 0)
    myDict.Add(88, 0)

    Console.WriteLine("Created New!")

End Sub

Public Sub New()

End Sub

End Class

When I'm cloning 1.000.000 of this class using this:

Public Clones As List(Of Test) = New List(Of Test)

Public Sub BenchmarkTest(cnt As Long)

    Clones.Clear()
    GC.Collect()

    Dim t As Long = Now.Ticks
    Dim tst As Test = New Test("")

    For x As Long = 1 To cnt
        Clones.Add(DeepCopy(tst))
    Next

    Console.WriteLine("Copied " + Clones .Count.ToString + " in " + ((Now.Ticks - t) / 10000).ToString + "ms (" + ((Now.Ticks - t) / 10000000).ToString + " secs)")

End Sub

Public Function DeepCopy(inputTest As Test) As Test

    Dim other As Test = New Test()
    other.Test1 = inputTest.Test1
    other.Test2 = inputTest.Test2 
    other.Test3 = inputTest.Test3 
    other.Test4 = inputTest.Test4 
    other.Test5 = inputTest.Test5 
    other.Test6 = inputTest.Test6 
    other.myDict = New Dictionary(Of Byte, Byte)(inputTest.myDict)
    Return other

End Function

And I open up the task manager to see how much memory my application is using, I see it's using 1,300+Mb

Well, according to my calculations, my class size has to be 136 bytes only. (64 dictionary entries (of byte, byte) = 128bytes + 8bytes(for test1 to test8) = 136bytes)

Multiplying it with 1 million should be about 130Mbytes, not 1300.

What is wrong with my calculation ? Why it's using almost 10 times higher memory?

Answer 1

You're over-simplifying your calculations. .NET objects come with necessary overhead and cannot be simply flattened to byte-by-byte equivalencies. (Your calculation reminds me of a C-style struct alignment.)

The main "grabber" of memory in your Test class is going to be the Dictionary. If you analyze your process step by step you will see that simply instantiating an empty dictionary will consume memory. When you start adding items things get a little more interesting. .NET collections do not grow in a linear fashion as you add items, that would be too inefficient. The collection will grow by some internally-defined scheme (sometimes a Fibonacci sequence, sometimes a simple doubling of current capacity) where an addition of enough items will assume another block of items will be added and reserve that memory.

I know this is all theoretical, abstract, high-level discussion and will not yield a specific answer but I mean to communicate that the issue is very complicated and getting exact calculation on memory utilization can be very difficult.

Here is a decent Code Project article that mentions some of the issues of performance and memory usage in Dictionaries.

Here is another interesting link from Simple-Talk. The comment section contains an interesting discussion on the growth strategies for .NET collections. Enjoy.

Quick example in C#. (This is dirty, don't take it to the bank, but demonstrates the point. See GC.GetTotalMemory)

Console.WriteLine("Bar is doing stuff.");
Console.WriteLine(GC.GetTotalMemory(true));
var dict = new Dictionary<byte, byte>();
Console.WriteLine(GC.GetTotalMemory(true));
dict.Add(8, 9);
Console.WriteLine(GC.GetTotalMemory(true));

Output:

53,328
53,980
54,052

Notice that an empty byte/byte dictionary takes up 652 bytes. Adding a single entry of byte/byte increases the dictionary size by 72 bytes... One thing you're not seeing is that each entry is actually represented internally by an instance of a DictionaryEntry class.