How to get size of a Numba Dictionary?

Question

I'm trying to get size of Numba typed dictionary in bytes:

from numba import njit
from numba.typed import Dict
from sys import getsizeof as gso
@njit
def get_dict(n):
    d = {0:0}
    for i in range(1,n):
        d[i] = i
    return d

print(gso(get_dict(10))
print(gso(get_dict(10000)))

In both cases getsizeof function returns 64 bytes. Obviously, size of Dictionary must depend on length.

If I transform typed dictionary into native Python dictionary using dict(), it works and returns 376 and 295016:

print(gso(dict(get_dict(10))))
print(gso(dict(get_dict(10000))))

How can I measure it?

Answer 1

Currently (numba 0.46.0) what you intended to do is very likely not possible with a DictType.

sys.getsizeof on containers is tricky at best and very misleading in the worst case. The problem is that getsizeof needs to reduce possibly very complicated implementation details to one integer number.

The first problem is that calling sys.getsizeof on a container typically only reports the size of the container not the contents of the container - or in case of an opaque container it only returns the size of the wrapper. What you encountered was the latter - DictType is just a wrapper for an opaque numba struct that's defined (probably) in c. So the 64 you're seeing is actually correct, that's the size of the wrapper. You can probably access the wrapped type, but given that it's hidden behind private attributes it's not intended for non-numba-code, so I won't go down that path - mostly because any answer relying on those kind of implementation details may be outdated at any time.

However sys.getsizeof requires deep understanding of the implementation details to be interpreted correctly. So even for the plain dict it's not obvious what the number represents. It's certainly the calculated memory in bytes of the container (without the contents) but it could also be a key-sharing dictionary (in your case it's not a key-sharing dictionary) where the number would be accurate but since part of the key-sharing dictionary are shared it's probably not the number you are looking for. As mentioned, it normally also doesn't account for the contents of the container but that's an implementation detail as well, for example numpy.array includes the size of the contents, while list, set, etc. don't. That's because a numpy array doesn't have "real contents" - at least it doesn't have Python objects as content.

So even if the DictType wrapper would report the size of the underlying dictionary, you would still need to know if it contained the contents in the dictionary, or as pointer to Python objects or (even more convoluted) as objects defined in another language (like C, or C++) to interpret the results correctly.

So my advise would be to not use sys.getsizeof, except out of curiosity or academic interest. And then only if you're willing to dig through all the implementation details (that may or may not change any time) in order to interpret the results correctly. If you're really interested in memory consumption you are often better off using a tool that tracks the memory usage of your program as a whole. That still has a lot of pitfalls (memory-reuse, unused memory-allocations) and requires a significant amount of knowledge how memory is used to interpret correctly (virtual memory, shared memory, and how memory is allocated) but it often yields a more realistic view how much memory your program actually uses.

import gc
import numba as nb
import psutil

@nb.njit
def get_dict(n):
    d = {0:0}
    for i in range(1,n):
        d[i] = i
    return d

get_dict(1)

gc.collect()
print(psutil.Process().memory_info())
d = get_dict(100_000)
gc.collect()
print(psutil.Process().memory_info())

Which gives on my computer:

pmem(rss=120696832, vms=100913152, num_page_faults=34254, peak_wset=120700928,
     wset=120696832, peak_paged_pool=724280, paged_pool=724280, peak_nonpaged_pool=1255376,
     nonpaged_pool=110224, pagefile=100913152, peak_pagefile=100913152, private=100913152)

pmem(rss=126820352, vms=107073536, num_page_faults=36717, peak_wset=129449984, 
     wset=126820352, peak_paged_pool=724280, paged_pool=724280, peak_nonpaged_pool=1255376, 
     nonpaged_pool=110216, pagefile=107073536, peak_pagefile=109703168, private=107073536)

Which shows that the program needed 6 123 520 bytes of memory (using the "rss" - resident set size) more after the call than it had allocated before.

And similar for a plain Python dictionary:

import gc
import psutil

gc.collect()
print(psutil.Process().memory_info())
d = {i: i for i in range(100_000)}
gc.collect()
print(psutil.Process().memory_info())
del d
gc.collect()

Which gave a difference of 8 552 448 bytes on my computer.

Note that these numbers represent the complete process, so treat these with caution. For example for small values (get_dict(10)) they return 4096 on my windows computer because that's the page-size of windows. The program there actually allocated more space than the dictionary needs because of the OS-restrictions.

However even with these pitfalls and restrictions that's still significantly more accurate if you're interested the memory-requirements of your program.

If you still (out of curiosity) want to know how much memory the DictType theoretically needs you should probably ask the numba developers to enhance numba so that they implement __sizeof__ for their Python wrappers so that the numbers are more representative. You could for example open an issue on their issue tracker or ask on their mailing list.

An alternative may be to use other third-party tools, for example pypmler, but I haven't used these myself so I don't know if they work in this case.