How are dictionary variables stored in memory?

Question

Can anyone explain me how these python dictionary, list variables stored in memory. I do know that in python memory management is done using heap and stack. But I really couldn't find a simple explanation on how memory is allocated when dictionary variable is created, is it created in stack frame or heap space?

Answer 1

Well lets look at the source code to figure it out!

From: https://github.com/python/cpython/blob/master/Objects/dictobject.c

static PyObject *
new_dict(PyDictKeysObject *keys, PyObject **values)
{
    PyDictObject *mp;
    ...
        mp = PyObject_GC_New(PyDictObject, &PyDict_Type);
    ...
    return (PyObject *)mp;
}

So a new dict object appears to be allocated using PyObject_GC_New()

From: https://github.com/python/cpython/blob/master/Doc/c-api/gcsupport.rst#id9

.. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type)

   Analogous to :c:func:`PyObject_New` but for container objects with the
   :const:`Py_TPFLAGS_HAVE_GC` flag set.

From: https://github.com/python/cpython/blob/master/Objects/object.c

PyObject *
_PyObject_New(PyTypeObject *tp)
{
    PyObject *op;
    op = (PyObject *) PyObject_MALLOC(_PyObject_SIZE(tp));
    if (op == NULL)
        return PyErr_NoMemory();
    return PyObject_INIT(op, tp);
}

From: https://github.com/python/cpython/blob/master/Objects/obmalloc.c

#define MALLOC_ALLOC {NULL, _PyMem_RawMalloc, _PyMem_RawCalloc, _PyMem_RawRealloc, _PyMem_RawFree}
#ifdef WITH_PYMALLOC
#  define PYMALLOC_ALLOC {NULL, _PyObject_Malloc, _PyObject_Calloc, _PyObject_Realloc, _PyObject_Free}
#endif

#define PYRAW_ALLOC MALLOC_ALLOC
#ifdef WITH_PYMALLOC
#  define PYOBJ_ALLOC PYMALLOC_ALLOC
#else
# define PYOBJ_ALLOC MALLOC_ALLOC
static PyMemAllocatorEx _PyObject = PYOBJ_ALLOC;
...

void *
PyObject_Malloc(size_t size)
{
    /* see PyMem_RawMalloc() */
    if (size > (size_t)PY_SSIZE_T_MAX)
        return NULL;
    return _PyObject.malloc(_PyObject.ctx, size);
}

I think its safe to assume at this point that these will call malloc, calloc, realloc, and free.

At this point, this is no longer a python question, but the answer is it is dependent on the OS as to whether malloc will allocate on the stack or the heap.