Why can I use up all the system's memory by repeatedly allocating and deallocating memory?

Question

In the simple example C++ code below, I allocate a lot of memory (~4GB) on the heap with new and release it again with delete. In top (or some other memory monitor), I can see that the 4GBs are not returned to the OS until the program is shutdown.

I learned (among others in my related question) that this is normal because the unused heap is returned to the OS in hunks, which does not need to be immediately after the delete.

I would expect that then the already deallocated memory is returned to the OS as soon as it requests more memory (e.g. when I start a second instance of the example program). Unfortunately, this does not seem to be the case: when I run a second instance of my example after the first instance deallocated the memory, the consumption of both instances rises to 8GB. I can repeat this with several instances until all the physical memory of the system is used up and it starts swapping and becoming unresponsive.

The output of top when running 4 instances of the example program (each waiting in std::cin after deleting the data) looks like this:

Mem:     16065M total,    15983M used,       82M free,        0M buffers
Swap:     2053M total,     1323M used,      730M free,      139M cached

Is that really the wanted behavior or am I missing something?

I can observe this on different Linux systems, but not on Mac OS 10.11. On Mac OS the memory is returned to the OS immediately after the delete, what I actually expected to happen.

#include <iostream>
#include <stdlib.h>
#include <fstream>
#include <vector>

class MyObject
{
public:
    MyObject(int r=1000)
    : array(new float[r])
    {
        for (int i = 0; i<r;i++)
        {
            array[i] = random();
        }
    }

    ~MyObject()
    {
        delete[] array;
    }

public:
    float* array;
};


int main(int argc,char*argv[])
{
    char a;
    const int count=1000000;

    std::cout<<"Start after input"<<std::endl;
    std::cin >> a;
    // /proc/PROC_ID/status at this point:
    // VmSize:     11468 kB
    // VmLck:          0 kB
    // VmHWM:        960 kB
    // VmRSS:        960 kB
    // VmData:       144 kB
    {
        std::vector<MyObject*> vec(count);
        for(int i=0; i<count; i++)
        {
            vec[i] = new MyObject;
        }

        std::cout<<"Release after input"<<std::endl;
        std::cin >> a;
        // VmSize:   3972420 kB
        // VmLck:          0 kB
        // VmHWM:    3962016 kB
        // VmRSS:    3962016 kB
        // VmData:   3961096 kB

        for (int i=0; i<count; i++)
        {
            delete vec[i];
            vec[i]=NULL;
        }
    }

    std::cout<<"Shutdown after input"<<std::endl;
    std::cin >> a;
    // VmSize:   3964604 kB
    // VmLck:          0 kB
    // VmHWM:    3962016 kB
    // VmRSS:    3954212 kB
    // VmData:   3953280 kB


    return 0;
}

Answer 1

Your platform's memory allocator doesn't handle this particular case the way you would like it to. Most likely, it only returns sufficiently large allocations to the operating system and these allocations are too small. If you have unusual requirements, you should select an allocator known to meet those requirements. Having a very large number of very small allocations that are freed at the same time is such an unusual application.

It may be as simple as a wrapper allocator that allocates blocks much larger than requested and splits them up. This should make the blocks large enough for your platform's allocator to map them one-to-one to address space requested from the operating system.