Why Does a Memory Leak not Continue after Peaking?

Question

I created an intentional memory leak to demonstrate a point to people who will shortly be learning pointers.

int main()
{
    while (1)
    {
        int *a = new int [2];
        //delete [] a;
    }
}

If this is run without the commented code, the memory stays low and doesn't rise, as expected. However, if this is run as is, then on a machine with 2GB of RAM, the memory usage rapidly rises to about 1.5GB, or whatever is not in use by the system. Once it hits this point though, the CPU usage (which was previously max) greatly falls, and the memory usage as well, down to about 100MB.

What exactly caused this intervening action (if there's something more specific than "Windows", that'd be great), and why does the program not take up the CPU it would looping, but not terminate either? It seems like it's stuck between the end of the loop and the end of main.

Windows XP, GCC, MinGW.

How are you measuring "memory usage" and "CPU usage"? Is it possible that your measuring device is measuring the working set size, or physical memory size instead of virtual memory size? — Robᵩ, Apr 02 '12 at 16:15
"intervening action" is probably due to crash that is captured by some handler and you don't see it, and clean-up being performed after crash. — Daniel Mošmondor, Apr 02 '12 at 16:17
"I'm using the default task manager." That reports a lot of info. What stat are you using? — David Heffernan, Apr 02 '12 at 16:19
I was just using the processes tab with the mem usage and CPU usage columns. Adding in virtual memory actually shows the virtual memory continuing to rise, though. I'm still not exactly sure what happens to the physical memory though. Also, adding in a try-catch with all exceptions produced nothing. — chris, Apr 02 '12 at 16:24
Ah, actually once I left it going around 3-5x as long as the physical memory reclaim, it crashed and returned 3. — chris, Apr 02 '12 at 16:26

score 4 · Accepted Answer · answered Apr 02 '12 at 16:22

4

What's probably happening is that your code allocates all available physical RAM. When it reaches that limit, the system starts to allocate space on the swap file for it. That means it's (nearly) constantly waiting on the disk, so its CPU usage drops to (almost) zero.

The system may easily keep track of the fact that it never actually writes to the memory it allocates, so when it needs to be stored on the swap file, it'll just make a small record basically saying "process X has N bytes of uninitialized storage" instead of actually copying all the data to the hard drive (but I'm not sure of that, and it may well depend on the exact system you're using).

answered Apr 02 '12 at 16:22

Jerry Coffin

476,176
80
629
1,111

Indeed. I knew about the virtual memory and how it causes extreme slowness. I didn't know it actually tried to move so much over. That CPU usage explanation sounds reasonable too. – chris Apr 02 '12 at 16:31
It fails at around 2GB indicating that physical address space is a limiting factor (i.e. 32-bit XP). – Eddie Edwards Apr 02 '12 at 16:31

score 1 · Answer 2 · answered Apr 02 '12 at 16:28

To paraphrase Inigo Montoya, "I don't think that means what you think that means." The Windows task manager doesn't display the memory usage data that you are looking for.

The "Mem Usge" column displays something related to the working set size (or the resident set size) of the process. That is, "Mem Usage" displays a number related to the amount of physical memory currently allocated to your proccess.

The "VM Size" column displays a number wholly unrelated to the virtual memory subsystem (it is actually the size of the private heaps allocated by the process.

Try using a different tool to visual virtual memory usage. I suggest Process Explorer.

I actually did use process explorer at home, but I could never get used to the design over task manager, and it's not very often I use features exclusive from task manager. — chris, Apr 02 '12 at 17:47

score 0 · Answer 3 · answered Apr 02 '12 at 16:22

I guess when the program exhausts the available physical memory, it starts to use on-disk (virtual) memory, and it becomes so slow, it seems as if it's inactive. Try adding some speed visualization:

int counter = 0;
while (1)
{
    int *a = new int [2];
    ++counter;
    if (counter % 1000000 == 0)
        std::cout << counter << '\n'
}

score 0 · Answer 4 · answered Apr 02 '12 at 16:26

The default Memory column in the task manager of XP is the size of the working set of the process (the amount of physical memory allocated to that process), not the actual memory usage.

http://msdn.microsoft.com/en-us/library/windows/desktop/ms684891%28v=vs.85%29.aspx

http://blogs.msdn.com/b/salvapatuel/archive/2007/10/13/memory-working-set-explored.aspx

score 0 · Answer 5 · edited May 23 '17 at 11:56

The "Mem Usage" column of the task manager is probably the "working set" as explained by a few answers in this question, although to be honest I still get confused how the task manager refers to memory as it changes from version to version. This value goes up/down as you are obviously not actually using much memory at any given time. If you look at the "VM Size" you should see it constantly increase until something bad happens.

You can also given Process Explorer a try which I find easily to understand in how it displays things.

score 0 · Answer 6 · answered Apr 02 '12 at 16:40

Several things: first, if you're only allocating 2 ints at a time, it could take hours before you notice that the total memory usage is going up because of it. And second, on a lot of systems, allocation doesn't commit until you actually access the memory; the address space may be reserved, but you don't really have the memory (and the program will crash if you try to access the memory and there isn't any available).

If you want to simulate a leak, I'd recommend allocating at least a page at a time, if not considerably more, and writing at least one byte in each allocated page.

Well, it's in an unforgiving loop. That loop executes many many times each second. Even just using 1, I saw the memory rising by a lot before my eyes. — chris, Apr 02 '12 at 17:49

Why Does a Memory Leak not Continue after Peaking?

6 Answers6