Do you think its a good idea not to deallocate memory for small programs?

Question

Let us say my system has 4GB of RAM, and a certain program only consumes 100 MB of memory throughout its execution, and also runs for a limited time, like only 30 seconds. Don't you think its a good idea (by improving performance) not to deallocate memory at all during the execution of that program? Those 100 MBs would be freed anyway at the termination of that program.

Answer 1

The most important part of deallocating memory in a short running program is fascilitating memory reuse. Your processor has a limited cache, probably only a few MBs, and if you continuously allocate more memory you can never take advantage of that.

If however you deallocate some memory, and then reallocate and reuse it, there's a good chance that on reuse the memory will already be "hot" and resident in cache. This means you won't incur the penalty of a cache miss and your program will run faster.

Of course you are trading off the deallocation/reallocation cost for cache misses, but continuously allocating more and more memory is guaranteed to incur lowest level cache misses. Each of these cache misses cost ~100 instructions on a modern processor, way less than the cost of an allocation/deallocation.

Edit

So I made a little test program to test this.

#include <stdio.h>
#include <stdlib.h>

process1(){
    char * ptr = malloc(1 << 20);
    int i = 0;
    while (i < (1<<20)) ptr[i++] = rand();
    free(ptr);
}

process2(){
    char * ptr = malloc(1 << 20);
    int i = 0;
    while (i < (1<<20)) ptr[i++] = rand();
}


main(){
    int i = 100;
    while(i--) process1();
    i = 100;
    while(i--) process2();
}

Both processes chew through 100 MB of data, the first one deallocates it as it goes, the second does not. I profiled this with valgrind's cachegrind tool. Here are the results

fn=process1
0 943720200 2 2 419430900 100 0 209715800 1638400 32767
fn=process2
0 943719900 0 0 419430800 100 0 209715700 1638400 1605784

Okay not that exciting, i'll translate. By avoiding the free you saved less than .1% of instructions cycles, and you incurred about a million and a half LL cache misses. If we use the standard cycle estimation formula CEst = lr + 10 Llm + 100 LLm thats about a 16% degradation in performance from avoiding the free in the function alone.

I reran in callgrind to get the full story (i'm not gonna post the detailed results because they are way more complicated than the cache grind ones) but when we include the full call to free the result is the same, a 16% more cycles are used when you don't call free.

Feel free to do this same test on your own program, but in this case the results are clear. The cost of managing dynamic memory is truly trivial compared to the cost of continuously refreshing the cache with new memory.

Also, I didn't mention this, but the direct comparison that should be made is the cost of the free to the cost of the cache misses. The frees cost in total 38930 cycles, the cache misses 157332000, you saved 39 thousand cycles, and paid for it with 150 million.

Answer 2

Don't you think its a good idea not to deallocate memory at all during the execution of that program?

Cutting corners is not a "good idea". I can think of plenty of reasons to deallocate memory; I can't think of quite so many to leave unused memory allocated. If you don't want to deal with memory issues, don't write in a language that doesn't have automatic garbage collection- or, since you're writing in C++, use smart pointers.

only consumes 100 MB of memory

That's the most egregious misuse of the word "only" I have ever seen. 100 MB is a serious chunk of change. That's ~1/40th of the memory you're using, and if even 40 other programs running on the system were programmed with the same perspective, at best you'd have the system come to a grinding halt because of swap delays, and at worst you'd have a complete system crash. Usually, there are more than 40 applications running on any given (modern) machine while it's turned on.

also runs for a limited time, like only 30 seconds

Again, 30 seconds is a long time for a computer. But, to clarify, do you mean 30 seconds ever, or 30 seconds every day? Because if it's a one-off and you're low on time, maybe that's an acceptable loss, but it certainly isn't good practice.

TL;DR?

If you have more important, immediate considerations (e.g. a product deadline is approaching) and you've got not other reasonable choice, do it. Otherwise, you've got no excuse. Hogging memory is never something you should feel good about; it can seriously slow down processes that really do need that memory. Yes, the O/S cleans up resources after the program ends. No, that does not give you carte blanche to write programs which abuse their resources.

Answer 3

No, it is not good programming style. Virtually all modern OS's will cleanup the memory afterwards, but it will remain a leak for as long as the program runs.

Answer 4

If you want to avoid overhead from cleanup you can implement an allocator that permits that while still maintaining good practices. For example, allocate a memory pool on program start-up, replace the global operator new with a version that takes memory from the pool, and replace global operator delete with a no-op. Then free the memory pool before program exit. You could also use more find grained pools to cut down on the deallocation overhead but still allow the program to run without continuously growing memory usage.

Another alternative is to simply use a faster implementation of new/delete. The built-in versions are traditionally slow, but they don't have to be.