9

I'm having a problem with free() on a struct in my C program. When I look at /proc//statm before and after the free it doesn't seem to reduce. Am I using free() wrong in this case, or am I reading /proc//statm wrong?

Here is a test case which yields the problem:

struct mystruct {
    unsigned int arr[10000];
};

void mem() {
    char buf[30];
    snprintf(buf, 30, "/proc/%u/statm", (unsigned)getpid());
    FILE* pf = fopen(buf, "r");
    if (pf) {
        unsigned size; //       total program size
        unsigned resident;//   resident set size
        unsigned share;//      shared pages
        unsigned text;//       text (code)
        unsigned lib;//        library
        unsigned data;//       data/stack
        unsigned dt;//         dirty pages (unused in Linux 2.6)
        fscanf(pf, "%u %u %u %u %u %u", &size, &resident, &share, &text, &lib,         &data);
        printf("Memory usage: Data = %d\n", data*sysconf(_SC_PAGESIZE));
    }
    fclose(pf);
}

int main(int argc, char **argv) {
    mem();
    struct mystruct *foo = (struct mystruct *)malloc(sizeof(struct mystruct));
    mem();
    free(foo);
    mem();
}

The output is:

Memory usage: Data = 278528
Memory usage: Data = 282624
Memory usage: Data = 282624

When I would expect it to be:

Memory usage: Data = 278528
Memory usage: Data = 282624
Memory usage: Data = 278528

I've done a similar test with malloc'ing a (char *), then free'ing it and it works fine. Is there something special about structs?

oprimus
  • 396
  • 3
  • 11
  • Who cares? Virtual memory is not a scarce resource and each process has its own address space anyway. – David Schwartz Aug 28 '11 at 11:55
  • possible duplicate of [Memory usage isn't decreasing when using free?](http://stackoverflow.com/questions/1556014/memory-usage-isnt-decreasing-when-using-free) – Cory Klein Jul 17 '13 at 22:40

4 Answers4

14

Your answer is right over here on Stack Overflow, but the short version is that, for very good reasons, the memory allocator does not return memory to the host OS but keeps it (internally in your program's data space) as a free list of some kind.

Some of the reasons the library keeps the memory are:

  • Interacting with the kernel is much slower than simply executing library code
  • The benefit would be small. Most programs have a steady-state or increasing memory footprint, so the time spent analyzing the heap looking for returnable memory would be completely wasted.
  • Internal fragmentation makes page-aligned blocks (the only thing that could be returned to the kernel) unlikely to exist, another reason not to slow the program down looking for something that won't be there.
  • Returning a page embedded in a free block would fragment the low and high parts of the block on either side of the page.
  • The few programs that do return large amounts of memory are likely to bypass malloc() and simply allocate and free pages anyway using mmap(2).
Community
  • 1
  • 1
DigitalRoss
  • 143,651
  • 25
  • 248
  • 329
  • Thanks. That answers my question. So I am free'ing it correctly, and I am reading /proc//statm correctly. – oprimus May 15 '11 at 00:47
  • Oh, right, yes your code all looks correct, but your OS transaction will never show the image size decreasing unless some library function unrelated to malloc/free (like dlopen) uses mmap to release something. – DigitalRoss May 15 '11 at 01:24
  • I think the reasons for not returning memory to the kernel are overrated. If you do it right, you can avoid almost all of the cost - the main tricks are only returning unused ranges larger than a moderately-large threshold, and on consolidation of adjacent free blocks, only returning the new block when `floor(log(size))` has increased. I have an implementation working this way very successfully, and several questions on SO that were related to its development. – R.. GitHub STOP HELPING ICE May 15 '11 at 04:02
  • Also note that almost all `malloc` implementations use `mmap` directly for very large allocations, and thus can and do return them to the kernel when they're freed. – R.. GitHub STOP HELPING ICE May 15 '11 at 04:02
  • Note that the expression `new_size^old_size>old_size` is true if and only if `new_size` uses more bits than `old_size`. You can change the right-hand side to `old_size/2` or `/4` for more fine-grained freeing with low impact on performance, if desired. – R.. GitHub STOP HELPING ICE May 15 '11 at 04:12
  • @R, Thanks for the update, those are very good points. These questions come up in countless variations like clockwork. The thing each question has in common is that the OP is not initially aware of the free list or the page alignment requirements and isn't thinking of the barrier that kernel interaction presents. People ask why they are able to store things in just-free'ed blocks, which tells you all you need to know about where to begin the explanation. Your points are good but they are more related to where a completely detailed explanation would end. – DigitalRoss May 16 '11 at 17:08
1

Whenever free actually releases the memory is implementation dependent. So maybe free is not returning the memory right away when it's a big chunk on memory. I don't think it has anything to do with structs.

Pablo Santa Cruz
  • 176,835
  • 32
  • 241
  • 292
1

Mainly for performance reasons, the allocated heap memory won't be returned to the OS after being freed. It will be marked as free though, and maybe later the kernel will get it back, or your program will allocate it and use it again.

I don't know what you used to alloc/free your (char *). The difference you've seen might be that your (char *) was allocated on the stack, and the release/free process it different than with the heap (stack memory management is a lot simpler).

ack__
  • 913
  • 1
  • 14
  • 39
0

It is up to the OS to really free your data and thus shrink the memory consumption of your program. You only tell it, that you won't use that memory anymore.

Christian Rau
  • 45,360
  • 10
  • 108
  • 185