16

So I have this program that allocates 256 MB of memory, and after the user presses ENTER it frees the memory and terminates. 

#include <stdio.h>
#include <stdlib.h>

int main(void) {
    char *p, s[2];

    p = malloc(256 * 1024 * 1024);
    if ( p == NULL) 
        exit(1);

    printf("Allocated"); 
    fgets(s, 2, stdin);
    free(p);
    return 0;
}

I ran this program multiple times and backgrounded each of them until there is no longer enough memory that can be allocated. However, that never happens. I ran a linux top command and even after running this program many times, the free memory never goes down by nearly as much as 256 MB.

However, on the other hand, if I use calloc instead of malloc then there is a HUGE difference:

p = calloc(256 * 1024 * 1024, 1);

Now if I run the program and background it, and repeat, every time I run it, the free memory goes down by 256 MB. Why is this? Why does malloc not cause the available free memory to change, but calloc does?

Ryan
  • 647
  • 2
  • 7
  • 17
  • Run the program via strace, and see what it does. (I'd expect _both_ versions to mmap /dev/zero anonymously, BTW) – wildplasser Nov 15 '13 at 01:08
  • 8
    It's called "Lazy allocation". http://stackoverflow.com/questions/712683/what-is-lazy-allocation – jaeheung Nov 15 '13 at 01:08
  • 2
    The short, oversimplified answer is that since you haven't actually used the memory from `malloc`, the computer doesn't need to actually give it to you yet. With `calloc` though, the memory needs to be used (for the zeroing out part), and hence the computer actually needs to give you all of it. – Dennis Meng Nov 15 '13 at 01:09
  • 2
    The `top` and `free` commands' "free memory" figures are meaningless. At best they're telling you about cache efficiency. The actual meaningful number, commit charge, can be found in `/proc/meminfo` as the `Committed_AS:` line. – R.. GitHub STOP HELPING ICE Nov 15 '13 at 01:25
  • @DennisMeng: this is only true for [extremely primitive `calloc` implementations](http://stackoverflow.com/a/4319790/2171120) – but Ryan is apparently using one of those. BSD omalloc, for example, does not usually access the memory. – mirabilos Mar 31 '14 at 20:49
  • @mirabilos I *did* say oversimplified... – Dennis Meng Apr 20 '14 at 19:12

3 Answers3

28

malloc() does not use memory. It allocates it.

After you allocate the memory, use it by assigning some data.

size_t Size = 256 * 1024 * 1024;
p = malloc(Size);
if (p != NULL) {
  memset(p, 123, Size);
}

Some platforms implement malloc() is such a way that the physical consumption of memory does not occur until that byte (or more likely a byte within a group or "page" of bytes) is accessed.

calloc() may or may not truly use the memory either. A system could map lots of memory to the same physical zeroed memory, at least until the data gets interesting. See Why malloc+memset is slower than calloc?

Toby Speight
  • 27,591
  • 48
  • 66
  • 103
chux - Reinstate Monica
  • 143,097
  • 13
  • 135
  • 256
13

The memory may not be really available, especially that you didn't do anything using p in your example except check for if it's NULL. From man malloc

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer. For more information, see the description of /proc/sys/vm/overcommit_memory and /proc/sys/vm/oom_adj in proc(5), and the Linux kernel source file Documentation /vm/overcommit-accounting.

Yu Hao
  • 119,891
  • 44
  • 235
  • 294
5

The calloc on your system† actually touches the memory by clearing it, and on many systems memory is not really allocated (and thus “used up”) until it is touched by the process to which it is allocated. So just doing malloc does not “use” the memory until you, well, use it.

† See comments

Arkku
  • 41,011
  • 10
  • 62
  • 84