Does malloc only assign space in linear region structure vm_area_t insteading of real memory

Question

I read in "Understanding the linux kernel" that when malloc is invokded in user space, the kernel only add an linear region in the vm_area_t structure, insteading of allocating space in memory, which is called ostponing the allocation, and this space allocated in linear region can only be used when page interrupts occur. But if no page can be assigned during the page interrupts, isn't the user cheated when it invokes a malloc?

This is known as *overcommitting*. See http://www.google.co.uk/search?aq=f&sourceid=chrome&ie=UTF-8&q=linux+overcommit. — Oliver Charlesworth, May 31 '12 at 09:53
There is `kalloc` for kernel memory allocations, and you should use it IMO. — Seçkin Savaşçı, Jun 02 '12 at 19:52

fhtuft · Answer 1 · 2012-06-03T00:03:01.517

Memory allocated by malloc is not in real memory before it is written to. calloc on the other hand allocate real memory at ones.

From man the man page:

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. This is a really bad bug. In case it turns out that the system is out of memory, one or more pro‐ cesses will be killed by the infamous OOM killer. In case Linux is employed under circumstances where it would be less desirable to sud‐ denly lose some randomly picked processes, and moreover the kernel version is sufficiently recent, one can switch off this overcommitting behavior using a command like:
   # echo 2 > /proc/sys/vm/overcommit_memory

looking around in source of glibc and found this for what I believe is calloc:

if (__builtin_expect (hook != NULL, 0)) {
    sz = bytes;
    mem = (*hook)(sz, RETURN_ADDRESS (0));
    if(mem == 0)
      return 0;
#ifdef HAVE_MEMCPY
    return memset(mem, 0, sz);
#else
    while(sz > 0) ((char*)mem)[--sz] = 0; /* rather inefficient */
    return mem;
#endif
  }

If I were calloc(), I would make the new page table entries point to /dev/zero, and set the new page flags to COW. That would not "fault in" the pages at calloc() time, but later on, when the pages are actually referenced. — wildplasser, May 31 '12 at 11:17
`calloc` definitely does not allocate real memory at once. On any good implementation it will use the fact that new anonymous pages are inherently COW copies of a zero-page, or equivalent. — R.. GitHub STOP HELPING ICE, May 31 '12 at 12:39

score 0 · Answer 2 · edited May 23 '17 at 12:04

The malloc() call wrapped by glibc will first allocate from the heap memory maintained by glibc. (see what happens in the kernel during malloc?) If glibc runs out heap memory, sys_brk is invoked to allocate more. The kernel only allocate the memory to user request when it first touches the allocate memory (mostly page fault handler). Thus, I guess calloc will returned a real allocated one because it initialized the page to zero (from man page), which touches the page by scrubbing its content. If the new page can't be allocated in the fault handler, which means the system has very low memory, either in heavy swap or is going to OOM.

One small thing is that Linux always scrub the memory when it allocates to user process, because it doesn't do that when the page is returned to buddy allocator.

Does malloc only assign space in linear region structure vm_area_t insteading of real memory

2 Answers2