calloc() slower than malloc() & memset()

Question

I would like to ask you a question. I have the following code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

#define XXX 1024*1024

int main()
{
        int *p;
        unsigned long x=0;
        while (1)
        {
                //p = (int *) calloc (1,XXX);
                p = (int *) malloc (XXX);
                memset (p,0,XXX);
                x++;
                printf ("%lu MB allocated.\n",x);
                sleep (1);
        }
        return 0;
}

If I run this code, everything runs as normal. Every second, a new MB is allocated in the memory. The problem I encounter is if I uncomment the calloc() line and comment the malloc() and memset() lines. From what I know, calloc() should initialize all bytes to zero in the allocated memory; the same thing that malloc() and memset() do.

When I run the code with calloc() (without the malloc() and memset()), an initial 1 MB is allocated (as it is normal), and then after some seconds (~10) another MB is allocated.

Why is this behaviour?

Thanks in advance!

I open a new terminal and run a script that outputs "free -m" every second. — Alex, Nov 06 '13 at 13:21
It executes every second for me when I run with `calloc` instead of `malloc` and `memset`, if this execution is how you're measuring it. — Leigh, Nov 06 '13 at 13:21
Try _using_ the memory when derived via `calloc()`. Even though the virtual memory is allocated, the physical memory may not be allocated and zeroed until you use it. — chux - Reinstate Monica, Nov 06 '13 at 13:25
I think I didn't explain my issue very well. Sorry for this. From what I know, Linux implements a COW mechanism on memory. For example, if I allocate 1 GB of memory using malloc, memory is not allocated by the kernel until i try to write something in that memory area. Calloc initializes that memory with 0, or this is what it should do. So if I allocate 1 GB of memory with calloc, the kernel immediately allocates that memory. — Alex, Nov 06 '13 at 13:25
Try accessing of modifying this `calloc()`'d memory in the program and see if the numbers you're seeing are different. The program you've listed doesn't access the memory at all so it's not necessarily being used at all. — Leigh, Nov 06 '13 at 13:33
calloc doesn't necessarily allocate physical memory immediately; it can just allocate pages and mark them as "clear when first accessed" or similar. — davmac, Nov 06 '13 at 13:35

score 10 · Accepted Answer · edited May 23 '17 at 12:33

10

From what I know, calloc() should initialize all bytes to zero in the allocated memory.

This is partly true based on my understanding of the calloc call.

It reserves the space but doesn't initialise all memory to zero. It will often or generally initialise one section to zero, and point all others to that; when memory is then modified or accessed within this block, it will initialise it to zero before using. It means that a calloc call of very large size doesn't set all of that memory to zero multiple times, but only when it actually needs to.

tl;dr: it's an OS theory trick where kernels will cheat. There's a longer description here: https://stackoverflow.com/a/2688522/2441252.

edited May 23 '17 at 12:33

Community

1
1

answered Nov 06 '13 at 13:26

Leigh

12,038
4
28
36

Interesting fact about calloc. But does this answer the thing why it takes ~10 sec for next allocation? or I'm not getting it. I'd be interested to know that as well. Thanks. – Diwakar Sharma Dec 10 '13 at 04:14
1

@DiwakarSharma that's because you didn't specify the environment. Maybe your OS uses [an extremely primitive `calloc` implementation that actually _does_ touch all the space](http://stackoverflow.com/a/4319790/2171120)? – mirabilos Mar 31 '14 at 20:47

calloc() slower than malloc() & memset()

1 Answers1