1

This is a followup to the question I just asked here.

I've created a simple program to help myself understand memory allocation, malloc() and free(). Notice the commented out free line. I created an intentional memory leak so I can watch the Windows reported "Mem Usage" bloat slowly to 1GB. But then I found something stranger. If I comment out the loop just above the free line, so that I don't initialize my storage block with random ints, it appears that the space doesn't actually get "claimed" by the program, from the OS. Why is this?

Sure, I haven't initialized it the block, but I have claimed it, so shouldn't the OS still see that the program is using 1GB, whether or not that GB is initialized?

#include <stdio.h>
#include <stdlib.h>

void alloc_one_meg() {
    int *pmeg = (int *) malloc(250000*sizeof(int));
    int *p = pmeg;

    int i;
    // for (i=0; i<250000; i++) /* removing this loop causes memory to not be used? */
    //    *p++ = rand();
    // free((void *)pmeg); /* removing this line causes memory leak! */
}

main()
{
    int i;
    for (i=0; i<1000; i++) {
        alloc_one_meg();
    }
}
Community
  • 1
  • 1
The111
  • 5,757
  • 4
  • 39
  • 55
  • Note: The memory will not be "a bunch of zeros"... – Oliver Charlesworth Dec 23 '11 at 01:43
  • 1
    actually malloc()ed memory is random/not initialized. calloc() will initialize to zeros. – kfmfe04 Dec 23 '11 at 01:43
  • Thanks Oli and kfm, I removed that line. I still have a lot to learn, but it was definitely an assumption and I see now a wrong one. The main question still remains. :-) – The111 Dec 23 '11 at 01:45
  • @kfmfe04: According to the C standard, memory returned by `malloc` has an *indeterminate value,* which has a fairly specific definition as of C99. – Dietrich Epp Dec 23 '11 at 02:05
  • @DietrichEpp I suppose the definition has a different meaning from random/not initialized? I'm interested to know what it is (always up for new knowledge!) - could you elaborate what "indeterminate value" actually means? – kfmfe04 Dec 23 '11 at 02:23
  • @kfmfe04: According to the C99 standard, "indeterminate" means that the value is unspecified or a is trap representation. "Initialized" means that the object has an initial value. Malloc always returns "initialized" memory, but there are no guarantees about what the value is. "Random" has a technical meaning, avoid using it as a synonym for "arbitrary" or "unspecified". In practice, malloc will either return memory filled with zeroes or filled with data previously used by the same program. – Dietrich Epp Dec 23 '11 at 05:02
  • @DietrichEpp ty for that detailed explanation: imho, that's a really scary use of the word, "initialized", as I have no idea who or what did the "initialization" or what the memory was "initialized" to. Just out of curiosity, given these definitions, is it possible to have "uninitialized" memory? – kfmfe04 Dec 23 '11 at 09:05
  • @kfmfe04 yes, `int i;` (when used in a function). Using the value of `i` without writing to it first results in undefined behaviour – Seth Carnegie Dec 23 '11 at 09:06
  • @SethCarnegie how is this different from "initialized" malloc memory besides that one is on the stack and one is from the heap? I'm not trying to be facetious here - just trying to understand the terminology. – kfmfe04 Dec 23 '11 at 09:09
  • @kfmfe04 I don't know _why_ it is different, other than it is _defined_ to be unititialised whereas `malloc` is _defined_ to return a pointer to a block of initialised memory. I know that that's not really a reason, but I don't know any more than that – Seth Carnegie Dec 23 '11 at 09:10
  • @kfmfe04: You can have memory that is not "explicitly initialized". If it has automatic storage duration (i.e., stack) it will have indeterminate value. If it has static storage duration (i.e., global / static), then it will be set to zero. I'm not really meaning to nitpick, I just wanted to share the terminology used in the standard for those who are curious. – Dietrich Epp Dec 23 '11 at 15:45

2 Answers2

7

Allocated memory can be in two states in Windows: Reserved, and Commited (see the documentation of VirtualAlloc about MEM_RESERVE: "Reserves a range of the process's virtual address space without allocating any actual physical storage in memory or in the paging file on disk.").

If you allocate memory but do not use it, it remains in the Reserved state, and the OS doesn't count that as used memory. When you try to use it (whether it is only on write, or on both read and write, I do not know, you might want to do a test to find out), it turns into Commited memory, and the OS counts it as used.

Also, the memory allocated by malloc will not be full of 0's (actually it may happen to be, but it's not guaranteed), because you have not initialised it.

Seth Carnegie
  • 73,875
  • 22
  • 181
  • 249
  • `actually it may happen to be, but it's not guaranteed` and most often isn't... at least in my experience. – Steve Dec 23 '11 at 01:59
  • @Steve for large allocations it "usually" is the case. See this question: http://stackoverflow.com/questions/8029584/why-does-malloc-initialize-the-values-to-0-in-gcc – Mysticial Dec 23 '11 at 02:03
  • @Mystical - Cool - makes sense (security) - guess I'm most accustom to more temporal allocations... thanks for the link. I appreciate it. – Steve Dec 23 '11 at 02:18
-1

it could be compiler optimization : the memory is not used at all, so a possible optimization is to not allocate this memory depening on compiler and on optimization options.

i tested your code, the line : free((void *)pmeg); does not cause any memory leak for me.

Hicham
  • 983
  • 6
  • 17
  • @Dietrich Epp : you are wrong : not allocate a memory never used is one of the possible optimizations done by compilers. it is an important optimization. – Hicham Dec 23 '11 at 02:15
  • So how does the compiler know 'malloc' is a function to allocate memory? As far as the C language goes, 'malloc' can be any function. The heap isn't managed by the compiler, it's (usually) managed by the OS and libc. There's no way a compiler can optimize that because it's not its job. – tangrs Dec 23 '11 at 02:31
  • @tangrs : the compiler knows malloc is allocation... in fact, there is an allocation optimization by the compiler !!! for a lot of languages. – Hicham Dec 23 '11 at 02:41
  • The only optimization the compiler can do is for static allocations. How does the compiler know 'malloc' is for allocation? I could simply not link libc and compile a program where 'malloc' doesn't do anything. The compiler can't optimize dynamic allocations. If you have a link that can prove otherwise, I'd be interested to have a look – tangrs Dec 23 '11 at 03:05
  • it is required for the compiler to know about allocation. search about it. look at my link. it is the same principle in other languages. sodont vote -1 if you dont know it is wrong ! – Hicham Dec 23 '11 at 03:14
  • The link you provided was for Java and languages that support garbage collection. That's different to C (since it doesn't have garbage collection built-in). The compiler can optimize Java dynamic allocation because it's in charge of garbage collection. C on the other hand can't. Heck the C compiler can't even tell which functions are for memory allocations. See http://pastebin.com/0a5eTRif which proves my point. – tangrs Dec 23 '11 at 03:15
  • it is not a issue of garbage collector : search for compiler allocation optimization in a good book of c (or c++ : neither of them have garbage collector) !!! . – Hicham Dec 23 '11 at 03:22
  • malloc and free are defined in c standard – Hicham Dec 23 '11 at 03:22
  • *siigh* Yes, it is defined in the C standard library. However, the C standard library is NOT tied to C language. That's the whole point of C - to be as flexible as possible. If you really want to see proof, try and get the compiler to optimize this out so it doesn't print 'I malloc'd'. http://pastebin.com/ewH9MqUh – tangrs Dec 23 '11 at 03:33
  • OP here: I would definitely be interested in an example of a "good book" which proves (or disproves) eharvest's point, though I think the topic is a bit over my head at this point. I am old-fashioned though and like books for learning. K&R doesn't teach ANY of this stuff, just language. eharvest's claim seems believable to me, but I don't know if it's true. One way would be to compare the assembly code with the C code (right?)... if you could read assembly... which I of course can't, but some day I hope to learn. – The111 Dec 23 '11 at 04:43
  • @eharvest: Your link is about the Java language, which is very different from the C language. If you can show me a C implementation which removes calls to `malloc`, I will retract my statement. – Dietrich Epp Dec 23 '11 at 04:51
  • why you say it does not exist when you do not know ?? hpc.lsu.edu/help/docs/optaps_cls.pdf, page 28 : the intel compiler perform "dead store elimination". digitalmars.com/ctg/ctgOptimizer.html : perform dead variable elimination. people dont know the topic and disturbing me with out of topic arguments directly from there personnal thinking outside reality without verifying what they say which is completly wrong. – Hicham Dec 23 '11 at 16:56