Malloc -> how much memory has been allocated?

Question

# include <stdio.h>
# include <stdbool.h>
# include <string.h>
# include <stdlib.h>

int main ()
{
  char * buffer;
  buffer = malloc (2);

  if (buffer == NULL){
    printf("big errors");
  }
  strcpy(buffer, "hello");
  printf("buffer is %s\n", buffer);
  free(buffer);
  return 0;
}

I allocated 2 bytes of memory to the pointer/char buffer yet if I assign the C-style string hello to it, it still prints the entire string, without giving me any errors. Why doesn't the compiler give me an error telling me there isn't enough memory allocated? I read a couple of questions that ask how to check how much memory malloc actually allocates but I didn't find a concrete answer. Shouldn't the free function have to know exactly how much memory is allocated to buffer?

Answer 1

The compiler doesn't know. This is the joy and terror of C. malloc belongs to the runtime. All the compilers knows is that you have told it that it returns a void*, it has no idea how much, or how much strcpy is going to copy.

Tools like valgrind detect some of these errors. Other programming languages make it harder to shoot yourself in the foot. Not C.

Answer 2

No production malloc() implementation should prevent you from trying to write past what you allocated. It is assumed that if you allocate 123 bytes, you will use all or less than what you allocated. malloc(), for efficiency sake, has to assume that a programmer is going to keep track of their pointers.

Using memory that you didn't explicitly and successfully ask malloc() to give you is undefined behavior. You might have asked for n bytes but got n + x, due to the malloc() implementation optimizing for byte alignment. Or you could be writing to a black hole. You never can know, that's why it's undefined behavior.

That being said ...

There are malloc() implementations that give you built in statistics and debugging, however these need to be used in lieu of the standard malloc() facility just like you would if you were using a garbage collected variety.

I've also seen variants designed strictly for LD_PRELOAD that expose a function to allow you to define a callback with at least one void pointer as an argument. That argument expects a structure that contains the statistical data. Other tools like electric fence will simply halt your program on the exact instruction that resulted in an overrun or access to invalid blocks. As @R.. points out in comments, that is great for debugging but horribly inefficient.

In all honesty or (as they say) 'at the end of the day' - it's much easier to use a heap profiler such as Valgrind and its associated tools (massif) in this case which will give you quite a bit of information. In this particular case, Valgrind would have pointed out the obvious - you wrote past the allocated boundary. In most cases, however when this is not intentional, a good profiler / error detector is priceless.

Using a profiler isn't always possible due to:

• Timing issues while running under a profiler (but those are common any time calls to malloc() are intercepted).
• Profiler is not available for your platform / arch
• The debug data (from a logging malloc()) must be an integral part of the program

We used a variant of the library that I linked in HelenOS (I'm not sure if they're still using it) for quite a while, as debugging at the VMM was known to cause insanity.

Still, think hard about future ramifications when considering a drop in replacement, when it comes to the malloc() facility you almost always want to use what the system ships.

Answer 3

How much malloc internally allocates is implementation-dependent and OS-dependent (e.g. multiples of 8 bytes or more). Your writing into the un-allocated bytes may lead to overwriting other variable's values even if your compiler and run-time dont detect the error. The free-function remembers the number of bytes allocated separate from the allocated region, for example in a free-list.

Answer 4

Why doesnt the compiler give me an error telling me there isnt enough memory allocated ?

C does not block you from using memory you should not. You can use that memory, but it is bad and result in Undefined Behaviour. You are writing in a place you should not. This program might appear as running correctly, but might later crash. This is UB. you do not know what might happen.

This is what is happening with your strcpy(). You write in place you do not own, but the language does not protect you from that. So you should make sure you always know what and where you are writing, or make sure you stop when you are about to exceed valid memory bounds.

I read a couple of questions that ask how to check how much memory malloc actually allocates but I didn't find a concrete answer. Shouldn't the 'free' function have to know how much memory is exactly allocated to 'buffer' ?

malloc() might allocate more memory than you request cause of bit padding.

More : http://en.wikipedia.org/wiki/Data_structure_alignment

free() free-s the exact same amount you allocated with malloc(), but it is not as smart as you think. Eg:

int main()
{
     char * ptr = malloc(10);
     if(ptr)
     {
         ++ptr; // Now on ptr+1
         free(ptr); // Undefined Behaviour
     }
}

You should always free() a pointer which points to the first block. Doing a free(0) is safe.

Answer 5

You've written past the end of the buffer you allocated. The result is undefined behavior. Some run time libraries with the right options have at least some ability to diagnose problems like this, but not all do, and even those that do only do so at run-time, and usually only when compiled with the correct options.

Answer 6

Malloc -> how much memory has been allocated?

When you allocate memory using malloc. On success it allocates memory and default allocation is 128k. first call to malloc gives you 128k.

what you requested is buffer = malloc (2); Though you requested 2 bytes. It has allocated 128k.

strcpy(buffer, "hello"); Allocated 128k chunk it started processing your request. "Hello" string can fit into this.

This pgm will make you clear.

int main()
{
int *p= (int *) malloc(2);---> request is only 2bytes

p[0]=100;
p[1]=200;
p[2]=300;
p[3]=400;
p[4]=500;

int i=0;

for(;i<5;i++,p++)enter code here printf("%d\t",*p);

}

On first call to malloc. It allocates 128k---> from that it process your request (2 bytes). The string "hello" can fit into it. Again when second call to malloc it process your request from 128k.

Beyond 128k it uses mmap interface. You can refer to man page of malloc.

Answer 7

There is no compiler/platform independent way of finding out how much memory malloc actually allocated. malloc will in general allocation slightly more than you ask it for see here:

http://41j.com/blog/2011/09/finding-out-how-much-memory-was-allocated/

On Linux you can use malloc_usable_size to find out how much memory you can use. On MacOS and other BSD platforms you can use malloc_size. The post linked above has complete examples of both these techniques.