How to detect that malloc() function will fail?

Question

In my C program I am trying to allocate some memory with malloc() function, like this:

char *buf = (char *)malloc(size);

but the problem is that malloc() always returns non-NULL pointer. Even if I try to allocate enormous (size is 1E+13) amount of memory, it returns valid buf pointer. Of course after that program crashes.

But how can I detect that requested amount of memory is too large and will be not available, if returned buf value is not NULL?

Edit:

In comments I see that my question may be not clear. So this is more expanded sample:

unsigned long size = very_large_calculated_value;
char *buf = (char *)malloc(size);
if (buf == NULL) i_know_it_fails;
...

but Xcode runs this code and buf is never NULL whatever requested size is. So, very soon program crashes. How can I detect memory allocation failure if buf is not NULL, but obviously unuseable?

Edit:

To those who marked the question as a duplicate: There is no answer for the question "How can I detect memory allocation failure?", because the solution like "change some settings in your OS" is not an answer - I am asking for C code to detect memory allocation error, or something like "it is not possible to make programmatically".

Answer 1

There's no way to predict the memory allocation failure. The only way is to check the return value of malloc() for null pointer.

It seems your question is really about memory overcommit done by the kernel. Using which the kernel never returns null pointer. The default is to always overcommit. So to disable it on Linux-like systems do:

echo 2 > /proc/sys/vm/overcommit_memory

Or you could do the same using sysctl:

sysctl vm.overcommit_memory=2

Both are equivalent.

The value 2 is to ensure that malloc returns null pointer in case the requested memory exceeds the available physical memory (plus swap space).

Answer 2

malloc() always returns non-NULL pointer

That's not quite true.

In case malloc() fails, it will return NULL. You need to check the return value of malloc() (the pointer) against NULL to ensure malloc() is success.

To quote the man page, (emphasis mine)

The malloc() and calloc() functions return a pointer to the allocated memory that is suitably aligned for any kind of variable. On error, these functions return NULL. [...]

---

Note; [Following the comments]

If you're talking about the optimistic allocation techniques used by malloc() to return the pointer, in that case, there is no standard way to check or predict the future failure, in case malloc() has returned non-NULL pointer. If you want to be sure about the availability of memory, you can consider using calloc() instead.

Answer 3

Expanding on BlueMoon's answer, here's what the man page for malloc has to say about overcomitting:

BUGS

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. This is a really bad bug. In case it turns out that the system is out of memory, one or more processes will be killed by the infamous OOM killer. In case Linux is employed under circumstances where it would be less desirable to suddenly lose some randomly picked processes, and moreover the kernel version is suf- ficiently recent, one can switch off this overcommitting behavior using a command like # echo 2 > /proc/sys/vm/overcommit_memory
See also the kernel Documentation directory, files vm/overcommit-accounting and sysctl/vm.txt.

Answer 4

After numerous tests it seems to be an Xcode debugging issue. The result is dependent on whether the breakpoints are used and on their position in code.

In general, if to remove all breakpoints then code runs as it should, without any problems. But if some breakpoints are inserted (influence of their position in code is unclear to me) then Xcode becomes unstable and crashes after memory allocation functions.