malloc'ing 1TB of memory doesn't fail?

Question

//gcc 5.4.0

#include  <stdio.h>

int main(void)
{
    // Try to reserve 1 TB of memory
    int *arr = malloc(1024 * 1024 * 1024 * 1024);
    // Unfortunately most systems will actually allow this (swapping etc.)
    if (arr == NULL) {
        printf("Was not able to reserve memory!\n");
    }
    printf("Everything is ok\n");

    return 0;
}

compiler options: -Wall -std=c99 -o a.out source_file.c

The output ("Everything is ok") is what confuses me. I obviously don't have 1TB of memory - and according to the C-Reference, malloc should return NULL if, and only if(!) something goes wrong.

This is clearly not happening here. Can somebody wiser explain?

Answer 1

1024 * 1024 * 1024 * 1024

These are integer constants of type int. When multiplied together, the operations will get carried out on type int. The result will be of type int.

Most likely an int cannot hold a value larger than 2^32 / 2 -1 = 2.14*10^9 on your system.

Meaning that you overflow a signed int and invoke undefined behavior. Anything can happen, including malloc allocating some random chunk of data.

Answer 2

according to the C-Reference, malloc should return NULL if, and only if(!) something goes wrong.

This is incorrect.

malloc() returns NULL under 2 conditions:

1. A null pointer if the memory could not be allocated.

2. On an allocation of 0, it is implementation defined if malloc() returns NULL or a non-NULL pointer (which can not be de-referenced).

OP's 1024 * 1024 * 1024 * 1024 overflows 32-bit int math that is undefined behavior or UB. @WhozCraig hint. A typical result is a truncated product which would be 0. OP's system is then returning a non-NULL pointer for a zero byte allocation as "Was not able to reserve memory!\n" was not printed.

If OP's platform uses a rare 64-bit int, the true allocation of memory may be deferred.

---

OP's is also missing out on the helpful advice of the compiler. Increase warning options. Certain the below will report a warning.

// -pedantic -Wall -Wextra -std=c99  -o a.out source_file.c

int *arr = malloc(1024 * 1024 * 1024 * 1024);
// warning: integer overflow in expression [-Woverflow]

---

Be sure the type of expression's math meets/exceeds the type of the target. @Olaf and reasons not to use 100 * 1000 * 1000

int *arr = malloc((size_t) 1024 * 1024 * 1024 * 1024);

Answer 3

From the malloc(3) manpage:

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available.

This, of course, assumes you're on Linux.

Answer 4

You don't have 1TB physical memory, but most OSs support virtual memory, backed by disk space. When malloc asks for memory the system allocates virtual memory. A part of it will be real memory the rest will be most probably a file on your HD.

Answer 5

1024 * 1024 * 1024 * 1024 most likely invokes undefined behaviour: 1024 is a int constant. Operations are done with the "largest" type of both operands, but at least int. Both operands are int, so the multiplication types are int * int -> int. The same for the following multiplications. I.e. the final result is int. On typical systems int is not more than 32 bits. The type of the parameter is irrelevant for the expression.

The result of the multiplication overflows an int (unless it is more than 41 bits), which invokes undefined behaviour. Anything can happen (often, but not guaranteed: the result is truncated, yielding 0).

To avoid this, you should use (size_t)1024 * 1024 * 1024 *1024. This will (according to conversion rules) will perform the calculations with whichever standard type size_t has on your system.

Beforehand use a static assertion to ensure size_t can represent the result:

_Static_assert(SIZE_MAX >= 1024ULL * 1024 * 1024 *1024, "size_t is too small");`

Without this the multiplication would be truncated on a too small size_t (rules are different for signed and unsigned integers!) and you end up passing 0.

According to the standard malloc(0) is implementation defined either NULL or a pointer which must not be used to access an object.

Keeping all this in mind, your code might return a non-null pointer, which must not be used for accesses. But as you just test it, it is fine, but useless.