5

Is there a limit to the amount of memory that can be allocated from a program? By that I mean, is there any protection from a program, for example, that allocates memory in an infinite loop?

When would the call to malloc() return a NULL pointer?

sherrellbc
  • 4,650
  • 9
  • 48
  • 77
  • possible duplicate of [maximum memory which malloc can allocate](http://stackoverflow.com/questions/2798330/maximum-memory-which-malloc-can-allocate) – Jacob Sep 11 '13 at 19:37
  • It will depend on your operating system. – jmstoker Sep 11 '13 at 19:38
  • This discussion becomes even more complicated when you consider demand-paging operating systems; so long as you don't write anything to the newly allocated memory it is *not* fully allocated. This means you can allocate 2 GiB of memory on systems with only 32 MiB of RAM and `malloc (...)` will return immediately... the second you read/write that memory for the first time the kernel finally does the true allocation. That's why failure of `malloc (...)` is more indicative of running out of address space than running out of actual storage. Needless to say real-time OS's don't use demand-paging :) – Andon M. Coleman Sep 12 '13 at 04:33

4 Answers4

8

Yes, there is a limit. What that limit is depends on many factors, including (but not limited to):

  • The instruction set of the program (32-bit binaries have a smaller address space than 64-bit binaries, for example).
  • How much memory the system has free. ("Memory" here includes virtual memory.)
  • Any artificial restrictions set by the system administrator or a privileged process (see, for example, setrlimit() and the (obsolete) ulimit() function).

When memory cannot be allocated, malloc() will return NULL. If the system is completely out of memory, your process may be terminated forcefully.

cdhowie
  • 158,093
  • 24
  • 286
  • 300
  • +1 for mentioning setrlimit(). See also ulimit (the bane of my existence for 4 years working with AIX, and another 2 years under Debian Linux -- both of which ship with artificially low defaults). – PaulProgrammer Sep 11 '13 at 19:41
  • @PaulProgrammer Forgot about ulimit, thanks for pointing that out. Added it to my answer. – cdhowie Sep 11 '13 at 19:43
6

From Wikipedia,

The largest possible memory block malloc can allocate depends on the host system, particularly the size of physical memory and the operating system implementation. Theoretically, the largest number should be the maximum value that can be held in a size_t type, which is an implementation-dependent unsigned integer representing the size of an area of memory. The maximum value is 2CHAR_BIT × sizeof(size_t) − 1, or the constant SIZE_MAX in the C99 standard.

Jacob
  • 34,255
  • 14
  • 110
  • 165
  • This is useful information, but is about the largest *single* allocation request that `malloc()` can satisfy. The question seems to be about a limit on the sum total of many allocations. – cdhowie Sep 11 '13 at 19:42
  • @Jacob what is `CHAR_BIT` ? Clion saying it is undeclared identifier ? or do you mean `sizeof(char)` ? – Abhishek Mane Oct 22 '21 at 08:29
  • @cdhowie I think that's also gives memory allocated to program only, isn't it ? That's what asked in Question. – Abhishek Mane Oct 22 '21 at 08:49
3

It depends on the operating system and the standard library.

On Linux,

  • When you run out of address space, malloc() will return NULL.

  • When you run out of both physical memory and swap space, the OOM killer will run and kill a process to free memory.

Dietrich Epp
  • 205,541
  • 37
  • 345
  • 415
  • Doesn't GNU malloc (which is typically used on linux) also have a max limit on single invocations, even if there is enough address space? – HAL9000 Oct 22 '21 at 12:15
  • It should be easy enough to find, if you dig through glibc. – Dietrich Epp Oct 22 '21 at 13:21
  • Did a test on my system (linux 5.4.72-gentoo, glibc-2.32, x86_64). `malloc` seems to have an upper limit of 8012111176 bytes, or 0x1dd8f1d48 in hex. – HAL9000 Oct 22 '21 at 20:43
  • @HAL9000: Is that a glibc limit or an mmap limit? – Dietrich Epp Oct 23 '21 at 18:46
  • Don't know, I just tried to see if there was an upper limit for `malloc` using bisection. – HAL9000 Oct 24 '21 at 21:12
  • If you find what's causing it, I'll add it to the answer. – Dietrich Epp Oct 25 '21 at 01:26
  • Digging through glibc is not easy, but i tried the same tests with `mmap`. It gave me an upper limit of 8012107776 bytes, or 0x1dd8f1000 in hex. This is a little bit more than my mem+swap. Not sure if that is related. – HAL9000 Oct 26 '21 at 00:16
0

I am tackling this issue in Reverse. See pointer stores addresses of memory blocks. If we able to find maximum address it can store then we can find memory allocated to our program.

Code

#include <stdio.h>

int main()
{

    void *p;
    printf("%zu",sizeof(p));
    return 0;
}

Output

8

Understanding:

pointer size is 8 bytes.
8 bytes -> 64 bits
Max address it can store/ last memory block address: 2^64-1
Memory block addresses: 0, 1, 2, 3, ... 2^64-1
Memory allocated to program: 2^64 byte
Abhishek Mane
  • 619
  • 7
  • 20
  • You're referring more to the _addressable space_ of a 64 bit pointer, which is not quite answering the original question here. But otherwise you are correct in that a 64bit pointer would contain the value of `2^64 - 1` if you were on a system that could satisfy a 64GB memory request. Though there is a host of problems with that scenario on a modern OS, the least being that 64GB of memory is a tremendously large value – sherrellbc Nov 01 '21 at 16:38
  • @sherrellbc see when we writing Code in `C++` we are just writing down as text when we compile then memory allocate for it's object code. So that's memory you are talking about right ? if we increased variables in memory while writing code then memory allocated to object code also increase. Then we are checking upper limit to that as pointer size is `8` bytes it can point upto `2^64-1` spaces means that's max amount of memory but that's theoretical I know normally we have RAM's around `16 GB` means `2^34-1` max address. This is what I think if I am wrong then please correct me. – Abhishek Mane Nov 01 '21 at 17:14