2

How can I get the largest size (continious) in bytes my application can allocate in one malloc call without returning out of memory? (biggest available chunk)

P.S.: I would like a general answer for both if I want to allocate it and if I don't want to allocate it (so this question does not need to be reasked).

Gizmo
  • 1,990
  • 1
  • 24
  • 50
  • 1
    Do you want to *actually* allocate the memory, or do you only need to find out what that maximum size is? – Greg Hewgill Jun 14 '15 at 22:02
  • On any reasonable modern system, the OS may consider using your entire hard disk for overflow. Do you want to take that into account? – Jongware Jun 14 '15 at 22:03
  • 1
    Jongware, i think OP wants to know, how to find out the maximal size of a CONTINUOUS memory chunk in ram, i do not think, the question deals with any paging issues. – icbytes Jun 14 '15 at 22:07
  • @icbytes summed it up correctly – Gizmo Jun 14 '15 at 22:12
  • 1
    On systems with *overcommit* or lazy allocation you *will* be in for a surprise with almost any answer. "Available memory" is a poorly defined concept on today's virtual memory systems. For example, it may change the millisecond after you asked for the value. – Jens Jun 14 '15 at 22:15
  • OS and C runtimes handle this in very different ways. There's nothing in the C language itself. If you specify an OS and the version of malloc(), the question is reasonable. Else it's meaningless. You can also try the brute force approach of a loop that allocates some arbitrary small amount, checks for NULL, then frees and tries twice the size repeatedly until fail. (If you start with 1 byte, 64 iterations are guaranteed to work.) But this is terribly dangerous because some mallocs don't map VM until you write it. – Gene Jun 14 '15 at 22:50
  • @Gene: That has already been answered. Did not work well, for multiple reason, not just the delayed allocation problem. – too honest for this site Jun 14 '15 at 23:09

2 Answers2

4

I presume you are asking about a (somewhat) modern desktop or server OS.

The problem is: any answer can be invalid already when the result is returned. Reasons are other processes, threads, memory fragmentation, etc. As others stated in comments already, an OS might even report more memory than available - inlcuding swap. The idea is that the allocated memory might only be sparsely used by a process, so only actually accesses memory pages will be provided as required, malloc() itself will not reserve any memory in advance.

It is also often not a good idea to allocate as much memory as possible, as that can result in exessive swapping or starving/thrashing other processes. So, just allocate the memory you actually need.

**Conclusion: ** Just forget about it! Allocate the memory you require, not more, not less. If you need dynamic memory, think about dynamic structures like lists and trees or use realloc() (but use it well thought).

Sidenote:

Do not feel tempted to try allocated blocks in increasing size until you get NULL returned. This can easily result in fragmentation, and you might not be able to allocate even the previous largest block size (for similar reasons as stated above).

For embedded OSes, there might be a completely different answer.

too honest for this site
  • 12,050
  • 4
  • 30
  • 52
-3

The following piece of code will provide an approximate answer, especially for real-time deterministic memory allocators found in realtime operating systems. It is not targeted for a general purpose OS:

#include <stdio.h>
#include <stdlib.h>

#define INCR 100000

int main(void){
    unsigned long i;
    for(i=1;;i=i+INCR) {
        void *ret = malloc(i);
        if (ret) {
            free(ret);
        }
        else {
            printf("malloc could not allocate memory of size: %ld bytes!\n", i);
            return 0;
        }
    }
}
Eugeniu Rosca
  • 5,177
  • 16
  • 45
  • 1
    That's only good in the rare case the maximum size is a power of two. But it won't find a maximum of 999Mb. – Jens Jun 14 '15 at 22:12
  • Ok, then one could try some smaller increments! – Eugeniu Rosca Jun 14 '15 at 22:13
  • 3
    Also, depending on how your system's allocator works, running a bunch of `malloc()` and `free()` calls may actually change the answer. –  Jun 14 '15 at 22:13
  • `malloc()` can do lazy allocation. It may be better to try with `calloc()` – pmg Jun 14 '15 at 22:14
  • Ok, now what can I say. I like this kind of downvotes, when nobody wants even an approximate solution for the problem :) – Eugeniu Rosca Jun 14 '15 at 22:20
  • 3
    There is no appropriate solution for modern OSses. Your approach follows somewhat the "Heisenberg uncertainty principle": observation changes the observed object - at best. Even worse: a good compiler might even detect you do not use the memory at all and does not even call malloc and free. (if asked, I would suspect gcc to do so). – too honest for this site Jun 14 '15 at 22:43
  • @Olaf I am coming from the RTOS domain and with somewhat deterministic behavior of the pool-based allocators, this empiric example is not bad at all. I nevertheless agree that for a general purpose OS it is simply not suitable. – Eugeniu Rosca Jun 14 '15 at 22:48
  • Surprise, surprise. Checkout my profile. Then read my answer, That is exactly the reason I differentiate here. However, even for an embedded system, it makes not much sense either, as either you _require_ that much memory, or it is optional. If you need it, you just fail on allocation or allocate it statically. If that is optional (e.g. dynamic receive buffer), you just try to alloc and if you fail, you're done. Still no sense in your algorithm, or just checking how much is avail (also, pools are most times limited to few block sizes without split/merge for RT reasons). – too honest for this site Jun 14 '15 at 22:52
  • Never required that on RT systems, either. Pools normally have defined fixed block sizes, And do not split/merge. And some of the problems with concurrency might apply as well. And for a true split/merge allocator, some of the problems listed for full grown OSses also might apply (fragementation, etc.). – too honest for this site Jun 14 '15 at 22:58
  • @Olaf This kind of test-cases can be extremely useful when modifying/improving your heap or pool-based allocators and I used them a lot. Now, a general purpose OS is a completely different story and I must have specified that from the very beginning. – Eugeniu Rosca Jun 14 '15 at 23:04
  • Well, I tested my heap implementation differently. Same for the pool. classical split/merge allocators are not used very much by me and others I know, as I prefer interrupt-safe alloc/free. – too honest for this site Jun 14 '15 at 23:06