What is the largest amount of memory I can allocate on my MacBook Pro?

Question

I'm trying to figure out how much memory I can allocate before the allocation will fail.

This simple C++ code allocates a buffer (of size 1024 bytes), assigns to the last five characters of the buffer, reports, and then deletes the buffer. It then doubles the size of the buffer and repeats until it fails.

Unless I'm missing something, the code is able to allocate up to 65 terabytes of memory before it fails on my MacBook Pro. Is this even possible? How can it allocate so much more memory than I have on the machine? I must be missing something simple.

int main(int argc, char *argv[])
{
        long long size=1024;
        long cnt=0;
        while (true)
        {
                char *buffer = new char[size];
                // Assume the alloc succeeded. We are looking for the failure after all.

                // Try to write to the allocated memory, may fail
                buffer[size-5] = 'T';
                buffer[size-4] = 'e';
                buffer[size-3] = 's';
                buffer[size-2] = 't';
                buffer[size-1] = '\0';

                // report
                if (cnt<10)
                        cout << "size[" << cnt << "]: " << (size/1024.) << "Kb ";
                else if (cnt<20)
                        cout << "size[" << cnt << "]: " << (size/1024./1024.) << "Mb ";
                else
                        cout << "size[" << cnt << "]: " << (size/1024./1024./1024.) << "Gi ";
                cout << "addr: 0x" << (long)buffer << " ";
                cout << "str: " << &buffer[size-5] << "\n";

                // cleanup
                delete [] buffer;

                // double size and continue
                size *= 2;
                cnt++;
        }
        return 0;
}

Answer 1

When you ask for memory, an operating system reserves the right not to actually give you that memory until you actually use it.

That's what's happening here: you're only ever using 5 bytes. My ZX81 from the 1980s could handle that.

Answer 2

MacOS X, like almost every modern operating system, uses "delayed allocation" for memory. When you call new, the OS doesn't actually allocate any memory. It simply makes a note that your program wants a certain amount of memory, and that memory area you want starts at a certain address. Memory is only actually allocated when your program tries to use it.

Further, memory is allocated in units called "pages". I believe MacOS X uses 4kb pages, so when your program writes to the end of the buffer, the OS gives you 4096 bytes there, while retaining the rest of the buffer as simply a "your program wants this memory" note.

As for why you're hitting the limit at 64 terabytes, it's because current x86-64 processors use 48-bit addressing. This gives 256 TB of address space, which is split evenly between the operating system and your program. Doubling the 64 TB allocation would exactly fit in your program's 128 TB half of the address space, except that the program is already taking up a little bit of it.

Answer 3

Virtual memory is the key to allocating more address space than you have physical RAM+swap space.

malloc uses the mmap(MAP_ANONYMOUS) system call to get pages from the OS. (Assuming OS X works like Linux, since they're both POSIX OSes). These pages are all copy-on-write mapped to a single physical zero page. i.e. they all read as zero with only a TLB miss (no page fault and no allocation of physical RAM). An x86 page is 4kiB. (I'm not mentioning hugepages because they're not relevant here).

Writing to any of those pages triggers a soft page fault for the kernel to handle the copy-on-write. The kernel allocates a zeroed page of physical memory and re-wires that virtual page to be backed by the physical page. On return from the page fault, the store is re-executed and succeeds this time.

So after allocating 64TiB and storing 5 bytes to the end of it, you've used one extra page of physical memory. (And added an entry to malloc's bookkeeping data, but that was probably already allocated and in a dirty page. In a similar question about multiple tiny allocations, malloc's bookkeeping data was what eventually used up all the space).

If you actually dirtied more pages than the system had RAM + swap, the kernel would have a problem because it's too late for malloc to return NULL. This is called "overcommit", and some OSes enable it by default while others don't. In Linux, it's configurable.

---

As Mark explains, you run out of steam at 64TiB because current x86-64 implementations only support 48-bit virtual addresses. The upper 16 bits need to be copies of bit 47. (i.e. an address is only canonical if the 64-bit value is the sign-extension of the low 48 bits).

This requirement stops programs from doing anything "clever" with the high bits, and then breaking on future hardware that does support even larger virtual address spaces.