2

I am having trouble allocating more than 2GB of contiguous memory with mmap in a 64-bit machine with plenty of memory. The following code happily runs. But when increasing the number of contiguously mmap'ed pages to slightly above 2GB mmap fails in my machine. 

#include <sys/mman.h>
#include <stdio.h>
#include <signal.h>
#include <stdlib.h>
#include <time.h>
#include <errno.h>
#include <unistd.h>

#define PAGE_SIZE (4*1024)
#define START_ADDRESS 0x8000000
#define START_ADDRESS2 0x800000000
#define N_ITERATIONS 524288

// one more iteration and mmap fails
//#define N_ITERATIONS 524289

void allocate(void* base_address)
{
    int i;
    for (i = 0; i < N_ITERATIONS; ++i) {
        void* current_addr = base_address + PAGE_SIZE * i;
        void* ret = mmap((void*)current_addr, PAGE_SIZE, PROT_EXEC|PROT_READ|PROT_WRITE, 
                MAP_NORESERVE | MAP_FIXED | MAP_PRIVATE|MAP_ANONYMOUS, 0, 0); 
        if (ret == MAP_FAILED) {
            fprintf(stderr, "Error mmap. errno: %d\n", errno);
            exit(-1);
        }   
        printf("%d\n", i); 
    }   
}

int main()
{
    allocate((void*)START_ADDRESS);
    allocate((void*)START_ADDRESS2);
    return 0;
}

ulimit -a

-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         unlimited
-m: resident set size (kbytes)      unlimited
-u: processes                       515550
-n: file descriptors                65535
-l: locked-in-memory size (kbytes)  unlimited
-v: address space (kbytes)          unlimited
-x: file locks                      unlimited
-i: pending signals                 515550
-q: bytes in POSIX msg queues       819200
-e: max nice                        0
-r: max rt priority                 0
-N 15:                              unlimited

What prevents the kernel from allocating more than 2GB of contiguous data?

JC1
  • 657
  • 6
  • 21
  • Well, having a look at the crystal ball, I hindsight your machine has some 32 bit signed limit. IOW: See [ask] and follow the advice. – too honest for this site May 18 '16 at 18:40
  • I'd start by looking at `ulimit`. Also make sure you're actually compiling for 64-bit, so `printf("%zu", sizeof(void*))` might interesting. – EOF May 18 '16 at 19:13
  • @EOF Yes, I have checked it is compiling in 64 bits. I have pasted the output of `ulimit` up there, everything seems fine to me. – JC1 May 19 '16 at 05:11
  • The problem might not be the total size, but the number of mmap sections. Have you tried to allocate bigger chunks instead? – Stian Skjelstad May 19 '16 at 07:38

1 Answers1

3

Your are overflowing your base_address with PAGE_SIZE * i.

Thus you are trying to allocate on address near 0xffffffff86000000.

Put a long long instead for i and it will work.

yayg
  • 309
  • 2
  • 12