linux mbind call fails on HUGETLB memory

Question

I'm trying to use mbind to bind my memory to a specific NUMA node. I noticed it fails if my memory is allocated with mmap using MAP_HUGETLB, but mbind succeeds if I dont try to MAP_HUGETLB.

I definitely want both HUGE pages and to map the memory to a specific NUMA node. AmI doing something wrong, or is this not supported?

Running on CentOS7

//mbind fails on this
void * buf = mmap( NULL, sz, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_HUGETLB, 0, 0 );
//works on this
void * buf2 = mmap( NULL, sz, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0 );

int numaNode = 1;
unsigned long nodemask = 0;
nodemask |= 1 << numaNode;
if( mbind( buf, sz, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0 ) < 0 )
    printf( "mbind buf failed: %s\n", strerror(errno) );

if( mbind( buf2, sz, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0 ) < 0 )
    printf( "mbind buf2 failed: %s\n", strerror(errno) );

Edit: to be clear the mmap works fine and my machine is configured with huge pages. I didn’t check the mmap return code here for brevity

Add error checking of each `mmap` call. And provide program output. https://stackoverflow.com/help/minimal-reproducible-example — Maxim Egorushkin, Jul 11 '22 at 23:35

Maxim Egorushkin · Answer 1 · 2022-07-06T00:09:05.837

-1

You don't check mmap return value for error. It fails to allocate huge pages unless you reserve them first.

Reserve huge pages with hugeadm, for example:

sudo hugeadm --pool-pages-min 2MB:4 --pool-pages-max 2MB:8 # 4-8 2MB pages
sudo hugeadm --pool-pages-min 1GB:2 --pool-pages-max 1GB:4 # 2-4 1GB pages

Another option is to use transparent huge pages without having to pass any extra flags to mmap. See Transparent Hugepage Support for full details.

edited Jul 06 '22 at 00:09

answered Jul 04 '22 at 00:20

Maxim Egorushkin

131,725
17
180
271

This doesnt answer the question. The question is about mbind failing, not mmap ( which succeeds in my case ) – ByteMe95 Jul 11 '22 at 16:10
@ByteMe95 I tried your code and `mmap` failed. – Maxim Egorushkin Jul 11 '22 at 23:34
this is most likely because you dont have hugepages configured on your machine – ByteMe95 Jul 13 '22 at 13:22

score -1 · Answer 2 · answered Jul 06 '22 at 09:34

-1

I think this artilcle arlready solved under, please check for it.

link

answered Jul 06 '22 at 09:34

Tiborcz Kiss

65
1
4

I dont see how this addresses my question about mbind() failing? – ByteMe95 Jul 07 '22 at 20:23

linux mbind call fails on HUGETLB memory

2 Answers2