How to restrict ALL memory allocation to one NUMA node

Question

I am new to NUMA-aware multithreaded programming. I am writing my code such that all the threads and their memory allocation are restricted to one node. At the beginning of the program, I make the following calls:

struct bitmask *bm = numa_parse_nodestring("0");
if (bm == 0) {
    exit(1);
}
numa_bind(bm);

My understanding is that a call to numa_bind in this way would bind all threads and all memory allocation to node 0.

Furthermore, when I start pthreads from this code, I bind them to specific CPUs using:

pthread_setaffinity_n

However, when I look at /proc//numa_maps, I can still see that certain libraries (e.g., libc) are bound to the memory on node 1. How can I make sure that all the memory required by the process is bound to node 0?

I would like to know to. But after reading [this post](https://stackoverflow.com/questions/12388918/shared-library-bottleneck-on-numa-machine) (which Chrstinane mentioned below), I think even if there is a way to ask kernel to load all the shared libraries to a node you specify, it might run into other performance issues -- other processes need to get the shared libraries from remote nodes. Transferring data from one node to another node is via QPI/UPI (or whatever it is on your machine), and it is slow and I would imagine that there is some locks for coherence purpose, slowing down your app — HCSF, Oct 19 '18 at 04:07
another way I heard is to drop the page cache (probably several times) and hopefully your application would be the first one requesting `libc` so that it gets loaded from disk to page cache on the NUMA node you want. — HCSF, Oct 19 '18 at 06:05

score 1 · Answer 1 · edited May 23 '17 at 12:33

1

Shared libraries like libc can't be bound to a memory bank specified by your process/application. Please see shared-library-numa

edited May 23 '17 at 12:33

Community

1
1

answered Sep 10 '13 at 13:10

Christiane Pousa

23
4

score 0 · Answer 2 · answered Dec 29 '16 at 16:04

Code would tend to get cached in the local processor's L3 cache. Since it's read-only it's unlikely to generate any traffic once it's been loaded into cache. I wouldn't bother with it too much, unless you have profiling information showing it does pose a problem.

How to restrict ALL memory allocation to one NUMA node

2 Answers2