30

Hopping from Java Garbage Collection, I came across JVM settings for NUMA. Curiously I wanted to check if my CentOS server has NUMA capabilities or not. Is there a *ix command or utility that could grab this info?

Donald_W
  • 1,773
  • 21
  • 35
pr4n
  • 2,918
  • 3
  • 30
  • 42

5 Answers5

35

I'm no expert here, but here's something:

Box 1, no NUMA:

~$ dmesg | grep -i numa
[    0.000000] No NUMA configuration found

Box 2, some NUMA:

~$ dmesg | grep -i numa
[    0.000000] NUMA: Initialized distance table, cnt=8
[    0.000000] NUMA: Node 4 [0,80000000) + [100000000,280000000) -> [0,280000000)
Nikolai Fetissov
  • 82,306
  • 11
  • 110
  • 171
  • Mine doesn't even say "No NUMA configuration", matches at all (linux 2.6.18 / centos)... – Kevin Jul 26 '13 at 06:18
  • 4
    `dmesg` for me also lacks any mention of "NUMA", because it's too early. `grep /var/log/dmesg` instead, as it is more likely to have the complete log. (And what does "NUMA turned off." mean?) – Thanatos Feb 24 '14 at 17:55
  • For me, `dmesg` also lacks mention of "NUMA", but I have no access to /var/log/dmesg which need root privilege. I run `find /proc|grep -i numa` and saw some `numa_maps` files. I guess this is also a symbol that NUMA is enabled. – Roun Jun 18 '14 at 18:28
  • 6
    The approach of checking for Numa related messages in dmesg is unreliable. Firstly, since dmesg reads the kernel ring buffer, the message may have gone by the time you grep for it. i.e. False negative. You can't distinguish this from the case where the kernel is either so early, or sufficiently stripped down that Numa messages are completely absent. – Donald_W Apr 25 '15 at 18:42
  • I never said it's reliable. It is just one of the tools. – Nikolai Fetissov Apr 26 '15 at 01:18
26

I think this previous question is similar: How to confirm NUMA?

In particular, you can review the NUMA man page here: http://man7.org/linux/man-pages/man7/numa.7.html

And from there you'll see:

$ find /proc -name numa_maps
/proc/1/task/1/numa_maps
/proc/1/numa_maps
/proc/2/task/2/numa_maps
/proc/2/numa_maps
/proc/3/task/3/numa_maps
[etc if you have numa]

And you can get more detail like so:

$ grep NUMA=y /boot/config-`uname -r`
CONFIG_NUMA=y
CONFIG_K8_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
CONFIG_ACPI_NUMA=y

$ numactl --hardware
available: 2 nodes (0-1)
node 0 size: 18156 MB
node 0 free: 9053 MB
node 1 size: 18180 MB
node 1 free: 6853 MB
node distances:
node   0   1
  0:  10  20
  1:  20  10
Community
  • 1
  • 1
Donald_W
  • 1,773
  • 21
  • 35
16

You can also get this info from lscpu command:

lscpu | grep -i numa
NUMA node(s):          2
NUMA node0 CPU(s):     0-19,40-59
NUMA node1 CPU(s):     20-39,60-79
Learner
  • 425
  • 5
  • 14
13

For Redhat 4,5,6 and 7 systems, one can try the following to determine if NUMA configuration is disabled:

numactl --show does not show multiple nodes

# numactl --show
policy: default
preferred node: current
physcpubind: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
cpubind: 0
nodebind: 0
membind: 0

or numactl --hardware does not list multiple nodes

# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
node 0 size: 524163 MB
node 0 free: 505253 MB
node distances:
node   0
  0:  10
Kadir
  • 1,664
  • 2
  • 19
  • 22
9

You can also just query the information from /sys (this is what tools like numactl do underneath). As others pointed out, using dmesg will be unreliable since it usually does not have unlimited buffering.

To find out how many NUMA nodes are currently available, do:

cat /sys/devices/system/node/online
0-3
zgerd
  • 1,080
  • 8
  • 17
  • Hi @zgerd ! Do you know if there is documentation with more in depth details related to files tree /sys/devices/system/node/. I looked for it in kernel docs but unfortunately there is no documentation related to this area. I would like to know what stats are stored in specific files in that dir. – Michal16511 Feb 01 '22 at 10:32
  • 1
    It's documented under https://www.kernel.org/doc/Documentation/ABI/stable/sysfs-devices-node – zgerd Apr 25 '22 at 03:38