Sorry, this is a duplicate question from ServerFault. I am posting this here since I didn't get any response there.
Question:
I am using isolcpus to isolate cores. I would like to bind specific threads to cores, but it is not working. The threads are moved to different cores after I bind them.
Cores 13, 14, and 15 are isolated:
$ cat /proc/cmdline
ro root=/dev/mapper/vg0-root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg0/swaprd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=137M@0M rd_NO_DM KEYBOARDTYPE=pc KEYTABLE=us rd_LVM_LV=vg0/root rhgb quiet audit=0 intel_idle.max_cstate=0 console=tty0 console=ttyS1,115200 printk.time=1 processor.max_cstate=1 idle=poll biosdevname=0 isolcpus=13-15
top -H -p pgrep -u prusr12 Ser -d 1 shows this: 5017 and 5018 should have been bound to 14 and 15 and 5014 and 5016 should have been on 13.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ P COMMAND
5017 prusr12 20 0 1312m 1.1g 1.1g R 99.9 0.9 9:53.93 5 Server-3.10.
5018 prusr12 20 0 1312m 1.1g 1.1g R 99.9 0.9 10:08.88 7 Server-3.10.
5014 prusr12 20 0 1312m 1.1g 1.1g S 0.0 0.9 0:00.40 2 Server-3.10.
5016 prusr12 20 0 1312m 1.1g 1.1g S 0.0 0.9 0:01.04 4 Server-3.10.
The command line is this:
sg devuser "taskset -c 13 /releases/3.10.0/bin/Server-3.10.0 -n X -e DEV -p DEFAULT > /logs/ServerDevPR_DEFAULT.out 2>&1 &"
There are 4 threads in the process. I want the main thread to start on 13, hence taskset -c 13. Then two threads are spawed and will bind them to 14 and 15. I see that the threads were bound to 14 and 15, but then they were moved to other cores. pthread_setaffinity_np() is being used to bind the threads to cores.
Log after I bind the threads to 14 and 15: CpuSet returned by pthread_getaffinity_np() contained:CPU 14 CpuSet returned by pthread_getaffinity_np() contained:CPU 15
System details:
$ uname -a
Linux host123 2.6.32-573.12.1.el6.x86_64 #1 SMP Mon Nov 23 12:55:32 EST 2015 x86_64 x86_64 x86_64 GNU/Linux
$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Stepping: 2
CPU MHz: 3199.847
BogoMIPS: 6399.06
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0-7
NUMA node1 CPU(s): 8-15
What could be going wrong? Thanks for your time.
