73

Is there any way in Linux to assign one CPU core to a particular given process and there should not be any other processes or interrupt handlers to be scheduled on this core?

I have read about process affinity in Linux Binding Processes to CPUs using the taskset utility but that's not solving my problem because it just try to affine the given process to that core but it is possible that other processes may be scheduled on this core and this is what I want to avoid.

Should we change the kernel code for scheduling?

Pang
  • 9,564
  • 146
  • 81
  • 122
akp
  • 1,753
  • 3
  • 18
  • 26
  • Try also to set the highest real-time priority to that process. – Dmytro Sirenko Nov 27 '12 at 11:28
  • 1
    What if we use http://linux.die.net/man/1/htop and affine all other processes to other CPUS, while our task to a specific cpu. Should work I guess. – Sunny R Gupta Nov 27 '12 at 11:30
  • 2
    @EarlGray so duing this will ensure that other processes or interrupt handlers will not be scheduled to run on the given cpu core? – akp Nov 27 '12 at 11:31
  • 1
    Why exactly do you ask? You want your process to run.... Why should it run specifically one some particular core???? What is making that core so unique??? Is it so different from another core on the same chip??? – Basile Starynkevitch Nov 27 '12 at 12:06
  • Related: http://stackoverflow.com/questions/2595735/prevent-linux-thread-from-being-interrupted-by-scheduler – Ciro Santilli OurBigBook.com Oct 25 '15 at 20:13
  • 2
    @BasileStarynkevitch Firstly I think the author just wanted his process to run on it's own core. Secondly some Intel cores do have subtle differences. – user997112 Jan 21 '19 at 16:46
  • "Why exactly do you ask? You want your process to run.... Why should it run specifically one some particular core???? What is making that core so unique??? Is it so different from another core on the same chip???" -- @BasileStarynkevitch: Could be useful to collect performance measurements. – Daniel Feb 05 '20 at 12:13

5 Answers5

82

Yes there is. In fact, there are two separate ways to do it :-)

Right now, the best way to accomplish what you want is to do the following:

  1. Add the parameter isolcpus=[cpu_number] to the Linux kernel command line from the boot loader during boot. This will instruct the Linux scheduler not to run any regular tasks on that CPU unless specifically requested using cpu affinity.

  2. Use IRQ affinity to set other CPUs to handle all interrupts so that your isolated CPU will not receive any interrupts.

  3. Use CPU affinity to fix your specific task to the isolated CPU.

This will give you the best that Linux can provide with regard to CPU isolation without out-of-tree and in-development patches.

Your task will still get interrupted from time to time by Linux code, including other tasks - such as the timer tick interrupt and the scheduler code, IPIs from other CPUs and stuff like work queue kernel threads, although the interruption should be quite minimal.

For an (almost) complete list of interruption sources, check out my page at https://github.com/gby/linux/wiki

The alternative method is to use cpusets which is way more elegant and dynamic but suffers from some weaknesses at this point in time (no migration of timers for example) which makes me recommend the old, crude but effective isolcpus parameter.

Note that work is currently being done by the Linux community to address all these issues and more to give even better isolation.

AstroFloyd
  • 405
  • 5
  • 14
gby
  • 14,900
  • 40
  • 57
  • 4
    Is it still true that isolcpus gives better isolation than cpusets, even when --kthread=on option is provided? – Cedomir Segulja Aug 10 '13 at 08:06
  • 3
    @CedomirSegulja I believe it does although the difference narrows over time. For example, if a kernel module registered a timer on a CPU and that timer is self registering (e.g. it registers itself again when it elapses), currently there is no simple way to move it off the would-be dedicated CPU. With isolcpus the chance that such a timer would be registered in the first place is much lower. – gby Aug 18 '13 at 11:52
  • 1
    @gby, does isolcpus parameter accept physical core id or logical? I've read kernel.org/doc/Documentation/kernel-parameters.txt however it's not 100% clear to me. Specifically I'm interested about Hyper-Threading mode, which gives 2 logical cores per 1 physical on my Xeon CPU, and what should I specify in icolspus. Thanks. – Mark Jan 19 '15 at 12:05
  • @Mark logical. Same ID that sched_setaffinity accepts – gby Jan 19 '15 at 12:34
  • @gby, thanks for response. Am I right assuming that with hyper-threading mode disabled, LogicalID == PhysID of the CPU ? – Mark Jan 19 '15 at 15:48
  • "Use IRQ affinity to set other CPUs" @gby, how to do it? – Gilgamesz Apr 24 '18 at 10:26
  • @Gilgamesz the kernel has a good doc on it: https://www.kernel.org/doc/Documentation/IRQ-affinity.txt – gby Apr 25 '18 at 08:27
  • [`isolcpus=` ... **`[Deprecated - use cpusets instead]`**](https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html) – Константин Ван Feb 03 '23 at 08:45
7

There is Redhat article talking about it. It modifies the boot parameter isolcpus.

And an old article written by Robert Love. And there is solution in that article.

All of a process' children receive the same CPU affinity mask as their parent.

Then, all we need to do is have init bind itself to one processor. All other processes, by nature of init being the root of the process tree and thus the superparent of all processes, are then likewise bound to the one processor.

Hasturkun
  • 35,395
  • 6
  • 71
  • 104
louxiu
  • 2,830
  • 3
  • 28
  • 42
6

Dedicate a Whole CPU Core to a Particular Program

While taskset allows a particular program to be assigned to certain CPUs, that does not mean that no other programs or processes will be scheduled on those CPUs. If you want to prevent this and dedicate a whole CPU core to a particular program, you can use "isolcpus" kernel parameter, which allows you to reserve the CPU core during boot.

Add the kernel parameter "isolcpus=" to the boot loader during boot or GRUB configuration file. Then the Linux scheduler will not schedule any regular process on the reserved CPU core(s), unless specifically requested with taskset. For example, to reserve CPU cores 0 and 1, add "isolcpus=0,1" kernel parameter. Upon boot, then use taskset to safely assign the reserved CPU cores to your program.

Source(s)

  1. http://xmodulo.com/2013/10/run-program-process-specific-cpu-cores-linux.html
  2. http://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re46.html
dotnix
  • 848
  • 9
  • 13
Vasu
  • 265
  • 1
  • 10
  • 18
4

Even if you follow the steps in gby's answer, kernel tasks are executed on the isolated CPU core. Work is underway in the linux RT_PREEMPT real time project to improve this. So if you are not using a bleeding edge real time kernel from RP_PREEMPT, it might not be possible to completely isolate a CPU core.

bofh.at
  • 550
  • 3
  • 6
  • yes u are right till now 100% isolation is not possible!!! as some APIC timer interrupt may execute on the given core,etc. but gby's answer give considerable isolation. – akp Nov 29 '12 at 07:34
0

As per documentation

The Linux scheduler will honor the given CPU affinity and the process will not run on any other CPUs.

There is no mention that specific processor will be given to process exclusively.

Sergei Nikulov
  • 5,029
  • 23
  • 36
  • 1
    I read that full CPU isolation is not implemented yet: http://lwn.net/Articles/520704/ (scroll down to "CPU isolation"). – Maxim Egorushkin Nov 27 '12 at 11:51
  • That's exactly what I mean. You just give a hint to scheduler to run process on specific core/processor. But it is not a way to exclusively allocate core/processor for specific proress. – Sergei Nikulov Nov 27 '12 at 12:44