2

From the kernel I can call local_irq_disable(). To my understanding it will disable the interrupts of the current CPU. And interrupts will remain disabled until I call local_irq_enable(). Please correct me if my understanding is incorrect.

If my understanding is correct, does it mean upon calling local_irq_disable() interrupt is also disabled for a process in the user space that is running on that same CPU?

More details:

I have a process running in the user space which I want to run without affected by interrupts and context switch. As it is not possible from the user space, I thought disabling interrupt and kernel preemption for a particular CPU from kernel will help in this case. Therefore, I wrote a simple device driver to disable kernel preemption and local interrupt by using the following code,

    int i = irqs_disabled();
    pr_info("before interrupt disable: %d\n", i);
    pr_info("module is loaded on processor: %d\n", smp_processor_id());
    id = get_cpu();
    
    message[1] = smp_processor_id() + '0';
    
    local_irq_disable(); 
    
    printk(KERN_INFO " Current CPU id is %c\n", message[1]);
    printk(KERN_INFO " local_irq_disable() called, Disable local interrupts\n");
    pr_info("After interrupt disable: %d\n", irqs_disabled());

output: $dmesg

[22690.997561] before interrupt disable: 0
[22690.997564]  Current CPU id is 1
[22690.997565]  local_irq_disable() called, Disable local interrupts
[22690.997566] After interrupt disable: 1

I think the output confirms that local_irq_disable() does disable local interrupts.

After I disable the kernel preemption and interrupts, In the userspace I use CPU_SET() to pin my process into that particular CPU. But after doing all these I'm still not getting the desired outcome. So, it seems like disabling interrupt of a particular CPU from kernel also disable interrupts for a user space process running on that CPU is not true. I'm confused.

I was looking for an answer to the above question but could not get any suitable answer.

0andriy
  • 4,183
  • 1
  • 24
  • 37
user45698746
  • 305
  • 2
  • 13
  • If you return to user-space with interrupts disabled, how would the kernel ever regain control? From the NMI watchdog using a perf counter event? (NMIs aren't blocked by clearing IF). Or would you depend on user-space that uses this to make a system call eventually? – Peter Cordes Sep 24 '20 at 06:51
  • But anyway, user-space's RFLAGS are saved separately, and restoring it would re-enable interrupts regardless of the kernel's RFLAGS having IF cleared. So maybe you could remove whatever checks `ptrace` does to stop user-space from clearing IF in the saved-RFLAGS of the task state. – Peter Cordes Sep 24 '20 at 06:52
  • @PeterCordes my device driver has another segment to restore interrupts by calling, **local_irq_enable() & put_cpu()** – user45698746 Sep 24 '20 at 06:57
  • So you do depend on user-space making another system call (into your driver) to runs that kernel code, as I speculated. Ok that makes sense. – Peter Cordes Sep 24 '20 at 06:59
  • Interurrupts are enabled on entry to userspace. See for example the comments for `exit_to_user_mode()` in "kernel/entry/common.c" (which is called with interrupts disabled): **Syscall/interupt exit enables interrupts, but the kernel state is interrupts disabled when this is invoked.** – Ian Abbott Sep 24 '20 at 11:33

1 Answers1

3

Duration of the CPU state with disabled interrupts should be short, because it affects the whole OS. For that reason allowing user space code to be run with disabled interrupts is considered as bad practice and is not supported by the Linux kernel.

It is responsibility of the kernel module to wrap by local_irq_disable / local_irq_enable only the kernel code. Sometimes the kernel itself could "fix" incorrect usage of these functions, but that fact shouldn't be relied upon when write a module.

I have a process running in the userspace which I want to run without affected by interrupts and context switch.

Protection from the context switch could be achieved by proper setting of scheduling policy, affinity and priority of the process. That way, the scheduler will never attempt to reschedule your process. There are several questions on Stack Overflow about making a CPU to be exclusive for a selected process.

As for interrupts, they shouldn't be disabled for a user code.

If user code accesses some hardware which should have interrupts disabled, then consider moving your code into the kernel space.

If even rare interrupts badly affect on the performance of your process or its timing, then try to reconfigure Linux kernel to be "more real time". There are also some boot-time configuration options, which could help in further reducing number of interrupts on a specific core(s). See e.g. that question: Why does using taskset to run a multi-threaded Linux program on a set of isolated cores cause all threads to run on one core?.

Note, that Linux kernel is not a base for real-time OS and never intended to be. So, if no configuration and boot settings could help you, consider to choose for your application another OS, which is real time.

Tsyvarev
  • 60,011
  • 17
  • 110
  • 153
  • 1
    Worth mentioning that `isolcpus=2` as a kernel command-line option will get the kernel to truly leave that CPU core (`#2`) alone, perhaps not even configuring it to handle any external interrupts. [Why does using taskset to run a multi-threaded Linux program on a set of isolated cores cause all threads to run on one core?](https://stackoverflow.com/q/36604360). With the right tickless kernel config, you can I think avoid timer interrupts as well. [tickless kernel , isolcpus,nohz\_full,and rcu\_nocbs](https://stackoverflow.com/q/38112585) – Peter Cordes Sep 25 '20 at 18:13
  • @PeterCordes: Thanks for the comment, I have added the first link from it into the answer post. As for avoiding timer interrupts, this is still not "avoiding **all interrupts**" as far as I know: for maintain RCU grace periods, Linux kernel still needs regular interrupts raised on every CPU core. Not sure whether RT configuration/patch makes this need unnecessary. – Tsyvarev Sep 25 '20 at 18:46