2

I am testing my multithreaded application in Linux RT multicore machine.

However during testing, we are observing that scheduling (created with SCHED_FIFO scheduling policy ) in Linux RT is not happening according to the SCHED_FIFO policy. We could see in multiple places that the higher priority thread execution is getting preempted by a lower priority thread.

Based on some research we did on the internet, we found that the following kernel parameters need to be changed from

/proc/sys/kernel/sched_rt_period_us containing 1000000
/proc/sys/kernel/sched_rt_runtime_us containing 950000

to

/proc/sys/kernel/sched_rt_period_us containing 1000000
/proc/sys/kernel/sched_rt_runtime_us containing 1000000

or

/proc/sys/kernel/sched_rt_period_us containing -1
/proc/sys/kernel/sched_rt_runtime_us containing -1

We tried doing both but still we are facing the problem sometimes. We are facing the issue even when higher priority thread is not suspended by any system call.

It would be great if you could let us know if you are aware of such problems in Linux RT scheduling and/or have any solutions to make the Linux RT scheduling deterministic based on priority.

There are no printfs or any system calls in the higher priority thread but still the higher priority thread is getting preempted by the lower priority thread.

Also I have made sure all the threads in the process are running on a single core using taskset command.

Houmles
  • 197
  • 11
Syed Aslam
  • 79
  • 7
  • Is it just a priority inversion (http://en.wikipedia.org/wiki/Priority_inversion)? Does the lower priority thread hold a lock the higher priority one needs? – Joe Oct 01 '14 at 13:17
  • No, the higher priority threads are not waiting for any locks from the lower prio threads. – Syed Aslam Oct 01 '14 at 13:22
  • And no libraries you use are waiting on any locks? – Joe Oct 02 '14 at 09:02
  • I think it might be the higher priority thread were not preempted but rather the lower priority threads were running on a different core (both running). Try to run on single core, with affinity control, and see if you get the results that you are expecting. – hesham_EE Sep 16 '16 at 16:52
  • @Syed Aslam : I have the same problem but with a very small MCVE. Could you have a look at [Linux not respecting SCHED_FIFO priority,normal or GDB execution](https://stackoverflow.com/q/64061391/3972710) and tell me what you think about , taken into account the experience you gathered from your question. – NGI Sep 26 '20 at 17:05

1 Answers1

0

There could be two reasons:

  • CPU throttling: the scheduler is designed to reserve some CPU time to non-RT tasks; you have already disabled it by acting on the /proc/sys/kernel/ entries
  • blocking: your high-priority task is blocking either
    • on some synchronization mechanism (e.g., mutex, semaphore) or
    • on some blocking call (e.g., malloc, printf, read, write, etc.)
Claudio
  • 10,614
  • 4
  • 31
  • 71
  • Unfortunately, we have checked for both the reasons and both are not present. there is no priority inverion nor the high prio threads are waiting for mutex/semaphore/prints/malloc/free – Syed Aslam Oct 01 '14 at 14:12