3

I have a process that is launched on a Linux-based machine with exactly two cores.

Let's assume my process is the only process in the system (I will ignore other processes and even the system's ones).

My process is divided to two parts:

  1. Critical performance code
  2. Low priority code

Also let's assume my main process was launched on Core 0, and I want to exclusively reserve Core 1 for the critical performance code.

I'd like to divide the question to two:

  1. How can I make sure that every thread in my process (including 3rd party libraries which I have linked my code with that might call pthread_create and etc.) will always open new threads on Core 0 ?

  2. How can I write a test that can verify that Core 1 is doing absolutely nothing besides the performance critical path ?

I am familiar with APIs such as:

pthread_setaffinity_np

that can set a specific thread affinity but I want to know if there is a more low level way to make sure even threads that 3rd party libraries create (from inside the process) will also be pinned to Core 0.

Perhaps I can set the default affinity for the process to be Core 0 and for a specific thread - pin it to Core 1?

Toby Speight
  • 27,591
  • 48
  • 66
  • 103
Itay Marom
  • 801
  • 6
  • 15
  • *Why* would you want to do this? I think I understand the idea of trying to control an essentially random algorithm. But what practical result do you hope to gain? – wallyk Jun 25 '15 at 18:29
  • @wallyk - For example, i don't want the critical performance code to suffer from cache trashing. I want the critical path core to do nothing besides handle my critical path code. – Itay Marom Jun 25 '15 at 18:36
  • 3
    Well, raise the priority of the critical performance code so that it is very likely to remain running on its current core. – Martin James Jun 25 '15 at 18:39
  • @MartinJames - I am not saying it won't help, but seems like a way to improve the situtation - not truely solve it... – Itay Marom Jun 25 '15 at 18:41
  • 1
    Not really my domain, but I couldn't resist. "Perhaps I can set the default affinity for the process to be Core 0 and for a specific thread - pin it to Core 1?" I think that's what I'd do too. But this doesn't prevent the kernel from running other processes on your "reserved" core. For that you need to pass a parameter to the kernel during boot. Or you could change the affinity of all other processes, but that's very inelegant, wouldn't you agree? – Brian Jun 25 '15 at 18:43
  • @Brian: Probably could change the affinity of the `init` process itself? – jxh Jun 25 '15 at 19:39
  • 2
    @jxh Not sure what you mean. What I meant was the [isolcpus](http://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re46.html) flag. Then you'd be guaranteed to be "in charge" of the cores/CPUs you reserved. – Brian Jun 25 '15 at 19:58

2 Answers2

2

You have already described the solution you want:

Perhaps I can set the default affinity for the process to be Core 0 and for a specific thread - pin it to Core 1?

But, perhaps the question is you are not sure how to achieve this.

Linux provides sched_setaffinity to set the affinity of the current process.

To get newly created threads to run on a specific core, the easiest way is to initialize a pthread_attr_t, and set the desired core affinity with pthread_attr_setaffinity_np.

jxh
  • 69,070
  • 8
  • 110
  • 193
  • So basically you are saying that after setting the affinity for the process to Core 0 - every thread, even ones implicitly created by other libraries will not be allowed to run on other cores, but threads that I create can be explicity set to Core 1 - right ? – Itay Marom Jun 25 '15 at 20:14
  • and how about testing for it ? do you have any idea on how to write a test that can make sure no threads are set to Core 1 ? – Itay Marom Jun 25 '15 at 20:16
  • two problems with that method: 1. i want a programmable test so it can be run nightly to verify 2. ps -elFH will only sample the status currently, so theoretically i can "miss" all the time because the other thread is very fast – Itay Marom Jun 26 '15 at 05:41
  • 1
    I have not tried to do what you wish to do. But you might be able to intercept the thread creation calls, and check the CPU mask being used. – jxh Jun 26 '15 at 06:03
-1

One of the solution is to install (if you do not have it already) and run Cpuset utility. Details can be found here

Slava
  • 43,454
  • 1
  • 47
  • 90