6

I'm trying to run a program on a dedicated core in Linux. (I know Jailhouse is a good way to do so, but I have to use off-the-shelf Linux. :-( )

Other processes, such as interrupt handlers, kernel threads, service progresses, may also run on the dedicated core occasionally. I want to disable as many such processes as possible. To do that, I need first pin down the list of processes that may run on the dedicated core.

My question is:

Is there any existing tools that I can use to trace the list of PIDs or processes that run on a specific core over a time interval?

Thank you very much for your time and help in this question!

Claudio
  • 10,614
  • 4
  • 31
  • 71
Mike
  • 1,841
  • 2
  • 18
  • 34
  • One way to do it could be to boot with `loglevel=7`, that will print **a lot** into the kernel log (`dmesg`). It will include the scheduler debug, which you can then analyse. – grochmal Jun 01 '16 at 19:00
  • @grochmal, I'm thinking if there is any perf like tools to do this. dmesg is ok but it may not capture all of the services. For example, no printk is allowed in ISR, which means we won't be able to capture it in dmesg... – Mike Jun 01 '16 at 19:16
  • 3
    Can you run systemtap? A small modification to the [process migrate tapset](https://sourceware.org/systemtap/examples/process/migrate.stp) may do what you want. – Mark Plotnick Jun 01 '16 at 19:42

3 Answers3

3

TL;DR Dirty hacky solution.

DISCLAIMER: At some point stops working "column: line too long" :-/

Copy this to: core-pids.sh

#!/bin/bash

TARGET_CPU=0

touch lastPIDs
touch CPU_PIDs

while true; do
  ps ax -o cpuid,pid | tail -n +2 | sort | xargs -n 2 | grep -E "^$TARGET_CPU" | awk '{print $2}' > lastPIDs
  for i in {1..100}; do printf "#\n" >> lastPIDs; done
  cp CPU_PIDs aux
  paste lastPIDs aux > CPU_PIDs
  column -t CPU_PIDs > CPU_PIDs.humanfriendly.tsv
  sleep 1
done

Then

chmod +x core-pids.sh
./core-pids.sh

Then open CPU_PIDs.humanfriendly.tsv with your favorite editor, and ¡inspect!

The key is in the "ps -o cpuid,pid" bit, for more detailed info, please comment. :D

Explanation

Infinite loop with

  • ps -o cpuid,pid | tail -n +2 | sort | xargs -n 2 | grep -E "^$TARGET_CPU" | awk '{print $2}' > lastPIDs
    • ps ax -o cpuid,pid
      • Show pid's associated to CPU
    • tail -n +2
      • remove headers
    • sort
      • sort by cpuid
    • xargs -n 2
      • remove white spaces at begging
    • grep -E "^$TARGET_CPU"
      • filter by CPU id
    • awk '{print $2}'
      • get pid column
    • > lastPIDs
      • output to file those las pid's for the target CPU id
  • for i in {1..10}; do printf "#\n" >> lastPIDs; done
    • hack for pretty .tsv print with the "columns -t" command
  • cp CPU_PIDs aux
    • CPU_PIDs holds the whole timeline, we copy it to aux file to allow the next command to use it as input and output
  • paste lastPIDs aux > CPU_PIDs
    • Append lastPIDs columns to the whole timeline file CPU_PIDs
  • column -t CPU_PIDs > CPU_PIDs.humanfriendly.tsv
    • pretty print whole timeline CPU_PIDs file

Attribution

Community
  • 1
  • 1
ElMesa
  • 928
  • 8
  • 19
1

The best way to obtain what you want is to operate as follows:

  1. Use the isolcpus= Linux kernel boot parameter to "free" one core from the Linux scheduler
  2. Disable the irqbalance daemon (in case it is executing)
  3. Set the IRQs affinities to the other cores by manually writing the CPU mask on /proc/irq/<irq_number>/smp_affinity
  4. Finally, run your program setting the affinity to the dedicated core through the taskset command.

In this case, such core will only execute your program. For checking, you can type ps -eLF and look at the PSR column (which specifies the CPU number).

Claudio
  • 10,614
  • 4
  • 31
  • 71
0

Not a direct answer to the question, but I am usually using perf context-switches software event to identify the perturbation of the system or other processes on my benchmarks

Manuel Selva
  • 18,554
  • 22
  • 89
  • 134