5

For some context, I'm profiling the execution of Memcached, and I would like to monitor dTLB misses during the execution of a specific function. Assuming that Memcached spawns multiple threads, each thread could potentially be executing the function in parallel. One particular solution I discovered, Perf features toggle events (Using perf probe to monitor performance stats during a particular function), should let me achieve this by setting probes on function entry and exit and toggling the event counter on/off on each probe respectively.

My question is:

(a) From my understanding, perf toggle events was included as part of a branch to Linux kernel 3.x. Has this been incorporated in recent LTS releases of Linux kernel 4.x? If not, are there any other alternatives?

(b) Another workaround I found is described here: performance monitoring for subset of process execution. However I'm not too sure if this will work correctly for the problem at hand. I'm concerned since Memcached is multi-threaded, having each thread spawn a new child process may cause too much overhead.

Any suggestions?

shreyas42
  • 53
  • 4

1 Answers1

2

I could only find the implementation of the toggle events feature in the /perf/core_toggle repo, which is maintained by the developer of the feature. You can probably compile that code and play with the feature yourself. You can find examples on how to use it here. However, I don't think it has been accepted yet in the main Linux repo for any version of the kernel.

If you want to measure the number of one or more events, then there are alternatives that are easy to use, but require adding a few lines of code to your codebase. You can programmatically use the perf interface or other third-party tools that offer such APIs such as PAPI and LIKWID.

Hadi Brais
  • 22,259
  • 3
  • 54
  • 95
  • 1
    Yes, the perf_toggle_events has been discontinued in recent linux kernel versions. I managed to reach out to the author of the feature and he confirmed as such. Right now I'm working on instrumenting my code using perf_event_open system call. I'll post an update to this post, once I get it to work correctly. – shreyas42 Mar 05 '19 at 10:23
  • @shreyas42 By discontinued you mean it was supported at some point but then later the feature was removed? The toggle events feature sounds very useful. Do you know for which kernel versions it is supported and why it was removed? – Hadi Brais Mar 05 '19 at 21:36
  • So in his mail, Jiri Olsa (author of the toggle_events) mentioned that work on the toggle events functionality has been abandoned and that no one is really looking at it right now. I'm not really sure why they decided so. I actually cloned the perf/core_toggle repo and built the linux kernel. If I remember correctly, its linux kernel version 3.12.x. It was a bit of a nuisance tbh, alot of issues like unsupported gcc-5 and so on. In the end, I decided not to use it seeing as how that kernel is around 6 years old at this point. – shreyas42 Mar 06 '19 at 04:10
  • @shreyas42 But the `perf/core_toggle` branch has never been merged into the main kernel repo at any point in time, right? BTW, if this answers your question, you can click on the check mark below the voting buttons. – Hadi Brais Mar 06 '19 at 04:58
  • Yeah I don't think it's been merged into the main kernel repo. – shreyas42 Mar 06 '19 at 05:29
  • 2
    [Is there a way to find performance of individual functions in a process using perf tool?](https://stackoverflow.com/a/65573180) mentions https://github.com/zyedidia/perforator which apparently uses ptrace and `perf_event_open`. – Peter Cordes Feb 08 '21 at 21:14