How can I get the Perf working in an Azure VM?

Question

I am trying to profile an application inside an Azure VM (Ubuntu 18.04) using the perf. But the perf stats doesn't provide all the statistics.

Performance counter stats for 'ls':

          0.78 msec task-clock                #    0.760 CPUs utilized          
             1      context-switches          #    0.001 M/sec                  
             0      cpu-migrations            #    0.000 K/sec                  
           106      page-faults               #    0.135 M/sec                  
<not supported>      cycles                                                      
<not supported>      instructions                                                
<not supported>      branches                                                    
<not supported>      branch-misses                                               

   0.001031061 seconds time elapsed

   0.001016000 seconds user
   0.000000000 seconds sys

And, also the perf list doesn't list any hardware events.

List of pre-defined events (to be used in -e):

alignment-faults                                   [Software event]
bpf-output                                         [Software event]
context-switches OR cs                             [Software event]
cpu-clock                                          [Software event]
cpu-migrations OR migrations                       [Software event]
dummy                                              [Software event]
emulation-faults                                   [Software event]
major-faults                                       [Software event]
minor-faults                                       [Software event]
page-faults OR faults                              [Software event]
task-clock                                         [Software event]

duration_time                                      [Tool event]

msr/pperf/                                         [Kernel PMU event]
msr/smi/                                           [Kernel PMU event]
msr/tsc/                                           [Kernel PMU event]

rNNN                                               [Raw hardware event descriptor]
cpu/t1=v1[,t2=v2,t3 ...]/modifier                  [Raw hardware event descriptor]

I read that the CPU performance counters are not enabled for the VMs source.

Is it possible to enable the hardware events in Azure VM? or is there any other way to find the instructions used while executing an application?

Most VMs don't virtualize and pass through the PMU HW performance counters. I think Linux KVM was working on it, but IDK if even that is done yet. — Peter Cordes, Sep 29 '21 at 19:30
If you don't care about timing, just counting instructions, you could use dynamic instrumentation or emulation. e.g. Intel's SDE has an instruction-mix counter that can even break things down by mnemonic. See [How do I monitor the amount of SIMD instruction usage](https://stackoverflow.com/q/60104698) for example. — Peter Cordes, Sep 29 '21 at 19:34
@PeterCordes Thanks for your input. I will try dynamic instrumentation. — Naren, Oct 04 '21 at 09:33
@Naren are you possibly able to write an answer outlining your dynamic instrumentation tests in detail? Or didn't this work out as expected? — Simon Sobisch, Jul 11 '22 at 19:31

score 1 · Answer 1 · answered Jul 11 '22 at 19:27

Question:

is there any other way to find the instructions used while executing an application?

Answer:
If you have time or can adjust the application to limit the checking to only parts of the program, then valgrind's callgrind tool will get you very accurate numbers, along with a deep analysis option (inspecting the call profile via kachegrind, for example); but applications using this commonly take 10-20x longer, which may be a big problem (perf stat -e instructions don't do much, if anything to the time taken).

Side-question: is perf usable on local virtual machines / cloud virtual guests?
It definitely is, but not on all configurations (and if you can't change the virtualization settings, like it seems to be the case for most Azure environments [only works if you can adjust the Hyper-V configuration or get those adjusted (please drop a comment if you found a way for that as I haven't)], you can't get perf counters).
Belong the known setups are Qemu, VMWare and others - in general you can check the documentation for VMs with RR, as this software works on hardware counters, too.

How can I get the Perf working in an Azure VM?

1 Answers1