0

This is an old problem as is demonstrated as in https://community.intel.com/t5/Analyzers/Unable-to-view-source-code-when-analyzing-results/td-p/1153210. I have tried all the listed methods, none of them works, and I cannot find any more solutions on the internet. Basically vtune cannot find the custom python source file no matter what is tried. I am using the most recently version as of speaking. Please let me whether there is a solution.

For example, if you run the following program.

def myfunc(*args):
    # Do a lot of things. 

if __name__ = '__main__':
    # Do something and call myfunc

Call this script main.py. Now use the newest vtune version (I have using Ubuntu 18.04), run the vtune-gui and basic hotspot analysis. You will not found any information on this file. However, a huge pile of information on Python and its other codes are found (related to your python environment). In theory, you should be able to find the source of main.py as well as cost on each line in that script. However, that is simply not happening.

Desired behavior: I would really like to find the source file and function in the top-down manual (or any really). Any advice is welcome.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
RanWang
  • 310
  • 2
  • 12
  • CPython is an interpreter. The CPU is never running machine code compiled from this source file, only ever machine code ahead-of-time compiled from CPython's C source files. Your program is *data* in the interpreter. If you want info about time spent by Python function, you need a Python profiler ([How can you profile a Python script?](https://stackoverflow.com/q/582336)), not VTune profiling the Python interpreter as it runs your program. There's no way for VTune to look inside the Python interpreter and know which Python function / line it's on when a perf event fires. – Peter Cordes Dec 26 '20 at 07:38
  • So basically no, you shouldn't be able to find perf samples for `main.py` because it's not running on the CPU, it's just data for another program. – Peter Cordes Dec 26 '20 at 07:39
  • 1
    @PeterCordes If you search Intel Vtune website with the keyword python, they always show that it can profile and find the python file. It also mentions that they can profile mixed language, which includes python. – RanWang Dec 28 '20 at 15:45
  • Ok, maybe I assumed wrong, and VTune can hook into the Python implementation. I haven't used VTune, just simpler tools like Linux `perf` that only do perf counters. Did you check that it's supposed to work with CPython, not a Python implementation like PyPy that JITs to machine code? – Peter Cordes Dec 28 '20 at 22:54
  • @PeterCordes Yes, it actually can only work with CPython. I should all the CPython details except for what I care about. Very peculiar taste indeed. – RanWang Dec 29 '20 at 17:05
  • VTune seems to be picky on Python versions. It works for me for Python 3.7, but not Python 3.9. – RoastDuck Dec 14 '22 at 14:18

1 Answers1

1

VTune offer full support for profiling python code and the tool should be able to display the source code in your python file as you expected. Could you please check if the function you are expecting to see in the VTune results, ran long enough?

Just to confirm that everything is working fine, I wrote a matrix multiplication code as shown below (don't worry about the accuracy of the code itself):

def matrix_mul(X, Y):

  result_matrix = [ [ 1 for i in range(len(X)) ] for j in range(len(Y[0])) ]

  # iterate through rows of X

  for i in range(len(X)):

    # iterate through columns of Y

    for j in range(len(Y[0])):

      # iterate through rows of Y

      for k in range(len(Y)):

        result_matrix[i][j] += X[i][k] * Y[k][j]



  return result_matrix

Then I called this function (matrix_mul) on my Ubuntu machine with large enough matrices so that the overall execution time was in the order of few seconds.

I used the below command to start profiling (you can also see the VTune version I used):

/opt/intel/oneapi/vtune/2021.1.1/bin64/vtune -collect hotspots -knob enable-stack-collection=true -data-limit=500 -ring-buffer=10 -app-working-dir /usr/bin -- python3 /home/johnypau/MyIntel/temp/Python_matrix_mul/mat_mul_method.py

Now open the VTune results in the GUI and under the bottom-up tab, order by "Module / Function / Call-stack" (or whatever preferred grouping is).

You should be able to see the the module (mat_mul_method.py in my case) and the function "matrix_mul". If you double click, VTune should be able to load the sources too.

Rahila T - Intel
  • 832
  • 4
  • 11