28

MacOSX Xcode Instruments is really great for profiling native code. However, I have also a huge chunk of Python calls in my callstacks. Can I somehow make Instruments Python-aware?

One solution I could think of is when it sees some PyEval_EvalFrameEx frame that it looks in its local variables / parameters to separate different Python calls and show me some call info.

I'm not asking about just a Python profiler. I want to profile my native code. But in this native code profiling, I want to add some further intelligence to analyze and translate the Python stack frames.

Albert
  • 65,406
  • 61
  • 242
  • 386
  • 3
    have you tried [RunSnakeRun](http://www.vrplumber.com/programming/runsnakerun/)? I've personally never used it, but it appears to be what you're looking for. – James Mertz Oct 20 '14 at 20:06
  • @KronoS: I think I have seen similar solutions for cProfile and exporters to KCacheGrind. However: (1) I would like to also see the native C code profile, i.e. both combined. (2) I also would like to esp. use Xcode Instruments. – Albert Oct 21 '14 at 08:28
  • 1
    Maybe I don't fully understand what you're asking, but maybe you could run your script with `python -m cProfile myscript.py` ? Then you could just save the output to a logfile – ollien Nov 08 '14 at 22:30
  • @ollien: Then I don't see the native (C/C++/ObjC) function calls. – Albert Nov 09 '14 at 15:01

3 Answers3

4

According to this stackoverflow answer, Instruments is a GUI front-end to dtrace. There is Apple Documentation confirming this and some OS-X specific articles on dtrace at Big Nerd Ranch among other places.

There are patches that can be applied to the CPython source before compiling it to instrument it for dtrace. It appears that there is or used to be support for automatically building a new python with dtrace in homebrew, but googling now, I'm not finding references for a homebrew recipe with dtrace provider support for current python releases (2.7.10, 3.4/3.5). I haven't tried, so maybe the current recipe just works with a --with-dtrace switch when building.

There is a talk from PyTexas 2013: dtrace, Python and You which talks about getting a python install with dtrace support included (specifically demonstrating with a Mac), and using dtrace on the command line.

I would think that once you had a python with dtrace support installed, when running it, you should be able to see and use it in Instruments. If you're adding a python interpreter to an OS X application (either as a .framework or some other form of linking), if that python had the dtrace patches applied before compilation, I would also think that it would be available to work with in dtrace. I've tried neither, but given what I know about dtrace, I believe it should work. If I confirm this is true, I will post back.

Community
  • 1
  • 1
Matt Anderson
  • 19,311
  • 11
  • 41
  • 57
2

There is no MacOSX instrument to profile Python code. Personally, I use cProfile. Its an internal profiler called cProfile. You can use it in either of the ways below:

import cProfile
cProfile.run('print "Hello World!"')

or

python -m cProfile your_own_script.py

The result would be something like:

>>> cProfile.run('print "Hello World!"')
Hello World!
         2 function calls in 0.000 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Edit: If you are looking for native level calls. Then you should move to strace(for Linux) and dtruss (for Mac OSX) or someother tool like that. Basically you can wrap your Python script in an strace call and you will be able to view all your "native" C/C++ calls. For other linux systems use: 

strace -fetrace=open python your_script.py

or if you are on Mac OSX: 

dtruss -f -t open python your_script.py

I hope this helps!

tricklepie
  • 62
  • 4
  • That is not the question. The question is about native code. – Albert Dec 10 '14 at 09:25
  • If you are looking for native level calls. Then you should move to `strace` or someother tool like that. Basically you can wrap your Python script in an `strace` call and you will be able to view all your "native" C/C++ calls. – tricklepie Dec 10 '14 at 17:01
  • So, how would I see the Python calls then in `strace`? I don't just want to see `PyEval_EvalFrameEx` or so there, I want to see the real Python function. (Also, I asked not about `strace` but about MacOSX Instruments, but that is less important.) – Albert Dec 11 '14 at 08:20
  • I found [Traceback](https://docs.python.org/2/library/traceback.html). Not sure if it helps. – tricklepie Dec 12 '14 at 16:31
  • That works only *in* Python, and prints the Python stacktrace. Not sure how this should work with `strace` or so. – Albert Dec 13 '14 at 16:21
  • @Albert The question is "MacOSX Instruments to profile Python code". If you want the question to be about native code, you should probably put native code in the question. A lot of people aren't going to read to the third paragraph of your description and they're going to miss the clarification. =) – rakslice Dec 24 '14 at 18:38
  • @rakslice: MacOSX Instruments is a native profiler. Do you know that tool? You can only do native profiling with it. Also, people really should read the full question before they want to answer. – Albert Dec 26 '14 at 12:36
0

There is a great new GUI profiler called PyVmMonitor. I haven't successfully got it to attach to my running app yet, but I've been using it to generate and analyze profiles from the command line like this: 

python /Applications/PyVmMonitor.app/Contents/MacOS/public_api/pyvmmonitor --profile=yappi my_app.py
Joseph Sheedy
  • 6,296
  • 4
  • 30
  • 31