0

In order to debug how a complicated library executes a function call, e.g. pd.Series(np.NaN)|pd.Series("True") I would like to generate a list of all states of the stack during the execution of that function. So for every line of Python code executed (even inside functions and functions called by functions), there should be one entry in the list of stack traces).

This is a bit like a profile generated by a sampling profiler. But in contrast, I don't want a stack trace for every millisecond, but for every line of Python code executed.

I know I can manually call this function using pdb, and repeatedly enter step to step into any function calls and where to print stack traces of the current state - but I want to do this automatically.

How can I automate this stack trace collection? Is there a package for this? If not, how do I automate that with, for example, pdb or another tool that does the job? Is there an accepted word for such a list of stack traces?

That stack list could be used for at least two purposes: a) quickly finding all code that is reached, reducing the scope for finding relevant lines, b) creating a "flamegraph" of execution.

Somewhat related questions:
What cool hacks can be done using sys.settrace?
sandboxing/running python code line by line

Cornelius Roemer
  • 3,772
  • 1
  • 24
  • 55
  • check this: [Print current call stack from a method in code](https://stackoverflow.com/questions/1156023/print-current-call-stack-from-a-method-in-code) – deadshot Feb 10 '23 at 10:33
  • @deadshot I apologize for being unclear in my question. I don't want one stacktrace at one point (which is what you are pointing to). I want the stack trace of _all_ steps during the execution of a function, as if I was manually pressing `s` and `w` and collecting the resulting stack traces into a list of stacktraces. – Cornelius Roemer Feb 10 '23 at 10:35

1 Answers1

0

Without using a debugger you can only get the stack trace of the current point of execution, i.e. the position of your code.

Therefore you have three options:

  • If you can change to called functions code, you can inject a tiny function that makes the stack trace. Look at the traceback module on how to do this.
  • Some libraries/frameworks have the ability to tune the logging options. Lookup the doc (or the source code) of the called functions if they do logging and you may enable this using the logging module.
  • Use a debugger.
Holger Waldmann
  • 101
  • 1
  • 3
  • Thanks! I clarified the question a bit, specifically, I want the stack trace for each line of Python code executed. That means only the debugger option is viable here. Is this such an odd thing to do that there is no prior art/package? – Cornelius Roemer Feb 10 '23 at 10:41
  • 1
    This is just what debuggers do. You may find a package that automates a debugger. But I do not know of such a package. – Holger Waldmann Feb 10 '23 at 10:45