8

I want to build a visual debugger, which helps programming students to see how expression evaluation takes place (how subexpressions get evaluated and "replaced" by their values, something like expression evaluation visualizer in Excel).

Looks like you can't step through this process with Python's pdb, as its finest step granularity is line of code. Is it somehow possible to step through Python bytecode? Any other ideas how to achieve this goal?

EDIT: I need a lightweight solution that can be built on top of CPython standard library.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Aivar
  • 6,814
  • 5
  • 46
  • 78
  • wouldn't that be something like pythons c level debugger [gdb](http://wiki.python.org/moin/DebuggingWithGdb)? –  Dec 13 '12 at 14:21
  • @X.Jacobs - I need a solution that can be built upon CPython default distribution and it should work in Windows as well as in OSX and Linux. – Aivar Dec 13 '12 at 14:24
  • [Debugging C/C++ and CPython using GDB 7′s new Python extension support](http://misspent.wordpress.com/2012/03/24/debugging-cc-and-cpython-using-gdb-7s-new-python-extension-support/) –  Dec 13 '12 at 14:33
  • @X.Jacobs -- The problem is that my students don't have GDB and I don't want to make them install it. – Aivar Dec 13 '12 at 14:59
  • ok, please be more explicit when posting –  Dec 13 '12 at 15:24

4 Answers4

2

I have a solution idea also myself -- I could instrument the code (or AST) by wrapping all (sub)expressions in a dummy method call, which does nothing more than returning its argument. Eg. 

x = f(sin(x + y))

becomes

x = dummy(f(dummy(sin(dummy(dummy(x) + dummy(y))))))

This way I'm guaranteed to be notified after each subexpression gets evaluated and I also get the values. I can also add extra location/AST information about which part of the expression is currently dealt with, eg:

... dummy(x, line=23, col=13, length=1) ...

Unfortunately this requires messing with AST and compilation ...

Aivar
  • 6,814
  • 5
  • 46
  • 78
  • Hi! It seems that I'm solving the same problem as you :) Could you please take a look at [my question](http://stackoverflow.com/questions/40639652/tracing-python-expression-evaluation-step-by-step). There's a bounty open. It seems that I'm looking for a part of code that you described here. I can try to dig into the code of your Thonny software (as I'd like to have something interoperating with Jupyter and I don't want the full debugger, I currently don't want to use Thonny), but I believe you will do it better :) – Ilya V. Schurov Nov 27 '16 at 21:22
  • @IlyaV.Schurov, the relevant code is here https://bitbucket.org/plas/thonny/src/master/thonny/backend.py?at=master&fileviewer=file-view-default , maybe you can reuse something from there. You probably also need https://bitbucket.org/plas/thonny/src/master/thonny/ast_utils.py?at=master&fileviewer=file-view-default because Python AST doesn't include enough (correct) information about where each AST node is located in the code. – Aivar Nov 30 '16 at 18:38
  • Thanks! Could you please copy your answer to [my question](http://stackoverflow.com/questions/40639652/tracing-python-expression-evaluation-step-by-step) to allow me considering it as a possible bounty winner? – Ilya V. Schurov Nov 30 '16 at 20:56
2

Have you tried pudb? http://pypi.python.org/pypi/pudb On a debian-like: apt-get install python-pudb

It attaches to pdb, so I guess this is not what you're looking for. At least, when you step in a function, it clearly appears which one you're in.

For teaching students, something that you could can be:

  • first, write the program using variables, composing using several steps,
  • debug this program using whichever decent python debugger (pdb, winpdb, pudb ...),
  • then, once the process is well understood, get rid of temporary variables, by combining the code into fewer lines, gradually, until you come to the final code.

I know, it is far to be perfect, but this is the best I can think of, at the moment.

Pierre
  • 530
  • 5
  • 13
1

Using pdb, any function call can be stepped into. For any other statement, pdb can print the values of the relevant names in the line. What additional functionality are you looking for that isn't covered?

If you're trying to 'step into' things like a list comprehension, that won't work from a pure Python perspective because it's a single opcode. At some point for every expression you'll need to tell your students 'and this is where Python goes into the C implementation and evaluates this...'.

jeffknupp
  • 5,966
  • 3
  • 28
  • 29
  • The trouble is when there are several function calls on a line. If I choose "step" command at this line then I'll find myself in one of those functions and it's not clear which one. If Python stepped into all functions then I could parse the expression in advance to find out in which order the function calls get evaluated. This would help me keep the track of what subexpression must be under evaluation at the moment. But unfortunately Python doesn't step into builtin/C functions and syntactically those function calls are indistinguishable from calls to user defined functions. – Aivar Dec 19 '12 at 22:09
  • It does step into all functions as long as you keep pressing 's'. And your complaint that 'Python doesn't step into builtin/C functions' is exactly my point: at some point, you descend into C and need a way to deal with that. That could be gdb or something else entirely, but it will _always_ happen. – jeffknupp Dec 19 '12 at 22:28
  • I don't want to step into C functions, I just want to know where I am in terms of original expression, but Python doesn't tell me this (see my other question: http://stackoverflow.com/questions/13952022/). I could count the number of function entries and keep track of the position myself, but the fact that some (unknown number) of function calls will be ignored by tracer, breaks my counting system. – Aivar Dec 20 '12 at 08:23
  • In that case, you'll need to set a breakpoint in the main opcode evaluation loop using gdb with Python support. Anything from a pure Python perspective seems like it will not be granular enough for your needs. Even a breakpoint on the main loop isn't enough, though, since many opcodes have a fast path where they predict what the next one will be and dispatch them without going through the loop again (see PyEval_EvalFrameEx in Python/ceval.c). In that case, you'd have to break where any and every opcode is evaluated – jeffknupp Dec 20 '12 at 12:29
1

You should check out reinteract, it's pretty simple and you could contribute to that

Aamir Rind
  • 38,793
  • 23
  • 126
  • 164
asdf
  • 11
  • 1