Use Python C-API to get the current size of value stack

Question

I'm playing with Python's C-API. Specifically, I want to see if I can inspect how many elements the value stack currently has. Here's my code:

#include <Python.h>
#include <frameobject.h>

int test() {
    PyFrameObject* f = PyEval_GetFrame();
    return (int)(f->f_stacktop - f->f_valuestack);
}

I'm not sure whether this can work, but this line exists in Python's source code, so I gave it a try.

This usually results in negative number, something like -547715639.

So clearly I'm doing it wrong, probably because what'e described in the documentation: "Frame evaluation usually NULLs it (f_stacktop)". What's the right way to do it, or is it even possible?

Can you provide some context for your use-case? Is it one-off? dev-time? production? profiling? debugging? etc... — Dima Tisnek, Apr 30 '20 at 00:28

score 0 · Accepted Answer · answered Apr 22 '20 at 04:29

0

It's impossible. See this answer.

In short, I should use

(int)(stack_pointer - f->f_valuestack);

However, stack_pointer is a local varaible inside the _PyEval_EvalFrameDefault function, thus can't be accessed from outside.

answered Apr 22 '20 at 04:29

laike9m

18,344
20
107
140

score 0 · Answer 2 · answered Apr 28 '20 at 07:18

0

Nothing is impossible ‍♀️

Here's how I get things from value stack in a compiled extension module:

    PyObject **stack_pointer = frame->f_stacktop;

    if (frame->f_stacktop == frame->f_valuestack) {
        PyErr_SetString(PyExc_ValueError, "stack is empty");
        return NULL;
    }

    if (!stack_pointer) {
        PyErr_SetString(PyExc_ValueError, "stack pointer is null?");
        return NULL;
    }

    // Top-most on the value stack ought to be the "receiver".
    PyObject *next = stack_pointer[-1];

Source: https://github.com/dimaqq/awaitwhat/blob/e242ea7502cd13ebf7aca16700240aa592467f3a/awaitwhat/what.c#L24-L37

And here's how to get it in Python using ctypes: https://stackoverflow.com/a/14034945/705086

answered Apr 28 '20 at 07:18

Dima Tisnek

11,241
4
68
120

Sorry, I didn't really get it. How do you get the size of value stack? – laike9m Apr 28 '20 at 07:32
The delta between stack top and value stack (in elements) is the number of values on the stack. Caveats: stack grows downwards, thus, first element is at `[-1]`; I think one of these values is not updated after every instruction (that's an optimisation), but it is updated when e.g. generator is suspended, probably if this frame calls a function, etc. – Dima Tisnek Apr 28 '20 at 08:47
1

This answer(https://stackoverflow.com/a/44443331/2142577) explains why most of the time `f_stacktop` is null, so I don't think this is a general solution. Your lib is targeting `aysnc/await` scenarios, which, `f_stacktop` happens to not be NULL. – laike9m Apr 28 '20 at 18:08
This make sense, while Python executes classical synchronous code, 1 eval frame ≅ 1 python frame, thus stack top is only kept on (C) stack. Generators and coroutines break this equality, so stack top must be persisted. If it's the earlier, then read it like `gdb` would. On Linux, you can do this at runtime with `bpftrace` or `oprofile`, batteries not included :) If your use-case is a one-off, build custom cpython with a few tweaks :) – Dima Tisnek Apr 30 '20 at 00:25
Yes. But anyway, I found a way that does not require knowing the stack size, so it's fine. It's for debugging purposes. – laike9m Apr 30 '20 at 05:44

Use Python C-API to get the current size of value stack

2 Answers2