What is the stack used for in CPython, if anything?

Question

As far as I understand:

The OS kernel (e.g. Linux) always allocates a stack for each system-level thread when a thread is created.
CPython is known for using a private heap for its objects, including presumably the call stack for Python subroutines.

If so, what is the stack used for in CPython, if anything?

score 1 · Accepted Answer · answered Jul 05 '20 at 06:37

CPython is an ordinary C program. There is no magic in running Python script / module / REPL / whatever: every piece of code must be read, parsed, interpreted — in a loop, until it's done. There is whole bunch of processor instructions behind every Python expression and statement.

Every "simple" top-level thing (parsing and production of bytecode, GIL management, attribute lookup, console I/O, etc) is very complex under the hood. If consists of functions, calling other functions, calling other functions... which means there is stack involved. Seriously, check it yourself: some of the source files span few thousand lines of code.

Just reaching the main loop of the interpreter is an adventure on it's own. Here is the gist, sewed from pieces from all around the code base:

#ifdef MS_WINDOWS
int wmain(int argc, wchar_t **argv)
{
    return Py_Main(argc, argv);
}
#else
// standard C entry point
#endif

int Py_Main(int argc, wchar_t **argv)
{
    _PyArgv args = /* ... */;
    return pymain_main(&args);
}

static int pymain_main(_PyArgv *args)
{
    // ... calling some initialization routines and checking for errors ...
    return Py_RunMain();
}

int Py_RunMain(void)
{
    int exitcode = 0;
    pymain_run_python(&exitcode);
    // ... clean-up ...
    return exitcode;
}

static void pymain_run_python(int *exitcode)
{
    // ... initializing interpreter state and startup config ...
    // ... determining main import path ...
    if (config->run_command) {
        *exitcode = pymain_run_command(config->run_command, &cf);
    }
    else if (config->run_module) {
        *exitcode = pymain_run_module(config->run_module, 1);
    }
    else if (main_importer_path != NULL) {
        *exitcode = pymain_run_module(L"__main__", 0);
    }
    else if (config->run_filename != NULL) {
        *exitcode = pymain_run_file(config, &cf);
    }
    else {
        *exitcode = pymain_run_stdin(config, &cf);
    }
    // ... clean-up
}

int PyRun_AnyFileExFlags(FILE *fp, const char *filename, int closeit, PyCompilerFlags *flags)
{
    // ... even more routing ...
    int err = PyRun_InteractiveLoopFlags(fp, filename, flags);
    // ...
}

int PyRun_InteractiveLoopFlags(FILE *fp, const char *filename_str, PyCompilerFlags *flags)
{
    // ... more initializing ...
    do {
        ret = PyRun_InteractiveOneObjectEx(fp, filename, flags);
        // ... error handling ...
    } while (ret != E_EOF);
    // ...
}

Got it, so just because CPython uses a **private heap** for Python objects, data structures across Python function calls, that doesn't preclude, of course, CPython from using the stack that the OS provides for those C calls, right? — Josh, Jul 05 '20 at 15:07
@Josh Python script is running on top of CPython (the interpreter), and CPython is running on top of the CPU. CPython cannot run on top of itself / on it's own python stack, because these stacks are incompatible, they use different data structures — one is defined by C, other is defined by CPython. — Shadows In Rain, Jul 05 '20 at 19:07
Also, CPython uses OS-provided stack not because it can, but because it has to. It does not mater by what means stack is provided. I mean, in theory it's possible to write C program that uses no stack, but that would be horrible exercise in masochism. — Shadows In Rain, Jul 05 '20 at 19:08

What is the stack used for in CPython, if anything?

1 Answers1