0

Does the OS have visibility into the call stack (e.g. calls made between functions) in CPython? E.g. In what way is the OS involved in the creation, retrieval and/or management of the Python stack and operations of its stack frames?

Context:

  • My understanding is that the Python interpreter does not support tail call recursion, so this seems to be something left to Python to handle.
  • Most OS impose a maximum limit on the size of a stack (e.g. I believe in Linux OS the maximum stack size is 8192 KB by default but it can be changed via e.g. ulimit), meaning the kernel clearly can get involved in at least limiting the size of the call stack.
Josh
  • 11,979
  • 17
  • 60
  • 96

1 Answers1

1

In what way is the OS involved in the creation, retrieval and/or management of the Python stack and operations of its stack frames?

It isn't. The stack frames are for the process to take care of, the kernel does not interfere.

My understanding is that the Python interpreter does not support tail call recursion, so this seems to be something left to Python to handle.

Well yes, it's Python's job to handle its own stack, regardless of tail recursion. The fact that Python does not support tail recursion may have some disadvantages for deep recursive calls, but the code could always be rewritten as iterative.

See also: What is the maximum recursion depth in Python, and how to increase it?

the kernel clearly can get involved in at least limiting the size of the call stack

Yes, indeed the kernel does limit the stack size. The way it is done is by allocating an invisible guard page just after the top of the stack: when the stack is full, making another call (thus adding another stack frame) will trigger reads and/or writes into the guard page, and the kernel will detect it and increase the stack size. This only happens up to a certain predefined amount though, after which the process is killed for exceeding the maximum allowed stack size.

See also: Stack memory management in Linux

Marco Bonelli
  • 63,369
  • 21
  • 118
  • 128
  • Thanks, so the kernel is involved insofar it presents it as a virtual memory space designated as `stack` (separate from `heap`) to simply limit its size and resolve any page faults? i.e. the kernel doesn't care what it is used for? i.e. whether a process uses it to store stack frames or any other type of data is irrelevant to the kernel? – Josh Jun 07 '20 at 14:16
  • @Josh mostly yes. It's still a special area and it's treated differently in many cases, but the process itself is responsible for its management. – Marco Bonelli Jun 07 '20 at 14:30