2

When I learned C/C++ I not only learned the syntax of the language and the semantics of the language constructs but also how the program was executed by the computer. I learned stuff like:

All local variables are declared in a stack frame, which is allocated each time the function is called. These frames are laid out on the call stack one right after the other, and they're 'unwound' when the function returns, thus quickly and efficiently 'destroying' the local variables for that function

This, in turn, helped me figure out why it's a bad idea to take the address of a local variable & return it to the calling function. In other words, understanding C/C++'s memory model and the environment in which the code executes helps develop a deeper understanding of how to write correct programs.

Another example: malloc/new allocate objects in the 'heap' (and not the stack), which both explains why they live beyond the end of the function that allocated them. Further, knowing that these functions/keywords return the memory address that the object is located at helped me to understand how things like linked lists work. (And how to figure out whether I need another * or -> or not.)

So now I'm learning Python and I'm looking for a concise, clear, yet thorough explanation of how Python programs manage their memory and execution environment.

Searching online hasn't been particularly fruitful - I might be using the wrong search terms, but there appears to be very little out there in general.

I've looked through https://docs.python.org/ and found it to be an excellent resource for syntax and semantics ("if you type X, then Y will happen"), but it doesn't really describe what the computer is doing 'under the hood'.

I've found several posts here on Stack Overflow ( such as this, this, and this ) but these all seem to focus on specific situations.

Does anyone know of a resource that actually explains what Python is doing under the hood'?

EDIT: I'm getting feedback from StackOverflow that this question may be a duplicate. The other question asks 'how does a .PY file get compiled (to bytecode) and then executed by the VM?' What I'm asking here is 'is there a page that explains how Python lays variables, objects, functions, etc out in memory AND explains how that's all used to actually run Python programs' (Sub-question: Is this an appropriate way to address the concern about a duplicate question? Would it be better to put this in a comment?)

Community
  • 1
  • 1
MikeTheTall
  • 3,214
  • 4
  • 31
  • 40
  • python is actually written in C so you would be able to get the source code for the language itself and read through it if that would help you understand how it works. – Tadhg McDonald-Jensen Apr 01 '16 at 16:34
  • You may also find [this Programmers question](http://programmers.stackexchange.com/q/246094/143097) helpful. – skrrgwasme Apr 01 '16 at 16:40
  • 2
    https://docs.python.org/3/faq/design.html?highlight=garbage%20collection#how-does-python-manage-memory has some pointers – Ilja Everilä Apr 01 '16 at 16:41
  • are you looking for something like this? You can get information about an objects memory location using the `id` builtin but there is no way to directly access an object by its id, instead the interpreter keeps track of how many references exist for a particular object and when there are no more references that object gets garbage collected which calls its `__del__` method for it to clean up it's resources such as a file closing the open io. – Tadhg McDonald-Jensen Apr 01 '16 at 16:41
  • Thanks, @llja! That's exactly the sort of thing that I'm looking for! This sentence in particular seems to explain why there's no real documentation about how things are implemented: _The details of Python memory management depend on the implementation_. It's interesting, though - they've got a question about 'how is a list implemented' below that which explains that it's a resizable array (and NOT a linked list), so it does seem like it's possible to know something about the implementation... – MikeTheTall Apr 01 '16 at 17:51
  • A list is guaranteed to be backed by a resizable array. What the page refers to is that the low-level memory management works by reference counting with cycle collector in CPython and Java GC in Jython. – Antti Haapala -- Слава Україні Apr 01 '16 at 21:07
  • @AnttiHaapala - where did you learn that? Did you look for that in particular, was it something that you stumbled across, or maybe there was a page somewhere that talks about how data types, stack frames, objects, arrays, etc, etc, are stored in Python? – MikeTheTall Apr 01 '16 at 21:14
  • No page in particular; stackoverflow, reading the docs here and there, and reading the C and Java source code :D – Antti Haapala -- Слава Україні Apr 01 '16 at 21:16
  • 1
    Start by reading the FAQ linked by @Ilja and the Python data model from Paul Evans' answer. – Antti Haapala -- Слава Україні Apr 01 '16 at 21:18

1 Answers1

1

With python what you want to know about what's going on under the hood straight away to be able to work efficiently with the language is quite different from C/C++ since it's a quite different language environment. What you want to get you head around is not so much the nitty-gritty of what's going on memory, but Python's Data Model.

Paul Evans
  • 27,315
  • 3
  • 37
  • 54
  • 2
    While this is a link only answer that doesn't really answer the original question, I completely agree that understanding the Data Model will help with python more than understanding the memory management. – Tadhg McDonald-Jensen Apr 01 '16 at 16:52