3

Say I define a function, which builds a list, and then prints the items of the list one by one (no practical use, just an example:

import os

def build_and_print():
    thingy = os.walk('some directory')
    for i in thingy:
        print i

if __name__ == '__main__:
    build_and_print()

If the thingy that is built is very large it could take up a lot of memory, at what point will it be released from memory?

Does python store the variable thingy until the script is finished running or until the function that builds/uses it is finished running?

Mathias711
  • 6,568
  • 4
  • 41
  • 58
Joe Smart
  • 751
  • 3
  • 10
  • 28

1 Answers1

1

Once a Variable goes out of scope, it is collected by the garbage collector.

You can see the collector code here. Go to collect function, there comments explain the process well.

Navneet
  • 4,543
  • 1
  • 19
  • 29
  • 2
    Yes and no. In this case yes, in general, no. – Martijn Pieters Sep 04 '15 at 10:42
  • 1
    Would you mind expanding on that @Martijn? – Joe Smart Sep 04 '15 at 10:44
  • Technically, once it is no longer referenced it is eligible for collection. If you return a closure from inside a function, it would be necessary to investigate exactly how the Python compiler treats unreferenced variables in the lexical scope to see if references are held for the lexical scope as a whole, or just the variables referenced. – Vatine Sep 04 '15 at 10:44
  • @JoeSmart ...it is *eligible to be* collected. It might never happen, or not immediately at least. – Peter Wood Sep 04 '15 at 10:47
  • @JoeSmart: Python uses *reference counting*. When the reference drops to 0 the object is removed from memory. Local names in a function are just references, these are cleared when the scope ends, but there could be *other* references to the object still. – Martijn Pieters Sep 04 '15 at 10:49
  • @PeterWood: in CPython, absent circular references, it happens immediately. – Martijn Pieters Sep 04 '15 at 10:49
  • @MartijnPieters so, to simplify (for my newbie programming mind) once there are no more references to a variable it is eligible for collection, which should happen pretty quickly so it won't hang around taking up resources? – Joe Smart Sep 04 '15 at 10:55
  • @JoeSmart: check. But take into account your OS won't reclaim the memory immediately; after all, process may require more memory again in a minute. But Python won't be using the memory for that object anymore. – Martijn Pieters Sep 04 '15 at 10:56
  • @MartijnPieters Cool, that's all I need to know for what I'm writing. Just trying to predict if I'll hit a mem error due to a few large variables, but the first ones fall out of scope before new ones are creating. Monitoring memory usage it seems pretty constant. Thanks for the help :) – Joe Smart Sep 04 '15 at 10:58
  • 1
    The code you link to has *nothing* to do with how most object deletions work. That function is used to track circular references (stuff the garbage collection must actually track). – Martijn Pieters Sep 04 '15 at 11:02
  • 1
    @JoeSmart: Bear in mind that when a Python object is freed its memory is returned to Python's memory arenas, it's not returned to the OS. Those memory arenas _may_ be released to the OS, but not necessarily. See [python - memory not being given back to kernel](http://stackoverflow.com/q/11957539/4014959) and the link on that page: [Why doesn't Python release the memory when I delete a large object?](http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm). – PM 2Ring Sep 04 '15 at 11:27