7

I would like to know why a file object opened using with statement or in a block, remains in scope after exit. Are <closed file> objects ever cleaned up?

>>> with open('test.txt','w') as f:
...     f.write('test')
...
>>> f
<closed file 'test.txt', mode 'w' at 0x00E014F0>
>>> f.close()


>>> if True:
...     fal = open('demo.txt','w')
...     fal.write('stuff')
...     fal.close()
...
>>> fal
<closed file 'demo.txt', mode 'w' at 0x00E015A0>
Michał Zaborowski
  • 3,911
  • 2
  • 19
  • 39
pyInTheSky
  • 1,459
  • 1
  • 9
  • 24

5 Answers5

16

In Python new scopes (aka namespaces) are only created for modules, classes and functions, but not for any other statement, especially not for with and if blocks. Identifiers bound within the body of with or for statements are consequently bound in the inner-most surrounding scope, which is the top-level scope of the interactive interpreter in your case. Identifiers are bound in a scope as long as this scope is valid, or until they are explicitly removed from the scope (by using del as in del fal).

Objects can only be cleaned up when they are not longer referenced. The actual moment, in which this object is really cleaned up, is however undefined. Python uses garbage collection for memory management, and doesn't enforce a specific strategy. In CPython, which uses reference counting, objects are immediately cleaned up, once the last reference goes out of scope. Alternative implementations like PyPy or Jython use more advanced garbage collectors, which clean up unreferenced objects at arbitrary points of time.

This means, that in your example the objects bound to f and fal are bascially never cleaned up, because the top-level scope of the interactive interpreter does naturally exist as long as the interpeter is running. Beware however, that this is not actually a problem, because they are nevertheless correctly closed and do not claim any file resource anymore, but only some memory.

4

The name will remain in scope untill you leave the scope.

If you want the object cleaned up, assign a different value to the name, like

f = None

This does not close the file!

It is a good habit to make the scopes limited by structuring the program in to functions ( and possibly classes ). That makes it more readable and makes unbound objects (no names in scope which refer to the object ) elegible for garbage collection.

This is usually not an issue when you're using the prompt though :)

extraneon
  • 23,575
  • 2
  • 47
  • 51
  • 1
    The file *is* closed. Even `repr` tells you that. But a closed file doesn't delete the handle and magically makes all references disappear. Otherwise good answer and actually touching the scope issue, so +1. –  Feb 16 '11 at 20:15
  • @delnan in this case it is. I just wanted to make explicit that a file disappearing from the context does not necessarilly close it. – extraneon Feb 16 '11 at 20:26
  • When used in a `with` block, is *will* be closed unless the interpreter crashes. But yes, just removing one reference to it generally won't cause it to be closed. –  Feb 16 '11 at 20:28
2

with and if don't create a new scope, local variables are only created within functions, classes, and modules. See What's the scope of a Python variable declared in an if statement?

Community
  • 1
  • 1
Wooble
  • 87,717
  • 12
  • 108
  • 131
0

The file object, like any other object, 'hangs around' for the duration of your program, until it's deleted (its __del__ method is called).

del f will cause f to be garbage collected by Python's garbage collector. This happens automatically when you exit a scope, your script terminates, or in your example, the interpreter session ends.

Aphex
  • 7,390
  • 5
  • 33
  • 54
  • 2
    This is misleading. Python will delete the object when there are no more references, and `del f` will only remove one reference to the object referenced by `f`. –  Feb 16 '11 at 20:14
  • Right, but I was referring to `del f` as part of his example, where it only has one reference (he COULD read into the intricacies of the python GC's reference-counting mechanism for a deeper explanation). – Aphex Feb 16 '11 at 20:17
  • If that's the case, then should the '__exit__ wrapper that's part of the with() statment, call __del__, I don't understand how scoping the variable inside the with statement, doesn't cause it to be fully cleaned up. In my example, there were no more references to 'f'. It was declared right within the 'with()' statement – pyInTheSky Feb 16 '11 at 20:19
  • 1
    `__del__()` is not necessarily invoked, when objects are deleted. The mere existence of this methods in the class namespace may actually prevent the object from being garbage-collected. –  Feb 16 '11 at 20:20
  • 2
    @pyInTheSky: It can't (you can't modify the caller's scope, which is a very good thing btw!) and there are times where you don't want it. And even if you don't need it around, it doesn't matter. You make two wrong assumptions: (1) That GC of the object is required for cleanup (all cleanup is in `__exit__`, only memory for the actual context manager isn't reclaimed) and (2) that `with` has it's own scope (it doesn't - in your example `f` is still around because of this). –  Feb 16 '11 at 20:22
  • @delnan: only 2 :p, thanks for the clarifications. At the end of the day, I wish i could do f.open() if it's going to stick around. – pyInTheSky Feb 16 '11 at 20:28
0

Python's with statement calls the file class's __exit__ method [which ,in the case of files, closes them]. It does not delete the object; it is cleaned up only when you explicitly clean it up or close the program/interpreter.

Foo Bah
  • 25,660
  • 5
  • 55
  • 79
  • 4
    ... which would be asynch with GC anyway. To expand, the variable is still around because `with` doesn't introduce it's own scope. –  Feb 16 '11 at 20:13