Does a File Object Automatically Close when its Reference Count Hits Zero?

Question

I was under the impression that file objects are immediately closed when their reference counts hit 0, hence the line:

foo = open('foo').read()

would get you the file's contents and immediately close the file. However, after reading the answer to Is close() necessary when using iterator on a Python file object I get the impression that this does not happen, and that calling .close() on a file object is always necessary.

Does the line above do what I think it's doing and even if it does, is it the Pythonic thing to do?

Are you asking about the Python file object or the OS-level underlying file object? The OS layer is separate from the Python layer. Which are you asking about? — S.Lott, Dec 02 '09 at 18:10
I would welcome an explanation of what happens at both levels. More specifically, I'd like to know if writing that line of code will potentially cause problems in my application. I was assuming when Python closes the file object that the underlying object is released as well. — Brent Newey, Dec 02 '09 at 18:22
On recent versions of Python, you can enable warnings (`-Wall`) to check for unclosed files. You'll get a `ResourceWarning` with details like `unclosed file <_io.TextIOWrapper name='foo' mode='r' encoding='UTF-8'>`. — wjandrea, May 30 '23 at 15:32

score 35 · Accepted Answer · answered Dec 02 '09 at 18:08

35

The answer is in the link you provided.

Garbage collector will close file when it destroys file object, but:

you don't really have control over when it happens.

While CPython uses reference counting to deterministically release resources (so you can predict when object will be destroyed) other versions don't have to. For example both Jython or IronPython use JVM and .NET garbage collector which release (and finalize) objects only when there is need to recover memory and might not do that for some object until the end of the program. And even for CPython GC algorithm may change in the future as reference counting isn't very efficient.
if exception is thrown when closing file on file object destruction, you can't really do anything about it because you won't know.

answered Dec 02 '09 at 18:08

Tomek Szpakowicz

14,063
3
33
55

1

So, can I infer in the general sense that relying on the specifics of how garbage collection is implemented in the version of Python that I'm using is a Bad Idea and not Pythonic, and thus that is why the line of code I have in the question is incorrect? – Brent Newey Dec 02 '09 at 18:28
1

@Brent: It isn't necessarily incorrect (if you __decide__ to depend on specific behaviour of your implementation). I don't know about Pythonic ;-). But in any managed environment relying on garbage collector (tuned to free just memory) to clean up other resources is wrong direction. Of course if it's just short script _and_ you simply read file (like in your example) it's OK to let GC or even OS close the file. – Tomek Szpakowicz Dec 02 '09 at 18:37

hughdbrown · Answer 2 · 2012-01-06T16:49:21.210

If you want to be sure, I'd write the code like this:

from __future__ import with_statement

with open('foo') as f:
    foo = f.read()

That way, your file closes as expected, even with exceptions.

Much later: here is some code with import dis to show how the compiler treats these differently.

>>> def foo(filename):
...     with open(filename) as f:
...         return f.read()
... 
>>> def bar(filename):
...     return open(filename).read()
... 
>>> from dis import dis
>>> 
>>> dis(foo)
  2           0 LOAD_GLOBAL              0 (open)
              3 LOAD_FAST                0 (filename)
              6 CALL_FUNCTION            1
              9 DUP_TOP             
             10 LOAD_ATTR                1 (__exit__)
             13 ROT_TWO             
             14 LOAD_ATTR                2 (__enter__)
             17 CALL_FUNCTION            0
             20 STORE_FAST               1 (_[1])
             23 SETUP_FINALLY           23 (to 49)
             26 LOAD_FAST                1 (_[1])
             29 DELETE_FAST              1 (_[1])
             32 STORE_FAST               2 (f)

  3          35 LOAD_FAST                2 (f)
             38 LOAD_ATTR                3 (read)
             41 CALL_FUNCTION            0
             44 RETURN_VALUE        
             45 POP_BLOCK           
             46 LOAD_CONST               0 (None)
        >>   49 WITH_CLEANUP        
             50 END_FINALLY         
             51 LOAD_CONST               0 (None)
             54 RETURN_VALUE        
>>> dis(bar)
  2           0 LOAD_GLOBAL              0 (open)
              3 LOAD_FAST                0 (filename)
              6 CALL_FUNCTION            1
              9 LOAD_ATTR                1 (read)
             12 CALL_FUNCTION            0
             15 RETURN_VALUE

I understand that, but I'm specifically asking if the code does what I think it does or not. If not, what does it do, if so, is it the right way? — Brent Newey, Dec 02 '09 at 18:02
As @Edward Loper and @tomekszpakowicz say, what happens will depend on the implementation, so whether the "code does what I think it does or not" will depend on where it is running. The solution I am providing is independent of the implementation. — hughdbrown, Dec 02 '09 at 19:44

score 12 · Answer 3 · answered Dec 02 '09 at 18:15

For the cpython implementation of python: yes, it is guaranteed to be closed when its reference count goes to zero.

For python as an abstract language (e.g., including Jython, IronPython, etc): no, it is not guaranteed to be closed. In particular, an implementation of Python may choose not to use reference counting, but to use some other form of GC.

References:

score 2 · Answer 4 · answered Dec 02 '09 at 18:05

2

No, Python optimize deleting unused objects so It may never close your file (OK at the end of your script at exit it will cleanup). @ hughdbrown have pointed out nice solution.

answered Dec 02 '09 at 18:05

przemo_li

350
2
4

Does a File Object Automatically Close when its Reference Count Hits Zero?

4 Answers4

Linked

Related