2

I have a line of python that splits a file by carriage return character:

lines = open(sFile, 'r').read().split("0d".decode('hex'))

Was this file is closed? If not, can I acquire the file handle somehow?

royhowie
  • 11,075
  • 14
  • 50
  • 67
stack user
  • 835
  • 1
  • 9
  • 28

3 Answers3

8

The short answer is "probably". The open file object should get garbage collected which will close the file. However, there are some circumstances where that might not be true and the open file handle can live on.

Best practice is to always close your file handles. The beauty of context managers is hard to over-estimate here:

with open(sFile) as input_file:
    lines = input_file.read().split('0d'.decode('hex'))

It has been asked in the comments if we can demonstrate that a file object is implicitly closed. I don't know the answer to that question, however, we can demonstrate that the file object can be reaped by the garbage collector before the current statement is even finished executing. Since it's fairly common knowledge that file objects are closed when reaped we can make some assumptions about what can happen in OP's case.

from __future__ import print_function
import weakref

def callback(*args):
    print('cleanup')

fobj = weakref.ref(open('test.dat'), callback)()
print(fobj)

If you'll notice, cleanup gets printed before None (and that the weakref.ref returns None) which means our file object has been reaped by the garbage collector before the reference is called1.

1Note, this is CPython behavior. Run it on pypy-2.6.1 and the file object is still alive at this point and cleanup never gets printed! This lends support to the claim that you should close your own file handles as there is definitely implementation dependence in when the GC runs (and therefore, when the file handle will be closed and reaped).

mgilson
  • 300,191
  • 65
  • 633
  • 696
  • 1
    @PM2Ring -- when I'm being lazy I use `f` or `fin` for input file handles -- unfortunately, that's not very easy to read. (I've had code reviews come back asking me about what "fin" is and if this is something to do with fish). So I'm trying to be less lazy these days. As you can see, old habits die hard. Fortunately, we have a great community around here and alecxe cleaned it up for me. :-). – mgilson Sep 02 '15 at 14:57
  • Do you think we can *demonstrate* that the file is implicitly closed by the gc during a clean up? Thanks. – alecxe Sep 02 '15 at 15:08
  • You can run `strace` (or your platform's equivalent) and watch for the call to `close`. – chepner Sep 02 '15 at 15:38
  • 1
    @alecxe -- See my edit. I have no _proof_ for you, but I do make some plausibility arguments anyway. You'll also notice that in trying to make my plausibility arguments, I've discovered some implementation dependent behavior here which further lends credence to the assertion that the *best* way to deal with this is to close it explicitly. – mgilson Sep 02 '15 at 15:47
  • @mgilson oh wow, that's a good example demonstrating the differences in how different interpreters handle it. Here is also an [interesting read](http://blog.lerner.co.il/dont-use-python-close-files-answer-depends/) with some experiments on CPython vs PyPy. And [here](http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies) is also something related. Thank you. – alecxe Sep 02 '15 at 15:55
6

The file may be closed for you implicitly by the garbage collector. There is more to it:

It's recommended to use with context manager when dealing with files or file-like objects:

It is good practice to use the with keyword when dealing with file objects. This has the advantage that the file is properly closed after its suite finishes, even if an exception is raised on the way. It is also much shorter than writing equivalent try-finally blocks

with open(sFile) as input_file:
    lines = input_file.read().split("0d".decode('hex'))
Community
  • 1
  • 1
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • I don't think your examples are equivalent. Creating a variable for the file object prevents garbage collection, as noted by mgilson. – jpmc26 Sep 02 '15 at 14:46
  • @jpmc26 yeah, I agree, I'll think about how to demonstrate it. Thanks. – alecxe Sep 02 '15 at 14:47
2

The with clause can take care of that for you.

This is the pythonic way to deal with files:

with open(sFile, 'r') as f:
    lines = f.read().split("0d".decode('hex'))
Alejandro Ricoveri
  • 887
  • 1
  • 7
  • 11