11

To read contents of a file:

data = open(filename, "r").read()

The open file immediately stops being referenced anywhere, so the file object will eventually close... and it shouldn't affect other programs using it, since the file is only open for reading, not writing.

EDIT: This has actually bitten me in a project I wrote - it prompted me to ask this question. File objects are cleaned up only when you run out of memory, not when you run out of file handles. So if you do this too often, you could end up running out of file descriptors and causing your IO attempts at opening files to throw exceptions.

Community
  • 1
  • 1
Claudiu
  • 224,032
  • 165
  • 485
  • 680
  • 3
    Note that this will read the whole file into memory, no matter how large it is. So make sure it's a file you can handle. Other than that, I agree with the answers. – balpha Sep 03 '09 at 14:23
  • @balpha: But the answers are conflicting. ;) (I presume you made the comment before all the answers were in.) – John Y Sep 04 '09 at 02:55

6 Answers6

30

Just for the record: This is only slightly longer, and closes the file immediately:

from __future__ import with_statement

with open(filename, "r") as f:
    data = f.read()
Andrew Hare
  • 344,730
  • 71
  • 640
  • 635
0x89
  • 2,940
  • 2
  • 31
  • 30
  • 5
    +1 I added the `import` just in case they are using Python 2.5 :) – Andrew Hare Sep 03 '09 at 14:26
  • A followup style question: is it really awful to do *with open("t1.py", "r") as f: f.read()* all on one line? I know it's not as readable, but so often, the reading in of the file is a very basic sort of thing, and the next guy reading the code really doesn't care how you did it. – Jenn D. Sep 03 '09 at 17:47
  • @Jenn D: One point of the `with` is to confine all the processing into a tidy scope. Putting `with open() as f: f.read()` into one line sort of defeats the scoping purpose of `with`. – S.Lott Sep 03 '09 at 18:19
  • 2
    That simply doesn't make sense. The scoping is identical, whether there's a newline in there or not. I don't like it as style, but scope doesn't enter into it. – Glenn Maynard Sep 03 '09 at 18:36
  • 1
    @Glenn: It's not the newline, it's the entire indented suite. The intent of `with` is to read and process the data in one tightly-bound context. Using a one-line `with this as that: variable= that.read()` is that you now have a side-effect from the with block, losing the tidy binding. – S.Lott Sep 04 '09 at 19:55
  • i think this is a matter of style. i could use your argument replacing `with` with `for`, and i often have one-liner for loops – Claudiu Jun 10 '10 at 04:05
7

It is true that it will close eventually, but eventually might not be soon enough. Especially if you're using this inside a loop, the system might run out of file handles before the GC gets to the file objects.

sepp2k
  • 363,768
  • 54
  • 674
  • 675
  • But if the fileobj is reference counted.. it goes to 0 directly and is deleted immediately? In CPython(?). – u0b34a0f6ae Sep 03 '09 at 19:03
  • @kaizer.se: it's still not necessarily immediately deleted. only when CPython needs some more memory. – Claudiu Sep 04 '09 at 16:44
  • Wait I was wrong. It *should* get deleted immediately since that's how reference counting works. Unless there's some cycle in the file object itself. Is this answer just wrong? If not then why aren't the objects removed right away? – Claudiu Nov 28 '14 at 19:58
4

The code works exactly as you say it does, but it's bad style nevertheless. Your code relies on assumptions which may be true now, but won't always be true. It's not impossible that your code will be run in a situation where the file being opened and not close does matter. Is it really worth that risk just to save 1 or 2 lines of code? I don't think so.

Imagist
  • 18,086
  • 12
  • 58
  • 77
3

No, it's perfectly reasonable Python style IMO, as per your reasoning.

Update: There are a lot of comments here about whether file objects get tidied up straight away or not. Rather than speculate, I did some digging. Here's what I see:

From a comment in Python's object.h:

The macros Py_INCREF(op) and Py_DECREF(op) are used to increment or decrement reference counts. Py_DECREF calls the object's deallocator function when the refcount falls to 0

Looking in Python's fileobject.c:

The function table for file objects points to function file_dealloc. This function calls close_the_file, which in turn closes the file.

So it seems reasonable to state that at the moment, on CPython, when there are no more references to a file object, it's closed without any delay. If you think this interpretation is wrong, please post a comment indicating why you feel that way.
Vinay Sajip
  • 95,872
  • 14
  • 179
  • 191
  • Reasonable, yet, but the OS resources may not be as totally available as you'd like. For instance, trying to remove the file immediately after reading it may not work because the OS resources are held by underlying C libraries even thought the Python `file` object has been garbage collected. Until the OS thinks the file is no longer in use, you may not be able to delete it. – S.Lott Sep 03 '09 at 14:32
  • @ S. Lott: are you saying that a garbage-collected `file` object is not closed (so that the OS knows it is no longer in use)? I would *expect* that deleting a file object closes the file, but I can't find anything in the documentation. – Eric O. Lebigot Sep 03 '09 at 14:58
  • 5
    My understanding is that CPython (the reference implementation that most people think of when they think Python) does in fact destroy all unreferenced objects as they leave scope. In that case, the file object leaves scope as soon as the read() operation completes. So it should do exactly what you want. That's not guaranteed behavior, however. Other implementations of Python (Jython I think is a prime example) may handle garbage collection differently. My instinct is to use the simple implementation if you know you're using CPython and don't care about portability. – Jason R. Coombs Sep 03 '09 at 15:12
  • The issue isn't garbage collection *per se*. It's the relationship between a deallocated object and any OS resources. I suspect (but don't know) that the object is simply deallocated and the underlying OS file handle is left dangling. – S.Lott Sep 03 '09 at 17:25
  • @Jason, that is wrong. My understanding is that garbage collection is never guaranteed to happen. – Kenan Banks Sep 03 '09 at 19:24
2

Even though it works as expected, I think it fails in two counts:

  1. Your code will not scale up seamlessly, because you are reading the entire file into memory, and this may or may not be necessarily what you want.
  2. According to the Zen of Python (try import this in the Python prompt to retrieve it) "explicit is better than implicit" and, by failing to explicitly close the file, you could confuse someone who, down the road, will be left with your code for maintenance.

It really helps being explicit! Python encourages explicit style.

Other than that, for a throwaway script, your style makes sense.

Maybe you will benefit from this answer.

Community
  • 1
  • 1
Escualo
  • 40,844
  • 23
  • 87
  • 135
1

looks fine to me.. I read files like that often.

Corey Goldberg
  • 59,062
  • 28
  • 129
  • 143