177

In Python, if you either open a file without calling close(), or close the file but not using try-finally or the "with" statement, is this a problem? Or does it suffice as a coding practice to rely on the Python garbage-collection to close all files? For example, if one does this:

for line in open("filename"):
    # ... do stuff ...

... is this a problem because the file can never be closed and an exception could occur that prevents it from being closed? Or will it definitely be closed at the conclusion of the for statement because the file goes out of scope?

tshepang
  • 12,111
  • 21
  • 91
  • 136
user553702
  • 2,819
  • 5
  • 23
  • 27
  • 17
    The file does ___not___ go out of scope at the end of the `for` block. Its reference count will go to zero, causing it to be closed automatically, but only functions, classes, and modules define scopes in Python, not other compound statements. – agf Sep 13 '11 at 00:42
  • 26
    It's not a problem unless it's a problem. At the OS level, any files opened by the script will be closed when the script exits, so you needn't worry about closing files in throwaway tool scripts. However, processes have a limit on the number of open files they can maintain, so long-lived or complex scripts may need to be more careful. In any case, it's a good habit to close your files. – Russell Borogove Sep 13 '11 at 02:09
  • 3
    @agf: You are right that the file doesn't go out of scope, but it's not related to the distinction between `for` blocks and functions/classes/modules. It's much simpler than that: objects don't have scopes, only names do. There is no name that refers to this object, so there is nothing here to stay in scope or go out of scope. – max Dec 30 '12 at 02:09
  • @max My comment is correcting his assumption that there is a scope associated with the `for` loop, and mentioning that the file gets closed for an entirely different reason. It doesn't get into _what scopes are_ in Python, as it's not relevant here. – agf Dec 30 '12 at 02:24
  • @max there's an implicit reference scoped to that for loop... this is an argument of semantics – Peter R Feb 16 '15 at 03:24

7 Answers7

151

In your example the file isn't guaranteed to be closed before the interpreter exits. In current versions of CPython the file will be closed at the end of the for loop because CPython uses reference counting as its primary garbage collection mechanism but that's an implementation detail, not a feature of the language. Other implementations of Python aren't guaranteed to work this way. For example IronPython, PyPy, and Jython don't use reference counting and therefore won't close the file at the end of the loop.

It's bad practice to rely on CPython's garbage collection implementation because it makes your code less portable. You might not have resource leaks if you use CPython, but if you ever switch to a Python implementation which doesn't use reference counting you'll need to go through all your code and make sure all your files are closed properly.

For your example use:

with open("filename") as f:
     for line in f:
        # ... do stuff ...
Peter Graham
  • 11,323
  • 7
  • 40
  • 42
  • 9
    Does using `with open() as f` automatically close the file after it is done? – Rohan Mar 25 '16 at 05:55
  • 29
    @Rohan yes, that is the little magic that the `with` statement provide, but of course for this magic to work the object must have the especial methods `__enter__` and `__exit__`, in the latter the object do the `close` and any other cleanup stuff that need to be done at the end of the `with` statement... – Copperfield Mar 26 '16 at 13:02
  • 4
    FYI: This answer only explains "when it would be closed" but does not explain "what if it stays open". For the latter, please read the "What would happen if a file stays open?" part in this answer (http://askubuntu.com/questions/701491/is-closing-a-file-after-having-opened-it-with-open-required-in-python) – RayLuo Aug 22 '16 at 16:58
  • Moreover, not closing files can result in truncated files as file contents have not been flushed. – Erwan Legrand Mar 17 '17 at 08:36
  • So if I don't close the file, will I get my memory back for sure once the program stops running? Or do I actually have to quit out of the entire interpreter? – Pro Q Jun 14 '17 at 00:26
  • I'm using Jython 2.7 and the WITH statement is not an option. I have to use open() and close(). – Drewdin Aug 23 '17 at 14:19
  • It's not even guaranteed to be closed in CPython. CPython does not guarantee that objects that still exist when the interpreter exits will have their `__del__` method called. For instance, if an exception were to be thrown in the body of the loop and not be caught, then the interpreter would exit whilst containing a reference to traceback of the last exception. In doing so it would have kept the file object alive and thus be unlikely to close (or flush) the file. – Dunes Oct 18 '18 at 15:09
27

Some Pythons will close files automatically when they are no longer referenced, while others will not and it's up to the O/S to close files when the Python interpreter exits.

Even for the Pythons that will close files for you, the timing is not guaranteed: it could be immediately, or it could be seconds/minutes/hours/days later.

So, while you may not experience problems with the Python you are using, it is definitely not good practice to leave your files open. In fact, in cpython 3 you will now get warnings that the system had to close files for you if you didn't do it.

Moral: Clean up after yourself. :)

Ethan Furman
  • 63,992
  • 20
  • 159
  • 237
  • 10
    Files get closed when they're no longer referenced in CPython, but that's not a language feature. If it was you could quite happily rely on it. – Peter Graham Sep 13 '11 at 02:01
12

Although it is quite safe to use such construct in this particular case, there are some caveats for generalising such practice:

  • run can potentially run out of file descriptors, although unlikely, imagine hunting a bug like that
  • you may not be able to delete said file on some systems, e.g. win32
  • if you run anything other than CPython, you don't know when file is closed for you
  • if you open the file in write or read-write mode, you don't know when data is flushed
Dima Tisnek
  • 11,241
  • 4
  • 68
  • 120
3

Hi It is very important to close your file descriptor in situation when you are going to use it's content in the same python script. I today itself realize after so long hecting debugging. The reason is content will be edited/removed/saved only after you close you file descriptor and changes are affected to file!

So suppose you have situation that you write content to a new file and then without closing fd you are using that file(not fd) in another shell command which reads its content. In this situation you will not get you contents for shell command as expected and if you try to debug you can't find the bug easily. you can also read more in my blog entry http://magnificentzps.blogspot.in/2014/04/importance-of-closing-file-descriptor.html

Zeel Shah
  • 462
  • 6
  • 15
  • Why not just seek to the beginning of the file using file.seek(0)? Closing then re-opening the same file might be more expensive. – ZeZNiQ Sep 07 '21 at 00:48
3

The file does get garbage collected, and hence closed. The GC determines when it gets closed, not you. Obviously, this is not a recommended practice because you might hit open file handle limit if you do not close files as soon as you finish using them. What if within that for loop of yours, you open more files and leave them lingering?

Nam Nguyen
  • 1,765
  • 9
  • 13
  • But if you opened other files within that for loop, it would still be the case that there would be more than one file open simultaneously whether you explicitly close any of them or not. Are you saying that the file isn't necessarily garbage-collected as soon as the file goes out of scope, thus it would be closed sooner if done explicitly? What about when an exception happens (when you use with/try-finally vs. not doing so)? – user553702 Sep 13 '11 at 00:43
  • 1
    In CPython, reference counting will cause it to be collected after the `for` statement -- you won't have to wait for the next garbage collection run. – agf Sep 13 '11 at 00:43
2

During the I/O process, data is buffered: this means that it is held in a temporary location before being written to the file.

Python doesn't flush the buffer—that is, write data to the file—until it's sure you're done writing. One way to do this is to close the file.

If you write to a file without closing, the data won't make it to the target file.

Sanket Nagrale
  • 21
  • 2
  • 2
    I don't think this is true at all in Linux. Files are created as soon as open("file", "w") and written into as soon as .write("whatever"). – sinekonata Sep 08 '20 at 23:50
  • 1
    @sinekonata That's factually false. It might be the case in some circumstances, but I've had cases where it didn't. Thus, don't rely on it. – Nearoo Mar 18 '22 at 11:41
-1

Python uses close() method to close the opened file. Once the file is closed, you cannot read/write data in that file again.

If you will try to access the same file again, it will raise ValueError since the file is already closed.

Python automatically closes the file, if the reference object has been assigned to some another file. Closing the file is a standard practice as it reduces the risk of being unwarrantedly modified.

One another way to solve this issue is.... with statement

If you open a file using with statement, a temporary variable gets reserved for use to access the file and it can only be accessed with the indented block. With statement itself calls the close() method after execution of indented code.

Syntax:

with open('file_name.text') as file:

    #some code here
Apoorv Gunjan Pathak
  • 65
  • 6