2

As an example, let's say we are checking a preferences.json file with the contents of

preferences.json 
{
    "background_color": "red"
}

Then we read/modify/write the preferences back using the Python json module. The following two examples both function the same except one uses the open() context manager and one doesn't.

change_pref_with_CM.py 
import json

# Load the user preferences
with open('preferences.json') as prefs_file:
    prefs = json.load(prefs_file)

print(prefs['background_color'])  # Prints 'red'
# Change the user's background_color preference to blue
prefs['background_color'] = 'blue'

# Save the user preferences
with open('preferences.json', 'w') as prefs_file:
    json.dump(prefs, prefs_file)
change_pref_without_CM.py 
import json

# Load the user preferences
prefs = json.load(open('preferences.json'))

print(prefs['background_color'])  # Prints 'red'
# Change the user's background_color preference to blue
prefs['background_color'] = 'blue'

# Save the user preferences
prefs = json.dump(prefs, open('preferences.json', 'w'))

For normal file reading/writing I always use a context manager and handle all logic inside the context manager. But in this example the file is being used just long enough to either populate a dict or write a dict to a file. So the example not using a context manager seems much cleaner to me. I worry that since the files are being opened "anonymously" there is nothing to call close() on and I don't know if the json module handles that with a context manager internally or if I would just be leaking file descriptors. I'm sure using a context manager just in case is the safe way to go, but at the same time I would like to know how the json module handles file descriptors (or is it file objects?)

I went looking for the source for the json module but after finding it and searching for "dump" and "load" and finding nothing I'm not sure what to check next. I'm not nearly proficient enough with C to understand what's going on in there.

Stack of Pancakes
  • 1,881
  • 18
  • 23
  • BTW, what are your thoughts on whether it would be fair/accurate to retitle this "*Is a context manager needed to force a close() if a program will exit right after the write?*"? I'd argue that that's a title that better communicates the question to its wider audience of Python developers writing short-lived programs -- the content of the question isn't really specific to JSON (that it's `json.write(prefs, prefs_file)` doing the writes vs `prefs_file.write("whatever")` is effectively immaterial for purposes of whether/when a flush is guaranteed to take place). – Charles Duffy Mar 21 '18 at 16:02
  • @CharlesDuffy I mean, it can be re-titled. But the initial intent of the question was to find how Python built-ins handle file descriptors and if things like the json module handles it by itself or if I still need to use a context manager, even for small read/writes. – Stack of Pancakes Mar 21 '18 at 16:40
  • closely related: https://stackoverflow.com/questions/7395542/is-explicitly-closing-files-important – VPfB Mar 21 '18 at 17:53

1 Answers1

2

Either of these are safe, but the version with the context manager is better practice.

Keep in mind that when your program exits, all writes are flushed regardless. That said, the context manager causes a close and flush to happen immediately, not on garbage collection, and not on exit. As such, it's that much more explicit, and will make sure things take effect immediately even if there's more (potentially long-running) code added to the end of your program.

That said, consider trying to make your writes truly atomic.

There's still a window here where the file has been opened for output but hasn't had new content flushed into it; if power was yanked, or you had a SIGKILL, or a reader just tried to pull in the contents of the file inside that window, you'd lose your data. See atomic writing to file with Python; personally, I favor the advice by @vog to use a purpose-built library that's aware of various platforms' idioms and best practices.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441