5

What is the "safest" way to write files in Python? I've heard about atomic file writing but I am not sure of how to do it and how to handle it.

Jk1
  • 11,233
  • 9
  • 54
  • 64
Arnaud Aliès
  • 1,079
  • 13
  • 26
  • 7
    Safe by what definition? –  Oct 16 '12 at 20:31
  • Safe mean safe: avoid file corruption and stuff like that – Arnaud Aliès Oct 16 '12 at 20:34
  • 2
    @mou I believe that's a genuinely Hard question. I'd search for how databases implement file locking. – millimoose Oct 16 '12 at 20:34
  • 3
    You probably want to write to a temporary file, then move the temporary file to the destination (overwriting any preexisting file). If the temporary file is created in the proper way and the move is atomic (as it should be), I think this is usually what's meant by "atomic file writing." – Isaac Oct 16 '12 at 20:35
  • 2
    You probably want to look at http://en.wikipedia.org/wiki/Write-ahead_logging and http://en.wikipedia.org/wiki/Journaling_file_system - but if it's for serious purposes, then a lot of study is recommended. Normally it's not something you need to care about as databases/filesystems take care of most of this for you. (And it's not generally Python specific, as mostly you would use atomic operations provided by the OS) – Jon Clements Oct 16 '12 at 20:39

3 Answers3

6

What you want is an atomic file replacement, so that there is never unfinished final file on the disk. There only exist complete new version or complete old version on the target location.

The method for Python is described here:

Atomic file replacement in Python

Community
  • 1
  • 1
Mikko Ohtamaa
  • 82,057
  • 50
  • 264
  • 435
2

You can write to a temp file and rename it, but there's a lot of gotchas doing it correctly. I prefer to use this nice library Atomic Writes.

Install it:

pip install atomicwrites==1.4.0 #check if there is a newer version

And then just use it as a context:

from atomicwrites import atomic_write

with atomic_write('foo.txt', overwrite=True) as f:
    f.write('Hello world.')
    # "foo.txt" doesn't exist yet will be created when closing context
neves
  • 33,186
  • 27
  • 159
  • 192
-1
with open("path", "w") as f:
    f.write("Hello, World")

The use of the with-Statement guarantees that the file is closed, no matter what happens (well it equals to a try .. finally).

dav1d
  • 5,917
  • 1
  • 33
  • 52
  • 2
    -1: This doesn't really answer any question beyond "how to make sure a file is closed after I write to it in Python". Which might be one possible interpretation of the OP's vague phrasing, but that calls for a comment, not for guessing an answer. – millimoose Oct 16 '12 at 20:39