I use pickle to save a Python dictionary to a file.
with open(FILENAME, 'wb') as f:
pickle.dump(DATA, f, protocol=pickle.HIGHEST_PROTOCOL)
It was all good until the disk run out of space on a shared server and my file became empty (0 byte).
Traceback (most recent call last):
File "****.py", line 81, in ****
with open(FILENAME, 'wb') as f:
OSError: [Errno 28] No space left on device
What is the best solution to prevent overwriting the previous data if the above error occurs?
Write to a temporary file (on the same filesystem!), and move it to the real destination when finished. Maybe with a fsync in between, to make sure the new data is really written.
If you go with @snild-dolkow's answer, keep in mind some interesting behaviour with shutil.move when working with files in different filesystems: How to move a file in Python?.
As has been noted in comments on other answers, shutil.move simply calls os.rename in most cases. However, if the destination is on a different disk than the source, it will instead copy and then delete the source file.
If you create the temporary file on the same filesystem like he suggests then you should be fine, but if you're worried, use os.rename instead, and it'll fail rather than copy cross filesystem (which could fail in the same way that you're trying to avoid).
