0

We recently hit a situation in our production ubuntu server that one of our important files were corrupted.

After investigation we found one of my python scripts who write content to that file wrote only part of the content as the node hit disk full scenario exactly at that time.

Can anybody suggest what is the best way to ensure the content is whether fully written or no content is written, so that the file will not land in corrupted state?

That part of my python script is as simple as

with open('workfile', 'w') as f:
     f.write("Write some thing")

I did some research, but couldn't get any good pythonic way to do.

And as my file is a growing file, I can't create a temporary file and rename as suggested in atomic writing to file with Python. I will loose old data. And it may make the operation atomic, but what I want is data fully written and not partly.

Even in writing to temporary file and rename, we are not sure that data is completely written to the temporary file.

Community
  • 1
  • 1
Joshi Sravan Kumar
  • 157
  • 1
  • 11
  • As suggested in the answer of http://stackoverflow.com/questions/2333872/atomic-writing-to-file-with-python I can't have a temporary file and rename, as my file is a growing file. I can't loose the old data – Joshi Sravan Kumar Jan 05 '17 at 10:24
  • Do you mean you're appending data onto an existing file? If that's the case, just copy the existing file to make your temporary file and start writing on the end of it. – SuperBiasedMan Jan 05 '17 at 10:28
  • Even in the temporary file case, how sure are we that the data is fully written to temp file before rename. This entire operation may make it atomic, but how to ensure the data fully written. More over I am scared that in future if there are multiple writers to this file and some consumers, this make it more in consistent availability of data and handling that is much complex – Joshi Sravan Kumar Jan 05 '17 at 10:34

0 Answers0