0

I'm using the ConfigParser module to read/write a .conf file from two Python scripts. One script is only reading data, while the other may also write to the .conf file. Considering this context is possible to end with corrupt data ? Or the ConfigParser is preventing such situations ?

Cheers

steve
  • 594
  • 4
  • 10
  • 23

1 Answers1

1

ConfigParser itself doesn't know how to open and write physical files and therefore cannot prevent races. You pass a file-like object to write() and it's up to you to make sure that files you're changing are updated atomically. On POSIX systems like Linux, this is typically done by writing to a temporary file and renaming it to the final name when the writing has completed.

An atomic rename requires that the source and destination file are on the same filesystem, and an easy way of guaranteeing this is making sure that the files are in the same directory.

import ConfigParser
import os
import tempfile

FILENAME = '/some/path/test.config'

config = ConfigParser.SafeConfigParser()
config.read([FILENAME])
config.add_section('foo')
with tempfile.NamedTemporaryFile(dir=os.path.dirname(FILENAME),
                                 delete=False) as tempfile:
  config.write(tempfile)
os.rename(tempfile.name, FILENAME)

This assumes that you only have one concurrent writer of the config file in addition to the one or more concurrent readers.

Doing an atomic replacement on Windows is less trivial. See e.g. Is an atomic file rename (with overwrite) possible on Windows?. Also relevant to the discussion is How to safely write to a file? – depending on the file system implementation (and its mount options) it's possible that the metadata change (the rename) completes before the data is persisted to the file.

Community
  • 1
  • 1
Magnus Bäck
  • 11,381
  • 3
  • 47
  • 59
  • Won't there be an exception/error thrown by the rename operation since the 'FILENAME' file is already in the directory ? – steve Dec 04 '13 at 14:39
  • This is true for Windows, yes. I've updated the answer to make that clear. – Magnus Bäck Dec 04 '13 at 14:55
  • Thanks a lot. However I'd need a platform independent solution, since my scripts may run on Windows as well as Linux machines. – steve Dec 04 '13 at 14:59
  • Still, I thought Python makes sure the rename operation is atomic on all platforms – steve Dec 04 '13 at 15:00
  • Python 3.3 has [os.replace()](http://docs.python.org/3.3/library/os.html#os.replace) which does what you want. See the discussion in http://bugs.python.org/issue8828. – Magnus Bäck Dec 04 '13 at 15:26