What Type of Settings File to Implement?

Question

I am making an auto-archiver/backup sort of thing in Python(http://sourceforge.net/projects/frozendirectory/). As of now it uses an ordinary text file to store its small amount of settings(but ever growing). I was thinking of changing it to an XML file so that it would be more flexible/scalable and thus easier to work with in the long run. However, when I asked a friend he recommended that I just keep it in a python file, such as settings.py and just use 'import settings' from the file that needs the settings when needed. He claimed that it takes less space and I would only have to write code to write to the settings file as opposed to worrying about both reading and writing.

His points were convincing but made me wonder why no other large programs used the technique he is recommending.

So anyways, I just wanted to know what you guys think. Should I go with XML, .py, or something else? Thanks in advance.

Answer 1

JSON is a good alternative to XML, it is also easy to read and write in Python.

Answer 2

I find ConfigObj really useful for this. It's similar to the "ini" format commmon on Windows, but more flexible.

You can set types and constraints for configuration flags. You can also merge default settings with user settings, ie. some settings can be missing from the user's config file, indicating that the defaults should be used. This makes upgrades pretty seamless, unless you need some magical reinterpretation of existing settings.

RabbitVCS has an example of a ConfigObj specification and this is the code that implements it.

Answer 3

His points were convincing but made me wonder why no other large programs used the technique he is recommending.

(Django uses settings.py)

For my own programs... I generally either use a Python file like you mentioned, or else I use ConfigParser.

It depends on who my audience is for the configuration file... a fellow python programmer or a non-programmer.

Answer 4

Who says large programs don't use this technique? :-) They do, when it makes sense.

Why is there a trend to use XML? well that's to address issues involving validation of data being supplied, ensuring that multiple data generators don't have to learn about each other, ensuring that multiple readers will interpret the data identically, to encapsulate the nature of the data in the datafile itself... and on and on and on...

Why did your friend tell you to avoid it? because for your application it may be way too cumbersome :-)

Answer 5

You friend might have given you the best idea... depends on your situation. It will be fast, it will be easy on you and the program, won't require summoning additional libraries (or heaven forbid, install one just for that).

The best argument against using import for settings file is that an "evil hacker" hypothetically may add some code to the file to interfere with the work of your program. Then again, somebody evil enough can replace your whole program with something else so... depends on the level of determination. You need to see if that is concern for whatever your program is targeted.

If you consider that unsuitable, I think you can keep your settings in plain text file if they are simple enough (after all reading INI-style settings is a one liner, a-la

settings = dict(s.split('=',1) for s in open('somefile'))

If your structures are more structured, either JSON or YAML should do. See also:

• python-human-readable-object-serialization
• how-to-process-a-yaml-stream-in-python
• best-way-to-parse-a-yaml-file

XML you can do... but that ain't human-friendly.

Answer 6

I would say that it depends on the complexity and the object or purpose of the settings being stored.

If it were me - I would use the following as the basis for an initial needs based assessment and go from there.

A) Does my data require multiple nested layers?

• YES ---> INI File, JSON, YAML

• NO ---> INI File, JSON, YAML

B) Do I need to store value arrays or other complex datatypes?

• YES ---> INI File, JSON, YAML

• NO ---> INI File, JSON, YAML

C) Will I need to map the arguments to a live stream?

• YES ---> INI File, JSON,YAML

• No ---> INI File, JSON, YAML

D) Do I want to interpolate or cross-reference key-values within the file?

• YES ---> INI File, JSON, YAML

• NO ---> INI File, JSON, YAML

In the end,

INI Files are meant to be simple and interpolating arguments is easy but it is difficult to get more complex when you need to.

JSON good all around, nice support for nested key-value or array objects and data type support. Good for web development or JS object integration. Interpolation difficult, hard to actively adjust nested key values without replacing the whole object.

YAML is good too, I really like the support for internal specification of python objects and modules. Some data types can be a bit weird or are not available for certain versions of python. Nesting of arrays is a little strange and nested key-value objects with a lot of keys can be tedious to outline.

What you choose depends on what your requirements are but as a rule I usually stay away from xml mainly because the keys cant provide information in the same way as JSON or YAML.

UPDATES:

I was just reviewing the YAML docs and I ran across the following information and I thought you might find it useful.

YAML can therefore be viewed as a natural superset of JSON, offering improved human readability and a more complete inform- ation model. This is also the case in practice; every JSON file is also a valid YAML file. This makes it easy to migrate from JSON to YAML if/when the additional features are required

Pretty cool huh? Anyway, given this information I would likely default to JSON and then if I needed to go over to YAML.