Configuration variables for a collection of scripts in Python

Question

I have a collection of scripts written in Python. Each of them can be executed independently. However, most of the time they should be executed one after the other, so there is a MainScript.py which calls them in the appropriate order. Each script has some configurable variables (let's call them Root_Dir, Data_Dir and LinWinFlag). If this collection of scripts is moved to a different computer, or different data needs to be processed, these variable values need to be changed. As there are many scripts this duplication is annoying and error-prone. I would like to group all configuration variables into a single file.

I tried making Config.py which would contain them as per this thread, but import Config produces ImportError: No module named Config because they are not part of a package.

Then I tried relying on variable inheritance: define them once in MainScript.py which calls all the others. This works, but I realized that each script would not be able to run on its own. To solve this, I tried adding useGlobal=True in MainScript.py and in other files:

if (useGlobal is None or useGlobal==False):
  # define all variables

But this fails when scripts are run standalone: NameError: name 'useGlobal' is not defined. The workaround is to define useGlobal and set it to False when running the scripts independently of MainScript.py. It there a more elegant solution?

Since you don't have `Config` set up as a Module you can still import its information using: `from config import *` — MrAlexBailey, Jan 16 '15 at 22:58
That does not work either: `ImportError: No module named Config` — Dženan, Jan 16 '15 at 23:34

volent · Answer 1 · 2015-01-20T08:56:07.707

The best way to do this is to use a configuration file placed in your home directory (~/.config/yourscript/config.json).

You can then load the file on start and provide default values if the file does not exist :

Example (config.py) :

import json

default_config = {
    "name": "volnt",
    "mail": "oh@hi.com"
}

def load_settings():
    settings = default_config
    try:
        with open("~/.config/yourscript/config.json", "r") as config_file:
            loaded_config = json.loads(config_file.read())
            for key in loaded_config:
                settings[key] = loaded_config[key]
    except IOError: # file does not exist
        pass
    return settings

For a configuration file it's a good idea to use json and not python, because it makes it easy to edit for people using your scripts.

Thanks for suggestion. Is there any advantage to this approach over ConfigParser module? — Dženan, Jan 19 '15 at 22:08
No, except that ConfigParser config files are using the .ini format. — volent, Jan 20 '15 at 06:41

score 1 · Answer 2 · answered Jan 16 '15 at 23:01

1

The idea is that python wants to access files - including the Config.py - primarily as part of a module.

The nice thing is that Python makes building modules (i.e. python packages) really easy - initializing it can be done by creating a

__init__.py

file in each directory you want as a module, a submodule, a subsubmodule, and so on.

So your import should go through if you have created this file.

If you have further questions, look at the excellent python documentation.

answered Jan 16 '15 at 23:01

cleros

4,005
1
20
30

That should be `__init__.py` – declension Jan 16 '15 at 23:03
Thanks - I noticed ... I forgot the auto-markup for underscore :-) – cleros Jan 16 '15 at 23:04
Creating `__init__.py` does not make the import error go away. – Dženan Jan 16 '15 at 23:28
hmm, strange. Could you please give us the folder-structure of your project (so, where is config.py, `__init__py` and the python file for the import)? the problem could be related to importing something from another level of the package-structure. – cleros Jan 17 '15 at 15:21
1

And another option - which could be more elegant in the first place - could be the configparser package. Look [here](https://docs.python.org/2/library/configparser.html) for more information. – cleros Jan 17 '15 at 15:25
All the Python files (MainScript.py, Config.py, OtherFile.py etc) are in a single folder (called Batch). This folder contains some other files (Results.csv etc) and subfolders (Input1 etc), if that matters. – Dženan Jan 19 '15 at 21:20

score 0 · Accepted Answer · answered Jan 19 '15 at 22:09

0

As suggested by cleros, ConfigParser module seems to be the closest thing to what I wanted (one-line statement in each file which would set up multiple variables).

answered Jan 19 '15 at 22:09

Dženan

3,329
3
31
44

Configuration variables for a collection of scripts in Python

3 Answers3