A configuration file that can be read by python and shell

Question

I have python scripts and shell scripts in the same folder which both need configuration. I currently have a config.py for my python scripts but I was wondering if it is possible to have a single configuration file which can be easily read by both python scripts and also shell scripts.

Can anyone give an example of the format of a configuration file best suited to being read by both python and shell.

I've used `YAML`, and recommend it. There is also `python-yaml`. — Faheem Mitha, Oct 17 '15 at 01:57

score 6 · Accepted Answer · edited May 23 '17 at 12:02

6

I think the simplest solution will be :

key1="value1"
key2="value2"
key3="value3"

in shell you just have to source this env file and in Python, it's easy to parse.

Spaces are not allowed around =

For Python, see this post : Emulating Bash 'source' in Python

edited May 23 '17 at 12:02

Community

1
1

answered Jan 21 '13 at 15:06

Gilles Quénot

173,512
41
224
223

score 4 · Answer 2 · edited Mar 13 '23 at 20:49

4

configobj, a third party library you can install with pip can help with this.

from configobj import ConfigObj
cfg = ConfigObj('/home/.aws/config')
access_key_id = cfg['aws_access_key_id']
secret_access_key = cfg['aws_secret_access_key']

Configobj's homepage: https://github.com/DiffSK/configobj

edited Mar 13 '23 at 20:49

Olsgaard

1,006
9
19

answered Oct 16 '15 at 10:20

Ilja

142
3
7

I think you should add [] before ['aws_access_key_id']. something like cfg[]['aws_access_key_id'] – Malgi Jun 24 '16 at 00:46

score 1 · Answer 3 · answered Jan 21 '13 at 15:07

1

This is valid in both shell and python:

NUMBER=42
STRING="Hello there"

what else do you need?

answered Jan 21 '13 at 15:07

Emanuele Paolini

9,912
3
38
64

I think the OP wishes for a method that doesn't break conventions. (I.e. don't open the file as a text file or start a subprocess or call `eval`) – JellicleCat Feb 11 '18 at 03:12
This is exactly what I've been doing. What I "need" now is to build/nest configuration. For example: BASE_DIR="/home/base" CONFIG_DIR="$BASE_DIR/config" How can I do this such that it's compatible with bash and Python and doesn't duplicate the config? That's what led me to this question. – CapoChino Apr 02 '21 at 21:53

score 0 · Answer 4 · answered May 23 '18 at 20:28

Keeping "config.py" rather than "config.sh" leads to some pretty code.

config.py

CONFIG_VAR = "value"
CONFIG_VAR2 = "value2"

script.py:

import config

CONFIG_VAR = config.CONFIG_VAR
CONFIG_VAR2 = config.CONFIG_VAR2

script.sh:

CONFIG_VAR="$(python-c 'import config;print(config.CONFIG_VAR)')"
CONFIG_VAR2="$(python-c 'import config;print(config.CONFIG_VAR2)')"

Plus, this is a lot more legible than trying to "Emulate Bash 'source' in Python" like Gilles answer requires.

score 0 · Answer 5 · answered Apr 14 '23 at 19:46

I've experimented with using file name as value name and file content as value.

─┰ .settings/
 ┠─ var1
 ┠─ var2
 ┠─ var3
 ┗─ var4

And in shell you can load settings:

var1=$(<.settings/var1)
var2=$(<.settings/var2)

And in Python

var1 = open('.settings/var1','r').read().strip()
var1 = open('.settings/var2','r').read().strip()

And you can trivially load all files into a dict

from glob import glob

settings = {name: open(name, 'r').read().strip() for name in glob('.settings/*')}

if you want numbers to not be strings, you can add a function for loading data

def read_setting(name):
    val = open('.settings/'+name,'r').read().strip()
    try:
        return(float(val)
    except ValueError:
        return(val)

A configuration file that can be read by python and shell

5 Answers5