0

As the title says, I am looking for a way to manage config files in python with options that should be accessible across multiple source files and multiple processes, with a (hopefully) human readable format, where I can also set some options programmatically.

I have tried/looked at the following:

  • Using a global.py: file and importing it everywhere. However, when I change any of the values, the change is not reflected in the other processes. I suppose this cannot be done as per this answer: Globals variables and Python multiprocessing. Is there a way to make this work?
  • configparser: This seems to be a really good option, though I don't understand the following: When I change some of the options after reading a file, how do I make sure this change is reflected in the other files/processes?
  • configargparser: I like this as well, since it is extremely customizable. However, I again have the same question as above.
  • yaml: This seems to be the most human readable option. In that, I can give lists and dicts as arguments without much of a problem. I still don't know how to use it across files and processes though. I guess one way would be to read everything in a main file, dump it somewhere before spawning other processes and read that dumped file in other processes. Is there a better way?
cauthon14
  • 269
  • 1
  • 3
  • 14
  • 1
    For the latter three: you'll just need to re-read the configuration file. – 9769953 May 26 '22 at 23:40
  • 1
    If you require a running program to change its function halfway through a change in a configuration way, you'll need some process to scan the configuration for changes and reread it accordingly. – 9769953 May 26 '22 at 23:41
  • yaml is a dangerous format, since it can easily get out of hand with its level of depth. For a configuration file, stick to something simple. configparser is very simple and straightforward, toml (if not gone overboard with nesting levels) is also good. But it really depends on your requirements, on what configurations you need. – 9769953 May 26 '22 at 23:43
  • It's not clear why/what you want to change during runtime of a process (there may be better solutions), and what goes in the configuration. Is it just key-values, with keys being strings and values being strings or numbers? Or is it more complicated, and if so, in what way? – 9769953 May 26 '22 at 23:44
  • @9769953, I need dicts and lists in my config. configparser doesn't seem to support that. Regarding changes, I don't need to change anything during the runtime. It's just that some of the options depend on the values of other options. So far, I have been setting them programmatically every time they are read. I wanted to know if I can do it once and forget about it. – cauthon14 May 26 '22 at 23:46
  • You might also want to consider JSON. – Nick May 27 '22 at 00:03
  • If you don't need changes during runtime, then your question about reflecting changes in other processes doesn't make sense: you just start those processes with the new configuration file. – 9769953 May 27 '22 at 00:09
  • 1
    If you have options depending on each other, and you have dict and lists, using a Python file and import it. Be aware that that is a risk if other people can also change options, since a Python file/module could do anything (but that's the same for your program or any dependencies). – 9769953 May 27 '22 at 00:10

0 Answers0