dynamically set default value from cfg file through argparse python

Question

I am trying to use argparse module. I am dealing with 2 configuration files:
1.default cfg file 2. File provided by user.

If no file provided at run time, read from default file. And if user provides a file, use the content of provided file.

My question is : I want to provide default field for each parser.add_agument, so if it's not provided by user,it will always have some default value. These values can be read from either of the file (i.e. default or user given file). I need to check which file is provided and then from default/user-given-file import *

My code:
get_args.py

class GetArgs:
   def get_args(self):
       parser = argparse.ArgumentParser(description='foo')
       #
       parser.add_argument(
        '-c', '--config', type=str, help='Provide a file', 
                          required=False, default='settings.cfg')
         #now note that below "default=version" will be from either "settings.cfg" or "user given file"

         # here may be i need to import from file ? 
        parser.add_argument(
        '-v','--version', type=str, help='API version', required=False, default=version)
        args = parser.parse_args()

        # Assign args to variables
        config=args.config
        version = args.version
        retutn config,version

code to read cfg file: read_file.py

import ConfigParser
configParser = ConfigParser.RawConfigParser()
configParser.read(file)
try:
    version= configParser.get('Set', 'version')
    ....
    ....
except ConfigParser.NoOptionError:
    print "Option not found in configuration file!"
    sys.exit(1)

something like:

1.python get_args.py                                      #reads "version" from settings.cfg
2.python get_args.py --config /path/to/file/file.cfg      #reads "version" file.cfg

It works perfect when i don't provide --config.

Is this possible to read dynamically?

I don't think what you want to achieve is possible, because the default value must be set before the arguments are parsed – this is for instance important to display the default values when you call `./get_args.py -h`. However, you could set the default value to `None` and later replace the values with values you read from either configuration file. — David Zwicker, Feb 02 '16 at 00:39
@DavidZwicker - Yes make sense that default value should be set before we execute script. — H. U., Feb 02 '16 at 00:45

hpaulj · Answer 1 · 2017-05-30T00:28:55.083

Populating a parser dynamically

Another question today got an answer that shows how to populate a parser from values in a dictionary: https://stackoverflow.com/a/35132772/901925

Ipython does something similar. It reads config files (default as well as user profile one(s)), and uses the config values to popular a parser. That gives the user several levels of control - default config, profile config, and finally, commandline. But the Ipython code is rather complex.

Editing argument values

Another useful point, is that each argument, or Action object, is available after creation, and may (within limits) be modified.

arg1 = parser.add_argument('--foo', default='test', ...)
# arg1 is an Action object with the various parameters
print arg1     # a display of some attributes
print arg1.default    # the value of the default as set above
arg1.default = 'different'  # assign a new value

So it is possible dynamically set the default values before parsing.

Prefix char file

Another useful tool is https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars

parser = argparse.ArgumentParser(fromfile_prefix_chars='@')

allows me to specify the name of a file prefaced with the character. It will read the files contents and splice the strings into the sys.argv list, just as though I had typed them it.

Filling in defaults after parsing

However if you want to specify the config file in the commandline, and then use its contents to set defaults for the parser, things get more complicated.

After parsing args will be a namespace object that contains the config file name along with all other arguments that it parsed.

You can see the exact contents of args with print args, or in dictionary form as vars(args). There's no hidden information about which values were set with the parser defaults and which were set by the commandline.

The cleanest, post parsing test for defaults is

if args.foo is None:
   # this has the default default

If the value is None it can only come from the default, not the commandline (there's no string that translates to None).

So I can imagine writing a function that iterates through the keys of vars(args), testing for None, and replacing it with the corresponding value from config. (Do you need help with that?).

If both config and args are cast as compatible dictionaries, then you could use the dictionary update method to transfer values from one to the other.

default=argparse.SUPPRESS can be used to keep an argument out of the namespace if the argument does not appear in the commandline. https://docs.python.org/3/library/argparse.html#default. This could be used along with the dictionary update, e.g.

config_dict.update(vars(args))

would only change config_dict for arguments given in the commandline.

Two stage parser

Another idea is to use two stage parsing. Define one parser that has the config_file argument, and no (or few) others. Run it with parse_known_args to the get the config_file value. Read the config file and use it to populate the 2nd stage parser (or at least to modify its defaults). Now run the 2nd stage parser to get the full namespace (I'd just ignore any config_file values in this args).

ChainMap

https://docs.python.org/3/library/collections.html#chainmap-examples-and-recipes

A newish collections class, ChainMap has an example of chained value lookup from a namespace, env, defaults, etc.

namespace = parser.parse_args() 
command_line_args = {k:v for k, v in vars(namespace).items() if v} 
combined = ChainMap(command_line_args, os.environ, defaults) 
print(combined['color'])

Other good ideas at Setting options from environment variables when using argparse