Best way to pass command line arguments via file in python

Question

I have a lot of arguments to pass to my main.py. It's easier to store them in a txt file. So, i would like to know best way of using "config" files to pass CL args. Shell script is not what i need, unfortunatly.

Answer 1

If you plan to use argparse, then fromfile_prefix_chars is designed to solve exactly this problem.

In your launching program, put all of the arguments, one per line, into a file. Pass @file.txt to your child program. In your child program, pass a fromfile_prefix_chars parameter to the ArgumentParser() constructor:

parser = argparse.ArgumentParser(fromfile_prefix_chars='@')

argparse takes care of the rest for you.

Here is an example:

from argparse import ArgumentParser

parser = ArgumentParser(fromfile_prefix_chars='@')
parser.add_argument('-f', '--foo')
parser.add_argument('--bar')
parser.add_argument('q', nargs='*')

ns = parser.parse_args()
print(ns)

The contents of foo.txt:

-f
1
--bar=2
q one
q two

The command line and the output:

$ python zz.py @foo.txt
Namespace(bar='2', foo='1', q=['q one', 'q two'])

Answer 2

Use configparser. It uses .ini files and it's really easy to use.

Config file:

[DEFAULT]
KeepAlive = 45
ForwardX11 = yes

Example Code:

>>> config = configparser.ConfigParser()
>>> config.sections()
[]
>>> config.read('example.ini')
>>> for key in config['bitbucket.org']: print(key)
...
keepalive
forwardx11
>>> default = config['default']
>>> default['keepalive']
'45'
>>> default['ForwardX11']
'yes'

Answer 3

Here is a simple function that converts any @foo argument into the contents of foo, one argument per line. After the conversion, you may use sys.argv in any of the normal ways.

import sys

def expand_arg_files(args):
    for arg in args:
       if arg.startswith('@'):
           with open(arg[1:]) as f:
               file_args = f.read().splitlines()
           yield from expand_arg_files(file_args)
       else:
           yield arg


sys.argv[:] = expand_arg_files(sys.argv[:])
print(sys.argv)

Notes:

• The generator delegation syntax requires Python3.3 or higher.
• You may have @ args inside the argument file. The expansion is recursive.