Python dependencies between groups using argparse

Question

I started to learn Python, and now I'm learning the great benefits of argparse. Using argparse, I have created two groups of arguments: group_list and group_simulate. Each of the groups has its own arguments -- the user can specify only one argument in each group (achieved using parser.add_mutually_exclusive_group()).

And now my target is present a syntax error if the user specified arguments from both groupgs and not from only one of them -- I want to achieve this by using the capabilities of argparse and not by writing a method that asks if this and this was specified print syntax error.

import argparse
parser = argparse.ArgumentParser(
        description='this is the description',
        epilog="This is the epilog",
        argument_default=argparse.SUPPRESS  
        )

parser.add_argument('-v', '--verbose', help='verbose', action='store_true', default=False)

group_list = parser.add_mutually_exclusive_group()
group_list.add_argument('-m', help='list only modules', action='store_const', dest='list', const='modules', default='all')
group_list.add_argument('-p', help='list only ports', action='store_const', dest='list', const='ports', default='all')
group_list.add_argument('--list', help='list only module or ports', choices=['modules','ports'], metavar='<modules/ports>', default='all')

group_simulate = parser.add_mutually_exclusive_group()
group_simulate.add_argument('-M', help='simulate module down', nargs=1, metavar='module_name', dest='simulate')
group_simulate.add_argument('-P', help='simulate FC port down', nargs=1, metavar='fc_port_name', dest='simulate')
group_simulate.add_argument('-I', help='simulate iSCSI port down', nargs=1, metavar='iSCSI_port_name', dest='simulate')
group_simulate.add_argument('--simulate', help='simulate module or port down', nargs=1, dest='simulate')

args = parser.parse_args()

print args

So talking more specifically:

allowed:

test.py
output: Namespace(list='all', verbose=False)
test.py -m
output: Namespace(list='modules', verbose=False)
test.py -P asfasf
output: Namespace(P=['asfasf'], list='all', verbose=False)

not allowed:

test.py -m -P asfsaf
expected output: <the help message>
test.py -P asfasf -m
expected output: <the help message>

I have tried to achieve the wanted target with the option of add_subparsers from argparse but without any success.

So my question is how to achieve this situation?

Answer 1

You can use a common mutually-exclusive-group as "root" of the two subgroups:

import argparse
parser = argparse.ArgumentParser(
        description='this is the description',
        epilog="This is the epilog",
        argument_default=argparse.SUPPRESS  
        )

parser.add_argument('-v', '--verbose', help='verbose', action='store_true', default=False)

root_group = parser.add_mutually_exclusive_group()

group_list = root_group.add_mutually_exclusive_group()
group_list.add_argument('-m', help='list only modules', action='store_const', dest='list', const='modules', default='all')
group_list.add_argument('-p', help='list only ports', action='store_const', dest='list', const='ports', default='all')
group_list.add_argument('--list', help='list only module or ports', choices=['modules','ports'], metavar='<modules/ports>', default='all')

group_simulate = root_group.add_mutually_exclusive_group()
group_simulate.add_argument('-M', help='simulate module down', nargs=1, metavar='module_name', dest='simulate')
group_simulate.add_argument('-P', help='simulate FC port down', nargs=1, metavar='fc_port_name', dest='simulate')
group_simulate.add_argument('-I', help='simulate iSCSI port down', nargs=1, metavar='iSCSI_port_name', dest='simulate')
group_simulate.add_argument('--simulate', help='simulate module or port down', nargs=1, dest='simulate')

args = parser.parse_args()

print args

Result:

$ python test.py -m -P asfafs
usage: test.py [-h] [-v] [[-m | -p | --list <modules/ports>]
                [-M module_name | -P fc_port_name | -I iSCSI_port_name | --simulate SIMULATE]
test.py: error: argument -P: not allowed with argument -m 

$ python test.py -m -p
usage: test.py [-h] [-v] [[-m | -p | --list <modules/ports>]
                [-M module_name | -P fc_port_name | -I iSCSI_port_name | --simulate SIMULATE]
test.py: error: argument -p: not allowed with argument -m

Answer 2

Use Docopt! You shouldn't have to write a usage doc and then spend hours trying to figure out how to get argparse to create it for you. If you know POSIX you know how to interpret a usage doc because it is a standard. Docopt know how to interpret usage docs that same as you do. We don't need an abstraction layer.

I think the OP has failed to describe their own intentions based on what I read in their help text. I'm going to try and speculate what they are trying to do.

---

test.py

"""
usage: test.py [-h | --version]
       test.py [-v] (-m | -p)
       test.py [-v] --list (modules | ports)
       test.py [-v] (-M <module_name> | -P <fc_port_name> | -I <iSCSI_port_name>)

this is the description

optional arguments:
  -h, --help                show this help message and exit
  -v, --verbose             verbose
  -m                        list only modules (same as --list modules)
  -p                        list only ports   (same as --list ports)
  --list                    list only module or ports
  -M module_name            simulate module down
  -P fc_port_name           simulate FC port down
  -I iSCSI_port_name        simulate iSCSI port down

This is the epilog
"""

from pprint import pprint
from docopt import docopt

def cli():
    arguments = docopt(__doc__, version='Super Tool 0.2')
    pprint(arguments)

if __name__ == '__main__':
    cli()

While it would be possible to communicate all of the usage in a single line with complex nested conditionals, this is more legible. This is why docopt makes so much sense. For a CLI program you want to make sure you communicate to the user clearly. Why learn some obscure module syntax in the hope that you can convince it to create the communication to the user for you? Take the time to look at other POSIX tools with option rules similar to your needs and copy-pasta.

Answer 3

A simpler version of this parser is

parser=argparse.ArgumentParser(description="this is the description",
                epilog='this is the epilog')
parser.add_argument('-v', '--vebose', action='count')
g1=parser.add_mutually_exclusive_group()
g1.add_argument('--list', help='list module or ports (default=%(default)s)', choices=['modules','ports','all'], default='all')
g1.add_argument('--simulate', '-M','-P','-C', help='simulate [module down/ FS port down/ iSCSI port down]', dest='simulate', metavar='module/port')

With a help that looks like:

usage: stack14660876.py [-h] [-v]
                        [--list {modules,ports,all} | --simulate module/port]

this is the description

optional arguments:
  -h, --help            show this help message and exit
  -v, --vebose
  --list {modules,ports,all}
                        list module or ports (default=all)
  --simulate module/port, -M module/port, -P module/port, -C module/port
                        simulate [module down/ FS port down/ iSCSI port down]

this is the epilog

Beside verbose (here I substituted a count) the OP sets to attributes, list and simulate. list has a default value of all, and can be set to modules or ports. -m and -p are just short cuts, and don't really add to the definition. Shortcuts can be handy when defining lots of options, especially if the options can be used together (e.g. -vpm). But here only one option is allowed (besides -v).

simulate takes an unconstrained string. The M/P/C options are just documentation conveniences, and don't constrain the values or add meaning.

This is a nice exercise in pushing the boundaries of argparse (or any other parser), but I think it is too complicated to be useful. Despite all the groupings it comes down to allowing only one option.

==========================

Comments about docopt and POSIX argument handling prompted me to look at C argument libraries. getopt is the old standard. Python has a functional equivalent, https://docs.python.org/2/library/getopt.html

The other parser in the GNU library is argp.

http://www.gnu.org/software/libc/manual/html_node/Argp.html

I haven't seen, yet, a clear description of what it adds to the getopt syntax. But the following paragraph is interesting.

Argp also provides the ability to merge several independently defined option parsers into one, mediating conflicts between them and making the result appear seamless. A library can export an argp option parser that user programs might employ in conjunction with their own option parsers, resulting in less work for the user programs. Some programs may use only argument parsers exported by libraries, thereby achieving consistent and efficient option-parsing for abstractions implemented by the libraries.

It sounds a bit like the argparse subparser mechanism. That is, there's some sort of meta-parser that can delegate the action to one (or more) subparsers. But in argparse subparsers have to be explicitly named by the user.

A possible extension is to have the meta-parser look at the context. For example in the OP case, if it sees any of [--list, -p, -m] use the list subparser, if any of the simulate arguments, use the simulate subparser. That might give some more powerful grouping tools. And it might be possible to implement that sort of thing with the stock argparse. You can create and run several different parsers on the same sys.argv.