Argparse suggests nonsensical order in help text usage line

Question

This program:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('files', metavar='INPUT', nargs='*',
    help='File(s) containing words to include. If none given, stdin will be used.')
parser.add_argument('-x', '--exclude', nargs='*',
    help='File(s) containing words to exclude.')
args = parser.parse_args()
print args.files
print args.exclude

produces this output when run in Python 2.7.9:

$ python prog.py --help
usage: prog.py [-h] [-x [EXCLUDE [EXCLUDE ...]]] [INPUT [INPUT ...]]

positional arguments:
  INPUT                 File(s) containing words to include. If
                        none given, stdin will be used.

optional arguments:
  -h, --help            show this help message and exit
  -x [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]
                        File(s) containing words to exclude.

However, that "help" output instructs the user to use a nonsensical ordering for the arguments. It is nonsensical because if the -x option is used, then no INPUT arguments will be detected.

Argparse ought instead to advise the user to use this ordering:

usage: prog.py [-h] [INPUT [INPUT ...]] [-x [EXCLUDE [EXCLUDE ...]]]

Two questions:

1. Is this a bug in argparse? (I think it is.)
2. Regardless of whether it is a bug, how can I fix it so that $ python prog.py --help will output the help text I desire (see above), preferably in as DRY a way as possible?

Answer 1

The simplest way is to add usage="..." to argparse.ArgumentParser().

By viewing the source of argparse, I found a way to resort arguments which might be a little bit dirty:

class MyHelpFormatter(argparse.HelpFormatter):

    def _format_actions_usage(self, actions, groups):
        actions.sort(key=lambda a: bool(a.option_strings and a.nargs != 0))
        return super(MyHelpFormatter, self)._format_actions_usage(actions, groups)

parser = argparse.ArgumentParser(formatter_class = MyHelpFormatter)

Answer 2

Add '-f', '--files' to the input option:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-f', '--files', metavar='INPUT', nargs='*', required=True,
    help='File(s) containing words to include. If none given, stdin will be used.')
parser.add_argument('-x', '--exclude', nargs='*',
    help='File(s) containing words to exclude.')
args = parser.parse_args()
print args.files

shows:

usage: argparse_test.py [-h] [-f [INPUT [INPUT ...]]]
                    [-x [EXCLUDE [EXCLUDE ...]]]

optional arguments:
  -h, --help            show this help message and exit
  -f [INPUT [INPUT ...]], --files [INPUT [INPUT ...]]
                        File(s) containing words to include. If none given,
                        stdin will be used.
  -x [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]
                    File(s) containing words to exclude.
print args.exclude

You can make 'files' required. From the docs:

In general, the argparse module assumes that flags like -f and --bar indicate optional arguments, which can always be omitted at the command line. To make an option required, True can be specified for the required= keyword argument to add_argument():

Answer 3

When generating the usage line, flagged arguments are placed first, positional after. That fits with common commandline usage. It makes no effort to evaluate whether that is the best choice or not.

The simplest way around this is to provide a custom usage parameter.

(there have be SO questions about changing the order of arguments in the usage line. The solution requires customization of the HelpFormatter class. Change argparse usage message argument order )

As you note, when the * positional is placed after the * optional, all arguments are assigned to the optional. You could use a '--' to separate the two lists of arguments.

There is a bug/issue with patch that should improve handling when the positional takes a known number of arguments (e.g. the default one). It does so by noting that the positional requires an argument, so it reserves one for it. But in the * * case, the positional is satisfied with none, so it has no way of knowing how to split the arguments between the 2.

I like the idea of turning that postional into a flagged argument. That should reduce the ambiguity inherent in lots of * arguments.