Why doesn't argparse let me add optional arguments in any order?

Question

In the code below, I cannot add the -v and -p arguments after the action (update, blacklist, auto), presumably due to the subparser. How can I make it so that I can add these optional arguments in any order?

parser = argparse.ArgumentParser(usage='pyfilter.py <file> <action> <options>')
parser.add_argument('file', help='blacklist file containing IPs', type=str)

subparsers = parser.add_subparsers(help='help', dest='action')

parser_update = subparsers.add_parser('update', help='update help')

parser_blacklist = subparsers.add_parser('blacklist', help='blacklist help')

parser_auto = subparsers.add_parser('auto', help='auto help')
parser_auto.add_argument('-i', '--interval', help='interval help')

parser.add_argument('-p', '--port', help='specify the port to block', type=int)
parser.add_argument('-v', '--verbose', help='write output to screen', nargs='?')
args = parser.parse_args()

According to the parser, this is valid:

python3.5 testfilter.py /etc/blacklist.lst -p 22 -v yes update

Whereas this yields errors for every argument following 'update':

python3.5 testfilter.py /etc/blacklist.lst update -p 22 -v yes

I think the issue is because you need to pass all the parameters to your `parser` before you move down into the `subparsers` otherwise, how is it meant to know which parser you're referring to? — Alan Kavanagh, Nov 21 '17 at 00:45
@AK47 It's fine as long as I declare them before the subparsers, but if I put them after the subparser I get an error. I want it to be like Bash where you can put optional arguments that are part of the main parser anywhere. — , Nov 21 '17 at 00:45
I don't think what you're trying to do is possible directly with `argparse`. Parsing out which options should go to the script itself (the main parser) versus a sub-command is actually very complex and difficult to get right, so `argparse` doesn't even try. — , Nov 21 '17 at 00:47
If you put the options after `update`, you're saying they should be options for `update`, not for the main parser. — user2357112, Nov 21 '17 at 00:47
@user2357112 Yes, I was hoping there was a way for argparse to detect that they're now for the main parser with something like nargs, although I don't think that will work. If there's no simple way to do it without additional libraries, I feel like it's unnecessary work to keep trying. — , Nov 21 '17 at 00:51
@Chase, "like bash"? Bash doesn't have any control over how programs it invokes behave, and inasmuch as GNU tools let you put options after positional arguments, doing so is an [active violation of POSIX utility syntax guidelines](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html#tag_12_02) -- see #9 in particular. — Charles Duffy, Nov 21 '17 at 00:53

score 0 · Answer 1 · answered Nov 21 '17 at 01:24

0

This is how your tree of parser looks like

parser
  |__ update
  |     |__ updateOpts
  |__ blacklist
  |__ auto
  |__ port
  |__ verbose

So if you do update, it will traverse down the updateOpts, and it cannot find port or verbose there

answered Nov 21 '17 at 01:24

user8734784

84
3

hpaulj · Accepted Answer · 2017-11-21T02:16:08.790

The main parser recognizes file (required), --port and --verbose, plus the subparser choice {'update','auto'}.

Once it encounters the subparser argument it delegates parsing to that parser. The main parser does not resume parsing after the subparser is done. It reports the strings the subparser couldn't handle as unrecognized.

So all strings meant for the main parser have to occur before the subparser is invoked.

You might get around this by using parse_known_args. Then the unrecognized args will be returned in an extras list, which could be handled with another parsing step.

Add top level argparse arguments after subparser args

How do you get argparse to choose a default subparser?

As a rule argparse tries to allow optionals (flagged) arguments in any order. There are some limits to that when there are variable narg positionals, and this case, where there are subparsers. This behavior makes sense once you know something about how the code is organized.

Why doesn't argparse let me add optional arguments in any order?

2 Answers2