How to share arguments between the toplevel and a subcommand

Question

In a program using argparse with subcommands, I want to add arguments to control verbosity. I would like these to act as "global" arguments, meaning it shouldn't matter whether they are passed before or after the subcommand. In other words, I want the following invocations to be interpreted the same way:

> ./test.py -v -v foo
> ./test.py foo -v -v
> ./test.py -v foo -v

Things I've tried

Adding the arguments to the parent parser and the subcommand parser:

main = arg.ArgumentParser()
main.add_argument ("-v", "--verbose", default = 0, action = "count")
s = main.add_subparsers()
foo = s.add_parser ("foo")
foo.add_argument ("-v", "--verbose", default = 0, action = "count")

print (main.parse_args().verbose)

This gives (all should be 2):

Invocation	Result
./test.py -v -v foo	0
./test.py foo -v -v	2
./test.py -v foo -v	1

Adding a common "parent" parser to both the main and subcommand parsers:

Inspired by this question, the difference being that in the other question only a single subcommand is active at a time, whereas here the main parser and the subcommand parser are both active simultaneously.

p = arg.ArgumentParser (add_help = False)
p.add_argument ("-v", "--verbose", default = 0, action = "count")
main = arg.ArgumentParser (parents = [ p ])
s = main.add_subparsers()
foo = s.add_parser ("foo", parents = [ p ])

print (main.parse_args().verbose)

This gives the same result as above.

Declaring the main parser as parent for the subcommand parser:

main = arg.ArgumentParser()
main.add_argument ("-v", "--verbose", default = 0, action = "count")
s = main.add_subparsers()
foo = s.add_parser ("foo", parents = [ main ], add_help = False)

print (main.parse_args().verbose)

This results in an infinite recursion where the foo subcommand expects itself:

> ./test.py -v -v foo
usage: test.py foo [-h] [-v] {foo} ...
test.py foo: error: too few arguments

Adding the arguments to the parent and subcommand parsers with different destinations:

main = arg.ArgumentParser()
main.add_argument ("-v", "--verbose", default = 0, action = "count", dest = "main_verbose")
s = main.add_subparsers()
foo = s.add_parser ("foo")
foo.add_argument ("-v", "--verbose", default = 0, action = "count")

args = main.parse_args()
print (args.main_verbose + args.verbose)

This works, but it adds complexity: now I need to combine the values for each common parameter and for as many layers as I have nested subcommands.

Is there a way to define "global" parameters, that will act the same no matter where they are put on the command-line?

Answer 1

The main parser and subparser don't 'share' arguments. Each is an independent parser with its own argument list. The parents mechanism just copies argument definitions (by reference) from parent to child. It does not alter parsing.

The main parser puts values in its namespace up to the point that the subparser is called. The subparser does its thing, parsing the remaining strings and putting the values in its own namespace. When done, the subparser namespace is copied to the main namespace.

The result is that for any arguments with a common dest, the subparser values (including default) override the main's values. Thus if you want to see what the main sets, you have use different dest.

Answer 2

One tool that I've used that has this behavior is the AWS CLI v2. With that command you can run

aws --profile myprofile s3 ls s3://mybucket
aws s3 ls s3://mybucket --profile myprofile

and you get the same result.

After perusing the aws cli v2 source code, mainly argparser.py and clidriver.py, it seems they create a main parser and the service parsers. The main parser handles aws [options] command subcommand and the service parsers handle aws s3 command. The service parsers are not parents of the main parser, though.

They first parse the arguments with the main parser and then pass remaining arguments to the service parser. Here is an example of this applied to your problem.

import argparse

main = argparse.ArgumentParser()
main.add_argument("-v", "--verbose", default=0, action="count")
main.add_argument("command")
sub = argparse.ArgumentParser()
sub.add_argument("operation")


def test_parser(cli_string):
    cli_args = cli_string.split()
    parsed_args, remaining = main.parse_known_args(cli_args)
    sub_args, remaining = sub.parse_known_args(remaining)
    print("Verbose:", parsed_args.verbose)
    print("Subcommand:", sub_args.operation)

calls = ["./test.py foo -v -v", "./test.py -v foo -v", "./test.py -v -v foo"]

for c in calls:
    print(c)
    test_parser(c)
    print()

which results in

./test.py foo -v -v
Verbose: 2
Subcommand: foo

./test.py -v foo -v
Verbose: 2
Subcommand: foo

./test.py -v -v foo
Verbose: 2
Subcommand: foo

I am not sure what other functionality you might need and how it would interact with this pattern, though.