Parsing argparse input bit by bit

Question

I am using Argparse to parse shell input to my Python function.

The tricky part is that this script first reads in a file that partially determines what kind of arguments are available to Argparse (it's a JSON file containing criteria by which the user can specify what data to output).

But before these arguments are added to my parser, I would like to read in some arguments relating to the file reading itself. (e.g. whether to fix the formatting of the input file). Kinda like this:

test.py (fix_formatting=True, **more arguments added later)

When I try to run args = parser.parse_args() twice, after the initial input and after adding more keys, things fall apart: Argparse quite predictably complains that some of the user input are unrecognized arguments:. I thought I might use subparsers to that end.

So I tried variations of (following the example in the docs as best as I could):

def main():
    parser = argparse.ArgumentParser()
    subparsers = parser.add_subparsers(help='sub-command help')
    settingsparser = subparsers.add_parser('settings') #i want a subparser called 'settings'
    settingsparser.add_argument('--fix_formatting', action='store_true') #this subparser shall have a --fix_formatting

Then I try to parse only the "settings" part like so:

    settings=parser.parse_args(['settings'])

This seems to work. But then I add my other keys and things break:

    keys=['alpha','beta','gamma','delta']
    for key in keys:
        parser.add_argument("--"+key, type=str, help="X")
    args = parser.parse_args()

If I parse any input for any of the arguments from keys, Argparse complains that I make an invalid choice: [...] (choose from 'settings'). Now I don't understand why I have to choose from "settings"; the docs say that the parse

will only contain attributes for the main parser and the subparser that was selected by the command line (and not any other subparsers)

• what is my error of understanding here?

• and if this is the wrong approach, how would one go about parsing one bit of input before another bit?

Any help is much appreciated!

Answer 1

parse_args calls parse_known_args. This returns the args namesparse along with a list of strings (from sys.argv) that it could not process (extras). parse_args raises this error if this list is not empty.

https://docs.python.org/3/library/argparse.html#partial-parsing

Thus parse_known_args is useful if you want to parse some of the input.

sys.argv remains unchanged. Subsequent calls to a parser (whether it was the original one or not) use that again, unless you pass the extras.

I don't think subparsers help you here. They aren't meant for delayed or two stage parsing. I'd suggest playing with the documentation examples for subparsers first.

---

To the main parser, the subparsers look like

subparsers = parser.add_argument('cmd', choices=['select',...])

In other words, it adds a positional argument where the choices are the subparser names that you define. That may help you see why it expects you to name select. Positionals are normally required.

(there's an exception to this in recent versions, https://stackoverflow.com/a/22994500/901925)