Python argparse: Processing Certain Arguments Before Adding Others

Question

Using python's argparse library, I would like to process the first few command-line arguments and use them to generate a list of choices for the other command-line arguments.

How can I process the first few arguments without argparse complaining about extra arguments that it doesn't expect (which I plan to add later)?

For example, I have a script that gets the username and password from the command-line, uses those to access available properties of an API, and then uses that list to restrict the values of a third argument:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('username', help='Your username.')
parser.add_argument('password', help='Your password.')
args = parser.parse_args() # Error here because a third argument exists on the command-line

response = requests.get(
    url='https://api.example.com/properties',
    auth=(args.username, args.password)
)

parser.add_argument(
    'property',
    choices=response.json()['properties'], # Validates the input
    help='The property you want to access.'
)
args = parser.parse_args()

I know I can just add all the arguments at once, and then validate the third argument myself manually, but I'm wondering if there's a way to natively do what I'm asking in the argparse library?

Did you check whether http://stackoverflow.com/questions/20165843/argparse-how-to-handle-variable-number-of-arguments-nargs answers your question? — Chih-Hsu Jack Lin, May 16 '17 at 00:51
`argparse` is as the name sugests a parser. It has functionalities to verify the data to parse, but this verification has is limits. Doing HTTP lookups is definitely outside of this limits. You have to do it in code. Also consider that your approach would not allow to display help without giving the credentials as arguments, but you don't know how until you see the help. — Klaus D., May 16 '17 at 01:12
@Chih-HsuJackLin although it doesn't directly answer my question, within the accepted answer I see the use of the `parse_known_args()` function (which I wasn't aware of), which I can use to parse chunks of arguments at a time. @KlausD. I see your point about incomplete help documentation. I'll stick to adding all the arguments at once and verifying it myself. Thank you both. — ezgoodey, May 16 '17 at 01:35
`parse_known_args` is the first thing that came to mind as I started to read your question. The relevant docs section: https://docs.python.org/3/library/argparse.html#partial-parsing. The `intermixed` subclass that I suggested in the linked SO is probably overkill. I haven't gotten much feedback on that since it was proposed. — hpaulj, May 16 '17 at 02:19

score 3 · Accepted Answer · answered May 16 '17 at 02:32

parser = argparse.ArgumentParser()
parser.add_argument('username', help='Your username.')
parser.add_argument('password', help='Your password.')
parser.add_argument(
    'property',
    help='The property you want to access.'
)
args = parser.parse_args() 

response = requests.get(
    url='https://api.example.com/properties',
    auth=(args.username, args.password)
)
allowed_properties = response.json()['properties']
if args.property not in allowed_properties:
    parser.error('property not in allowed properties')

choices is not a complicated test. I'd recommend parsing all inputs, and then testing for this special case. Overall I think gives you better control, and better help and error display.

The parse_known_args approach is good to learn and occasionally use, but I don't think it's that great for this case. You don't get any extra points for embedding all argument testing in the parser.

Python argparse: Processing Certain Arguments Before Adding Others

1 Answers1