Parse arguments to get an argument value, or environment variable value, and raise exception if neither

Question

I wish to use the argparse module to set the value of a server URL. As well as being set via an argument, the vale can also be in an environment variable, though the argument should always take precedence.

To achieve this, I've added an argument to the parser with the environment variable value as the default, and a custom type. However, if the argument is omitted, the custom type=EnvironDefault is not initialised. This means that if default value is None, that value is used regardless, rather than an exception being raised as expected.

I have found a 10 year old question that suggests it is necessary to check the value of the argument after parsing the arguments, but I'm hoping that in the interim a solution has been added to argparse.

Example 1 - Success - Environment variable but no argument

foo@bar:~$ export SERVER_URL="https://some.server.co.uk"
foo@bar:~$ python mycli
Namespace(url='https://some.server.co.uk')

Example 2 - Success - Environment variable and argument

foo@bar:~$ export SERVER_URL="https://some.server.co.uk"
foo@bar:~$ python mycli --url "https://different.server.co.uk"
Namespace(url='https://different.server.co.uk')

Example 3 - Failure - No environment variable or argument

foo@bar:~$ unset SERVER_URL
foo@bar:~$ python mycli
Namespace(url=None)

Expected response

usage: mycli [-h] [--url URL]
mycli: error: argument --url: Server URL must be provided via either the CLI or the environment variable SERVER_URL.

Example Argument Parser

import argparse
import os

class EnvironDefault(str):

    def __init__(self, url: str) -> None:
        err_msg = (
            'Server URL must be provided via either the CLI or the '
            'environment variable SERVER_URL.')
        if not url:
            raise argparse.ArgumentTypeError(err_msg)

parser = argparse.ArgumentParser()
parser.add_argument(
    '--url', type=EnvironDefault, default=os.environ.get('SERVER_URL'),
    help='Server URL. Overrides environment variable SERVER_URL.')

print(parser.parse_args())

Default argument values are assigned directly to the destination, not passed to the type first. Since you won't know if no argument is provided until after parsing is done, you'll have to either override `ArgumentParser.parse_args`, or content yourself with checking the return value of `parse_args`. — chepner, Oct 18 '22 at 13:27
(Or dynamically decide if the argument is required or not; see my answer below.) — chepner, Oct 18 '22 at 13:38
Thanks for your comments. Your answer below has steered me straight, so thanks for that. However, I'm curious about your suggestion of overriding `ArgumentParser.parse_args` - presumably I'd have to dig further into the `ArgumentParser` object, as it doesn't look like there is anything in `parse_args` that I can override to pass the default value to the type first. — David Gard, Oct 18 '22 at 13:52

score 3 · Accepted Answer · edited Oct 18 '22 at 14:01

3

When the program starts, you can check immediately if SERVER_URL is available, and use that fact to decide if --url is required or not.

parser = argparse.ArgumentParser()
parser.add_argument(
    '--url', required='SERVER_URL' not in os.environ, default=os.environ.get('SERVER_URL'),
    help='Server URL. Overrides environment variable SERVER_URL.')
    
print(parser.parse_args())

If SERVER_URL is not in the environment, the default value of None will be ignored, because --url is required. If it is in the environment, then the default value will be used.

Note that default values are not processed by your type. If you do use a custom type, it should be applied to the default explicitly. For example -

parser.add_argument('--value', 
                    required='VALUE' not in os.environ,                               
                    type=int,
                    default=int(os.environ.get('VALUE')))

edited Oct 18 '22 at 14:01

David Gard

11,225
36
115
227

answered Oct 18 '22 at 13:38

chepner

497,756
71
530
681

It's not something I've used in the past to solve this problem--just thought of it now--so there might be some issues I haven't thought of yet. – chepner Oct 18 '22 at 13:42
Both of your suggestions here work, so thanks for that. The only thing of note is that on your second suggestion (explicitly applying the type to the default), the `argparse.ArgumentTypeError` exception is caught outside of `argparse`, so you don't get the nicely formatted error with the usage. – David Gard Oct 18 '22 at 13:50
True, but it's really "your" responsibility (not `argparse`'s) to ensure that the default value is correct. – chepner Oct 18 '22 at 14:20
Agreed, just mentioning it for prosperity in case others stumble across this. – David Gard Oct 18 '22 at 14:50