How can I test whether my code is throwing the appropriate argparse exceptions?

Question

From this great answer I learned to put argument parsing into its own function to simplify unit testing.

From this answer I learned that sometimes you need to throw your own parser errors to get argparse to perform the behaviour you want. E.g.:

if not (args.process or args.upload):
    parser.error('No action requested, add -process or -upload')

But it is hard to test if this does what it should since throwing the parser error also exits the program. So something like this TestCase won't work:

def test_no_action_error(self):
    '''Test if no action produces correct error'''
    with self.assertRaises(ArgumentError) as cm:
        args = parse_args(' ')
    self.assertEqual('No action requested, add -process or -upload', str(cm.exception))

The comments from the first question suggest this question. But I don't follow how to use this code within a testing file.

@wim nothing wrong with that, I just couldn't find a question here on SO that said to do that (and how) — raphael, Nov 30 '16 at 21:47

score 3 · Accepted Answer · edited May 23 '17 at 12:25

After a bit of hacking away I've found something that will pass testing. Suggestions to remove cruft welcome.

In my main program I defined parse_args with some extra keyword args to be used for testing only.

def parse_args(args, prog = None, usage = None):
    PARSER = argparse.ArgumentParser(prog=prog, usage=usage)
    ....

Then in the testing class for testing the parser, adding these parameters to suppress usage and help information on an error as much as possible.

class ArgParseTestCase(unittest.TestCase):
    def __init__(self, *args, **kwargs):
        self.testing_params = {'prog':'TESTING', 'usage':''}
        super(ArgParseTestCase, self).__init__(*args, **kwargs)

In the testing file defined this context manager from this answer:

from contextlib import contextmanager
from io import StringIO

@contextmanager
def capture_sys_output():
    capture_out, capture_err = StringIO(), StringIO()
    current_out, current_err = sys.stdout, sys.stderr
    try:
        sys.stdout, sys.stderr = capture_out, capture_err
        yield capture_out, capture_err
    finally:
        sys.stdout, sys.stderr = current_out, current_err

And then modified the test in my question above to be something like:

def test_no_action_error(self):
    '''Test if no action produces correct error'''
    with self.assertRaises(SystemExit) as cm, capture_sys_output() as (stdout, stderr):
        args = parse_args([' '], **self.testing_params)
    self.assertEqual(2, cm.exception.code)
    self.assertEqual('usage: \n TESTING: error: No action requested, add -process or -upload',
                     stderr.getvalue())

Now the extra text at the start of the assertEqual isn't pretty... but the test passes so I'm happy.

score 0 · Answer 2 · answered Dec 01 '16 at 01:54

test/test_argparse.py does some of this kind of testing:

For example:

class TestArgumentTypeError(TestCase):

    def test_argument_type_error(self):

        def spam(string):
            raise argparse.ArgumentTypeError('spam!')

        parser = ErrorRaisingArgumentParser(prog='PROG', add_help=False)
        parser.add_argument('x', type=spam)
        with self.assertRaises(ArgumentParserError) as cm:
            parser.parse_args(['XXX'])
        self.assertEqual('usage: PROG x\nPROG: error: argument x: spam!\n',
                         cm.exception.stderr)

But the key to this the ErrorRaisingArgumentParser subclass defined near the start of the file.

class ErrorRaisingArgumentParser(argparse.ArgumentParser):

    def parse_args(self, *args, **kwargs):
        parse_args = super(ErrorRaisingArgumentParser, self).parse_args
        return stderr_to_parser_error(parse_args, *args, **kwargs)

    def exit(self, *args, **kwargs):
        exit = super(ErrorRaisingArgumentParser, self).exit
        return stderr_to_parser_error(exit, *args, **kwargs)

    def error(self, *args, **kwargs):
        error = super(ErrorRaisingArgumentParser, self).error
        return stderr_to_parser_error(error, *args, **kwargs)

See that file for details. With stderr redirection it gets a bit complicated. Maybe more than really needed.

I think I see where this might be going. But it also seems like it involves essentially duplicating the definition of the parser between the code and the testingcode (the first testing code block adds arguments to the parser). — raphael, Dec 01 '16 at 15:40
I didn't pick the example to closely match your case; it's testing a different type of error. For your case there are 2 issues - catching or redirecting the `sys.exit`, and catching the stderr message that goes with it. — hpaulj, Dec 01 '16 at 17:14
I found a solution that works for me, see [here](http://stackoverflow.com/a/40916320/4047679) — raphael, Dec 01 '16 at 18:16

score 0 · Answer 3 · answered Apr 15 '21 at 14:12

The easiest way using pytest would be the following:

with pytest.raises(SystemExit) as e:
    parse_args(...)

assert isinstance(e.value.__context__, argparse.ArgumentError)
assert 'expected err msg' in e.value.__context__.message

We need this workaround as argparse will exit error code 2 which means that a SystemExit will be raised.

How can I test whether my code is throwing the appropriate argparse exceptions?

3 Answers3

Linked