3

I am writing unit tests for a python program using the unittest framework. One of the functions of the program is connecting to an external REST API using the requests library. If there is a connection error or timeout, I would like the function to retry up to 3 times before failing. As such I have written a test which uses a mock (actually httpretty) to replace the external API and raises requests.ConnectionError twice before returning something.

class APITests(unittest.TestCase):

    def mock_response(self, uri, body='OK', method=httpretty.GET,
                      status_code=200, error=None,
                      error_repeats=None):
        """
        Function to register HTTPretty response
        """
        responses = []

        if error:
            def callback(request, uri, headers):
                raise error

            if error_repeats:
                responses += [httpretty.Response(body=callback)]*error_repeats
                responses += [httpretty.Response(body=body,
                                                 status=status_code)]
            else:
                responses += [httpretty.Response(body=callback)]

        else:
            responses += [httpretty.Response(body=body, status=status_code)]

        httpretty.register_uri(method, uri, responses=responses)

    @httpretty.activate
    def test_get_connection_error_then_success_doesnt_raise(self):
        self.mock_response(
                'http://irrelevant.com',
                error=requests.ConnectionError,
                error_repeats=2
            )
        api._get('http://irrelevant.com')

This works fine and the test passes when an exception is not raised no the third attempt but the two exceptions that are raised (intentionally) and caught and handled by the code are printed to the console, polluting the test output. Is there a clean way to stop this happening?

Further info

Here is the method(s) I am testing

def _get(self, url, retries=3):
    while retries > 0:
        try:
            r = requests.get(url)
            try:
                r.raise_for_status()
                return r
            except requests.HTTPError as e:
                self._handle_HTTP_error(e)
        except (requests.ConnectionError, requests.Timeout) as e:
            retries -= 1
            if not retries:
                self._handle_connection_error(e)

def _handle_connection_error(self, error):
    raise requests.ConnectionError('Could not connect to API')

The console output is:

Exception in thread Thread-23:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/lukecaldwell/Documents/Repos/other/AidTrends/venv/lib/python2.7/site-packages/httpretty/core.py", line 637, in fill_filekind
    headers
  File "/Users/lukecaldwell/Documents/Repos/other/AidTrends/aidtrends/tests/test_OECD.py", line 131, in callback
    raise error
ConnectionError
lac
  • 755
  • 10
  • 19
  • Any exception that gets raised, should fail your test. Since that apparently doesn't happen, you're probably catching the exception, but still *printing* the exception message (as a string, not as an exception). Are you printing it to stderr though? You could redirect the output by re-assigning sys.stdout or sys.stderr temporarily, –  Apr 08 '16 at 06:42
  • The code I'm testing has a loop in it that catches the exception a maximum of two times before failing on the third loop. The test above is intended to simulate that. I don't print the exception message at all. I'll add the tested code now to make that clear. – lac Apr 08 '16 at 18:20
  • If you're not printing anything, how then does it clutter the test output. You also say "the two exceptions that are raised (intentionally) and caught and handled by the code are printed to the console". –  Apr 09 '16 at 01:16
  • Or do you just want to catch the *third* time when the exception is raised as an *expected* exception? –  Apr 09 '16 at 01:17
  • I don't intentionally print anything, but something in the unittest framework sees the exception and displays it in the console (e.g. as shown at the end of the question), this is what I mean by cluttering the test output. There should be no exceptions raised as it should catch the first two and the third time is a success. – lac Apr 09 '16 at 01:24
  • If you're not printing anything, and the unit test still shows an exception being raised, than either the unit test is incorrect or your function doesn't work the way you want to. In both cases, it's not about silencing the output, but interpreting the output. –  Apr 09 '16 at 01:31
  • Note that your exception appears to occur somewhere else than in the given function; I see `fill_filekind`. There is also the call to `Response(body=calback)`, but I don't see that anywhere: `body` isn't in `_get`, so that's used elsewhere (to raise the mock exception). Besides, `Response` with a capital is a tad non-pythonic nomenclature for a function (but for a class, it only gets initialized; no methods called). Things are going wrong somewhere else that is not shown in the question –  Apr 09 '16 at 01:37
  • The exception is called by the `mock_response` function in the test so you are right - it is not in the code being tested. But thats the point as it is simulating `requests.get` throwing a `ConnectionError`. `Response` is an httpretty class not a function. – lac Apr 09 '16 at 01:41
  • I looked at the httpretty code, and `Response` is in fact a classmethod. From a quick glance, If just find the httpretty interface confusing. –  Apr 09 '16 at 04:36
  • Nevertheless, while your question is confusingly phrased, I think you simple want to catch an exception. Which is simply done by [`assertRaises`](https://docs.python.org/2/library/unittest.html#unittest.TestCase.assertRaises). –  Apr 09 '16 at 04:38
  • Hi @Evert thanks for your responses but I don't think that is what I want, there is a separate test that checks that the exception is raised when the connection error happens 3+ times. This uses `assertRaises`. The test in question is checking that an exception is **not** raised by my code. If I use assert raises it will fail because the actual tested code catches the exceptions. What I don't understand is why details of the exceptions it catches are being printed to the console anyway. – lac Apr 09 '16 at 15:59
  • "What I don't understand is why details of the exceptions it catches are being printed to the console anyway." I guess that the crux of it, right? My current best guess would be that 1/ httpretty outsources its functionality to a thread, 2/ the ConnectionError gets raised inside the thread, 3/ the exception message as a whole gets piped to the caller through the `fill_filekind` method, and written to stdout, 4/ you get to see the traceback even if you catch the exception. –  Apr 10 '16 at 01:53
  • With my above comment: does the exception traceback show if you run your test (including use of httpretty) without the unittest framework? Just as a single script that uses httpretty for a mock response. Then httpretty may be to blame for the output. –  Apr 10 '16 at 01:55
  • You should also test whether the traceback is written to stdout or stderr. If it's written to stdout, you may be able to redirect stdout for that one test (as per my first comment), assuming that failed tests are written to stderr by the unittest framework. If it's written to stderr, things may become more annoying: you could catch all output in a buffer, and try to remove anything that matches the unwanted traceback. Alternatively, you could find a way to shut up httpretty, but I didn't see an option there (assuming httpretty is the cause of the unwanted output). –  Apr 10 '16 at 01:59

0 Answers0