Preserve response context testing flask app with pytest

Question

I'm testing a flask application with py.test with the following code:

response = flask_app_test_client.post('/users', data=json.dumps(user))
assert response.status_code == 201
assert response.content_type == 'application/json'
assert isinstance(response.json, dict)
assert set(response.json.keys()) >= {'id', 'status', 'created_at', 'updated_at'}
assert response.json['name'] == user['name']
assert response.json['status'] == 'pending'

When some assertion fails I'm getting something like this:

            response = test_client.post('/users', data=json.dumps(user))
    >       assert response.status_code == 201
    E       assert 400 == 201
    E        +  where 400 = <JSONResponse streamed [400 BAD REQUEST]>.status_code
    ============== 1 failed, 3 passed in 0.10 seconds ===================

I do a lot of TDD so I expect my test fails frequently while developing. My problem is the assertion error message is kind of useless without the rest of the response data (body, headers, etc).

I only get in the output that the response.status_code is 400 but I don't get the error description that is in the response body: {"errors": ["username is already taken", "email is required"]}. Ideally I would like a full dump of the request and response (headers + body) when an assertion fails.

How I can print a summary of the response on each failed assertion?

I usually take advantage of the print cappabilites of the `assert` keyword, so I put `response.json` after the assert. I would extend you assert to make it look like `assert response.status_code == 201, reponse.json`, then you will see what was returned by an endpoint when the assertion fails. If the response body is not much of help then you need to debug by putting breaking point into endpoint code. Take a look at https://stackoverflow.com/a/5142453/7320870 — wiaterb, Apr 18 '20 at 10:42

score 0 · Answer 1 · answered Apr 15 '16 at 19:43

0

Assert statement graamar

assert response.status_code == 201, "Anything you want"

You can be as verbose as you want. You can also use UnitTest's suite of helper methods - without test case classes through this bit of abuse - https://github.com/nose-devs/nose2/blob/master/nose2/tools/such.py#L34

answered Apr 15 '16 at 19:43

David

17,673
10
68
97

Indeed, right now I'm adding response.json() on every assert but doesn't seem the right approach. Can't I just enhanced the assert somehow to print custom stuff? I'm pretty sure I'm missing something here... – pablomolnar Apr 15 '16 at 19:51
@pablomolnar My apologies, working near a deadline and misread your question. If it is just this specific use-case, perhaps `try{assert block}except AssertionError: print(more.data); raise` ? – David Apr 15 '16 at 21:35

pablomolnar · Answer 2 · 2016-04-18T18:18:27.880

I'm came up with two different solutions.

Solution #1: try/catch

try:
    assert response.status_code == 201
    assert response.content_type == 'application/json'
    assert isinstance(response.json, dict)
    assert set(response.json.keys()) >= {'id', 'status', 'created_at', 'updated_at'}
    assert response.json['name'] == user['name']
    assert response.json['status'] == 'pending'
except AssertionError as e:
    except AssertionError as e:
    raise ResponseAssertionError(e, response)

class ResponseAssertionError(AssertionError):
    def __init__(self, e, response):
        response_dump = "\n +  where full response was:\n" \
                        "HTTP/1.1 {}\n" \
                        "{}{}\n".format(response.status, response.headers, response.json)

        self.args = (e.args[0] + response_dump,)

Solution #2: no try/catch needed (if repr is too long sometimes is cut off...)

Extend and override Flask response object

import json
class JSONResponse(Response):

    def __repr__(self):
        headers = {}
        while len(self.headers) > 0:
            tuple_ = self.headers.popitem()
            headers[tuple_[0]] = tuple_[1]

        data = {
            'status': self.status,
            'headers': headers,
            'body': self.json
        }
        return json.dumps(data)

and

    @pytest.fixture(scope='session')
    def test_client(flask_app):
        flask_app.response_class = JSONResponse
        return flask_app.test_client()

Out of curiosity which do you prefer? First form seemed more ideal as you wouldn't change application code to make the test output useful but seems cleaner then having to wrap your tests. — David, Apr 22 '16 at 14:19

score 0 · Answer 3 · answered Apr 17 '20 at 14:27

0

I know this is an older question, but Pytest has an option --pdb that will pop you into a PDB shell should your test fail. Very handy way to "just look around" rather than having to pass tons of stuff to an exception message.

answered Apr 17 '20 at 14:27

vulpxn

731
1
6
14

Preserve response context testing flask app with pytest

3 Answers3