10

I would like to write some testcase to exercise object_check in isinstance(obj, requests.Response) logic. After I create Mock data as return value for requests.post. The type for mock data is always be Mock class. In that way, how can I rewrite mock data so mock data can be type of requests.Response? so I can exercise line d = obj.json()?

from unittest.mock import patch, Mock
import unittest
import requests
from requests.exceptions import HTTPError
import pytest
def object_check(obj):
    if isinstance(obj, bytes):
        d = ujson.decode(obj.decode())
    elif isinstance(obj, requests.Response):
        d = obj.json()
    else:
        raise ValueError('invalid type')
    return d

def service_post(params):
    """
    trivial function that does a GET request
    against google, checks the status of the
    result and returns the raw content
    """
    url = "https://www.iamdomain.com"
    params = {'number': 1234, 'user_id': 1, 'name': 'john'}
    resp = requests.post(url, data=params)
    return object_check(resp)

@patch.object(requests, 'post')
def test_service_post(mock_request_post):
    data = {'number': 0000, 'user_id': 0, 'name': 'john'}
    def res():
        r = Mock()
        r.status_code.return_value = 200
        r.json.return_value = data
        return r
    mock_request_post.return_value = res()
    assert data == service_post(data)
hoefling
  • 59,418
  • 12
  • 147
  • 194
jacobcan118
  • 7,797
  • 12
  • 50
  • 95

3 Answers3

20

You could do this:

@patch.object(requests, 'post')
def test_service_post(mock_request_post):
    data = {'number': 0000, 'user_id': 0, 'name': 'john'}
    def res():
        r = requests.Response()
        r.status_code = 200
        def json_func():
            return data
        r.json = json_func
        return r
    mock_request_post.return_value = res()
    assert data == service_post(data)

Test then passed for me when I ran it locally. Be aware that Mock is a mini-smell.

I used to be a big fan of Mock. As I've grown as a dev, though, I really try to avoid it. It can trick you into some really bad design, and they can be really hard to maintain (especially since you're modifying your Mock to hold return values). Mock can also create a false sense of security (your test will continue to pass even if the web services changes dramatically, so you might explode in prod). I don't think you really need it here. Two alternatives:

  1. You could hit whatever service you're trying to hit, and serialize (save) that response out with pickle, and store to disk (save it in your test suite). Then have your unit test read it back in and use the actual response object. You'd still have to patch over requests.post, but at least the return values will be lined up for you and you won't have to add or modify them as your needs/application grows.
  2. Just hit the web. Forget the patch entirely: just do the POST in your test and check the response. Of course, this might be slow, and will only work if you have internet. And you'll get goofy purists who will tell you to never to do this in a unit test. Maybe move it to an integration test if you run into one of those puristy people. But seriously, there's no substitute for doing what you're actually going to do in prod. The upside to doing this is that if the web service changes, then you'll know about it right away and can fix your code. Downside is it can slow down your test suite, and it's a potentially unreliable test (if the webservice is down, your test will fail...but it might actually be good to know that).

I recommend if the webservice is unstable (i.e liable to change), use option 2. Else, use option 1. Or do some combination of both (Mock and patch for a unit test, and hit the service on an integration test). Only you can decide!

HTH, good luck!

Matt Messersmith
  • 12,939
  • 6
  • 51
  • 52
  • thank you..it is great suggestion. for option 1, what is the best way for me "serialize (save) that response out with pickle"? like your code example? – jacobcan118 Aug 05 '18 at 03:58
  • 3
    `pickle.dump(your_object, your_file_descriptor)` and `pickle.load("your_filename")`. You can see more here: https://stackoverflow.com/questions/4530611/saving-and-loading-objects-and-using-pickle and the pickle docs are here: https://docs.python.org/3/library/pickle.html. You'll want to actually hit the webservice, and then insert the dump logic right after `elif isinstance(obj, requests.Response):`. Similarly, when you do the test, you read that `requests.Response` object back in from disk and pass it around everywhere you want to run unit tests that need it as a dependency – Matt Messersmith Aug 06 '18 at 13:18
  • 1
    Excellent! Glad to have helped. – Matt Messersmith Aug 06 '18 at 21:31
  • 2
    Hearing someone point out how silly it is to mock something that you should be checking before it gets to prod is really encouranging. I feel like I'm the crazy one for thinking that Live API calls in testing functions are ok. – Dave Liu May 13 '19 at 23:19
  • @DaveLiu no, you are the sane one, I like Matt had the misfortune to work with code that was mocked from toes to the top of the head. Then if you rewrite 90% of code, the only fix for tests to make them pass is to update mock paths, nevermind tests will make no sense from now on. – Drachenfels Dec 07 '20 at 18:47
  • 2
    @MattMessersmith I know you gave this solution like 3 years ago man, but I wanted to thank you. I had been struggling with this for like 11 hours when I came across this post. You saved my bacon. Thank you – Derek1st Nov 07 '22 at 15:22
3

If you want to mock the text or content @property value use PropertyMock around the text

@patch.object(requests, 'post')
def test_service_post(mock_request_post):
    data = {'number': 0000, 'user_id': 0, 'name': 'john'}
    def res():
        r = requests.Response()
        r.status_code = 200
        type(r).text = mock.PropertyMock(return_value=my_text)  # property mock
        def json_func():
            return data
        r.json = json_func
        return r
    mock_request_post.return_value = res()
    assert data == service_post(data)
drt
  • 735
  • 6
  • 16
  • I downvote this because it produces buggy behavior. Using this approach makes any other test that uses request to generate this mocked `my_text`. I implemented this solution and a month later I am spending 3 hours trying to debug my code till I found that this was the problem. – Iván Aug 10 '22 at 18:42
2

Use the spec argument when instantiating the mock:

>>> from unittest.mock import Mock
>>> from requests import Response
>>> m = Mock(spec=Response)
>>> m.__class__
requests.models.Response
>>> isinstance(m, Response)
True

Also note that r.status_code.return_value = 200 will not work with speccing; set the value directly instead:

r.status_code = 200
hoefling
  • 59,418
  • 12
  • 147
  • 194
  • but `isinstance(m, request.Response)` == False? that is my question – jacobcan118 Aug 05 '18 at 04:00
  • I'm not sure what your question is. There is no such class `requests.Response` (apart from `request.Response` you wrote; I assume this is only a typo). There is a class `requests.models.Response` which is **exported** in `requests.__init__.py` via `__all__`. So `isinstance(m, requests.Response)` will return `True`. – hoefling Aug 05 '18 at 05:39