mocking of url for unittesting

Question

I want to mock urllib.urlopen for creating unit test

def url_test(self):
  response = urllib2.urlopen(test_url)
  body = response.read()
  if body:
    return body.split(':')[0]

config.py

 test_url = "localhost"

I want to mock the url_test() function but I do not understand how to mock the value of test_url. Because when I am trying to unit test the function it says me "connection refused"

this is what i tried.

@patch('urllib.urlopen')
def url_test(self, m_url):
  m_response = m_url.return_value
  m_response.read.return_value = 'Some body value:you wanted to return'
  self.assertTrue(url_test(), "Failed")
  m_url.assert_called_with('localhost')

You don't mock `test_url`, you mock `urllib2.urlopen` and assert that it gets *called* with `test_url`. — jonrsharpe, Jun 16 '16 at 12:13
Try getting hints from [Mocking urllib2.urlopen().read() for different responses](http://stackoverflow.com/questions/19203627/mocking-urllib2-urlopen-read-for-different-responses) and a search on the terms `mock urllib2.urlopen` and I suggest also to understand the comment from @jonrsharpe as what you really want to mock ;-) — Dilettant, Jun 16 '16 at 12:18

Martijn Pieters · Accepted Answer · 2016-06-19T08:57:51.037

2

You would mock any external system, which here is urllib2. Assuming you are using the unittest.mock library (backported to Python 2 as the mock project):

with mock.patch('urllib2.urlopen') as urlopen_mock:
    mock_response = urlopen_mock.return_value
    mock_response.read.return_value = 'Some body value:you wanted to return'

    # call the method being tested
    result = someobject.url_test()

    # make assertion about the return value and that the code tried to use 
    # a specific URL
    urlopen_mock.assert_called_with('localhost')
    self.assertEqual(result, 'Some body value')

In your update you mock the wrong location:

@patch('urllib.urlopen')

Your code uses urllib2, not urllib.

edited Jun 19 '16 at 08:57

answered Jun 16 '16 at 12:14

Martijn Pieters

1,048,767
296
4,058
3,343

it gives me connection refused error when I am calling the method to be tested ! – sam Jun 16 '16 at 12:22
@sam: then you are not mocking the right place or not at the right time. Are you sure you have a mock in place for `urllib2.urlopen()` when that error happens? – Martijn Pieters Jun 16 '16 at 12:23
when the actual method to be tested is getting called, its calling localhost and not mocked response is retrieved. :( – sam Jun 16 '16 at 12:25
@sam: right, so the mock is not in place. There are 3 possibilities here: Something *else* is calling that method at some other point when there is no mock in place, you haven't understood how I used `with` here to use `mock.patch()` as a context manager and you didn't keep the indentation (so the mock is no longer in place by the time the method is called) or the wrong place was patched (e.g. you are using a wrapper around `urllib2.url_open` in a submodule of a package that just happens to also be called `urllib2`). – Martijn Pieters Jun 16 '16 at 12:28
@sam: I can't see your dev setup, I don't have your codebase. The answer I posted here works (I use this in production regularly), but I can't help you debug why your setup is different. Sorry. – Martijn Pieters Jun 16 '16 at 12:29
Hi @Martijn: I have updated my question, what I am trying to do by understanding ur answer. but still getting the error. – sam Jun 16 '16 at 16:09
@sam: you are essentially doing the same thing as what I proposed in my answer. I don't see anything out of the ordinary there. – Martijn Pieters Jun 16 '16 at 19:26
result gives me mock object and not the value – sam Jun 19 '16 at 05:00
1

@sam: In your question, you mocked `urllib.urlopen`, not `urllib2.urlopen`, by the way (only noticed that now). If you corrected that and still see a mock being returned, what is the `repr()` value of that mock object? It tells me what path was taken to get it, and that helps me diagnose what may be wrong. – Martijn Pieters Jun 19 '16 at 08:56

score 0 · Answer 2 · answered Jun 16 '16 at 16:31

For mocking web requests in Python, I highly recommend HTTPretty.

A simple example for what you want to do would look something like:

@httpretty.activate
def test_url():
    httpretty.register_uri(httpretty.GET, test_url,
                           body='some body value:some other value',
                           content_type="text/plain")

    self.assertEqual(url_test(), 'some_body_value')

There are a lot of complicated things and gotchas that come with mocking URL requests, and HTTPretty does a pretty good job of making them happen behind the scenes.

In terms of your function, consider making test_url a parameter of the method instead of a global variable - it makes testing significantly easier.

mocking of url for unittesting

2 Answers2