using wsgiref.simple_server in unittests

Question

I have some functions like this one:


URL = 'http://localhost:8080'
def func():
    response = urlopen(URL)
    return process(response)

And i want to test it with unittest.

I did something like this:


from wsgiref.simple_server import make_server
def app_200_hello(environ,start_response):
    stdout = StringIO('Hello world')
    start_response("200 OK", [('Content-Type','text/plain')])
    return [stdout.getvalue()]

s = make_server('localhost', 8080, app_200_hello)

class TestFunc(unittest.TestCase):
    def setUp(self):
        s.handle_request()

    def test1(self):
        r = func()
        assert r, something

if __name__ == '__main__':
    unittest.main()

At setUp() my tests stop because s.handle_request() waits for request. How i can get around that? Run s.handle_request() in another thread? or maybe there is another solution?

EDIT: I want to test "func" function, not "app_200_hello"

Ali Afshar · Answer 1 · 2009-03-11T13:55:06.707

7

If you are testing a WSGI application, I can strongly recommend werkzeug.test which gets around these issues by testing the application itself without a server:

from werkzeug.test import Client

# then in your test case
def test1(self):
    client = Client(app_200_hello)
    appiter, status, headers = client.open()
    assert ''.join(appiter) == 'Hello World'
    assert status == '200 OK'

This approach just removes the need for a WSGI server altogether.

Of course if you did want to start a server, you would have to use a separate thread or a process, but then you have to make sure you stop it afterwards. However, it strikes me that the only time you would want to test with a real server is for production integration testing, in which case your server won't be wsgiref, it will be a real server that you probably won't have to start-stop like this.

edited Mar 11 '09 at 13:55

answered Mar 11 '09 at 13:44

Ali Afshar

40,967
12
95
109

Thanks. It is look like what i want. But werkzeug is too big. Is Client class http://dev.pocoo.org/projects/werkzeug/browser/werkzeug/test.py#L551 independent from other part of werkzeug package? – Mykola Kharechko Mar 11 '09 at 14:12
No it's not independent, but it is not a big dependency for development only. In fact it's tiny. – Ali Afshar Mar 11 '09 at 14:21
Oh. I tangled. I don't want to test app_200_hello. I want to test "func" – Mykola Kharechko Mar 11 '09 at 14:23

score 5 · Answer 2 · answered Jul 03 '11 at 17:49

Use multiprocessing to start the server in a separate process

in setUp do something like:

self.port = 8000
server = make_server('', self.port, make_my_wsgi_ap())
self.server_process = multiprocessing.Process(target=server.serve_forever)
self.server_process.start()

then in tearDown do:

self.server_process.terminate()
self.server_process.join()
del(self.server_process)

I've found that if you don't explicitly put the del() in there then subsequent server instances may have a problem with the port already being used.

This is the best way to do it IMO. – vangheem May 22 '15 at 14:43 — vangheem, May 22 '15 at 14:43

score 4 · Answer 3 · answered Mar 11 '09 at 13:59

4

Your server must be a separate process.

You'll want to start it with subprocess.Popen()

If you're using Python 2.6, you can then kill the subprocess during tearDown.

def setUp( self ):
    self.server= subprocess.Popen( "python","myserver","etc." )
def tearDown( self ):
    self.server.kill()

If you're not using Python 2.6, killing the server can be unpleasant.

answered Mar 11 '09 at 13:59

S.Lott

384,516
81
508
779

I don't think it is a good idea. As for me it will be very hard for system to start/stop python after each test. And i don't want to write for tests some complex wsgi application. I want to write many simple wsgi applications for many situations. – Mykola Kharechko Mar 11 '09 at 14:07
Since tearDown stops the WSGI server, Python ends normally. Very easy for system to stop Python. – S.Lott Mar 11 '09 at 14:29
wsgiref.simple_server must be started as a separate process to be useful. – S.Lott Mar 11 '09 at 14:36
+1 Example given in question [Why does wsgiref.simple_server report content-type of request as 'text/plain' while none was sent?](http://stackoverflow.com/questions/5924166/) – Piotr Dobrogost May 07 '11 at 21:44

score 2 · Answer 4 · answered Mar 11 '09 at 16:28

You could also provide a mock version of urlopen that doesn't actually run the server.

Assuming your original code was in mycode.py, in your test code you'd do something like:



import mycode

class TestFunc(unittest.TestCase):
    def setUp(self):
        # patch mycode, to use mock version of urlopen
        self._original_urlopen = mycode.urlopen
        mycode.urlopen=self.mock_urlopen

    def tearDown(self):
        # unpatch urlopen
        mycode.urlopen=self._original_urlopen

    def mock_urlopen(self,url):
        # return whatever data you need urlopen to return

    def test1(self):
        r = func()
        assert r, something

if __name__ == '__main__':
    unittest.main()

This is known as "monkey patching" which some frown upon, but for testing your code it can make life a lot easier, as you don't need to change your original code to make it testable.

score 0 · Answer 5 · answered Mar 11 '09 at 14:58

My solution:


URL = 'http://localhost:8085'
def func():
    response = urlopen(URL)
    return response.read()

import unittest
from wsgiref.simple_server import WSGIServer, WSGIRequestHandler
import threading
from urllib2 import urlopen
from cStringIO import StringIO

def app_200_hello(environ,start_response):
    stdout = StringIO('Hello world')
    start_response("200 OK", [('Content-Type','text/plain')])
    return [stdout.getvalue()]

server = WSGIServer(('localhost', 8085), WSGIRequestHandler)
server.set_app(app_200_hello)

t = threading.Thread(target=server.serve_forever)
t.start()

class TestFunc(unittest.TestCase):
    def setUp(self):
        pass

    def test1(self):
        r = func()
        self.assertEqual(r, 'Hello world')

    def __del__(self):
        server.shutdown()

if __name__ == '__main__':
    unittest.main()

I start "server" in another thread and shutdown it at TestFunc destructor.

Aren't you worried about leaving the server in a bad state after each test, and that affecting the tests? How would you test cookies for example. If you have to do this, please consider using processes, and doing it per-request. — Ali Afshar, Mar 11 '09 at 15:45

using wsgiref.simple_server in unittests

5 Answers5