Using Pytest and Mock to test view that queries database

Question

I'm trying to write a unittest for a view that my Django app uses. The view itself takes data from the database by way of a custom model (code snippet of the view below).

views.py

def calibrator_data(calid,code):
    data = []
    sources, times = zip(*DataSource.objects.filter(event__name=code).values_list('id','timestamp').order_by('timestamp'))
    points  = Datapoint.objects.filter(data__in=sources)
    people = Decision.objects.filter(source__id=calid,planet__name=code,value='D',current=True).values_list('person__username',flat=True).distinct()
    norm = dict((key,0) for key in sources)
    for pid in people:
        cal = []
        sc = dict(points.filter(user__username=pid,pointtype='S').values_list('data__id','value'))
        bg = dict(points.filter(user__username=pid,pointtype='B').values_list('data__id','value'))
        c = dict(points.filter(user__username=pid,pointtype='C',coorder__source__id=calid).values_list('data__id','value'))
        sc_norm = dict(norm.items() + sc.items())
        bg_norm = dict(norm.items() + bg.items())
        c_norm = dict(norm.items() + c.items())
        for v in sources:
            try:
                cal.append((sc_norm[v]- bg_norm[v])/(c_norm[v] - bg_norm[v]))
            except:
                cal.append(0)
        data.append(cal)
    return data,[timegm(s.timetuple())+1e-6*s.microsecond for s in times],list(people)

And the test that I've attempted to write.

test_reduc.py

pytestmark = pytest.mark.django_db

@pytest.mark.django_db
class TestDataReduction(TestCase):

    pytestmark = pytest.mark.django_db

     ################################################################################
     ############################ Testing calibrator_data ###########################
     ################################################################################

    def test_calibrator_data(self):

        mock_source = MagicMock(spec=DataSource)
        mock_times = MagicMock(spec=DataSource)
        mock_source.return_value = array([random.randint(0,10)])
        mock_times.return_value = datetime.now()

        mock_points = MagicMock(spec=Datapoint)
        mock_points.user = []

        mock_people = MagicMock(spec=Decision)
        mock_people.data = []

        calid = 22
        code = 'corot2b'

        self.output = calibrator_data(calid,code)

        assert type(self.output[0])==type([])

The test keeps failing with the error:

    =============================================== test session starts ===============================================
platform darwin -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.2
rootdir: /Users/tomasjames/Documents/citsciportal/app, inifile: pytest.ini
plugins: django
collected 1 items 

agentex/tests/test_reduc.py F

==================================================== FAILURES =====================================================
_____________________________________ TestDataReduction.test_calibrator_data ______________________________________

self = <agentex.tests.test_reduc.TestDataReduction testMethod=test_calibrator_data>

    def test_calibrator_data(self):

        mock_source = MagicMock(spec=DataSource)
        mock_times = MagicMock(spec=DataSource)
        mock_source.return_value = array([random.randint(0,10)])
        mock_times.return_value = datetime.now()

        mock_points = MagicMock(spec=Datapoint)
        mock_points.user = []

        mock_people = MagicMock(spec=Decision)
        mock_people.data = []

        calid = 22
        code = 'corot2b'

>       self.output = calibrator_data(calid,code)

agentex/tests/test_reduc.py:51: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

calid = 22, code = 'corot2b'

    def calibrator_data(calid,code):
        data = []
>       sources, times = zip(*DataSource.objects.filter(event__name=code).values_list('id','timestamp').order_by('timestamp'))
E       ValueError: need more than 0 values to unpack

agentex/datareduc.py:56: ValueError
============================================ 1 failed in 7.69 seconds =============================================

This is my first ever attempt at writing any sort of test (as you can probably see) and it's a challenging one. I think the error is arising because views.py is still trying to access the database in the test environment (which runs with a blank database) - the timings would seem to confirm this. My attempts at mocking the variables sources, times, points and people doesn't seem to have worked however. I've attempted to assign them to the variables I know the database query yields to save having to mock the entire database/QuerySet.

Is this the incorrect way of implementing the test? I can't spot where I'm going wrong.

Thanks in advance!

score 2 · Accepted Answer · edited May 23 '17 at 11:58

You've missed one key component of using mock to override methods. You need to use mock as a method decorator to basically monkey match your method to be able to do what you would like.

You'll want to write something that looks like this. (Note: Haven't tested this at all, but should lead you in the right direction).

@pytest.mark.django_db
class TestDataReduction(TestCase):

    @mock.patch(your_module.xyz.DataSource)
    @mock.patch(your_module.xyz.Datapoint)
    @mock.patch(your_module.xyz.Decision)
    def test_calibrator_data(self, mock_source, mock_points,
                             mock_people):

        mock_source.objects.filter.return_value.values.return_value.order_by.return_value = [array([random.randint(0,10)]), datetime.now()]

        mock_points.objects.filter.return_value = []

        mock_people.objects.filter.values_list.return_value.distinct.return_value = []

        calid = 22
        code = 'corot2b'

        self.output = calibrator_data(calid,code)

        assert type(self.output[0])==type([])

You are also going to need to mock out whatever you want the return values to be your multiple calls to points.filter. One way to do this would be by using side effects. There is a pretty good example here: https://stackoverflow.com/a/7665754/2022511

In addition to my post that you've already looked at (https://www.chicagodjango.com/blog/quick-introduction-mock/), there is more information about using mock.patch in this blog post: http://fgimian.github.io/blog/2014/04/10/using-the-python-mock-library-to-fake-regular-functions-during-tests/

Using Pytest and Mock to test view that queries database

1 Answers1