How to properly patch boto3 calls in unit test

Question

I'm new to Python unit testing, and I want to mock calls to the boto3 3rd party library. Here's my stripped down code:

real_code.py:

import boto3

def temp_get_variable(var_name):
  return boto3.client('ssm').get_parameter(Name=var_name)['Parameter']['Value']

test_real_code.py:

import unittest
from datetime import datetime
from unittest.mock import patch

import real_code

class TestRealCode(unittest.TestCase):

    @patch('patching_config.boto3.client')
    def test_get_variable(self, mock_boto_client):

        response = {
            'Parameter': {
                'Name': 'MyTestParameterName',
                'Type': 'String',
                'Value': 'myValue',
                'Version': 123,
                'Selector': 'asdf',
                'SourceResult': 'asdf',
                'LastModifiedDate': datetime(2019, 7, 16),
                'ARN': 'asdf'
            }
        }

        mock_boto_client.get_variable.return_value = response

        result_value = real_code.get_variable("MyTestParameterName")

        self.assertEqual("myValue", result_value)

When I run it the test fails with

Expected :myValue
Actual   :<MagicMock name='client().get_parameter().__getitem__().__getitem__()' id='2040071816528'>

What am I doing wrong? I thought by setting mock_boto_client.get_variable.return_value = response it would mock out the call and return my canned response instead. I don't understand why I am getting a MagicMock object instead of the return value I tried to set. I'd like to set up my test so that when the call to get_parameter is made with specific parameters, the mock returns the canned response I specified in the test.

I would recommend checking out moto: https://github.com/spulec/moto — jordanm, Jul 16 '19 at 22:32

score 8 · Accepted Answer · edited Jul 16 '19 at 23:48

8

There are two issues with your test code. The first is that when your mock object mock_boto_client called, it returns a new mock object. This means that the object that get_parameter() is being called on is different than the one you are attempting to set a return value on. You can have it return itself with the following:

mock_boto_client.return_value = mock_boto_client

You can also use a different mock object:

foo = MagicMock()
mock_boto_client.return_value = foo

The second issue that you have is that you are mocking the wrong method call. mock_boto_client.get_variable.return_value should be mock_boto_client.get_parameter.return_value. Here is the test updated and working:

import unittest
from datetime import datetime
from unittest.mock import patch

import real_code

class TestRealCode(unittest.TestCase):

    @patch('boto3.client')
    def test_get_variable(self, mock_boto_client):

        response = {
            'Parameter': {
                'Name': 'MyTestParameterName',
                'Type': 'String',
                'Value': 'myValue',
                'Version': 123,
                'Selector': 'asdf',
                'SourceResult': 'asdf',
                'LastModifiedDate': datetime(2019, 7, 16),
                'ARN': 'asdf'
            }
        }

        mock_boto_client.return_value = mock_boto_client
        mock_boto_client.get_parameter.return_value = response

        result_value = real_code.get_variable("MyTestParameterName")

        self.assertEqual("myValue", result_value)

edited Jul 16 '19 at 23:48

Shawn

8,374
5
37
60

answered Jul 16 '19 at 22:31

jordanm

33,009
7
61
76

I think I understand the explanation for the first issue. So the @patch line is essentially just replacing the "client" member of the boto3 object with the MagicMock object referenced by the mock_boto_client variable...which means I still need to set up what happens when that MagicMock object is invoked. Is that the correct understanding? – Shawn Jul 17 '19 at 00:03
@Shawn Close - it's replacing the `client` *function* of the boto3 *module*. By default mock creates a new mock object for all return values that are not explicitly set. – jordanm Jul 17 '19 at 12:47
@jordanm I'm having a hard time understanding what you mean. I can see that the client function in the boto3 client is called by the get_parameter method. But I don't see how `mock_boto_client.get_parameter.return_value = response` does this. Can you share anymore insight into this please? – Preet Sangha Aug 25 '23 at 00:27
1

@PreetSangha That's the magic that the `mock` module does. Setting that return_value attribute tells `mock` to return the assigned value when the method `get_parameter` is called on the mock object. See the relevant parts of the doc on return value here: https://docs.python.org/3/library/unittest.mock.html#the-mock-class – jordanm Aug 25 '23 at 01:16
@jordanm Thank you. So to reiterate (line 1) boto3.client serves up the mock entity which returns itself (a mockable entity) when anything is called on it. We then (line 2) attach a fake result to the mocked method get_parameter. – Preet Sangha Aug 25 '23 at 23:39
1

@PreetSangha That's correct. – jordanm Aug 26 '23 at 02:15

How to properly patch boto3 calls in unit test

1 Answers1

Linked