How can I simulate a ParameterNotFound boto3 exception in my unit test?

Question

I want to test some error handling logic, so I want to simulate a the specific exception type in my unit test. I am mocking the call to boto3, but I want to make that mock to raise a ParameterNotFound exception. The code I am testing follows this pattern:

boto3_client = boto3.client('ssm')
try:
    temp_var = boto3_client.get_parameter(Name="Something not found")['Parameter']['Value']
except boto3_client.exceptions.ParameterNotFound:
    ... [logic I want to test]

I have created a unittest mock, but I don't know how to make it raise the exception as this ParameterNotFound exception. I tried the following, but it doesn't work because it gets "exceptions must derive from the base class" when evaluating the except clause:

@patch('patching_config.boto3.client')
def test_sample(self, mock_boto3_client):
        mock_boto3_client.return_value = mock_boto3_client

        def get_parameter_side_effect(**kwargs):
            raise boto3.client.exceptions.ParameterNotFound()

        mock_boto3_client.get_parameter.side_effect = get_parameter_side_effect

How can I simulate a ParameterNotFound boto3 exception in my unit test?

This looks like the same issue as the last question I answered of yours. You are right that it's `side_effect`, but you need another mock object that's returned by the `client()` function. — jordanm, Jul 26 '19 at 21:53
Isn't this line doing that? mock_boto3_client.return_value = mock_boto3_client — Shawn, Jul 26 '19 at 22:02
Sorry, I missed that. I will see what's going on with your example — jordanm, Jul 26 '19 at 22:03
I tested your example and besides the fact that boto3 doesn't provide a `ParameterNotFound` exception, the example worked. Is there more going that's missing from this example? — jordanm, Jul 26 '19 at 22:11
I get `exceptions must derive from BaseException`, but I can see that the side_effect is working because I added a print statement to it. — jordanm, Jul 26 '19 at 22:16
It's hard to be sure from this fragment, but I have a feeling you're patching the wrong thing. If the first snippet is top-level, you need to do `patch('my_module.boto3_client')` — Norrius, Jul 26 '19 at 22:26

Shawn · Accepted Answer · 2021-07-23T19:17:03.870

I think the problem was my misunderstanding of how boto3 is raising exceptions. I found an explanation here: https://github.com/boto/boto3/issues/1262 under "Structure of a ClientError"

Structure of a ClientError

Within ClientError (but not BotoCoreError), there will be an operation_name attribute (should be a str) and a response attribute (should be a dict). The response attribute should have the following form (example from a malformed ec2.DescribeImages call):

and also here: https://codeday.me/en/qa/20190306/12210.html

{
    "Error": {
        "Code": "InvalidParameterValue",
        "Message": "The filter 'asdfasdf' is invalid"
    },
    "ResponseMetadata": {
        "RequestId": "aaaabbbb-cccc-dddd-eeee-ffff00001111",
        "HTTPStatusCode": 400,
        "HTTPHeaders": {
            "transfer-encoding": "chunked",
            "date": "Fri, 01 Jan 2100 00:00:00 GMT",
            "connection": "close",
            "server": "AmazonEC2"
        },
        "RetryAttempts": 0
    }
}

It sounds like the exception is thrown as a ClientError which has a ParameterNotFound error code, so I need to change things to

from botocore.exceptions import ClientError

and then

except ClientError as e:

and in the mocking I need to raise a ClientError instead which has the ParameterNotFound as the Code:

raise botocore.exceptions.ClientError({"Error": {"Code": "ParameterNotFound",
                                                 "Message": "Parameter was not found"}}, 
                                      'get_parameter')

My experience differs from what is listed in the following answers: https://stackoverflow.com/a/50958676/230055 and https://stackoverflow.com/a/48956137/230055. When I tried `except boto3_client.exceptions.ParameterNotFound` I got "exceptions must derive from the base class," so I question whether those other answers are really still valid. Ultimately, what I listed in this answer above is what worked for me. — Shawn, Jul 30 '19 at 16:06

How can I simulate a ParameterNotFound boto3 exception in my unit test?

1 Answers1