Moto does not appear to be mocking aws interactions in a pytest

Question

Say I want to mock the following:

session = boto3.Session(profile_name=profile)
resource = session.resource('iam')
iam_users = resource.users.all()
policies = resource.policies.filter(Scope='AWS', OnlyAttached=True, PolicyUsageFilter='PermissionsPolicy')

How do I go about starting to mock this with in pytest? I could create mocked objects by creating a dummy class and the necessary attributes, but I suspect that's the wrong approach.

Some additional details, here's what I'm trying to test out:

def test_check_aws_profile(self, mocker):
    mocked_boto3 = mocker.patch('myapp.services.utils.boto3.Session')
    mocker.patch(mocked_boto3.client.get_caller_identity.get, return_value='foo-account-id')
    assert 'foo-account-id' == my_func('foo')

#in myapp.services.utils.py
def my_func(profile):
    session = boto3.Session(profile_name=profile)
    client = session.client('sts')
    aws_account_number = client.get_caller_identity().get('Account')
    return aws_account_number

But I can't quite seem to be able to get this patched correctly. I'm trying to make it so that I can patch session and the function calls in that method

I tried using moto and got this:

@mock_sts
def test_check_aws_profile(self):
    session = boto3.Session(profile_name='foo')
    client = session.client('sts')
    client.get_caller_identity().get('Account')

But I'm running into

>           raise ProfileNotFound(profile=profile_name)
E           botocore.exceptions.ProfileNotFound: The config profile (foo) could not be found

So it seems like it's not mocking anything :|

Edit:

Turns out you need to have the mocked credentials in a config and credentials file for this to work.

Answer 1

If you want to use moto, you can use the AWS_SHARED_CREDENTIALS_FILE environment variable, to point it to a dummy credentials file which can be kept in the tests folder. You can define your profiles there. Example:

Files: test_stuff.py. dummy_aws_credentials

test_stuff.py:

import os
from pathlib import Path
import boto3
import pytest
from moto import mock_sts


@pytest.fixture(scope='module')
def aws_credentials():
    """Mocked AWS Credentials for moto."""
    moto_credentials_file_path = Path(__file__).parent.absolute() / 'dummy_aws_credentials'
    os.environ['AWS_SHARED_CREDENTIALS_FILE'] = str(moto_credentials_file_path)


@mock_sts
def test_check_aws_profile(aws_credentials):
    session = boto3.Session(profile_name='foo')
    client = session.client('sts')
    client.get_caller_identity().get('Account')

dummy_aws_credentials:

[foo]
aws_access_key_id = mock
aws_secret_access_key = mock

Answer 2

I'm not sure what exactly you want, so I'll give you something to start.

You let unittest.mock to mock everything for you, for example. (Useful reading: https://docs.python.org/3/library/unittest.mock.html)

module.py:

import boto3

def function():
    session = boto3.Session(profile_name="foobar")
    client = session.resource("sts")
    return client.get_caller_identity().get('Account')

test_module.py:

from unittest.mock import patch

import module

@patch("module.boto3")  # this creates mock which is passed to test below
def test_function(mocked_boto):
    # mocks below are magically created by unittest.mock when they are accessed
    mocked_session = mocked_boto.Session()
    mocked_client = mocked_session.resource()
    mocked_identity = mocked_client.get_caller_identity()

    # now mock the return value of .get()
    mocked_identity.get.return_value = "foo-bar-baz"

    result = module.function()
    assert result == "foo-bar-baz"

    # we can make sure mocks were called properly, for example
    mocked_identity.get.assert_called_once_with("Account")

Results of pytest run:

$ pytest
================================ test session starts ================================
platform darwin -- Python 3.7.6, pytest-5.3.2, py-1.8.1, pluggy-0.13.1
rootdir: /private/tmp/one
collected 1 item                                                                    

test_module.py .                                                              [100%]

================================= 1 passed in 0.09s =================================

I would also recommend to install pytest-socket and run pytest --disable-socket to make sure your tests do not talk with network by accident.

Answer 3

Although there is nothing wrong with manually patching boto using mock.patch, you could also consider using a higher level testing utility like moto.