2

I have a UserRepository class that I am writing unittests for

class UserRepository(object):
    """
    Repository that handles storage and retrieval of models.User objects
    in and from the datastore.

    """
    def create(self, user):
        """
        Create the given user in the datastore if it doesn't exist yet.

        Args:
            user: The user to create.

        Returns:
            The created user.

        Raises:
            exc.DuplicateEntity: If the desired phonenumber is
                already taken.

        """
        duplicate_user = models.User.query(models.User.phonenumber == user.phonenumber).fetch()
        if duplicate_user:
            raise exc.DuplicateEntity()

        user.put()
        return user

I have these tests for it

class UserServiceTest(unittest.TestCase):
    """Tests for the UserService."""
    def setUp(self):
        """
        Called before tests are run.

        """
        self.user_repo = repositories.UserRepository()
        #self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)

    def test_create(self):
        """
        Test if the create method creates a user.

        """
        ndb.delete_multi(models.User.query().fetch(keys_only=True))

        user = models.User(phonenumber='+31612345678#',
                           email='tim@castelijns.nl',
                           password='1234abcd')

        ret_user = self.user_repo.create(user)
        self.assertEqual(ret_user, user)

    def test_create_duplicate_fails(self):
        """
        Test if attempting to create a user with an existing phonenumber
        fails.

        """
        ndb.delete_multi(models.User.query().fetch(keys_only=True))

        user = models.User(phonenumber='+31612345678#',
                           email='tim@castelijns.nl',
                           password='1234abcd')

        self.user_repo.create(user)

        with self.assertRaises(exc.DuplicateEntity):
            self.user_repo.create(user)

The ndb.delete_multi(models.User.query().fetch(keys_only=True)) is to clear existing users from the test environment so test cases don't have influence on one another.

This is the custom exception

class DuplicateEntity(Exception):
    """Exception to raise when trying to create a duplicate entity."""

I run the tests with

$ nosetests --with-gae

It outputs

======================================================================
FAIL: Test if attempting to create a user with an existing phonenumber
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tests/dal/test_repositories.py", line 53, in test_create_duplicate_fails
    self.user_repo.create(user)
AssertionError: DuplicateEntity not raised

----------------------------------------------------------------------
Ran 2 tests in 0.080s

FAILED (failures=1)

Which is unexpected, because the 2nd call to .create here should raise the exception since there already is a user with that phonenumber.

I'm sure the code works, because I've tested it live.

What's also weird is that if I add a call to .create above the with statement, it does raise the exception:

self.user_repo.create(user)
self.user_repo.create(user)

with self.assertRaises(exc.DuplicateEntity):
    self.user_repo.create(user)

So it's raised on the 3rd call, but not the 2nd.

I have a feeling it's related to the datastore consistency policy, as documented here:

The PseudoRandomHRConsistencyPolicy class lets you control the likelihood of a write applying before each global (non-ancestor) query. By setting the probability to 0%, we are instructing the datastore stub to operate with the maximum amount of eventual consistency. Maximum eventual consistency means writes will commit but always fail to apply, so global (non-ancestor) queries will consistently fail to see changes.

however I don't know how nosegae handles this. Is it even configurable? nosegae doesn't have alot of documentation.

How can I work around (or fix) this?

Dan McGrath
  • 41,220
  • 11
  • 99
  • 130
Tim
  • 41,901
  • 18
  • 127
  • 145
  • Setting up the datastore stub with a different consistency-policy is nothing to do with whether you use nosegae or not - the code to set that up should go in your test base-class. – Greg Dec 30 '14 at 16:59

1 Answers1

2

Your problem is you are using a query to test for duplicates and that isn't guaranteed to work due to eventual consistency. You will note that the tests in the document you referred to all use ancestor queries which do guarantee consistency.

My take is this, it is expected and correct behavior (the AssertionError: DuplicateEntity not raised error) and highlights a problem with you model/approach.

Dave W. Smith
  • 24,318
  • 4
  • 40
  • 46
Tim Hoffman
  • 12,976
  • 1
  • 17
  • 29
  • Do you have any insights as to why the exception is raised on the 3rd call? Is it that when the 3rd call is made, enough time has passed for the put() in the 1st call to have been processed? – Tim Dec 30 '14 at 13:19
  • You can set the consistency to 100% on gae's 'builtin' testbed to make it work during unittests, do you know if that can be done using nosegae? – Tim Dec 30 '14 at 13:21
  • Setting consistency to 100% makes the symptom go away, but leaves the underlying problem. – Dave W. Smith Dec 30 '14 at 19:49
  • Yeah, I wasn't going to suggest setting consistency policy to make the problem go away ;-) It would still be there . – Tim Hoffman Dec 30 '14 at 22:44