How do I confirm entities are saved with GAE's Eventual Consistency?

Question

I'm trying to create tests to verify that my entities are being saved in the database. When I put breakpoints in the post function, I can see that the customer count changes after the record is saved. I read https://cloud.google.com/appengine/docs/python/tools/localunittesting#Python_Writing_High_Replication_Datastore_tests

From what I understood, the tests were failing because of Eventual Consistency and the way to get around that was to change the PseudoRandomHRConsistencyPolicy settings.

policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)

And when I ran the test again I got the same error.

What am I doing wrong with creating these tests?

> /Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py(137)post()  
-> customer.put()  
(Pdb) l  
134             query = Customer.query()  
135             orig_customer_count = query.count()  
136             import pdb; pdb.set_trace()  
137  ->         customer.put()  
138             import pdb; pdb.set_trace()  
139             query_params = {'leadbook_name': leadbook_name}  
140             self.redirect('/?' + urllib.urlencode(query_params))  
141       
142     config = {}  
(Pdb) orig_customer_count  
5  
(Pdb) c  
> /Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py(139)post()  
-> query_params = {'leadbook_name': leadbook_name}  
(Pdb) l  
134             query = Customer.query()  
135             orig_customer_count = query.count()  
136             import pdb; pdb.set_trace()  
137             customer.put()  
138             import pdb; pdb.set_trace()  
139  ->         query_params = {'leadbook_name': leadbook_name}  
140             self.redirect('/?' + urllib.urlencode(query_params))  
141       
142     config = {}  
143     config['webapp2_extras.sessions'] = {  
144         'secret_key': 'my-super-secret-key',  
(Pdb) query.count()  
6

The entities also show up in the Datastore Viewer.

However, my test keeps failing.

$ nosetests --with-gae  
F  
======================================================================  
FAIL: test_guest_can_submit_contact_info (dermalfillersecrets.functional_tests.NewVisitorTest)  
----------------------------------------------------------------------  
Traceback (most recent call last):  
  File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/functional_tests.py", line 80, in test_guest_can_submit_contact_info  
    self.assertNotEqual(orig_custs, query.count())  
AssertionError: 0 == 0

This is the functional_tests.py file contents:

import os, sys  
sys.path.append("/usr/local/google_appengine")  
sys.path.append("/usr/local/google_appengine/lib/yaml/lib")  
sys.path.append("/usr/local/google_appengine/lib/webapp2-2.5.2")  
sys.path.append("/usr/local/google_appengine/lib/django-1.5")  
sys.path.append("/usr/local/google_appengine/lib/cherrypy")  
sys.path.append("/usr/local/google_appengine/lib/concurrent")  
sys.path.append("/usr/local/google_appengine/lib/docker")  
sys.path.append("/usr/local/google_appengine/lib/requests")  
sys.path.append("/usr/local/google_appengine/lib/websocket")  
sys.path.append("/usr/local/google_appengine/lib/fancy_urllib")  
sys.path.append("/usr/local/google_appengine/lib/antlr3")  

import unittest  
from selenium import webdriver  
from google.appengine.api import memcache  
from google.appengine.ext import db  
from google.appengine.ext import testbed  
import dev_appserver    
from google.appengine.tools.devappserver2 import devappserver2  


class NewVisitorTest(unittest.TestCase):  

    def setUp(self):  
        self.testbed = testbed.Testbed()  
        self.testbed.activate()  
        #self.testbed.setup_env(app_id='dermalfillersecrets')  
        self.testbed.init_user_stub()  
        ####################################################
        # this sets testbed to imitate strong consistency 
        from google.appengine.datastore import datastore_stub_util
        policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)
        self.testbed.init_datastore_v3_stub(consistency_policy=policy)
        self.testbed.init_memcache_stub() 
        ####################################################

        # setup the dev_appserver  
        APP_CONFIGS = ['app.yaml']  

        self.browser = webdriver.Firefox()  
        self.browser.implicitly_wait(3)  

    def tearDown(self):  
        self.browser.quit()  
        self.testbed.deactivate()  

    def test_guest_can_submit_contact_info(self):  
        from main import Customer  
        query = Customer.query()  
        orig_custs = query.count()  
        self.browser.get('http://localhost:8080')  
        self.browser.find_element_by_name('id_name').send_keys("Kallie Wheelock")  
        self.browser.find_element_by_name('id_street').send_keys("123 main st")  
        self.browser.find_element_by_name('id_phone').send_keys('(404)555-1212')  
        self.browser.find_element_by_name('id_zip').send_keys("30306")  
        self.browser.find_element_by_name('submit').submit()  
        # this should return 1 more record  
        #import pdb; pdb.set_trace()  
        query = Customer.query()   
        self.assertNotEqual(orig_custs, query.count())  
        assert(Customer.query(Customer.name == "Kallie Wheelock").get())  
        # Delete the Customer record  
        Customer.query(Customer.name =="Kallie Wheelock").delete()

Unless you need to test frontend javascript, you are probably better off with webtest (https://pypi.python.org/pypi/WebTest/) than with Selenium. Webtest is faster and easier. — new name, Feb 12 '15 at 14:45
I will be testing javascript eventually, this is just a sandbox. — BryanWheelock, Feb 13 '15 at 16:38

score 5 · Answer 1 · answered Feb 11 '15 at 22:54

5

The PseudoRandomHRConsistencyPolicy is not helping you here because your selenium test is submitting a live html form and the subsequent db update happening on the server which is outside scope of your policy.

What you testing here is the end to end testing not the unit test per se. So your selenium test should take care of the real world scenario and should wait for a predefined period of time before comparing the counts.

answered Feb 11 '15 at 22:54

gipsy

3,859
1
13
21

I added a wait time of 10 seconds before the assertNotEqual and the test still failed. : – BryanWheelock Feb 13 '15 at 16:23
import time; time.sleep(10) – BryanWheelock Feb 13 '15 at 16:24
You may have to show us what is happening on the handler at http://localhost:8080 Is it really creating a new Customer there ? etc. – gipsy Feb 13 '15 at 17:37
I can see the form being filled out during the test and I also see it's being submitted. – BryanWheelock Feb 13 '15 at 17:44
Please shere the handler code including the routing. – gipsy Feb 13 '15 at 17:57

score 2 · Answer 2 · answered Feb 12 '15 at 00:55

There's nothing wrong with strong/eventual consistency, but the design of your tests is wrong. Why you're trying to deal with devappserver in your tests by yourself? Why you're trying to remove entities in the end of the test? Each test should be isolated from each other and start from empty datastore with some possible initializations.

Please, use the latest version of NoseGAE plugin. Here's two simple tests about strong/eventual consistency:

import unittest
from google.appengine.ext import ndb
from google.appengine.datastore import datastore_stub_util


class Foo(ndb.Model):
  pass


class TestEventualConsistency(unittest.TestCase):
  nosegae_datastore_v3 = True
  nosegae_datastore_v3_kwargs = {
    'consistency_policy': datastore_stub_util.PseudoRandomHRConsistencyPolicy(
      probability=0)}

  def test_eventual_consistency(self):
    self.assertEqual(Foo.query().count(), 0)
    Foo().put()
    self.assertEqual(Foo.query().count(), 0)


class TestStrongConsistency(unittest.TestCase):
  nosegae_datastore_v3 = True
  nosegae_datastore_v3_kwargs = {
    'consistency_policy': datastore_stub_util.PseudoRandomHRConsistencyPolicy(
      probability=1)}

  def test_strong_consistency(self):
    self.assertEqual(Foo.query().count(), 0)
    Foo().put()
    self.assertEqual(Foo.query().count(), 1)

Notice that I don't have anything about GAE paths, dev_appserver, etc. You still can control testbed by yourself, but better configure it with nosegae_*. (read about this in plugin documentation)

And as I remember, it will work even if you will programmatically fill your HTML form, but its not unittests anymore though.

I think that the OP's intent was to simply test that his code to save the entity works, and it can be done by simply retrieving the entity by key. What you propose is a test on whether PseudoRandomHRConsistencyPolicy works (i.e. if probability is set to zero entity is not saved, but if probability is set to 1 entity will save). — Andrei Volgin, Feb 12 '15 at 01:24
Sure, he should receive it by key if its possible, but if he has the only one option to use non-ancestor query (and he does not care about strong consistency) -- ignoring the eventual consistency is a good option. — Dmytro Sadovnychyi, Feb 12 '15 at 01:27
I'm not sure how to receive a Key from a entity saved via Selenium. How would I do that? — BryanWheelock, Feb 13 '15 at 16:30
Set a custom key by monkey patching it from test, but I don't think that it will help as it does not works with strong consistency. There's a problem with something else. — Dmytro Sadovnychyi, Feb 13 '15 at 23:00

dlebech · Answer 3 · 2015-02-11T20:47:27.970

1

Try using ancestor queries to get strong consistency instead of eventual consistency. From the docs:

Ancestor queries allow you to make strongly consistent queries to the datastore...

If this does not work, the next thing I would try is to not reuse the query object but create a new the second time.

If this does not work either, my guess is that something else is wrong. I am not familiar with browser test, but I have used webtest with great success for testing web endpoints and have not had any consistency issues while unit testing.

edited Feb 11 '15 at 20:47

answered Feb 11 '15 at 20:24

dlebech

1,817
14
27

There's nothing about to use ancestor queries, he's asking how to make PseudoRandomHRConsistencyPolicy ignore eventual consistency. – Dmytro Sadovnychyi Feb 12 '15 at 01:04
@DmitrySadovnychyi - No, this is not what he is asking. He is using the policy as a "way to get around" the eventual consistency issue - in his own words. – Andrei Volgin Feb 12 '15 at 01:16

Andrei Volgin · Answer 4 · 2015-02-12T16:02:23.847

1

Queries are eventually consistent (unless an ancestor is set), but a get operation is always consistent.

If your objective is to simply test the code for writing an entity, you can insert an entity in this test and check if you can retrieve this entity using its key.

edited Feb 12 '15 at 16:02

answered Feb 11 '15 at 21:43

Andrei Volgin

40,755
6
49
58

How do I confirm entities are saved with GAE's Eventual Consistency?

4 Answers4