Why is the entity I create in setUp() not accessible via nosetests?

Question

I still can't seem to get Nosetest to run properly. The dev_appserver runs fine when started from the command line, but when I attempt to start it from the functional_tests.py it fails.

I create a Client entity in the setUp(), but it's not accessable from the test.

How do I step into pdb debugger with dev_appserver starting up in the test?
I have tried to put pdb() breakpoints in the code. While the code stops execution, I can't step into the debugger. I'm not even sure how to see the output.

$ nosetests

INFO     2015-02-24 19:08:56,172 devappserver2.py:726] Skipping SDK update check.    
INFO     2015-02-24 19:08:56,242 api_server.py:172] Starting API server at: http://localhost:62049    
INFO     2015-02-24 19:08:56,247 dispatcher.py:186] Starting module "default" running at: http://localhost:8080    
INFO     2015-02-24 19:08:56,249 admin_server.py:118] Starting admin server at: http://localhost:8000    
ERROR    2015-02-24 19:09:00,307 webapp2.py:1552] 'NoneType' object has no attribute 'key'    
Traceback (most recent call last):    
  File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__    
    rv = self.handle_exception(request, response, e)    
  File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__    
    rv = self.router.dispatch(request, response)    
  File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher    
    return route.handler_adapter(request, response)    
  File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__    
    return handler.dispatch()    
  File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 18, in dispatch    
    webapp2.RequestHandler.dispatch(self)    
  File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch    
    return self.handle_exception(e, self.app.debug)    
  File "/Users/Bryan/Desktop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch    
    return method(*args, **kwargs)    
  File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/main.py", line 95, in get    
    self.session['client'] = client.key.urlsafe()    
AttributeError: 'NoneType' object has no attribute 'key'    
INFO     2015-02-24 19:09:00,314 module.py:737] default: "GET / HTTP/1.1" 500 2354    
INFO     2015-02-24 19:09:00,377 module.py:737] default: "GET /favicon.ico HTTP/1.1" 200 8348    
INFO     2015-02-24 19:09:00,381 module.py:737] default: "GET /favicon.ico HTTP/1.1" 304 -    
EINFO     2015-02-24 19:09:08,482 shutdown.py:45] Shutting down.    
INFO     2015-02-24 19:09:08,483 api_server.py:588] Applying all pending transactions and saving the datastore    

======================================================================    
ERROR: test_guest_can_submit_contact_info (dermalfillersecrets.functional_tests.NewVisitorTest)    
----------------------------------------------------------------------    
Traceback (most recent call last):    
  File "/Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/functional_tests.py", line 88, in test_guest_can_submit_contact_info    
    self.browser.find_element_by_name('id_name').send_keys("Kallie Wheelock")    
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 302, in find_element_by_name    
    return self.find_element(by=By.NAME, value=name)    
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element    
    {'using': by, 'value': value})['value']    
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute    
    self.error_handler.check_response(response)    
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response    
    raise exception_class(message, screen, stacktrace)    
NoSuchElementException: Message: Unable to locate element: {"method":"name","selector":"id_name"}    
Stacktrace:    
    at FirefoxDriver.prototype.findElementInternal_ (file:///var/folders/mw/0y88j8_54bjc93d_lg3120qw0000gp/T/tmpSjWZ6W/extensions/fxdriver@googlecode.com/components/driver-component.js:9641:26)    
    at fxdriver.Timer.prototype.setTimeout/<.notify (file:///var/folders/mw/0y88j8_54bjc93d_lg3120qw0000gp/T/tmpSjWZ6W/extensions/fxdriver@googlecode.com/components/driver-component.js:548:5)    

Here is the code in functional_tests.py

import sys, os, subprocess, time, unittest, shlex      
sys.path.append("/usr/local/google_appengine")       
sys.path.append("/usr/local/google_appengine/lib/yaml/lib")       
sys.path.append("/usr/local/google_appengine/lib/webapp2-2.5.2")       
sys.path.append("/usr/local/google_appengine/lib/django-1.5")       
sys.path.append("/usr/local/google_appengine/lib/cherrypy")       
sys.path.append("/usr/local/google_appengine/lib/concurrent")       
sys.path.append("/usr/local/google_appengine/lib/docker")       
sys.path.append("/usr/local/google_appengine/lib/requests")       
sys.path.append("/usr/local/google_appengine/lib/websocket")       
sys.path.append("/usr/local/google_appengine/lib/fancy_urllib")       
sys.path.append("/usr/local/google_appengine/lib/antlr3")       

from selenium import webdriver       
from google.appengine.api import memcache, apiproxy_stub, apiproxy_stub_map      
from google.appengine.ext import db       
from google.appengine.ext import testbed       
import dev_appserver         
from google.appengine.tools.devappserver2 import devappserver2       


class NewVisitorTest(unittest.TestCase):       

    def setUp(self):       
        # Start the dev server        
        cmd = "/usr/local/bin/dev_appserver.py /Users/Bryan/work/GoogleAppEngine/dermalfillersecrets/app.yaml --port 8080 --storage_path /tmp/datastore --clear_datastore --skip_sdk_update_check"       
        self.dev_appserver = subprocess.Popen(shlex.split(cmd),      
                                              stdout=subprocess.PIPE)     
        time.sleep(2) # Important, let dev_appserver start up     

        self.testbed = testbed.Testbed()   
        self.testbed.setup_env(app_id="dev~myapp")       
        self.testbed.activate()       
        #self.testbed.setup_env(app_id='dermalfillersecrets')       
        self.testbed.init_user_stub()       
        # Create a consistency policy with a probability of 1,  
        #  the datastore should be available.  
        self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=1)  
        # Initialize the datastore stub with this policy.  
        self.testbed.init_datastore_v3_stub(datastore_file="/tmp/datastore/datastore.db", use_sqlite=True, consistency_policy=self.policy)       
        self.testbed.init_memcache_stub()       
        self.datastore_stub = apiproxy_stub_map.apiproxy.GetStub('datastore_v3')     

        # setup the dev_appserver       
        APP_CONFIGS = ['app.yaml']       

        # setup client to make sure      
        from main import Client      
        if not ( Client.query( Client.name == "Bryan Wheelock").get()):     
            logging.info("create Admin")     
            client = Client(     
            email = "bryan@mail.com",     
            name = "Bryan Wheelock",     
            street1 = "555 Main St",     
            street2 = "unit 1",     
            city = "Atlanta",     
            zipcode = 99999,     
            phone = "(888)555-1212"     
            ).put()  
            # this sleep is to allow eventual consistency to propogate     
            time.sleep(2)

        self.browser = webdriver.Firefox()       
        self.browser.implicitly_wait(3)       

    def tearDown(self):       
        self.browser.quit()       
        self.testbed.deactivate()       
        self.dev_appserver.terminate()     

    def test_guest_can_submit_contact_info(self):      
        from main import Client, Customer        
        client = Client.query( Client.name == "Bryan Wheelock").get()       
        orig_customer_count = Customer.query(ancestor=client.key).count()       
        self.browser.get('http://localhost:8080')     
        time.sleep(5)     
        self.browser.find_element_by_name('id_name').send_keys("Kallie Wheelock")       
        self.browser.find_element_by_name('id_street').send_keys("123 main st")       
        self.browser.find_element_by_name('id_phone').send_keys('(404)555-1212')       
        self.browser.find_element_by_name('id_zip').send_keys("30306")       
        self.browser.find_element_by_name('submit').submit()       
        # the time delay is to allow eventual consisenency to happen.     
        time.sleep(4)       
        assert(Customer.query(Customer.name == "Kallie Wheelock").get())     
        # this should return 1 more record     
        final_customer_count = Customer.query(ancestor=client.key).count()       
        self.assertNotEqual(orig_customer_count, final_customer_count)       

        # Delete the Customer record       
        Customer.query(Customer.name =="Kallie Wheelock").delete()   

Here's the code in main.py:

import os  
import urllib  
import logging  

from google.appengine.api import users  
from google.appengine.ext import ndb  

import jinja2  
import webapp2  
from webapp2_extras import sessions  

class BaseHandler(webapp2.RequestHandler):  
    def dispatch(self):  
        self.session_store = sessions.get_store(request=self.request)  

        try:  
            # dispatch the request  
            webapp2.RequestHandler.dispatch(self)  
        finally:  
            # save all sessions  
            self.session_store.save_sessions(self.response)  

    @webapp2.cached_property  
    def session(self):  
        # Returns a session using the default cookie key.  
        return self.session_store.get_session()  

JINJA_ENVIRONMENT = jinja2.Environment(  
    loader = jinja2.FileSystemLoader(os.path.dirname(__file__)),  
    extensions=['jinja2.ext.autoescape'],  
    autoescape=True)  

DEFAULT_LEADBOOK_NAME = 'whatsmyname'  

def leadbook_key(leadbook_name=DEFAULT_LEADBOOK_NAME):  
    """Constructs a Datastore key for a LeadBook entity with leadbook_name."""  
    return ndb.Key('LeadBook', leadbook_name)  

class Client(ndb.Model):  
    email =  ndb.StringProperty()   
    name = ndb.StringProperty(indexed=True)  
    street1 = ndb.StringProperty()  
    street2 = ndb.StringProperty()  
    city = ndb.StringProperty()  
    zipcode = ndb.IntegerProperty()  
    phone = ndb.StringProperty()  
    signup = ndb.DateTimeProperty(auto_now_add=True)  

# this just creates a Client to use  
if not ( Client.query( Client.name == "Bryan Wheelock").get()):  
    client = Client(  
    email = "bryan@mail.com",  
    name = "Bryan Wheelock",  
    street1 = "555 Main St",  
    street2 = "unit 1",  
    city = "Atlanta",  
    zipcode = 99999,  
    phone = "(888)555-1212"  
    ).put()  

class Customer(ndb.Model):  
    # I commented out client property because using Ancestor Query( limited to 1 write per second)  
    #client = ndb.KeyProperty(kind=Client)  
    #email = ndb.StringProperty(indexed=True)  
    name = ndb.StringProperty(indexed=True)  
    street1 = ndb.StringProperty()  
    street2 = ndb.StringProperty()  
    city = ndb.StringProperty()  
    zipcode = ndb.IntegerProperty()  
    phone = ndb.StringProperty()  
    signup = ndb.DateTimeProperty(auto_now_add=True)  

class MainPage(BaseHandler):  
    def get(self):  
        leadbook_name = self.request.get('leadbook_name',  
                                            DEFAULT_LEADBOOK_NAME)  

        # This should be the Client record that shows the info of the owner of the local clinic  
        # the question is how do I get the site to show the correct Client?  
        client = Client.query( Client.name == "Bryan Wheelock").get()  
        self.session['client'] = client.key.urlsafe()  

        template_values = {  
            'client': client,  
            'leadbook_name': urllib.quote_plus(leadbook_name),  
        }  

        template = JINJA_ENVIRONMENT.get_template('index.html')  

        self.response.write(template.render(template_values))  

class LeadBook(BaseHandler):  
    def post(self):  
        leadbook_name = self.request.get('leadbook_name',  
                                          DEFAULT_LEADBOOK_NAME)  

        client = ndb.Key(urlsafe=self.session['client']).get()  

        customer = Customer( parent = client.key)  
        customer.name = self.request.get('id_name')  
        customer.street1 = self.request.get('id_street')  
        customer.phone = self.request.get('id_phone')  
        customer.zipcode = int(self.request.get('id_zip'))  
        # show original number of customer to show the code works  
        starting_customer_count = Customer.query(ancestor=client.key).count()  
        #import pdb; pdb.set_trace()  
        customer.put()  
        # This should return the record  
        assert(Customer.query(Customer.name == "Kallie Wheelock").get())   
        final_customer_count = Customer.query(ancestor=client.key).count()  
        #import pdb; pdb.set_trace()  
        query_params = {'leadbook_name': leadbook_name}  
        self.redirect('/?' + urllib.urlencode(query_params))  

config = {}  
config['webapp2_extras.sessions'] = {  
    'secret_key': 'my-super-secret-key',  
}  
application = webapp2.WSGIApplication([  
    ('/', MainPage),  
    ('/sign', LeadBook),  
], config = config,  
    debug=True)

Answer 1

The problem is hinted at by your comment:

# this sleep is to allow eventual consistency to propogate

That's really not how it works. Eventual consistency has nothing to do with time, and the way it is emulated in the local datastore has even less; in the tests, the datastore testbed implements a policy whereby the initial read almost always fails. The documentation explains how you can tweak the policy in your test; one shortcut - again, that only works in tests and not in production - is to do an explicit .get() after saving, which will always make the entity visible.