4

I have an Python appengine app using datastore with ndb API and I want to do background work and store results into the datastore so appengine uses it.

I wanted to use GCE or my computer to do so, but ndb API is not available outside appengine and the alternative seems to be gcloud.datastore API which is very different.

How do you guarantee that what you push (with gcloud API) is consistent with what you get (ie: matches a ndb entity) ?

I can't do unit-tests because the local server is not the same (gcd vs dev_appserver). Here is a workaround (but in Java).

Should I replace ndb code by gcloud.datastore in appengine to ensure consistency (but loosing ndb advantages like auto caching...) ?

Is there an obvious solution I'm missing ? If someone had the same issue, how did you handle it ?

Thanks

Community
  • 1
  • 1
nicoxx
  • 111
  • 2

1 Answers1

3

If you're really worried about consistency and you're using fancy features of ndb, you should probably look into using the Remote API for App Engine, which effectively let's you run arbitrary code via a remote (HTTP) interface. It might help you get the job done, but remember that the CPU cycles you're using will be in GAE -- not in GCE.

If you're willing to wait a while, we're working on porting the ndb API to run against the Cloud Datastore API which would mean the same code you run in App Engine will work outside of App Engine (on your local machine or inside Google Compute Engine).

The gcloud.datastore (gcloud-python) API is much more low-level, so you should have even more control over the data that ends up in the Datastore. It wasn't built to be identical to ndb (and therefore doesn't have some of the fancy stuff like derived fields or geo-points as first-class citizens), however ndb stores those fields using it's own Python logic so you should be able to write safely if you're comfortable with the lower-level data representations.

JJ Geewax
  • 10,342
  • 1
  • 37
  • 49