6

I'm developing a web app with Google's AppEngine. I'd like to iterate on the code locally using dev_appserver.py. But it's hard to do this without all the data in my deployed app's datastore. I currently run a script to populate the local datastore, but it takes on the order of 15-20 minutes to populate.

Is it possible for dev_appserver.py to connect to my deployed app's datastore?

danvk
  • 15,863
  • 5
  • 72
  • 116

2 Answers2

8

Yeah, it is possible.

First, turn on remote-api in app.yaml and deploy the application on production.

builtins:
- remote_api: on

Then, for example in appengine_config.py:

import os
from google.appengine.ext.remote_api import remote_api_stub
from google.appengine.datastore.entity_pb import Reference

remote_api_stub.ConfigureRemoteApi(app_id=None, path='/_ah/remote_api',
                                   auth_func=lambda: ('email', 'password'),
                                   servername='appid.appspot.com')

if os.environ['SERVER_SOFTWARE'].startswith('Development'):
  Reference.app = lambda *args: os.environ['APPLICATION_ID'].replace('dev~', 's~')

If you have old application ID you may need to edit .replace('dev'...) part.

Dmytro Sadovnychyi
  • 6,171
  • 5
  • 33
  • 60
  • 1
    Dmitry, I upvoted your answer because think it is better than my own! I'll leave my answer standing though in case somebody wants to consider different approaches. – Martin Berends Mar 06 '14 at 04:28
  • Read this before you use remote API. https://developers.google.com/appengine/articles/remote_api?csw=1 – voscausa Mar 06 '14 at 11:03
  • Thanks for the response. I'm having trouble getting this to work, however. Whenever I access the datastore, I get one of these errors: File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore_types.py", line 2099, in ReferenceToKeyValue key.app(), key.name_space()] TypeError: () takes no arguments (1 given) OR BadRequestError: app s~foo cannot access app dev~foo's data. Suggestions? – danvk Mar 10 '14 at 17:02
  • Try to use "lambda *args:" instead of "lambda:" in the last line. I can see second error time to time too.. Need some another way to change app id here. – Dmytro Sadovnychyi Mar 10 '14 at 19:39
  • I traced the "cannot accees app dev~foo's data" error back to memcache. I added a flag to toggle whether memcache is used and turned it off in development mode. This made the errors go away. – danvk Mar 10 '14 at 20:20
  • After some more testing, that wasn't entirely true -- I'm still seeing the data ownership error with memcache disabled, just less frequently. – danvk Mar 10 '14 at 20:55
  • Just want to confirm, It is not working for me. When I looked at the post date it is near about 2years ago. So is it still working now? – Vijay Shegokar Oct 14 '16 at 09:00
  • This is not working anymore because of ClientLogin has been officially deprecated. https://developers.google.com/accounts/docs/AuthForInstalledApps This part is needed because of authentication – dieend Jan 17 '17 at 06:48
1

Yes it is possible using Google Cloud Datastore. You will need a flag in your application to control which datastore you are using at any particular time, and an additional set of data model classes as an interface to the Cloud Datastore instance.

Martin Berends
  • 3,948
  • 2
  • 16
  • 19