Unable to share share Taskqueue and Datastore among services for development local machine- Google App Engine Java Standard Environment

Question

GAE application with multiple services:

module_1 - default
module_2 - backend using datastore
module_3 - Taskqueues and cron job

module_2 is added as dependency in module_3 and using same datastore instance used in module_2. All modules (1, 2, 3 and datastore) are in different virtual machine deployed in test and production.

Same code I am deploying in local development machine, using command gradle appengineRun all three modules <project_home>/module_<1/2/3>$ gradle appengineRun, all project creating own datastore and taskqueues.

Have tried following:

1. dispatcher.xml

<?xml version="1.0" encoding="UTF-8"?>
<dispatch-entries>
    <dispatch>
        <!-- Default module serves the typical web resources and all static resources. -->
        <url>*/favicon.ico</url>
        <module>module_1</module>
    </dispatch>
    <dispatch>
        <!-- Default module serves simple hostname request. -->
        <url>simple-sample.appspot.com/</url>
        <module>module_1</module>
    </dispatch>
    <dispatch>
        <!-- Send all mobile traffic to the mobile frontend. -->
        <url>*/v2/*</url>
        <module>module_2</module>
    </dispatch>
    <dispatch>
        <!-- Send all work to the one static backend. -->
        <url>*/v3/*</url>
        <module>module_3</module>
    </dispatch>
</dispatch-entries>

2. Converting backend to services: Have not tried this, this would require lot of refactor, like movie cron.xml and queue.xml from module_3 to module_1. As this system working with same configuration in stage and production, there must be way to make it work in local.

3. Appengine local datastore location config for Java DevServer

Answer 1

Regardless of actually converting your backends to services or not, what you need to keep in mind is that the queue, datastore index, cron and dispatch configurations are application-level configs, shared by all app's services (it doesn't matter from which service they are actually deployed). But when developing locally the development server may need some those files actually present in each service (and contain configurations consistent across services). See somehow related Do I need to define datastore-indexes in every microservice(module) which uses it or just in root application?. A similar approach can be applied to other app-level shared configs, as needed.

Since your services share the datastore, for proper local development execution you also need a single datastore emulation. That means either:

• running all services inside the same local development server with the datastore emulation performed by that server, possibly by using different ports as per your #3 reference, or
• running each service in its own development server, but have all those development servers not emulate the datastore by themselves and instead connect to a single instance of a cloud Datastore Emulator, see (python context) Is it possible to start two dev_appserver.py connecting to the same google cloud datastore emulator?

But I'm not a Java user, I'm not sure exactly if/how these approaches would work for java, the development server is quite different than the python one I'm using.

Side note: you don't need to add dispatch rules for the default module - that's where requests not matching any other dispatch rules are being sent by default anyways.