TL;DR - There might be a way to do this based off earlier version of the documentation, but this is not described in the latest version of the documentation.
Approach based on the latest documentation
As per the latest documentation regarding Push queues (and like you already mentioned), you can configure the target module, version in the queue.xml per queue. If specified, the request for the task is sent to the specified target. As you already described, this is a static configuration and does not really answer your question (describing just for the sake of completeness).
<?xml version="1.0" encoding="UTF-8"?>
<queue-entries>
<queue>
<name>queue-blue</name>
<target>v2.task-module</target>
</queue>
<queue>
<name>queue-red</name>
<rate>1/s</rate>
</queue>
</queue-entries>
In the documentation for creating tasks and specifying worker service, it describes about the target chosen for the task but does not clearly describe how to specify it.
When a task is popped off its queue, the Task Queue service sends it
on to a worker service. Every task has a target and a url, which
determine what service and handler will ultimately perform the task.
target
The target specifies the service that will receive the HTTP request to
perform the task. It is a string that specifies a
service/version/instance in any one of the canonical forms. The most
often-used ones are:
service
version.service
instance.version.service
The target string is prepended to the domain name of your app. There
are three ways to set the target for a task:
- Explicity declare the target when you construct the task.
- Include a target directive when you define a queue in the queue.xml, as in the definition of queue-blue above. All tasks added to a queue
with a target will use that target, even if a different target was
assigned to the task at construction time.
- If no target is specified according to either of the previous two methods, then the task's target is the version of the service that
enqueues it. Note that if you enqueue a task from the default service
and version in this manner, and the default version changes before the
task executes, it will run in the new default version.
url
The url selects one of the handlers in the target service, which will
perform the task.
The url should match one of the handler URL patterns in the target
service. The url can include query parameters if the tasks's method is
GET or PULL. If no url is specified the default URL
/_ah/queue/[QUEUE_NAME] is used, where [QUEUE_NAME] is the name of the
task's queue.
Based off the above documentation, if you look at TaskOptions.Builder, there is no method to specify the target for the task. This could either indicate a missing documentation on how the target should be specified or just that there is no way any more to specify the target dynamically when adding the task to the queue. Look at the approach described below based on the earlier documentation.
Approach based on earlier documentation
DISCLAIMER: What I mention here looks to be based on outdated information (I've cited the sources below) and hence might not work as expected and/or break in the future.
You can use the Host header to specify the module, version and instance information which will get the request.
To specify module and version, you can do:
Queue queue = QueueFactory.getQueue("QUEUE_NAME");
queue.add(TaskOptions.Builder
.withUrl("/url/path")
.param("key", "PARAM")
.header("Host",
ModulesServiceFactory.getModulesService().getVersionHostname("MODULE_NAME", "VERSION")));
To specify instance you can do:
Queue queue = QueueFactory.getQueue("QUEUE_NAME");
queue.add(TaskOptions.Builder
.withUrl("/url/path")
.param("key", "PARAM")
.header("Host",
ModulesServiceFactory.getModulesService().getInstanceHostname("MODULE_NAME", "VERSION", "INSTANCE_NAME"));
I could not find this info in the current version of the App Engine documentation, but using the Wayback machine I found this info from an earlier version of the doc from Jan 1 2016 which describes the Push task execution. It has been discussed in a different context in this github issue as well.
Push task execution
App Engine executes push tasks by sending HTTP POST requests to your
app. Specifying a programmatic asynchronous callback as an HTTP
request is sometimes called a web hook. The web hook model enables
efficient parallel processing.
The task's URL determines the handler for the task and the module that
runs the handler.
The handler is determined by the path part of the URL (the
forward-slash separated string following the hostname), which is
specified by the url parameter in the TaskOptions that you include in
your call to the Queue.add() method. The url must be relative and
local to your application's root directory.
The module and version in
which the handler runs is determined by:
- The "Host" header parameter
in the TaskOptions that you include in your call to the Queue.add()
method.
- The target directive in the queue.xml or queue.yaml file.
If you do not specify any of these parameters, the task
will run in the same module/version in which it was enqueued, subject
to these rules:
If the default version of the app enqueues a task, the task will run
on the default version. Note that if the app enqueues a task and the
default version is changed before the task actually runs, the task
will be executed in the new default version.
If a non-default version
enqueues a task, the task will always run on that same version.
Note:
If you are using modules along with a dispatch file, a task's URL may
be intercepted and re-routed to another module.
The namespace in which
a push task runs is determined when the task is added to the queue. By
default, a task will run in the current namespace of the process that
created the task. You can override this behavior by explicitly setting
the namespace before adding a task to a queue, as described on the
multitenancy page.
