What's the proper YAML to start a service in Google App Engine?

Question

Using Google App Engine, I have an application myapp as a default service that adds a task to a task queue and launch a background service worker called optimize. Although myapp is running fine, unfortunately I always see a POST 404 error in the myapp log when the AppEngine task queue tries to launch the URL /optimize-dot-myapp.appspot.com/index.php/optimize. Of course because of the 404 error the task queue keeps retrying. My current optimize.yaml file contains the following. Any thoughts?

# optimize.yaml configuration for Google App Engine
# Full details at: https://cloud.google.com/appengine/docs/php/config/appref
runtime: php55
api_version: 1
service: optimize

handlers:
# Serve php scripts.
- url: /index.php/optimize
  script: index.php/optimize

The default app.yaml file contains the following:

# app.yaml configuration for Google App Engine
# Full details at: https://cloud.google.com/appengine/docs/php/config/appref
runtime: php55
api_version: 1

handlers:
# Serve php scripts.
- url: /(.+\.php).*
  script: \1
- url: /
  script: index.php

# All URLs beginning with /assets are treated as paths to
# static files in the assets/ directory.
- url: /assets
  static_dir: assets

In case it's useful, the optimize worker is started in the default task queue with the following PHP:

// Start the background worker
// API details: https://cloud.google.com/appengine/docs/php/refdocs/classes/google.appengine.api.taskqueue.PushTask
$url = '/optimize-dot-myapp.appspot.com/index.php/optimize';
$task = new PushTask($url, $param);
$task_name = $task->add();

Answer 1

You have collisions in the URL path patterns. The /index.php/optimize path matches both the /index.php/optimize url pattern from optimize.yaml and the /(.+\.php).* pattern from app.yaml. Probably the request ends up in the default service instead of the optimize one. Easy to confirm: check the app logs, you can select a specific service and you'll see which service got the request.

I would add a dispatch.yaml file to clarify things and eliminate the possibility of ambiguous routing (no need to specify the default module, anything not matching dispatch rules is sent to the default module):

application: my_app
dispatch:
  - url: "*/optimize/*"
    module: optimize

Then adjust the url patterns accordingly in optimize.yaml (they should all start with /optimize):

- url: /optimize/index.php
  script: index.php

Note: the index.php file mentioned above would be in the optimize service dir, not in the default service one. Assuming here that each service has its own dir, as mentioned in Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?

And in the task enqueueing code the url should only contain the request path, not the hostname (which is interpreted as part of the path, thus causing a mismatch with the handler's url pattern). You want:

$url = '/optimize/index.php';

Answer 2

In Google App Engine, applications define task queues in a configuration file called queue.yaml. You can use queue.yaml to configure both push queues and pull queues.

The following a basic example that defines a named queue and overrides the default processing rate:

queue:
- name: my-push-queue
  rate: 1/s

The following is a more complex example of a queue.yaml configuration that demonstrates setting up task retries and modifying the default processing rate.

queue:
- name: fooqueue
  rate: 1/s
  retry_parameters:
    task_retry_limit: 7
    task_age_limit: 2d
- name: barqueue
  rate: 1/s
  retry_parameters:
    min_backoff_seconds: 10
    max_backoff_seconds: 200
    max_doublings: 0
- name: bazqueue
  rate: 1/s
  retry_parameters:
    min_backoff_seconds: 10
    max_backoff_seconds: 200
    max_doublings: 3

Please read this documentation for further details