5

I am trying to port to google app engine modules, a previously long-running job ( running in 'backends' ) .

A sample module1.yaml is present below.

application: myuniqueapp
module:  module1
version: 1
runtime: python27
api_version: 1
threadsafe: true
instance_class: F4_1G
automatic_scaling:
  max_idle_instances: 1

handlers:
- url: /data
  static_dir: data
  application_readable: true

- url: /.*
  script: app.application

The code to submit to this (from a front-end instance), via taskqueue is this: 

taskqueue.add(url='/tasks/do_my_task',
            target='1.module1')

This submits the right task without an issue. The task gets executed by the module1 as well.

But it gets killed by the 10th minute, with the DeadlineExceededError. This is a long running task and runs for longer than 10 minutes ( like how it used to work for 'backends' ).

What configuration change needs to be done, for the task executing in a module to be > 10 minutes ?

rakesh.usenet
  • 75
  • 1
  • 6

3 Answers3

6

You simply need to choose manual or basic scaling for your module:

https://developers.google.com/appengine/docs/java/modules/#Java_Instance_scaling_and_class

Andrei Volgin
  • 40,755
  • 6
  • 49
  • 58
  • 1
    Just make sure you omit or set appropriately. The [example for ``](https://cloud.google.com/appengine/docs/java/modules/#Java_Instance_scaling_and_class) is 10 minutes. I made the mistake of copying that verbatim and hit a 10 minute DeadlineExceededError. – Michael Osofsky Nov 06 '14 at 04:02
2

You have a few options, but currently there is no way of getting around the 10 minute request deadline for App Engine auto-scaling modules. Simply put, you need to make your task run faster, or you need to run it on a service that has no deadline.

Faster:

  • Chunk the task into smaller processes and queue multiple tasks.
  • Perform optimizations on the Task and make it run under 10 minutes.

No Deadline:

  • Create another auto-scaling/manual app engine service to send the task
  • Change your module from auto-scaling to manual or basic scaling.
  • Create another auto-scaling/manual app engine service to send the task to.
Antoine Vo
  • 559
  • 4
  • 10
1

Google appengine also provides backinstance classes where a task can run upto 24 hours.Re-route the tasks request to run on backinstance which will solve the 10 min issue.Hope this will solve your issue

Dipu Muraleedharan
  • 43
  • 1
  • 8
  • I believe Google Backends are being deprecated: https://cloud.google.com/appengine/docs/deprecations Unless you are talking about something else? – Antoine Vo Mar 03 '20 at 19:21
  • @AntoineVo , I referred Google backinstance servers.Please find more details here:https://cloud.google.com/appengine/docs/standard#instance_classes – Dipu Muraleedharan Mar 04 '20 at 06:50