1

I have recently converted an app from using backends to modules. I was using backends for long running requests, primarily get requests to external url's (using the requests library). Since changing my app such that everything runs on the single default module I am getting DeadlineExceededError's. i.e.

DeadlineExceededError('The API call urlfetch.Fetch() took too long to respond and was cancelled.

I have read in the appengine documentation that there is a:

60-second deadline for HTTP requests, 10-minute deadline for tasks

The requests I am running are run as a task (using deferred.defer(...) ). However the task starts at, for example, 15:51:54.489 and finishes at 15:51:59.600 - a grand total of 5 seconds before throwing the error.

In app.yml automatic scaling enabled as follows:

automatic_scaling:
  min_idle_instances: 1
  max_idle_instances: 1
  max_pending_latency: 1s

It is not clear to me why this deadline error is being thrown. Could you please let me know how I can avoid throwing this error?

user714852
  • 2,054
  • 4
  • 30
  • 52
  • 1
    Logs for such exceptions might not be very explicit, you may want to check out this thread with interpretations about them: http://stackoverflow.com/questions/30918582/deferred-task-request-deadline-exceeded-but-work-never-started?noredirect=1#comment50132217_30918582 – Dan Cornilescu Sep 17 '15 at 17:12

1 Answers1

3

As the exception mentions, the error is from urlfetch.Fetch, which every outbound HTTP request is routed through on App Engine, even if you use urllib, requests, or other libraries. And the default deadline for urlfetch is 5 seconds. So the problem is that your HTTP requests to external URLs are timing out. It seems unrelated to your switch from backends to regular instances. Probably a coincidence.

This page has an overview of the various types of errors behind DeadlineExceededError.

  • Problem seems to have been solved by setting urlfetch.set_default_fetch_deadline(600) in the requests init.py and in appengine_config.py. – user714852 Sep 18 '15 at 09:51