19

I'm trying to have Django (on top of GAE) fetch data from another web service. I'm often hit with error like this:

ApplicationError: 2 timed out Request

Method: GET

Request URL:http://localhost:8080/

Exception Type: DownloadError

Exception Value: ApplicationError: 2 timed out

Exception Location: /google_appengine/google/appengine/api/urlfetch.py in _get_fetch_result, line 325

It feels as if it will time out only after 12 seconds (I'm not sure, but it's really short).

Question: how can I set a longer timeout?

GRex
  • 417
  • 1
  • 5
  • 7

4 Answers4

27

Seeing as this is a Python question, I thought I'd provide a Python answer for anyone who comes across this problem.

Just import urlfetch and then define a deadline before doing anything else in your code:

from google.appengine.api import urlfetch

urlfetch.set_default_fetch_deadline(60)
rescdsk
  • 8,739
  • 4
  • 36
  • 32
Alex Young
  • 369
  • 3
  • 4
24

You can set it using the deadline argument of the fetch function. From the docs:

The deadline can be up to a maximum of 60 seconds for request handlers and 10 minutes for tasks queue and cron job handlers. If deadline is None, the deadline is set to 5 seconds.

Edit: looks like this has changed now. From here:

You can set a deadline for a request, the most amount of time the service will wait for a response. By default, the deadline for a fetch is 5 seconds. You can adjust the default deadline for requests using the urlfetch.set_default_fetch_deadline() function.

And this page lists the default timeout values:

Currently, there are several errors named DeadlineExceededError for the Python runtime:

  • google.appengine.runtime.DeadlineExceededError: raised if the overall request times out, typically after 60 seconds, or 10 minutes for task queue requests.
  • google.appengine.runtime.apiproxy_errors.DeadlineExceededError: raised if an RPC exceeded its deadline. This is typically 5 seconds, but it is settable for some APIs using the 'deadline' option.
  • google.appengine.api.urlfetch_errors.DeadlineExceededError: raised if the URLFetch times out.
Mark Bell
  • 28,985
  • 26
  • 118
  • 145
  • Ok, may I know why you decided to answer the question using Java when the OP clearly stated he's using Django? You didn't provide the python equivalent :( – kassold May 21 '14 at 20:27
  • Someone else had edited my answer two years after I gave it and added in a Java code snippet for some reason... if you look at the hyperlinks in the first line, they link to the Python documentation. There is a Python example given in Alex Young's answer here anyway. – Mark Bell May 24 '14 at 18:28
  • The documents you link to no longer include any mention of a maximum of 60 seconds. Has that limitation been deprecated? – conradlee Jul 16 '15 at 19:38
7

For Go, you might want to try below code.

// createClient is urlfetch.Client with Deadline
func createClient(context appengine.Context, t time.Duration) *http.Client {
    return &http.Client{
        Transport: &urlfetch.Transport{
            Context:  context,
            Deadline: t,
        },
    }
}

Here is how to use it.

// urlfetch
client := createClient(c, time.Second*60)
gosharplite
  • 111
  • 2
  • 7
  • I realize this comment is from 2013 but the API has changed and urlfetch,Transport no longer has a Deadline field. To set the deadline, set it on the context instead using: `ctx, _ := context.WithDeadline(context, time.Second * 60)`. The problem with this approach is that you also limit the time available for all subsequent requests. What I mean is that if you poll bigquery in a task (that has a 10 minute timeout) and set the deadline to 60 seconds you are really limiting the polling period to 60 seconds since the deadline is set on the context that is set on the client passed to BigQuery. – gabrielf Nov 23 '17 at 12:30
-2

It seems short but you have to know that the timeout of a request on GAE is around 30 seconds. As you probably need to do some operations on the response of your urlfetch, there's no need to have a timeout more than 10 seconds I think.