2

I have a task handler that is making a batch request to the Google Calendar API. After 5 seconds, the request fails with DeadlineExceededError: The API call urlfetch.Fetch() took too long to respond and was cancelled. I have changed urlfetch.set_default_fetch_deadline(60) near where I make the batch request, as suggested here but it does not seem to make a difference: the deadline seems to remain 5 seconds.

I am using the Python Google API Client library which sits on top of oauth2client and httplib2. But my understanding is that GAE intercepts the underlying calls to use urlfetch.Fetch. This is what the stack trace seems to show as well.

Can you see any reason why urlfetch.set_default_fetch_deadline does not seem to be working?

EDIT:

This is the code used to build the batch request:

# note `http` is a oauth2client authorized http client
cal = apiclient.discovery.build('calendar','v3',http=http)
req = cal.new_batch_http_request(callback=_callback)
for event in events:   # anything larger than ~5 events in batch takes >5 secs
  req.add( 
    cal.events().patch(calendarId=calid, eventId=event["id"], body=self._value) 
  )
urlfetch.set_default_fetch_deadline(60)  # has no effect
req.execute()
Community
  • 1
  • 1
Eric G
  • 1,282
  • 8
  • 18

2 Answers2

1

So, urlfetch.set_default_fetch_deadline() did eventually work for me. The problem was my underlying http client (oauth2client / httplib2) was essentially stored in a global. Once I created it in the task handler thread the set_default_fetch_deadline worked.

Eric G
  • 1,282
  • 8
  • 18
0

Try adding the deadline parameter:

my_result = urlfetch.fetch(my_url, deadline=15)

GAEfan
  • 11,244
  • 2
  • 17
  • 33
  • Thanks, but I'm not using urlfetch directly -- but the [api client library](https://developers.google.com/api-client-library/python/). AFAIK there is no direct way to pass down a deadline param, especially since that library doesn't use urlfetch except in the GAE enivronment. – Eric G Oct 04 '16 at 19:53
  • Can you provide the code you use to make the request? You may be able to pass kwargs, like deadline. – GAEfan Oct 04 '16 at 20:06
  • Done. I noticed there is a `num_retries` kwarg to the [`execute` method](https://google.github.io/google-api-python-client/docs/epy/googleapiclient.http.HttpRequest-class.html#execute) but that doesn't help me here. – Eric G Oct 04 '16 at 20:25
  • 1
    From the docs: "You can adjust the default deadline by using the `urlfetch.set_default_fetch_deadline()` function. This function stores the new default deadline on a thread-local variable, so it must be set for each request, for example, in a custom middleware." – GAEfan Oct 04 '16 at 20:51
  • I am setting that, but it's not affecting the deadline of the request - whether I set it before or after creating the request object. I think the problem may be the underlying http client is not thread-local. But when I make it thread-local, it attempts to refresh the access token and gets an AccessTokenRefreshError... I'm going to update some of these libraries and see if it helps before anything else. – Eric G Oct 04 '16 at 21:59