1

According to the official docs, the timeout for a http request via URLFetch on Google App Engine is 10 minutes if issued from a task:

https://developers.google.com/appengine/docs/java/urlfetch/overview#Quotas_and_Limits

Unfortunately I'm experiencing a 250 seconds (4:10 min) timeout no matter what

I've setup as simple php test script running on Apache (but I've tested it with Lighttpd as well) that I'm calling from GAE that just waits 300 seconds and then returns

Http hook:

echo "Starting to wait\n";

$waited = 0;
while($waited < 300) {
  sleep(5);
  $waited += 5;
  echo "Waited so far $waited seconds\n";
}

echo "all done\n";

The call always fails after approx 250 seconds and throws the following error in the GAE logs

IOException : Could not fetch URL ...

which isn't even a timeout related exception

On the other side the web server records a successful call with http return code 200

The Java code I'm using to make the call is as follows

HTTPRequest httpReq = new HTTPRequest(new URL(http://example.com/very-slow.php),  HTTPMethod.GET, FetchOptions.Builder.allowTruncate().doNotValidateCertificate().setDeadline(3600d));

HTTPResponse resp = null;

try {
Future<HTTPResponse> futureResp = urlService.fetchAsync(httpReq);

log.info("Aync Call request lodged, waiting for a response");

resp = futureResp.get();

log.info("Aync Call completed");

} catch(Throwable th) {
log.warning("URLFetch execution error: " + th.getMessage());
}

I've tried playing around with the setDeadline method specifying all sorts of different values without much luck. If I specify a value lower than 250 seconds, it's used and a timeout exception is thrown. Anything above 250 seconds is ignored and a generic IOException is thrown instead.

I'm pretty sure this looks like a GAE bug and after looking at the bug list on google code I freaked out at how many unresolved bugs are just pending there for years...

Very disappointed by Google App Engine at this stage...

Update

  1. Only the production environment is affected. On the development server I experience no problems. Timeouts are generally not enforced on the dev server though...
  2. I've tried with both the sync and async URLFetch methods: same result.
  3. I've tried with both the low-level GAE-proprietary URLFetch or the java.net URLConnection objects: same result
Stefano Fratini
  • 3,741
  • 2
  • 18
  • 14
  • You might want to mention if this is on the devserver or deployed as behaviour can be very different on each. – Paul Collingwood Dec 24 '12 at 15:37
  • There are two different timeouts: connect timeout and read timeout. I could not find the break-down of 10 minutes limit, but you may try to compare two scenarios: very slow but steady response vs wait for some time and then respond. – Andrei Volgin Dec 24 '12 at 21:45
  • @AndreiVolgin Good point. Using the low level URLFetch interface it is possible to specify only a "deadline" parameter (which most likely accounts for both). Using the URLConnection java.net standard class you can specify both. I was using the URLConnection way before and experienced the same problem no matter what I used as connectionTimeout or readTimeout values... It clearly seems to me like a GAE bug but the question is: am I the only one experiencing this? – Stefano Fratini Dec 25 '12 at 22:28

0 Answers0