"Timeout while fetching" URLFetch GAE/J

Question

I'm using the XMLReader to simply read a feed like below.

URLConnection urlConnection = url.openConnection(); 
XmlReader reader = new XmlReader(urlConnection);

When this is called I receive within 5 seconds an IOException "Timeout while fetching". So I tried to set the timeouts to the max. (10 sec) but still no luck and still an IOExeption in 5 sec.

urlConnection.setConnectTimeout(10000);

(the max is stated in documentation: http://code.google.com/intl/nl-NL/appengine/docs/java/urlfetch/overview.html)

Seems that the size of the feed is too large. When I call a smaller feed it works properly. Is there any workaround or solution for this? I need to be able to call larger feeds.

score 11 · Accepted Answer · edited May 23 '17 at 10:30

11

You should use setReadTimeout method that sets the read deadline:

urlConnection.setReadTimeout(10000); //10 Sec

You should be able to download larger feeds in 10 seconds.
If you still have problem, try to fiddle with this different approach.

edited May 23 '17 at 10:30

Community

1
1

answered Nov 08 '10 at 14:31

systempuntoout

71,966
47
171
241

1

Thanks for the quick response. I tried it, unfortunately still the same Exception. It seems there is totally no response within the 5 seconds. – Ben Groot Nov 08 '10 at 14:55
1

@Ben sounds odd to me; how many seconds does it take to get this feed via browser for example? – systempuntoout Nov 08 '10 at 15:14
It takes about 10 sec. in the browser (FireFox/Chrome/IE) – Ben Groot Nov 08 '10 at 15:42
1

@Ben wow, that's a fat feed. If the server supports "range requests" you could try to asynch fetch smaller chunks of feed; I don't know if it does fit for this kind of data though. – systempuntoout Nov 08 '10 at 15:56
I'll try, thanks. Although, I still think it's strange that the timeout time is not influenced by the setConnectionTimeout or setReadTimeout. – Ben Groot Nov 08 '10 at 16:08
@Ben wait, setReadTimeout(10000) sets the deadline to the max (10sec) and, as you said, feed takes 10 seconds even from the browser; it sounds reasonable that GAE raises a Deadline exception on urlfetch. – systempuntoout Nov 08 '10 at 16:15

score 0 · Answer 2 · edited May 23 '17 at 12:26

The reason is:

If no data is available for the read timeout period, exception can be thrown. From the doc of Oracle

A SocketTimeoutException can be thrown when reading from the returned input stream if the read timeout expires before data is available for read.

By the way, ReadTimeout is different with ConnectTimeout, the read timeout is the timeout to get data from the host, see different connection timeout and read timeout

So as @systempuntoout answer, need to set read timeout.

"Timeout while fetching" URLFetch GAE/J

2 Answers2

Linked