3

In Python 2.6, a new "timeout" parameter was added to the httplib.HTTPConnection class: http://docs.python.org/library/httplib.html#httplib.HTTPConnection

However, this is only a timeout for the connection to a server. I am looking to set a timeout value for the request, not the connection. This doesn't seem to be supported by httplib.

Is there any way to emulate this behavior?

Corey Goldberg
  • 59,062
  • 28
  • 129
  • 143
  • related: [Read timeout using either urllib2 or any other http library](http://stackoverflow.com/q/9548869/4279) – jfs Sep 21 '15 at 01:05

4 Answers4

9

You can set a global socket timeout (*):

import socket

timeout = 10
socket.setdefaulttimeout(timeout)

(*) EDIT: As people in the comments correctly point out: This is technically true, but it only has predictable results for tasks that involve a single socket operation. An HTTP request consists of multiple socket operations (e.g. DNS requests or other things that might be abstracted away from an HTTP client). The timeout of the overall operation becomes unpredictable because of that.

Tomalak
  • 332,285
  • 67
  • 532
  • 628
  • if I set a socket timeout, will all HTTP requests sent from httplib timeout after the value I set? – Corey Goldberg Nov 05 '08 at 16:49
  • This should affect *all* socket operations. – Tomalak Nov 05 '08 at 17:01
  • 2
    Beware of HTTPS requests. The client might get stuck indefinitely because of a known bug in 2.6.x http://bugs.python.org/issue5103 (timeout is ignored during the handshake process in SSL) – GabiMe Oct 25 '09 at 13:06
  • @GabiMe I don't think this is a problem with Python 2.6.1. I checked the ssl.py code and it doesn't look like the code in the bug report. – Mark Lakata Mar 01 '13 at 18:54
  • 1
    This doesn't work for me. It is hanging at the connection.request(). – Mark Lakata Mar 02 '13 at 01:56
  • @Mark The documentation says (emphasis mine) *"Set the default timeout in seconds (float) for **new** socket objects."* – Tomalak Mar 02 '13 at 08:36
  • 1
    @MarkLakata: [HTTPConnection.request not respecting timeout?](http://stackoverflow.com/q/28669744/4279) – jfs Feb 23 '15 at 10:57
  • 1
    @J.F.Sebastian has a point. A single HTTP connection may actual involve many socket stages (ie DNS). If the socket timeout is 10 seconds, it may take longer for the complete HTTP request with a hostname specified. J.F. claims it takes 3x to completely timeout. See his link. – Mark Lakata Feb 23 '15 at 17:39
  • @MarkLakata I agree. Given the fact that this answer is truly ancient... How do we proceed? Edit? Ask the OP to accept another one? Mark the question as duplicate? – Tomalak Feb 23 '15 at 18:05
  • @MarkLakata: It takes 15x (not 3x) in my example. The reason is not due the time it takes to make a single DNS request, the reason is that a single hostname may correspond to *multiple* ip addresses that are tried (to connect) sequentially in turn (the more ips to try the longer it takes). It is more prominent for `google.com` hostname because it corresponds to many ips. – jfs Feb 24 '15 at 04:15
  • @J.F.Sebastian I took 30 seconds and divided by 10 seconds to get 3x. – Mark Lakata Feb 24 '15 at 15:25
  • @MarkLakata: [timeout=2 (seconds): 30/2=15x](http://stackoverflow.com/a/28674109/4279) – jfs Feb 24 '15 at 15:29
7

No, there isn't.

It's because the HTTP spec does not provide anything for the client to specify time-to-live information with a HTTP request. You can do this only on TCP level, as you mentioned.

On the other hand, the server may inform the client about timeout situations with HTTP status codes 408 Request Timeout resp. 504 Gateway Timeout.

mkoeller
  • 4,469
  • 23
  • 30
5

You could also use settimeout on the socket of the connection (works with Python 2.5):

connection = HTTPConnection('slow.service.com')
connection.request(...)
connection.sock.settimeout(5.0)
response = connection.getresponse()
response.read()
connection.close()

If the server cannot send the response within 5 seconds, a socket.error will be raised.

Pankrat
  • 5,206
  • 4
  • 31
  • 37
  • the error is actually `ssl.SSLError` in python 2.7 as far as I can tell – Matt Dodge Jul 18 '12 at 01:23
  • 2
    this does not set a timeout for the connection.request(). And you can't move the connection.sock.settimeout(5.0) up one line, because connection.sock is None – Mark Lakata Mar 01 '13 at 18:53
0

When you initiate the connection, it is posible to give as third argument the timout value.

connection = HTTPConnection(<IP of URL>, <port or None>, <timeout>)
...

is a float in seconds.

connection = HTTPConnection('slow.service.com', None, 20.0)
Bob Baeck
  • 81
  • 1
  • 3