13

I'm using Python's urllib2 to send an HTTP post:

import socket, urllib, urllib2

socket.setdefaulttimeout(15)    

postdata = urllib.urlencode({'value1' : 'a string', 'value2' : 'another string'})
headers = {
    'User-Agent': 'Agent',
    'Content-Type': 'application/x-www-form-urlencoded',
    'Accept': 'text/html, */*',
}

try: 
    request = urllib2.Request('http://www.example.com', postData, headers)
    response = urllib2.urlopen(request)
except urllib2.HTTPError, e:
    # Handle here
except urllib2.URLError, e:
    # Handle here
except httplib.HTTPException, e:
    # Handle here

Occasionally a network issue results in the call to urlopen never returning. We see other errors (including timeouts) handled correctly by the except block and have a call to socket.setdefaulttimeout() but there are still instances where the urlopen will never return.

I know it's never returning because we have some log lines in our actual code which get called before and after, and when this problem occurs only the calls before are made and the script hangs forever.

What's the best way to detect/handle this?

jathanism
  • 33,067
  • 9
  • 68
  • 86
davidmytton
  • 38,604
  • 37
  • 87
  • 93

1 Answers1

10

You can use signals, first set a handler for your signal

import signal
...
def handler(signum, frame):
    print 'Signal handler called with signal', signum
...
signal.signal(signal.SIGALRM, handler)

and put an alarm just before the urlopen call

signal.alarm(5)
response = urllib2.urlopen(request)
signal.alarm(0) # Disable the signal

after 5 seconds (or the time you desire) the OS will call the handler if the alarm is not disable (if urlopen never returns). More info about signal module: http://docs.python.org/library/signal.html

Manuel
  • 2,236
  • 2
  • 18
  • 28
  • 1
    ...and then in the handler I can throw an exception which will be caught by the outside try/except block. – davidmytton Apr 07 '11 at 21:51
  • 2
    This does NOT work on Windows (`signal.alarm()` is not supported) – Sébastien Oct 02 '11 at 12:38
  • 1
    @Sebastien,[here](http://stackoverflow.com/questions/644073/signal-alarm-replacement-in-windows-python) you can find a discussion about signal.alarm() in Windows systems – Manuel Oct 05 '11 at 08:26
  • Doesn't work on Linux. Unplug the network cable and execute. `signal.alarm(5)` is being ignored. – Viktor Joras Jan 30 '19 at 15:23
  • Hi @ViktorJoras, I just tried in ubuntu 18.04.1 with both Python 2.7.15 and 3.6.7, and it works. Maybe you should open a new question, providing more details about your problem. – Manuel Feb 03 '19 at 22:55