except ConnectionError or TimeoutError not working

Question

In case of a connection error, I want Python to wait and re-try. Here's the relevant code, where "link" is some link:

import requests
import urllib.request
import urllib.parse

from random import randint

try:
    r=requests.get(link)

except ConnectionError or TimeoutError:
    print("Will retry again in a little bit")
    time.sleep(randint(2500,3000))
    r=requests.get(link)

Except I still periodically get a connection error. And I never see the text "Will retry again in a little bit" so I know the code is not re-trying. What am I doing wrong? I'm pasting parts of the error code below in case I'm misreading the error. TIA!

TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))

During handling of the above exception, another exception occurred:

requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))

score 1 · Answer 1 · answered Jul 22 '20 at 20:24

For me, using a custom user agent in the request fixes this issue. With this method you spoof your browser.

Works:

url = "https://www.nasdaq.com/market-activity/stocks/amd"
headers = {'User-Agent': 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4'}
response = requests.get(url, headers=headers)

Doesn't work:

url = "https://www.nasdaq.com/market-activity/stocks/amd"
response = requests.get(url)

ConorSheehan1 · Answer 2 · 2017-09-08T13:10:52.287

0

I think you should use

except (ConnectionError, TimeoutError) as e:
    print("Will retry again in a little bit")
    time.sleep(randint(2500,3000))
    r=requests.get(link)

See this similar question, or check the docs.

edited Sep 08 '17 at 13:10

answered Sep 08 '17 at 13:00

ConorSheehan1

1,640
1
18
30

Thanks! What will "pass" do in this case? Cause the code under "try" to re-run again? Also, I do want to make the script wait a non-trivial amount before re-trying, and your suggestion doesn't seem to do that. – RandomCat Sep 08 '17 at 13:05
pass does nothing, you should replace it with your code. You should keep whatever code you have within the except block, i.e. your time.sleep call etc. I've edited my answer to use your code in the except block. Sorry for the confusion. – ConorSheehan1 Sep 08 '17 at 13:08
That makes sense, thanks! I updated my code and am re-running it just to make sure this fixed the issue. – RandomCat Sep 08 '17 at 13:13
Is there really a reason to assign an alias `e` in this case? You never use it again. `except (ConnectionError, TimeoutError):` will suffice. – pstatix Sep 08 '17 at 13:21
I tried this version, and it didn't fix the issue. Still broke down on a Connection/TimeoutError without printing "Will retry again in a little bit" and re-trying. – RandomCat Sep 11 '17 at 16:15
Can you paste the entire error message? was the line that raised the error definitely in the try block? are you definitely catching the right type of error? – ConorSheehan1 Sep 12 '17 at 10:33
Yes, definitely catching the right error. I don't have the full error code anymore because the solution suggested by t.m.adam below is working. I agree it's weird that your approach didn't work because I thought this is how the syntax for try-except was supposed to work. – RandomCat Sep 12 '17 at 13:44
Actually, I was wrong. It's not printing the retry statement ("Will retry again in a little bit"). The Exception text in e is "('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))" Is that not a TimeoutError? – RandomCat Sep 14 '17 at 13:37

t.m.adam · Accepted Answer · 2017-09-09T08:56:47.940

0

The second request is not inside a try block so exceptions are not caught. Also in the try-except block you're not catching other exceptions that may occur.
You could use a loop to attempt a connection two times, and break if the request is successful.

for _ in range(2):
    try:
        r = requests.get(link)
        break
    except (ConnectionError, TimeoutError):
        print("Will retry again in a little bit")
    except Exception as e:
        print(e)
    time.sleep(randint(2500,3000))

edited Sep 09 '17 at 08:56

answered Sep 08 '17 at 13:43

t.m.adam

15,106
3
32
52

Thanks, will try this! Question: will this loop always sleep for 2500-3000 seconds or only if a connection error occurs? I want the latter, but it looks like this code is doing the former. – RandomCat Sep 11 '17 at 16:16
Only on error. If the request is successful the `break` statement breaks the loop before `time.sleep` is called. – t.m.adam Sep 11 '17 at 16:23

score 0 · Answer 4 · answered Feb 14 '22 at 15:37

0

I had the same problem. It turns out that urlib3 relies on socket.py, which raises an OSError. So, you need to catch that:

try:
    r = requests.get(link)
except OSError as e:
    print("There as an error: {}".format(e))

answered Feb 14 '22 at 15:37

B. Robinson

106
1
6

except ConnectionError or TimeoutError not working

4 Answers4