nesting a retry loop into a try except block python API

Question

I have the following try except block that works well for me:

try:
    r = requests.post(endpoint,data=json.dumps(report_params),headers=headers)
    r.raise_for_status()
except requests.exceptions.HTTPError as errhttp:
    print ("Http Error:",errhttp)
    mbody = "The process encountered the following HTTP error: " + str(errhttp)
    sendmail(fromaddr, toaddr, msubject, mbody)
except requests.exceptions.ConnectionError as errconn:
    print ("Error Connecting:",errconn)
    mbody = "The process encountered the following connection error: " + str(errconn)
    sendmail(fromaddr, toaddr, msubject, mbody)
except requests.exceptions.Timeout as errtimeout:
    print ("Timeout Error:",errtimeout)
    mbody = "The process timed out after 3 tries and gave the error: " + str(errtimeout)
    sendmail(fromaddr, toaddr, msubject, mbody)
except requests.exceptions.RequestException as err:
    print ("No idea what the hell happened",err)
    mbody = "The process encountered the following unexpected error: " + str(err)
    sendmail(fromaddr, toaddr, msubject, mbody)

I would like to add a retry loop or function that retries 3 times on a connection error or timeout error

I have found something on SO that should work:

tries = 3
for i in range(tries):
    try:
        do_the_thing()
    except KeyError as e:
        if i < tries - 1: # i is zero indexed
            continue
        else:
            raise
    break

I am still learning try except blocks and therefore not very good at them. How could I put something like that loop inside the above block so that it looks like this:

try:
    r = requests.post(endpoint,data=json.dumps(report_params),headers=headers)
    r.raise_for_status()
except requests.exceptions.HTTPError as errhttp:
    print ("Http Error:",errhttp)
    mbody = "The process encountered the following HTTP error: " + str(errhttp)
    sendmail(fromaddr, toaddr, msubject, mbody)
except requests.exceptions.ConnectionError as errconn:

#### RETRY 3 TIMES AND IF SUCCESSFUL, CONTINUE TO NEXT EXCEPT STATEMENT.  IF UNSUCCESSFUL, SEND THE EMAIL

    print ("Error Connecting:",errconn)
    mbody = "The process failed to connect after 3 tries and gave the following error: " + str(errconn)
    sendmail(fromaddr, toaddr, msubject, mbody)
except requests.exceptions.Timeout as errtimeout:

#### RETRY 3 TIMES AND IF SUCCESSFUL, CONTINUE TO NEXT EXCEPT STATEMENT.  IF UNSUCCESSFUL, SEND THE EMAIL

    print ("Timeout Error:",errtimeout)
    mbody = "The process timed out after 3 tries and gave the error: " + str(errtimeout)
    sendmail(fromaddr, toaddr, msubject, mbody)
except requests.exceptions.RequestException as err:
    print ("No idea what the hell happened",err)
    mbody = "The process encountered the following unexpected error: " + str(err)
    sendmail(fromaddr, toaddr, msubject, mbody)

Thank you!

You want to *retry* the `requests.post(endpoint,...` call three times? — wwii, May 05 '21 at 16:34
Is [Python: Try three times a function until all failed](https://stackoverflow.com/questions/25106556/python-try-three-times-a-function-until-all-failed) what you are trying to do? — wwii, May 05 '21 at 16:37
i am trying to do it using base python. as i don't know if our servers have some of these packages installed. — zabada, May 05 '21 at 16:39
The accepted answer uses `functools` which **is** *base* python. But does that Q&A capture the essence of what you are after? — wwii, May 05 '21 at 16:51
I believe it does from looking at it, but i've never used wraps, and don't know how to go about nesting it. thats my main question is how to nest a loop and have it continue if successful. — zabada, May 05 '21 at 17:09
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/231993/discussion-between-zabada-and-wwii). — zabada, May 05 '21 at 17:10

wwii · Accepted Answer · 2021-05-05T21:50:40.537

For the ConnectionError and Timeout wrap a try/except in a while loop and keep track of number of tries in the except clause. return the result if successful or re-raise the exception if the limit is reached. Put all that in a function.

def retry(max_tries=3):
    n = 0
    while True:
        try:
            r = requests.post(endpoint,data=json.dumps(report_params),headers=headers)
            r.raise_for_status()
            return r
        except (ConnectionError,Timeout) as err:
            n += 1
            if n < max_tries:
                continue
            raise

To use it, call the function in a try/except block. In the except clause craft messages based on the type of exception raised.

try:
    r = retry()
except (HTTPError,RequestException,ConnectionError,Timeout) as err:
    print(err)
    # craft a message based on type of error:
    if isinstance(err,ConnectionError):
        # msg = ...
        pass
    elif isinstance(err,Timeout):
        # msg = ...
        pass
    elif isinstance(err,HTTPError):
        # msg = ...
        pass
    if isinstance(err,RequestException):
        # msg = ...
        pass

Or you might want to put all of the message crafting into the function.

class Special(Exception):
    pass

def retry(max_tries=3):
    n = 0
    msg = ''
    while True:
        try:
            r = requests.post(endpoint,data=json.dumps(report_params),headers=headers)
            r.raise_for_status()
            return r
        except (ConnectionError,Timeout) as err:
            # three tries
            n += 1
            if n < max_tries:
                continue
            # craft a message based on type of error:
            if isinstance(err,ConnectionError):
                msg = f'{type(err)} occurred: {str(err)}'
                pass
            elif isinstance(err,Timeout):
                # msg = ...
                pass
            raise
        except (HTTPError,RequestException) as err:
            # craft a message based on type of error:
            if isinstance(err,HTTPError):
                # msg = ...
                pass
            if isinstance(err,RequestException):
                # msg = ...
                pass
        finally:
            raise Special(msg)

Making the usage simpler

try:
    r = retry()
except Special as err:
    print(err.args)
    # send email with err.args

Ideas borrowed from inspired by:
is there a pythonic way to try something up to a maximum number of times?
Retry function in Python
Python: Try three times a function until all failed

nesting a retry loop into a try except block python API

1 Answers1