1

Below are snippets of my codes, what I wanted to know is hypothetically if function main2() throws out an error for some reason, how do I get my exception to run the same function again say 3 times before it breaks?

Just to add here, any of the functions could throw an error (not just main2()) also I might have not just 3 but many more functions

import numpy as np

def main():
    np.load('File.csv')

def main1():
    np.load('File1.csv')

def main2():
    np.load('File2.csv')

for i in range(1, 10):
    try:
        main()
        main2()
        main3()
    except Exception as e:
        print e
    else:
        break
MockinJay
  • 86
  • 1
  • 7
user13412850
  • 509
  • 1
  • 5
  • 16

5 Answers5

7

You could do it with python retry decorator

@retry((Exception), tries=3, delay=0, backoff=0)
def main2():
   np.load('File2.csv')

This would work the same way as if you would write:

error_counter = 0
    def main2():
       try:
          np.load('File2.csv')
       except:
          if error_counter < 3
             error_counter += 1
             main2()
          raise Exception("Will not try again, have tried 3 times")  
       error_counter = 0

If you want to make it robust and clean, you should go for the first solution. The first solution you can reuse in a large enterprise project and due to the backoff time it can take disk load,user load network issues into consideration with backoff/delay time.

If you don't use a time delay, you will make all the 3 tries in just a second. That is fine for some kind of exceptions but when having network issues, or disk issues you want a more sophisticated solution.

Also, consider to not catch all exceptions, it is a bad practice to cache all. More info, why it is bad

Fredrik
  • 484
  • 4
  • 13
  • thanks I just amended my original question. its not just main2() that could throw as error it was a hypothetical example. any of the 3 functions could throw an error and tbh very likely I will have probably 6-7 functions. so I would need to flexibility to re-run whichever function is throwing an error – user13412850 Jun 15 '20 at 14:11
  • You can use the decorator above each function that needs it and specify what exception you want to cache. I only included main2 as an example based on original question. – Fredrik Jun 15 '20 at 14:13
  • that seems to be working for me, I will have to test it a bit more but looks good so far. thanks agian. – user13412850 Jun 15 '20 at 16:03
4

Here's an idiom you can try:

for _ in range(3):  # try 3 times
    try:
        main2()
        break       # as soon as it works, break out of the loop
    except Exception as e:
        print e
        continue    # otherwise, try again
else:               # if the loop exited normally, e.g. if all 3 attempts failed
    pass
    # do_something...

Note the indentation. The else here is attached to the for, not the try.

Green Cloak Guy
  • 23,793
  • 4
  • 33
  • 53
  • thanks, I think what I need here is the flexibility that any of the three functions could throw an error (not just main2()). and tbh honest I might have even more functions over time so how would that work? – user13412850 Jun 15 '20 at 14:07
  • Have a different `try`/`except` fixture for each of them? Depends on what your three functions do, and how they connect with each other. One thing you could do is make this into a function where you pass in `n` number of times to try and `main2` the function to run, and then just call that function on whatever function you want to run. Which would allow you to run it more concisely. – Green Cloak Guy Jun 15 '20 at 15:28
1

Unfortunately, implementing a custom retry decorator can often be a bit of a pain. If you want to tweak their logic or adjust them, it can actually get quite complicated quite fast. There's a Python library out there called Backoff-Utils that supports very robust and easily-extensible retry / backoff strategies (full disclosure: I'm biased, since I'm the author of that library).

In your hypothetical question, you could use the library in a decorator-based strategy:

from backoff_utils import backoff, apply_backoff, strategies

@apply_backoff(strategies.Fixed, max_tries = 3, catch_exceptions = [type(ValueError)])
def main2():
    # np.load('File2.csv')
    raise ValueError
    print("In main2")

or you could use it in a function-based strategy when calling main2():

result = backoff(main2,
                 max_tries = 3,
                 catch_exceptions = [type(ValueError)],
                 strategy = strategies.Fixed)

Of course, the code snippet above is specifically designed to do exactly what you described up above. It uses a linear strategy (just retrying 3 times, with a default delay of 1 second between attempts).

Using the library, you can employ any number of other retry / delay strategies, including Exponential Backoff, Fibonnaci, Linear Progression, and Polynomial. You can also customize and create your own delay strategies, as well. And you can incorporate custom success / failure handlers, and different alternative paths for different types of situations.

Of course, all of this flexibility is overkill for your particular use case - you don't need that much. But it may be easier than worrying about copying/pasting/maintaining your own retry decorator and gives you additional built-in options if you find you need more sophisticated retry strategies down the road.

In case it's helpful, you can find some pretty detailed documentation here: https://backoff-utils.readthedocs.io/en/latest/index.html

Hope this helps!

Chris Modzelewski
  • 1,351
  • 10
  • 10
0

You can try

for i in range(1, 10):
    error_max = 3
    error_counter = 0
    try:
        main()
        try:
            main2()
        except Exception as e:
            counter += 1
            if counter == 3:
                raise e
            else:
                continue
        main3()
    except Exception as e:
        print e
    else:
        break

This code will run function main2() until it will get 3 errors and on the first 2 it will make the loop run again.

Leo Arad
  • 4,452
  • 2
  • 6
  • 17
0

You should handle all the errors within specific functions, otherwise if handling errors of all the functions together, the function coming prior to other functions will throw error and the control will execute the except block skipping rest of the code below it in the try block. Try it yourself:

def main():
    # np.load('File.csv')
    raise ValueError
    print("In main")

def main1():
    # np.load('File1.csv')
    raise ValueError
    print("In main1")

def main2():
    # np.load('File2.csv')
    raise ValueError
    print("In main2")

for i in range(1, 10):
    try:
        main()
        main2()
        main3()
    except Exception as e:
        print(e)
    else:
        break

Try commenting raised errors in the functions in different order. And when the errors are handled within each function then every function is executed without any skipping of the remaining functions in the loop

MockinJay
  • 86
  • 1
  • 7