Restarting a program after exception

Question

I have a program that queries an API every few seconds. Each response triggers a few functions which themselves make some calls to websites and such -- calls that I don't want to blindly trust to succeed. If I catch an exception in foo(), for example, or even in a function that foo() calls, is it possible to restart the program entirely in the except block? Essentially, I want to call queryRepeatedly() upon an exception in one of its sub-functions, without keeping the previous call on the stack.

Of course, I could return marker values and solve this another way, but the program is structured in a way such that the above approach seems much simpler and cleaner.

# Sample "main" function that I want to call
def queryRepeatedly():
    while True:
        foo()
        bar()
        baz()
        time.sleep(15)

def foo():
    # do something
    try:
        foo2() # makes a urllib2 call that I don't trust
    except:
        #restart queryRepeatedly

queryRepeatedly()

Nope, calling `queryRepeatedly` from `foo` will cause the stack to grow with each exception thrown, until you run out of memory. — Kevin, Jul 08 '13 at 18:08

score 33 · Accepted Answer · edited Jul 20 '22 at 02:49

To restart anything, just use a while loop outside the try. For example:

def foo():
    while True:
        try:
            foo2()
        except:
            pass
        else:
            break

And if you want to pass the exception up the chain, just do this in the outer function instead of the inner function:

def queryRepeatedly():
    while True:
        while True:
            try:
                foo()
                bar()
                baz()
            except:
                pass
            else:
                break
        time.sleep(15)

def foo():
    foo2()

All that indentation is a little hard to read, but it's easy to refactor this:

def queryAttempt()
    foo()
    bar()
    baz()

def queryOnce():
    while True:
        try:
            queryAttempt()
        except:
            pass
        else:
            break

def queryRepeatedly():
    while True:
        queryOnce()
        time.sleep(15)

But if you think about it, you can also merge the two while loops into one. The use of continue may be a bit confusing, but see if you like it better:

def queryRepeatedly():
    while True:
        try:
            foo()
            bar()
            baz()
        except:
            continue
        time.sleep(15)

Thanks, very thorough. For logging purposes, would you suggest to use try, except blocks within `foo`, `bar` and `baz` that log the error then just call `raise`? — vroomfondel, Jul 08 '13 at 18:16
@rogaos: That depends. Is there anything that `foo` knows about `foo2`, that the higher level doesn't/shouldn't know? If not, don't put the lower-level `try`. For example, if you're just going to `except Exception as e: logging.error('Caught {!r}'.format(e)`, don't repeat that three times; just do it once at the higher level. — abarnert, Jul 08 '13 at 18:40

djechlin · Answer 2 · 2013-07-08T18:36:07.823

6

Refactor this - you'll get a stackoverflow error sooner or later if you have enough failures.

queryRepeatedly should just be query. It should return void and throw exceptions on failures.

Wrap in something that looks like this, your true queryRepeatedly function?

while True:
    try:
        query()
    except:
        #handle
    time.sleep(15)

All looping, no recursion needed.

Note that you must think carefully about how much of your program you need to restart. From your question it sounded like your actual problem was ensuring the query could try again if it sporadically fails, which is what my solution ensures. But if you want to clean up program resources - say, bounce SQL connections, which may have broken - then you need to think more carefully about how much of your program you need to "restart." In general you need to understand why your query failed to know what to fix, and in the extreme case, the right thing to do is an email or SMS to someone on call who can inspect the situation and write an appropriate patch or fix.

edited Jul 08 '13 at 18:36

answered Jul 08 '13 at 18:06

djechlin

59,258
35
162
290

But what can he put in the `#handle` part? That's the tricky bit, and you haven't answered it. – abarnert Jul 08 '13 at 18:12
1

`void`? What is this `void` you speak of? – user2357112 Jul 08 '13 at 18:12
@abarnert the OP did not ask anything about that. Probably log over AWS SNS then set up a monitor that sends email alerts when there are too many failures in some unit of time. – djechlin Jul 08 '13 at 18:13
@user2357112 means doesn't return anything. `void` would be the type in C-like languages. – djechlin Jul 08 '13 at 18:14
@djechlin: The OP asked how to restart the whole thing on an exception. Your code doesn't do that—unless you put something in the `#handle` part to make it happen. Which you didn't. – abarnert Jul 08 '13 at 18:14
@abarnert okay, sort of. It depends what OP means by "restart entirely." Ultimately that's not possible since there must be some higher laye of the program responsible for restarting the lower layer. The factorization I proposed seems to be a "deep" enough restart for what the OP is actually asking. – djechlin Jul 08 '13 at 18:16
@djechlin +1 for telling me that completely restarting the program is "ultimately impossible." You should add that to your answer -- I think it's a helpful tidbit. – vroomfondel Jul 08 '13 at 18:32
@djechlin: It's pretty obvious that what he wants to restart is the since `query` loop, which is not only not impossible, but easy. And your answer doesn't show any restart at all, so I don't know how you could think it's a deep enough restart. – abarnert Jul 08 '13 at 18:36
@abarnert I don't actually know which loop you're talking about. – djechlin Jul 08 '13 at 18:38
@djechlin: There's only one loop in his program, the `while True` in `queryRepeatedly`. He obviously wants to start that over again. – abarnert Jul 08 '13 at 18:41
@abarnert AFAICT, yes, for the purpose of ensuring the query continues to run every 15 seconds even if errors occur. Which is why I refactored. – djechlin Jul 08 '13 at 18:42
You're both right - the truth is that my question wasn't entirely clear in that there is some setup that I want to run before I call queryRepeatedly. I felt that abarnert's answer fit the question a little bit better, but djechlin's was very helpful as well. – vroomfondel Jul 08 '13 at 18:44

score 3 · Answer 3 · edited Jun 22 '22 at 20:51

First make two files.

One file called run.py and one called forever.py and put them in the same folder.

Go to your terminal within that folder and type chmod +x forever.py

run.py

whatever code you want to run

forever.py

#!/usr/local/lib/python3.7
from subprocess import Popen
import sys

filename = sys.argv[1]
while True:
    print("\nStarting " + filename)
    p = Popen("python3 " + filename, shell=True)
    p.wait()

Open a terminal window from the folder and type this:

python3 ./forever.py run.py

to start run.py and if it fails or has an exception, it'll just start over again.

You now have a template to make sure if a file crashes or has an exception, you can restart it without being around. If this helps you, please give me a vote!

score -5 · Answer 4 · answered Jul 08 '13 at 18:06

-5

In your exception make a recursive call

except:
      queryRepeatedly()

answered Jul 08 '13 at 18:06

sedavidw

11,116
13
61
95

1

This works, sort of, but is bad design, as it leads to an infinite recursion over, say, a few days. – djechlin Jul 08 '13 at 18:07
That will create a stack overflow, as `queryRepeatedly()` contains a `while True` loop. – 2rs2ts Jul 08 '13 at 18:08
1

Thanks, but I asked to do it without recursion – vroomfondel Jul 08 '13 at 18:08
6

I actually am grateful for this answer as forewarning of how not to do it – davidhood2 Oct 05 '16 at 10:58

Restarting a program after exception

4 Answers4

Linked