0

I have a python function that runs other functions.

def main():
    func1(a,b)
    func2(*args,*kwargs)
    func3()

Now I want to apply exceptions on main function. If there was an exception in any of the functions inside main, the function should not stop but continue executing next line. In other words, I want the below functionality

def main():
    try:
        func1()
    except:
        pass
    try:
        func2()
    except:
        pass
    try:
        func3()
    except:
        pass

So is there any way to loop through each statement inside main function and apply exceptions on each line.

for line in main_function:
    try:
        line
    except:
        pass

I just don't want to write exceptions inside the main function.

Note : How to prevent try catching every possible line in python? this question comes close to solving this problem, but I can't figure out how to loop through lines in a function.

If you have any other way to do this other than looping, that would help too.

Uchiha Madara
  • 984
  • 5
  • 16
  • 35

2 Answers2

2

You could use a callback, like this:

def main(list_of_funcs):
    for func in list_of_funcs:
        try:
           func()
        except Exception as e:
           print(e)

if __name__ == "__main__":
    main([func1, func2, func3])
ConorSheehan1
  • 1,640
  • 1
  • 18
  • 30
  • Is there a way to dynamically generate the list_of _functions ? Because the `main` function is generated automatically and I can't edit it in run time. Also the functions inside `main` can have arguments. – Uchiha Madara Oct 12 '17 at 08:35
  • Python functions are higher order functions, so you can replace them at runtime! It's the same mechanism that allows you to pass the list_of_functions to the main function. That said, if you really want to dynamically generate list_of_functions at runtime, you could try looking into the inspect module. See [this question](https://stackoverflow.com/questions/427453/how-can-i-get-the-source-code-of-a-python-function) as an example of reading the source of a function. I think that will only work if the function source is written to a file though. – ConorSheehan1 Oct 12 '17 at 12:59
  • Yeah I tried inspect module too. But I needed to filter all the comments and even after that it fails if a statement is broke down to 2 or more line when executing with `eval`. So it isn't reliable. – Uchiha Madara Oct 12 '17 at 13:09
2

What you want is on option that exists in some languages where an exception handler can choose to proceed on next exception. This used to lead to poor code and AFAIK has never been implemented in Python. The rationale behind is that you must explicitely say how you want to process an exception and where you want to continue.

In your case, assuming that you have a function called main that only calls other function and is generated automatically, my advice would be to post process it between its generation and its execution. The inspect module can even allow to do it at run time:

def filter_exc(func):
    src = inspect.getsource(func)
    lines = src.split('\n')
    out = lines[0] + "\n"
    for line in lines[1:]:
        m = re.match('(\s*)(.*)', line)
        lead, text = m.groups()
        # ignore comments and empty lines
        if not (text.startswith('#') or text.strip() == ""):
            out += lead + "try:\n"
            out += lead + "    " + text + "\n"
            out += lead + "except:\n" + lead + "    pass\n"
    return out

You can then use the evil exec (the input in only the source from your function):

exec(filter_exc(main))  # replaces main with the filtered version
main()                  # will ignore exceptions

After your comment, you want a more robust solution that can cope with multi line statements and comments. In that case, you need to actually parse the source and modify the parsed tree. ast module to the rescue:

class ExceptFilter(ast.NodeTransformer):
    def visit_Expr(self, node):       
        self.generic_visit(node)
        if isinstance(node.value, ast.Call):  # filter all function calls
            # print(node.value.func.id)
            # use a dummy try block
            n = ast.parse("""try:
  f()
except:
  pass""").body[0]
            n.body[0] = node                  # make the try call the real function
            return n                          # and use it
        return node                           # keep other nodes unchanged

With that example code:

def func1():
    print('foo')


def func2():
    raise Exception("Test")

def func3(x):
    print("f3", x)


def main():
    func1()
    # this is a comment
    a = 1
    if a == 1:  # this is a multi line statement
        func2() 
    func3("bar")

we get:

>>> node = ast.parse(inspect.getsource(main))
>>> exec(compile(ExceptFilter().visit(node), "", mode="exec"))
>>> main()
foo
f3 bar

In that case, the unparsed node(*) write as:

def main():
    try:
        func1()
    except:
        pass
    a = 1
    if (a == 1):
        try:
            func2()
        except:
            pass
    try:
        func3('bar')
    except:
        pass

Alternatively it is also possible to wrap every top level expression:

>>> node = ast.parse(inspect.getsource(main))
>>> for i in range(len(node.body[0].body)): # process top level expressions
    n = ast.parse("""try:
  f()
except:
  pass""").body[0]
    n.body[0] = node.body[0].body[i]
    node.body[0].body[i] = n

>>> exec(compile(node, "", mode="exec"))
>>> main()
foo
f3 bar

Here the unparsed tree writes:

def main():
    try:
        func1()
    except:
        pass
    try:
        a = 1
    except:
        pass
    try:
        if (a == 1):
            func2()
    except:
        pass
    try:
        func3('bar')
    except:
        pass

BEWARE: there is an interesting corner case if you use exec(compile(... in a function. By default exec(code) is exec(code, globals(), locals()). At top level, local and global dictionary is the same dictionary, so the top level function is correctly replaced. But if you do the same in a function, you only create a local function with the same name that can only be called from the function (it will go out of scope when the function will return) as locals()['main'](). So you must either alter the global function by passing explicitely the global dictionary:

exec(compile(ExceptFilter().visit(node), "", mode="exec"), globals(),  globals())

or return the modified function without altering the original one:

def myfun():
    # print(main)
    node = ast.parse(inspect.getsource(main))
    exec(compile(ExceptFilter().visit(node), "", mode="exec"))
    # print(main, locals()['main'], globals()['main'])
    return locals()['main']

>>> m2 = myfun()
>>> m2()
foo
f3 bar

(*) Python 3.6 contains an unparser in Tools/parser, but a simpler to use version exists in pypi

Serge Ballesta
  • 143,923
  • 11
  • 122
  • 252
  • I tried that, it becomes messy when the source has comments and multi-line statements. I could use a way that can return each statement of the function as it passes, or a decorator to dynamically insert try-except between each statement – Uchiha Madara Oct 13 '17 at 07:12
  • @UchihaMadara: Your example only contained single line expressions... But you are right, my code filters out comments but would break on multi line statements. – Serge Ballesta Oct 13 '17 at 07:23
  • This is working magically for all the functions I tried so far. I have a couple of questions. – Uchiha Madara Oct 13 '17 at 11:17
  • (1) is there a way to check the source code of the final function ? `inspect.getsource(main)` is throwing `OSError: source code not available`. (2) we defined `visit_Expr` but never used it anywhere and still it works. how ? (3) can we modify this code to cast try-except on every source line (not only functions) of the `main` function ? – Uchiha Madara Oct 13 '17 at 11:29
  • 1
    @UchihaMadara:(1): there is no real source for `main` because it has been compiled directly from a parse tree. You will need an unparser (one exists in pypi) to build source from the parsed tree. (2) this is the magic of ast.NodeTransformer, that method is applied to any node of type ast.Expr. More references in the doc. (3) Would not have sense of every line, but it is possible to try-except every top level (optionaly multi-line) expression. See my edit. – Serge Ballesta Oct 13 '17 at 12:34
  • Yes this works fine. Thanks @Serge Ballesta. There seems to be an issue with `exec` which isn't working when I use it inside a function. – Uchiha Madara Oct 13 '17 at 13:25
  • `def myfun(): print(main); node = ast.parse(inspect.getsource(main)); exec(compile(ExceptFilter().visit(node), "", mode="exec")) ; print(main) ` this function prints same function object before and after `exec`. Which means `exec` isn't working when used inside an other class. Its working fine when used outside of class. any ideas ? – Uchiha Madara Oct 13 '17 at 13:38
  • @UchihaMadara: I could understand the problem and propose 2 different workarounds... – Serge Ballesta Oct 13 '17 at 15:44