Python context manager to handle multiple exceptions in a series

Question

I am wanting to use a context manager to catch not just one Exception, like in this answer, but an arbitrary number of Exceptions in series, so that the following code will perform custom code to handle the ModuleNotFoundError three times:

with handle_ModuleNotFoundErrors():
    import this_is_not_installed
    import neither_is_this
    import none_of_these_are_installed

So far, I have the following code, but it will only handle the first ModuleNotFoundError:

from contextlib import contextmanager
import re


@contextmanager
def handle_ModuleNotFoundErrors():
    try:
        yield
    except ModuleNotFoundError as e:
        failed_module = re.search(r"'((?:\\'|[^'])+)'", e.msg).group(1)
        print(f'Handling {failed_module}...')


with handle_ModuleNotFoundErrors():
    import asdfasdf1
    import asdfasdf2
    import asdfasdf3

# Handling asdfasdf1...

A correct answer would have the additional output...

# Handling asdfasdf2...
# Handling asdfasdf3...

Answer 1

Context managers and the with statement alone cannot do that: once an exception takes place, execution will jump to the end of the with block, and, regardless of it being dealt with in the __exit__ call of the context manager, there is no way of resuming the code where it raised an exception.

However, with some other resources of the language, a similar final behavior maybe possible. The way I can imagine doing this is using the with statement to setup debugging with sys.settrace in the target block, and execute each line in a "supervised" way in code related to the context manager.

It turns out, after some trial and error, that settrace cannot be used to suppress exceptions altogether - as the code being traced run in a frame "above" the one running the tracing code itself: one can't add a try/except block around a step in the traced code.

I even tried it in a weird way: copy the source code of each line about to be executed, and run it in a exec function inside the tracing function (the exec can be placed in a try/except block): this can get some results, but it turns out that trying to skip that actual code in the traced function (possible by assigining to frame.f_lineno directly) will cause the next line of code to skip being traced - and if it causes an exception, it raises.

Also, for lines starting loops, if blocks, and such, there is no way the "exec"ing of the source code out of context can keep the proper flow control.

There are ways to complicate this approach, and eventually it could work: one could go down there, and just let lines with flow control statements run "natively", and agree to have any other statement which could raise an exception run twice: once in the controled tracing code, and another one in its "native" traced frame - so fixable exceptions could be fixed before the exception is raised in the native frame. (in the example case, it would involve creating a module dynamically so that ImportError would not be raised)

All in all: not a suitable approach.

Another approach, if one only cares about controlling and suppressing import errors, is feasible, and a lot less hacky: Python's import machinery is highly extensible, and properly used can allow arbitrary import statements to run any code as a fallback. Unfortunatelly, doing it "by the book" can be some work - the docs for importlib (https://docs.python.org/3/library/importlib.html ) open up with a list of 11 PEP documents which describe the importing mechanism.

I have code that does great hacking import by using a shortcut rather than hooking into proper places, and it even does that using a context manager (with statement) - feel free to adapt that: https://github.com/jsbueno/extradict/blob/main/extradict/map_getter.py

(it temporarily monkey patches the builtin __import__ function so that objects existing in the current scope can be used as sources of attributes to be imported, as if they were packages or modules, to the current namespace)