6

I want to implement a way to repeat a section of code as many times as it's needed using a context manager only, because of its pretty syntax. Like this:

with try_until_success(attempts=10):
    command1()
    command2()
    command3()

The commands must be executed once if no errors happen. And they should be executed again if an error occurred, until 10 attempts has passed, if so the error must be raised. For example, it can be useful to reconnect to a data base. The syntax I represented is literal, I do not want to modify it (so do not suggest me to replace it with a kind of for of while statements).

Is there a way to implement try_until_success in Python to do what I want?

What I tried is:

from contextlib import contextmanager

@contextmanager
def try_until_success(attempts=None):
    counter = 0

    while True:
        try:
            yield
        except Exception as exc:
            pass
        else:
            break

        counter += 1
        if attempts is not None and counter >= attempts:
            raise exc

And this gives me the error:

RuntimeError: generator didn't stop after throw()

I know, there are many ways to reach what I need using a loop instead of with-statement or with the help of a decorator. But both have syntax disadvantages. For example, in case of a loop I have to insert try-except block, and in case of a decorator I have to define a new function.

I have already looked at the questions:

How do I make a contextmanager with a loop inside?

Conditionally skipping the body of Python With statement

They did not help in my question.

Fomalhaut
  • 8,590
  • 8
  • 51
  • 95
  • 1
    The problem with reinventing the way a well-understood and well-defined keyword operates is that you easily make it really hard for other people to understand and use your code. I'd strongly recommend not going this route and instead implement your code with loops and try-catch blocks, making it explicit ([The zen of Python](https://www.python.org/dev/peps/pep-0020/) states _"explicit is better than implicit"_). – Ted Klein Bergman Nov 10 '20 at 14:03
  • 1
    If you want to do a thing repeatedly, that's what a loop is for. Context managers don't support being used as loops. – user2357112 Nov 10 '20 at 14:03
  • @TedKleinBergman If you find it implicit, so why in Python I can do the same in decorators? – Fomalhaut Nov 10 '20 at 14:06
  • @user2357112supportsMonica I do not refuse loops. I just want to implement a loop once inside of try_until_success. – Fomalhaut Nov 10 '20 at 14:08
  • 1
    @Fomalhaut I don't understand what you mean. You can do everything in Python. If you want `a += 1` to start up a multithreaded game, you can. But you have to code with people's expectations in mind. So use constructs for their intended purpose and only break them if it's really necessary. – Ted Klein Bergman Nov 10 '20 at 14:08
  • 1
    The decorator typically *creates* a new function that might wrap a loop. The `with` statement cannot do that; it can call an `__enter__` method, but that method gets no information about the body of the `with` statement. – chepner Nov 10 '20 at 14:09
  • Context managers do exactly three things: run code on entry, run code on exit, and optionally suppress exceptions. They don't get to control execution of the `with` body. This isn't like Ruby's `yield`, where you get to create your own control flow constructs. – user2357112 Nov 10 '20 at 14:11
  • The `contextmanager` decorator doesn't change how the `with` statement works; it only provides a different model for defining an object with appropriate `__enter__` and `__exit__` methods. – chepner Nov 10 '20 at 14:11
  • 1
    @Fomalhaut I've been trying to think of similar examples, and the closest I can think of is how [timeouts are implemented in Trio](https://trio.readthedocs.io/en/stable/reference-core.html#cancellation-api-details). they only do cancellation, but a similar method to your name seems useful, just not compatible with existing semantics – Sam Mason Nov 10 '20 at 14:22

3 Answers3

2

This goes against how context managers were designed to work, you'd likely have to resort to non-standard tricks like patching the bytecode to do this.

See the official docs on the with statement and the original PEP 343 for how they are expanded. It might help you understand why this isn't going to be officially supported, and maybe why other commenters are generally saying this is a bad thing to try and do.

As an example of something that might work, maybe try:

class try_until_success:
    def __init__(self, attempts):
        self.attempts = attempts
        self.attempt = 0
        self.done = False
        self.failures = []
    
    def __iter__(self):
        while not self.done and self.attempt < self.attempts:
            i = self.attempt
            yield self
            assert i != self.attempt, "attempt not attempted"
    
        if self.done:
            return
        
        if self.failures:
            raise Exception("failures occurred", self.failures)
        
    def __enter__(self):
        self.attempt += 1
    
    def __exit__(self, _ext, exc, _tb):
        if exc:
            self.failures.append(exc)
            return True
        
        self.done = True

for attempt in try_until_success(attempts=10):
    with attempt:
        command1()
        command2()
        command3()

you'd probably want to separate out the context manager from the iterator (to help prevent incorrect usage) but it sort of does something similar to what you were after

Sam Mason
  • 15,216
  • 1
  • 41
  • 60
2

The problem is that the body of the with statement does not run within the call to try_until_success. That function returns an object with a __enter__ method; that __enter__ method calls and returns, then the body of the with statement is executed. There is no provision for wrapping the body in any kind of loop that would allow it to be repeated once the end of the with statement is reached.

chepner
  • 497,756
  • 71
  • 530
  • 681
1

Is there a way to implement try_until_success in Python to do what I want?

Yes. You don't need to make it a context manager. Just make it a function accepting a function:

def try_until_success(command, attempts=1):
    for _ in range(attempts):
        try:
            return command()
        except Exception as exc:
            err = exc

    raise err

And then the syntax is still pretty clear, no for or while statements - not even with:

attempts = 10
try_until_success(command1, attempts)
try_until_success(command2, attempts)
try_until_success(command3, attempts)
Tomerikoo
  • 18,379
  • 16
  • 47
  • 61