Try finally block with generator in python

Question

Can someone explain me the idea of generator and try except in this code:

from contextlib import contextmanager
 
@contextmanager
def file_open(path):
    try:
        f_obj = open(path, 'w')
        yield f_obj
    except OSError:
        print("We had an error!")
    finally:
        print('Closing file')
        f_obj.close()
 
 
if __name__ == '__main__':
    with file_open('test.txt') as fobj:
        fobj.write('Testing context managers')

As I know, finally is always executed regardless of correctness of the expression in try. So in my opinion this code should work like this: if we haven't exceptions, we open file, go to generator and the we go to finally block and return from the function. But I can't understand how generator works in this code. We used it only once and that's why we can't write all the text in the file. But I think my thoughts are incorrect. WHy?

It is unclear precisely what behavior you have, and what behavior you want. But my initial guess is to `yield from` the file object, rather than simply yielding it. — Cresht, Apr 26 '22 at 22:56
In the `try` block, your code opens the given path in write mode. Then the `f_obj` is yielded by the generator. Then the `f_obj` is closed. If an error occurred when opening the file, the error message is printed, and then you attempt to close the `f_obj` which may or may not have properly opened or even be defined. — ddejohn, Apr 26 '22 at 22:58
@Cresht This is how the example [here](https://book.pythontips.com/en/latest/context_managers.html#implementing-a-context-manager-as-a-generator) does it. It just uses `yield`, not `yield from`. — Barmar, Apr 26 '22 at 23:04
@Cresht `yield`ing the resource object is the correct usage for `contextlib.contextmanager`. See the example in the [docs](https://docs.python.org/3/library/contextlib.html#contextlib.contextmanager) — Brian61354270, Apr 26 '22 at 23:05
However, `try/finally` should be around the `yield`, not around `f_obj = open()`. — Barmar, Apr 26 '22 at 23:05

ShadowRanger · Accepted Answer · 2022-04-26T23:37:48.447

So, one, your implementation is incorrect. You'll try to close the open file object even if it failed to open, which is a problem. What you need to do in this case is:

@contextmanager
def file_open(path):
    try:
        f_obj = open(path, 'w')
        try:
            yield f_obj
        finally:
            print('Closing file')
            f_obj.close()         
    except OSError:
        print("We had an error!")

or more simply:

@contextmanager
def file_open(path):
    try:
        with open(path, 'w') as f_obj:
            yield f_obj
            print('Closing file')
    except OSError:
        print("We had an error!")

To "how do generators in general work?" I'll refer you to the existing question on that topic. This specific case is complicated because using the @contextlib.contextmanager decorator repurposes generators for a largely unrelated purpose, using the fact that they innately pause in two cases:

On creation (until the first value is requested)
On each yield (when each subsequent value is requested)

to implement context management.

contextmanager just abuses this to make a class like this (actual source code is rather more complicated to cover edge cases):

class contextmanager:
    def __init__(self, gen):
        self.gen = gen  # Receives generator in initial state
    def __enter__(self):
        return next(self.gen)  # Advances to first yield, returning the value it yields
    def __exit__(self, *args):
        if args[0] is not None:
            self.gen.throw(*args)  # Plus some complicated handling to ensure it did the right thing
        else:
            try:
                next(self.gen)      # Check if it yielded more than once
            except StopIteration:
                pass                # Expected to only yield once
            else:
                raise RuntimeError(...)  # Oops, it yielded more than once, that's not supposed to happen

allowing the coroutine elements of generators to back a simpler way to write simple context managers.

But as I understand we will never use generator again because of we call `finally` that will close our file. Than how we can write all the text in file if we use generator only once? — jjjjake, Apr 26 '22 at 23:41
I mean why we use generator here? What it gives us? What if we won't use it? — jjjjake, Apr 26 '22 at 23:50
@jjjjake: You use the function with a `with` statement; the `@contextmanager`-decorated generator is a cheap/easy way to define something that works with `with` statements. I already explained why it's used that way; it gives an easy way to write a function that can run halfway (the `__enter__` part of context management run on entering a `with` block), *stop*, then run the rest of the way at some later point (the `__exit__` run on leaving a `with` block). While you're in the `with` block, the `open` has occurred, but the `close` hasn't, so you can do all the `write`s then. — ShadowRanger, Apr 27 '22 at 00:24
@jjjjake: In short, the generator is "run" twice, once to do everything up to (and including) the `yield`, then a second time to do everything after the `yield`. A regular function can't put a pause in the middle where arbitrary things happen before it resumes (well, `async` stuff allows this, but it didn't exist when `contextlib.contextmanager` was invented, and it has a whole ecosystem associated with it so you can't use it without pulling in a bunch of other stuff), but a generator involves such an arbitrarily long pause after *every* `yield`. — ShadowRanger, Apr 27 '22 at 00:30

Try finally block with generator in python

1 Answers1