Python equivalent of golang's defer statement

Question

How would one implement something that works like the defer statement from go in python?

Defer pushes a function call to a stack. When the function containing the defer statement returns, the defered function calls are popped and executed one by one, in the scope that the defer statement was inside in the first place. Defer statements look like function calls, but are not executed until they are popped.

Go example of how it works:

func main() {
    fmt.Println("counting")

    var a *int
    for i := 0; i < 10; i++ {
        a = &i
        defer fmt.Println(*a, i)
    }

    x := 42
    a = &x

    fmt.Println("done")
}

Outputs:

counting
done
9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2
1 1
0 0

Go example of a usecase:

var m sync.Mutex
func someFunction() {
    m.Lock()
    defer m.Unlock()
    // Whatever you want, with as many return statements as you want, wherever.
    // Simply forget that you ever locked a mutex, or that you have to remember to release it again.
}

Answer 1

To emulate defer fmt.Println(*a, i) example, you could use contextlib.ExitStack:

#!/usr/bin/env python3
from contextlib import ExitStack
from functools import partial

print("counting")
with ExitStack() as stack:
    for i in range(10):
        a = i
        stack.callback(partial(print, a, i))

    x = 42
    a = x
    print("done")

Output

counting
done
9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2
1 1
0 0

It is easy to emulate the mutex case:

def some_function(lock=Lock()):
    with lock:
        # whatever

Answer 2

Python's with statement serves a similar purpose to Go's defer.

The similar code in Python is:

mutex = Lock()

def someFunction():
    with mutex:
        # Whatever you want, with as many return statements
        # as you want, wherever. Simply forget that you ever
        # locked a mutex, or that you have to remember to 
        # release it again.

Answer 3

I've made one there (compatible with 2.x):

@defers_collector
def func():
    f = open('file.txt', 'w')
    defer(lambda: f.close())

    defer(lambda : print("Defer called!"))

    def my_defer():
    recover()

    defer(lambda: my_defer())

    print("Ok )")
    panic("WTF?")

    print("Never printed (((")


func()
print("Recovered!")

Source of defers_collector is:

# Go-style error handling

import inspect
import sys

def panic(x):
    raise Exception(x)

def defer(x):
    for f in inspect.stack():
    if '__defers__' in f[0].f_locals:
        f[0].f_locals['__defers__'].append(x)
        break

def recover():
    val = None
    for f in inspect.stack():
    loc = f[0].f_locals
    if f[3] == '__exit__' and '__suppress__' in loc:
        val = loc['exc_value']
        loc['__suppress__'].append(True)
        break
    return val

class DefersContainer(object):
    def __init__(self):
    # List for sustain refer in shallow clone
    self.defers = []

    def append(self, defer):
    self.defers.append(defer)

    def __enter__(self):
    pass

    def __exit__(self, exc_type, exc_value, traceback):
    __suppress__ = []
    for d in reversed(self.defers):
        try:
            d()
        except:
            __suppress__ = []
            exc_type, exc_value, traceback = sys.exc_info()
    return __suppress__


def defers_collector(func):
    def __wrap__(*args, **kwargs):
    __defers__ = DefersContainer()
    with __defers__:
        func(*args, **kwargs)
    return __wrap__

Answer 4

A defer implementation partly inspired by @DenisKolodin answer is available as part of pygolang, 2:

   wc = wcfs.join(zurl)    │     wc = wcfs.join(zurl)
   defer(wc.close)         │     try:
                           │        ...
   ...                     │        ...
   ...                     │        ...
   ...                     │     finally:
                           │        wc.close()

Answer 5

This complement to jfs' answer pushes the ExitStack idea a bit further with the help of decorators:

@with_exit_stack
def counting(n, stack):
    for i in range(n):
        stack.callback(print, i)


@with_exit_stack
def locking(lock, stack):
    stack.enter_context(lock)
    # whatever

with_exit_stack is defined as follows:

import functools
import contextlib

def with_exit_stack(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        with contextlib.ExitStack() as stack:
            return func(*args, **kwargs, stack=stack)

    return wrapper

Answer 6

I've tried to make an equivalent for fun (only tested as a proof of concept)

Here is is:

import os
import inspect


class defer:
    """
    Proof of concept for a python equivalent of golang's defer statement

    Note that the callback order is probably not guaranteed

    """
    def __init__(self, callback, *args, **kwargs):
        self.callback = callback
        self.args = args
        self.kwargs = kwargs

        # Add a reference to self in the caller variables so our __del__
        # method will be called when the function goes out of scope
        caller = inspect.currentframe().f_back
        caller.f_locals[b'_' + os.urandom(48)] = self

    def __del__(self):
        self.callback(*self.args, **self.kwargs)

Usage example:

def main():
    first()
    second()

def first():
    print('- first')
    defer(lambda: print('   - deferred'))
    print('- first exit')

def second():
    print('- second')      

if __name__ == '__main__':
    main()

Answer 7

Little late to the party but I built a pip-installable library that does just this! Check out python-defer.

from defer import defer

def foo():
  print(", world!") in defer
  print("Hello", end="")
  # do something that might fail...
  assert 1 + 1 == 3

$ python foo.py
Hello, World!
Traceback (most recent call last):
  File "foo.py", line 7, in <module>
    assert 1 + 1 == 3
AssertionError