11

So recently I understand the concept of function closure.

def outer():
    somevar = []
    assert "somevar" in locals() and not "somevar" in globals()
    def inner():
        assert "somevar" in locals() and not "somevar" in globals()
        somevar.append(5)
        return somevar
    return inner

function = outer()
somevar_returned = function()
assert id(somevar_returned) == id(function.func_closure[0].cell_contents)

As much as I understand, the objective of function closure is to keep an active reference to the object, in order to avoid garbage collection of this object. This is why the following works fine :

del outer
somevar_returned_2 = function()
assert id(somevar_returned) == id(function.func_closure[0].cell_contents)
assert id(somevar_returned) == id(somevar_returned_2)

The thing is (always as much as I understood) before the execution of the inner function, Python rebuild the locals variables dictionary. This dictionary will contains :

  • the function's closure names associated to their cell contents
  • the function's parameters names associated to their default value or the parameter given (and it can overwrite the precedent names)

The question is where do Python store the name binding of the closure ? I can't find it anywhere.

Note: the function's attributes :

>>> print "\n".join("%-16s : %s" % (e, getattr(function, e)) for e in dir(function) if not e.startswith("_") and e != "func_globals")
func_closure     : (<cell at 0x2b919f6bc050: list object at [...]>,)
func_code        : <code object inner at [...], file "<stdin>", line 4>
func_defaults    : None
func_dict        : {}
func_doc         : None
func_name        : inner
FunkySayu
  • 7,641
  • 10
  • 38
  • 61

1 Answers1

12

This depends on the python implementation. I assume you mean CPython.

The __code__ (or func_code) has a co_freevars attribute that contains the name of all non-local variables (they are called "free vars" as if a python function was a logical formula where the arguments and local variables are quantified variables)

From these various attribute you can obtain a mapping from local and non-local names to cells.

In [35]: function.__code__.co_freevars
Out[35]: ('somevar',)

The co_varnames attribute lists all locally define names:

In [36]: function.__code__.co_varnames
Out[36]: ()
In [37]: def outer():
    ...:     somevar = ["stackoverflow"]
    ...:     def inner():
    ...:         x = 1
    ...:         somevar.append(5)
    ...:         return somevar
    ...:     return inner
    ...: 
    ...: function = outer()

In [38]: function.__code__.co_varnames
Out[38]: ('x',)

While co_cellvars says which local names are used by inner functions:

In [43]: outer.__code__.co_cellvars
Out[43]: ('somevar',)

All closure functions have __closure__ attribute. This attribute returns a tuple of cell objects. And The cell object has cell_contents attribute which stores the value of variable.

In [44]: function.__closure__
Out[44]: (<cell at 0x7f4e06b002b8: list object at 0x7f4e06b522c8>,)
In [45]: function.__closure__[0].cell_contents
Out[45]: ["stackoverflow"]
f9n
  • 93
  • 1
  • 5
Bakuriu
  • 98,325
  • 22
  • 197
  • 231
  • 2
    This is a really good answer. Can you just validate / explain how the local dictionnary is rebuild from the parameters and the closures names ? – FunkySayu Aug 26 '15 at 08:53
  • 1
    @FunkySayu I'm looking into it. It seems like `locals()` will call [`PyEval_GetLocals`](https://hg.python.org/cpython/file/tip/Python/ceval.c#l4433) which calls [`PyFrame_FastToLocalsWithError`](https://hg.python.org/cpython/file/tip/Objects/frameobject.c#l867). The thing I don't understand about it is that it references a `co_localsplus` attribute that doesn't seem to be available from python. Anyway you can see that it uses `co_varnames`, `co_freevars` etc. to build the `locals` dictionary. – Bakuriu Aug 26 '15 at 09:05
  • 1
    Maybe it would be great to open another question about it ? Your answer already fit with the question. – FunkySayu Aug 26 '15 at 09:08
  • Added tags `cpython` and `python-2.x`. – FunkySayu Aug 26 '15 at 09:11