Why does my generator expression raise NameError on class attributes?

Question

class test(object):
    data = 1
    dataSet = tuple(data for i in range(2))

This snippet of code is the simplest way to get a NameError with a <genexpr>.

Thanks to Jim, I know the simplest way to fix it is to simply use a lambda :

class test(object):
    data = 1
    dataSet = (lambda d = data: tuple(d for i in range(2)))()

I know I could also decorate the class to modify make the tuple when the class object is fully created.

def tuple_data(cls):
    setattr(cls, 'dataSet', tuple(getattr(cls, 'dataSet')))
    return cls

and now:

@tuple_data
class test(object):
    data =  1
    dataSet = (test.data for i in range(2))

I'm mainly trying to understand the computing behind the scenes.

By Jim's answer, the <genexpr> cannot access the class data since it doesn't exist when the <genexpr> is evaluated. By this answer, the <genexpr> just doesn't have access to the class' namespace, only to its own local scope. Problem with this is a <genexpr> uses LOAD_GLOBAL which is rather confusing.

On top of that, Jim's solutions are working but I cannot understand how exactly. I guess the decorator solutions works simply because the decorator modify the class functionnality either once it is fully created or when it is called to be instantiated. On the other hand, I cannot understand why the lambda is working.

Answer 1

You could get this to work by not using a comprehension and instead multiplying a single element list by the amount you need to initialize the tuple:

data = tuple([dataSet] * 2)

Alternatively, for highlighting how ugly but interesting lambdas can be, passing it as an argument to a lambda function and calling it:

data = (lambda dataSet=dataSet: tuple(dataSet for i in range(2)))()

should do the trick.

---

As for the reasoning behind this, I cannot seem to find any hard documentation, PEP 289 simply states that users are encouraged to create generator expressions only inside functions and fall-back to function generators for more complex scenarios. It just seems to be something that just is that way.

It is apparently compiled this way. I went ahead and examined the code object for a class that uses a generator expression to see what was going on, first, define a class:

cls = """
class test(object):
    i = 10
    d = tuple(i for j in range(2))
"""

Then, compile it:

cls_code = compile(cls, "stdin", mode = 'exec')

and now, we disassemble it and examine the contents of the class definition which is located in cls_code.co_consts[0]:

import dis
dis.dis(cls_code.co_consts[0])
  2           0 LOAD_NAME                0 (__name__)
              3 STORE_NAME               1 (__module__)
              6 LOAD_CONST               0 ('test')
              9 STORE_NAME               2 (__qualname__)

  3          12 LOAD_CONST               1 (10)
             15 STORE_NAME               3 (i)

  4          18 LOAD_NAME                4 (tuple)
             21 LOAD_CONST               2 (<code object <genexpr> at 0x7ff3e08584b0, file "stdin", line 4>)
             24 LOAD_CONST               3 ('test.<genexpr>')
             27 MAKE_FUNCTION            0
             30 LOAD_NAME                5 (range)
             33 LOAD_CONST               4 (2)
             36 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             39 GET_ITER
             40 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             43 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             46 STORE_NAME               6 (d)
             49 LOAD_CONST               5 (None)
             52 RETURN_VALUE

We see that inside the class, loading and storing is done via the LOAD_NAME opcode (which uses the co_names tuple which contains the names used inside the code object).

The generator expression, on the other hand, loads names differently, the code object for the generator expression is located in in cls_code.co_consts[0].co_consts[2] (it is a constant in the code object for the class definition, if we disassemble it, we can see that it uses LOAD_GLOBAL when it tries to look up i:

dis.dis(cls_code.co_consts[0].co_consts[2])
  4           0 LOAD_FAST                0 (.0)
        >>    3 FOR_ITER                11 (to 17)
              6 STORE_FAST               1 (j)
              9 LOAD_GLOBAL              0 (i)
             12 YIELD_VALUE
             13 POP_TOP
             14 JUMP_ABSOLUTE            3
        >>   17 LOAD_CONST               0 (None)
             20 RETURN_VALUE

---

Another way around this is by noticing that you're actually 'executing' the generator expression because you wrapped it in a tuple call, since it is executing before the class has been created, a reference of the form class_name.attr will fail.

If you defined your class without the tuple call, you could get it to work like this:

class test(object):
    dataSet = damage("ranged", "energy", 1, 0, 0.5, 0, 0, -0.5, 0,    0, False)
    data = (test.dataSet for i in range(2)

And then, after the class has been created, you can now wrap it in a tuple call and the look up for test.dataSet will succeed:

test.data = tuple(test.data)

If this is something you don't want to do manually, you could always create a decorator to do it for you automatically:

def tuple_data(cls):
    setattr(cls, 'data', tuple(getattr(cls, 'data')))
    return cls

and now:

@tuple_data
class test(object):
    dataSet = damage("ranged", "energy", 1, 0, 0.5, 0, 0, -0.5, 0,    0, False)
    data = (test.dataSet for i in range(2)

Will pass data to tuple after the class test has been created.