Why do I get this NameError in a generator within a Python class definition?

Question

In Python 3.5.0 this code:

a = (1,2)
class Foo(object):
    b = (3,4)
    c = tuple((i,j) for j in b for i in a)
    d = tuple((i,j) for i in a for j in b)

produces:

Traceback (most recent call last):
  File "genexprtest.py", line 2, in <module>
    class Foo(object):
  File "genexprtest.py", line 5, in Foo
    d = tuple((i,j) for i in a for j in b)
  File "genexprtest.py", line 5, in <genexpr>
    d = tuple((i,j) for i in a for j in b)
NameError: name 'b' is not defined

Why do I get this error? And why do I not get this error on the previous line?

Answer 1

This is because the expression for i in a has a local variable scope, and expression for j in b is inside the scope, thus, no b is found.
Actually, if you write c = tuple((i, j) for i in a for j in b), it will throw the same exception.

The solution is put b into scope of class definition (as you already did) and refer it by self.b.

Answer 2

I spent ages experimenting and I have a theory about why you're getting this error. I'm not certain but this does explain why it works for c and not for d. I hope this helps you, comment if you disagree :)

def Tuple(this):
    print(a) # this always works
    try:
        print(b) # this always gives an error
    except NameError:
        print("...b is not defined")
    try:
        return tuple(this) # this only gives an error for d and e
    except NameError:
        print("...couldn't make it a tuple")


a = (1,2)     
class Foo(object):
    b = (3,4)
    c = Tuple((i,j) for j in b for i in a)
    d = Tuple((i,j) for i in a for j in b)
    e = Tuple((i,j,k) for i in a for j in b for k in (5, 6))
    f = Tuple((i,j,k) for j in b for i in (5, 6) for k in a)

    print("\nc:", c,"\nd:", d,"\ne:", e,"\nf:", f)

What happened: every time I called the Tuple() function, b was not defined, but a was always defined. This explains why you get an error for d and e but it doesn't explain why c and f work even though b is 'not defined'

My theory: The first for loop is calculated before the whole thing is converted into a tuple. For example, if you tried to do this: Tuple((a, b, c) for a in loop1, for b in loop2 for c in loop3), in the Foo class it would calculate for a in loop1 first, then it would move to the foo and calculate the loops 2 and 3.

In summary:

1. does first for loop
2. moves to tuple function
3. does the remaining loops
4. the error occurs if a variable in the 2nd or 3rd loop is in class Foo

Answer 3

In my opinion, the error arises because b is defined as a class variable. To properly use it, you need to treat it as such (self.b). Also, you should use constructor:

a = (1, 2)

class Foo(object):
    def __init__(self):
        self.b = (3, 4)
        self.c = tuple((i, j) for j in self.b for i in a)
        self.d = tuple((i, j) for i in a for j in self.b)

This is a clearer code. And it behaves properly. Hope it helps.

EDIT: if you don't want to use __init__, there is also a possibility to get c and d using methods:

a = (1, 2)

class Foo(object):
    b = (3, 4)

    def get_c(self):
        return tuple((i, j) for j in self.b for i in a)

    def get_d(self):
        return tuple((i, j) for i in a for j in self.b)

This also works perfectly fine. You can try both implementations like this:

inst = Foo()
# 1st one
print(inst.c)
print(inst.d)
# 2nd one
print(inst.get_c())
print(inst.get_d())

Answer 4

The solution to your specific case is to use itertools:

d = tuple(itertools.product(a, b))

The explanation for the seemingly unexpected behavior is twofold:

1. Bare class attributes such as b are only accessible in the root class scope. See pep 227:

   Names in class scope are not accessible. Names are resolved in the innermost enclosing function scope. If a class definition occurs in a chain of nested scopes, the resolution process skips class definitions.

2. Nested loops in generators don't function like you might expect. The first loop is actually the outermost and the second the innermost. From python docs:

   Subsequent for clauses cannot be evaluated immediately since they may depend on the previous for loop. For example: (x*y for x in range(10) for y in bar(x)).

The second point can be illustrated with added line-breaks.

d = tuple((i,j) 
    for i in a
        for j in b)

This means that b is actually referenced from the inner loop (nested scope) and thus a NameError is thrown. In the first generator however, the reference is in the outer one which works fine.