Why is a Python generator confusing its scope with global in an exec'd script?

Question

Ok, so I'm working in an environment where a config script for a tool is an exec'd python script. The exec call is something like this:

outer.py:

exec(open("inner.py").read(), globals(), {})

Now, I want to do some relatively basic iteration within the exec'd script. In this case, doing work when some values aren't in the whitelist:

inner.py:

items = (
  'foo/bar',
  'foo/baz',
  'foof',
  'barf/fizz',
)

whitelist = (
  'foo/',
)

for key in items:
  try:
    # Not terribly efficient, but who cares; computers are fast.
    next(True for prefix in whitelist if key.startswith(prefix))

  # Do some work here when the item doesn't match the whitelist.
  except StopIteration:
    print("%10s isn't in the whitelist!" % key)

---

Running python inner.py yields the expected result:

      foof isn't in the whitelist!
 barf/fizz isn't in the whitelist!

Here's the strange part: Running python outer.py seems to show that the interpreter is confused about the scope of the generator:

Traceback (most recent call last):
  File "outer.py", line 1, in <module>
    exec(open("inner.py").read(), globals(), {})
  File "<string>", line 15, in <module>
  File "<string>", line 15, in <genexpr>
NameError: global name 'key' is not defined

---

Some other notes:

• You can print(key) just fine inside the for loop (before the generator is run).

• Replacing the empty dict with locals() in the exec line of outer.py resolves the issue, but that code is out of my control.

• I'm running the OS X built Python 2.7.2 (Mountain Lion).

Answer 1

Dodgy indeed.

From the doc:

If two separate objects are given as globals and locals, the code will be executed as if it were embedded in a class definition.

(Note that when you run exec(..., globals(), locals()) at module level, this does not apply, because globals() is locals(), i.e. not two separate objects).

This indicates you can simply reproduce the problem by running this script:

class A(object):

  items = (
    'foo/bar',
    'foo/baz',
    'foof',
    'barf/fizz',
  )

  whitelist = (
    'foo/',
  )

  for key in items:
    try:
      # Not terribly efficient, but who cares; computers are fast.
      next(True for prefix in whitelist if key.startswith(prefix))
      # Found!
      print(key)
    except StopIteration:
      pass

"Ok, but why do I get the error here?"

So glad you asked. The answer is here.

Answer 2

Why don't you just add an inner for loop there:

for key in items:
    for prefix in whitelist:
        if key.startswith(prefix):
            print(key)

I'm pretty sure you won't get that error and it's much simple/easier to read.