globals and locals in python exec()

Question

I'm trying to run a piece of python code using exec.

my_code = """
class A(object):
  pass

print 'locals: %s' % locals()
print 'A: %s' % A

class B(object):
  a_ref = A
"""

global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)

print local_env

which results in the following output

locals: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
  File "python_test.py", line 16, in <module>
    exec(my_code_AST, global_env, local_env)
  File "My Code", line 8, in <module>
  File "My Code", line 9, in B
NameError: name 'A' is not defined

However, if I change the code to this -

my_code = """
class A(object):
  pass

print 'locals: %s' % locals()
print 'A: %s' % A

class B(A):
  pass
"""

global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)

print local_env

then it works fine - giving the following output -

locals: {'A': <class 'A'>}
A: <class 'A'>
{'A': <class 'A'>, 'B': <class 'B'>}

Clearly A is present and accessible - what's going wrong in the first piece of code? I'm using 2.6.5, cheers,

Colin

* UPDATE 1 *

If I check the locals() inside the class -

my_code = """
class A(object):
  pass

print 'locals: %s' % locals()
print 'A: %s' % A

class B(object):
  print locals()
  a_ref = A
"""

global_env = {}
local_env = {}
my_code_AST = compile(my_code, "My Code", "exec")
exec(my_code_AST, global_env, local_env)

print local_env

Then it becomes clear that locals() is not the same in both places -

locals: {'A': <class 'A'>}
A: <class 'A'>
{'__module__': '__builtin__'}
Traceback (most recent call last):
  File "python_test.py", line 16, in <module>
    exec(my_code_AST, global_env, local_env)
  File "My Code", line 8, in <module>
  File "My Code", line 10, in B
NameError: name 'A' is not defined

However, if I do this, there is no problem -

def f():
  class A(object):
    pass

  class B(object):
    a_ref = A

f()

print 'Finished OK'

* UPDATE 2 *

ok, so the docs here - http://docs.python.org/reference/executionmodel.html

'A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution. The namespace of the class definition becomes the attribute dictionary of the class. Names defined at the class scope are not visible in methods.'

It seems to me that 'A' should be made available as a free variable within the executable statement that is the definition of B, and this happens when we call f() above, but not when we use exec(). This can be more easily shown with the following -

my_code = """
class A(object):
  pass

print 'locals in body: %s' % locals()
print 'A: %s' % A

def f():
  print 'A in f: %s' % A

f()

class B(object):
  a_ref = A
"""

which outputs

locals in body: {'A': <class 'A'>}
A: <class 'A'>
Traceback (most recent call last):
  File "python_test.py", line 20, in <module>
    exec(my_code_AST, global_env, local_env)
  File "My Code", line 11, in <module>
  File "My Code", line 9, in f
NameError: global name 'A' is not defined

So I guess the new question is - why aren't those locals being exposed as free variables in functions and class definitions - it seems like a pretty standard closure scenario.

It seems to be the same issue as in this question: http://stackoverflow.com/questions/2749655/why-are-closures-broken-within-exec — interjay, May 25 '10 at 12:38
thanks for the pointer - I'm not a python guru, but it appears that when I print the locals(), A *has* been compiled to a local variable - i.e. it does know what to do with it. The answer in the question you highlighted says - 'There's no way for compile to know that a is a freevar, so it compiles it to a global reference' The problem here seems to be that locals() is redefined inside the body of B when using exec, but not when using a function (see update in question)? Could easily be my misunderstanding of the implications of that answer though... — hawkett, May 25 '10 at 12:55
I've added a second update to the question which clarifies things a fair bit - at least what is happening, but still not why. — hawkett, May 25 '10 at 14:09

score 20 · Accepted Answer · answered May 25 '10 at 15:41

Well, I believe it's either an implementation bug or an undocumented design decision. The crux of the issue is that a name-binding operation in the module-scope should bind to a global variable. The way it is achieved is that when in the module level, globals() IS locals() (try that one out in the interpreter), so when you do any name-binding, it assigns it, as usual, to the locals() dictionary, which is also the globals, hence a global variable is created.

When you look up a variable, you first check your current locals, and if the name is not found, you recursively check locals of containing scopes for the variable name until you find the variable or reach the module-scope. If you reach that, you check the globals, which are supposed to be the module scope's locals.

>>> exec(compile("import sys\nprint sys._getframe().f_code.co_name", "blah", "exec"), {}, {})
<module>
>>> exec("a = 1\nclass A(object):\n\tprint a\n", {}, {})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 2, in <module>
  File "<string>", line 3, in A
NameError: name 'a' is not defined
>>> d = {}
>>> exec("a = 1\nclass A(object):\n\tprint a\n", d,d)
1

This behavior is why inheritance worked (The name-lookup used code object's scope locals(), which indeed had A in it).

In the end, it's an ugly hack in the CPython implementation, special-casing globals lookup. It also causes some nonsensical artifical situations - e.g.:

>>> def f():
...     global a
...     a = 1
...
>>> f()
>>> 'a' in locals()
True

Please note that this is all my inference based on messing with the interpreter while reading section 4.1 (Naming and binding) of the python language reference. While this isn't definitive (I haven't opened CPython's sources), I'm fairly sure I'm correct about the behavior.

Ok - thanks for that info - that's a hassle :) the main reason I wanted to use locals like that, was to get all the stuff defined in the code string, without all the other stuff that python puts in the globals. If I 'print globals()' inside that code, its a big dictionary of stuff, which makes sense, but now I'm at a loss as to how to get just the stuff that was defined in the code string into a dictionary - i.e. a dictionary with just {'A', , 'B', } in it. I don't want to have to manually strip everything out, and I don't know in advance what's in the code string. — hawkett, May 25 '10 at 16:30
I'm going to mark this question answered - I filed a python bug here - http://bugs.python.org/issue8819 - cheers. — hawkett, May 25 '10 at 16:50
The bug I filed was a duplicate. I've made an argument for this to be fixed in 2.6+ here - http://bugs.python.org/issue991196 — hawkett, May 26 '10 at 08:52
Looks like this is now a 'wont fix' issue. I think that the exec API is very misleading with bad defaults. — Bryce Guinta, Apr 11 '18 at 05:44
I recommend [runpy](https://docs.python.org/3/library/runpy.html). Much better api — Bryce Guinta, Apr 11 '18 at 07:13

score 10 · Answer 2 · edited Sep 27 '12 at 00:47

10

After print locals() and globals(),you will find the reason why exec throws "not defined" exception, and you can try this

d = dict(locals(), **globals())
exec (code, d, d)

edited Sep 27 '12 at 00:47

John Conde

217,595
99
455
496

answered Aug 01 '12 at 07:19

Zoe

101
1
3

This is the only response where you can ship and receive globals / locals back. Fantastic answer. You can then use d as the basis of accessing variables from the child code. d['status'] if you had status={} in your code. Thank you @Zoe – Dovy Mar 31 '17 at 14:45

score 3 · Answer 3 · answered May 27 '12 at 00:59

If your question is how to get the exec statement to behave like the file scope, I followed some hints in the linked question and bug and got it working by passing a single dictionary for globals and locals. Apparently the file scope is a special case where local declarations are automatically placed in the global scope.

exec code in dict()

score 2 · Answer 4 · answered Feb 28 '13 at 02:02

my_code = """
class A(object):
    pass

class B(object):
    a = A
"""

my_code_AST = compile(my_code, "My Code", "exec")
extra_global = "hi"
global_env = {}
exec my_code_AST in global_env
print "global_env.keys() =", global_env.keys()
print "B.a =", global_env["B"].a

prints

global_env.keys() = ['__builtins__', 'A', 'B']
B.a = <class 'A'>

Hawkett, you say,

the main reason I wanted to use locals like that, was to get all the stuff defined in the code string, without all the other stuff that python puts in the globals.

With exec, if your globals don't have __builtins__ defined, exec adds one item, __builtins__ to your globals, so you get A, B, and __builtins__. __builtins__ itself is a big dictionary, but it's always the same one element to delete (as long as you wait until your code is finished using it before you delete it!). Documented under exec() here.

The docs for eval under built in functions say

If the globals dictionary is present and lacks ‘builtins’, the current globals are copied into globals before expression is parsed.

But actually it seems only to copy __builtins__ in.

(And n.b. what everyone else said: either set globals and locals the same or say exec my_code_AST in global_env without a separate local_env.)

globals and locals in python exec()

4 Answers4

Linked

Related