Python `exec()` fails to load a class if it is referenced in a local import

Question

Let's say we have a class Foo in file foo.py like this:

class Foo:
    """Hi, I'm a class"""

Now I want to load that class from another file

from foo import Foo
assert Foo()

great!

Now I know it's not best practice, but we can also use exec to load the class:

exec(open('foo.py').read())
assert 'Foo' in locals()

nice!

Now let's write a function that gives us a choice, which method we want to use to load the class:

def load_class(exec_load=True):
    if exec_load:
        exec(open('foo.py').read())
        clazz = locals()['Foo']
    else:
        from foo import Foo
        clazz = Foo
    return clazz

Now when I run load_class(), I get a KeyError for Foo, which means the class was not loaded. This is unexpected, why wouldn't I get to load the class?

However, when I uncomment the local import statement from foo import Foo like this

def load_class(exec_load=True):
    if exec_load:
        exec(open('foo.py').read())
        clazz = locals()['Foo']
    else:
        #from foo import Foo
        clazz = Foo
    return clazz

it works just fine. Even though that line is never reached! load_class(exec_load=False) of course won't work now because of the missing import...

But how is this possible? How can this one import statement that is never even executed prevent the class from being loaded in the first place?

You really shouldn't be "loading" the class with `exec` in the first place. That causes *way* more problems than you realize, not just this problem. — user2357112, Dec 18 '19 at 18:15
`from foo import Foo` is a fancy assignment statement; its mere presence makes `Foo` a local variable, rather than a global. Further, `exec` without explicit namespace arguments defaults to using the global namespace for both. (That doesn't explain why `globals()['Foo']` raises the same `KeyError`, though, hence this comment instead of an answer.) — chepner, Dec 18 '19 at 18:20
Well, if it was up to me, I wouldn't use exec at all, but we all got to deal with a messy code base from time to time and at the very least I want to understand what's going on here — klamann, Dec 18 '19 at 18:23
[Have you read this](https://stackoverflow.com/questions/1463306/how-does-exec-work-with-locals)? — Izaak van Dongen, Dec 18 '19 at 18:26
@IzaakvanDongen Ah, ok, that clears up my question. (I also misread the documentation for `exec`). With the `import` statement, `Foo` is a local name, not a global name. But `exec` is setting the value of `Foo` using something that doesn't actually update `load_class`'s local scope. — chepner, Dec 18 '19 at 18:35
@chepner: A note: `exec` with no argument for either globals or locals `dict`s defaults to `globals()` and `locals()`, it doesn't default to `globals()` for both. It's only when you explicitly pass a globals `dict` and *not* a locals `dict` that it use the same `dict` for both. — ShadowRanger, Dec 18 '19 at 18:37
In the context of @IzaakvanDongen's link, this particular problem could be solved in the simplest way by simply replacing the entire contents of the `else` block with just `from foo import Foo as clazz`, which avoids a conflict between real locals and the fake "locals" returned by `locals()`. But it's still slightly wonky, as it relies on writing to the `dict` returned by `locals()` (which is not officially supported); probably best to just make a temporary `dict`, e.g. in the `if` block, do `fakelocals = {}`, then do `exec(open('foo.py').read(), fakelocals)`, `clazz = fakelocals['Foo']`. — ShadowRanger, Dec 18 '19 at 18:41
I've added a duplicate which, while the question isn't really related, contains an answer that addresses the local nature of `Foo` due to the `import` statement. — chepner, Dec 18 '19 at 18:49
thanks for the replies and the solutions to the problem. One thing I still don't understand: Why does `from foo import Foo` shadow the name `Foo` in `locals()`, even though it is a local import statement that is not even reached? — klamann, Dec 18 '19 at 18:51
For the same reason if you try to call a built-in before you shadow it in a function it will results in an `UnboundLocalError`. — Jordan Brière, Dec 18 '19 at 18:54
@JordanBrière wow, I totally wasn't aware that this would happen. Even if the assignment to the built-in function is hidden behind an `if False: ...`, it still fails. But in combination with `exec`, I couldn't possibly notice without being aware of this :/ — klamann, Dec 18 '19 at 19:05
Yes, that is because the names are already reserved as local by the compiler (print this at the top of your function: `sys._getframe().f_code.co_varnames`) and I believe the `from` and `import` statements prevent shadowing of these names behind the scenes, most likely caused by some kind of optimization. — Jordan Brière, Dec 18 '19 at 19:09
Another solution: add `global Foo` as first statement in the function. It's all about scope, this FAQ entry explains it quite well: https://docs.python.org/3/faq/programming.html#why-am-i-getting-an-unboundlocalerror-when-the-variable-has-a-value — klamann, Dec 18 '19 at 20:27
This wouldn't have the same behaviour, though. Because now your assignment would be done globally, rather than remaining local. — Jordan Brière, Dec 18 '19 at 20:38

score 2 · Answer 1 · answered Dec 18 '19 at 18:30

If you want to dynamically load objects from a file then you should be using runpy.run_path instead of exec as there is a lot more you have to do than just exec'ing the file in order for it to work properly. E.g.

from runpy import run_path

def load_class(exec_load=True):
    if exec_load:
        clazz = run_path('foo.py')['Foo']
    else:
        from foo import Foo
        clazz = Foo
    return clazz

Python `exec()` fails to load a class if it is referenced in a local import

1 Answers1