How to make modules from exec("import xxx") in a Python function available?

Question

I have written myself a simple function that sieves through my Python folder and look for where a possible module is. What I want to do is simple. I pass a string of module import and the function will find the folder of the module, cd there, and import it to whichever environment I am working in, e.g.:

anyimport('from fun_abc import *')

Originally I tried:

class anyimport(object):
    def __init__(self, importmodule, pythonpath='/home/user/Python', finddir=finddir):

        ##################################################################    
        ### A BUNCH OF CODES SCANNING THE DIRECTORY AND LOCATE THE ONE ###
        ##################################################################
        ### "pointdir" is where the directory of the file is ###
        ### "evalstr" is a string that looks like this : ---
        ### 'from yourmodule import *'

        os.chdir(pointdir)
        exec evalstr

As I coded the whole thing in iPython Notebook, it works. So the problem slipped by me. Then I found it does not work properly because the modules the function imports stay in the function's local variable space.

Then I found this Stack Overflow discussion "In Python, why doesn't an import in an exec in a function work?". Thus I changed the code to the following:

class anyimport(object):
    def __init__(self, importmodule, pythonpath='/home/user/Python', finddir=finddir):

        ##################################################################    
        ### A BUNCH OF CODES SCANNING THE DIRECTORY AND LOCATE THE ONE ###
        ##################################################################
        ### "pointdir" is where the directory of the file is ###
        ### "evalstr" is a string that looks like this : ---
        ### 'from yourmodule import *'

        sys.path.append(os.path.join(os.path.dirname(__file__), pointdir))
        exec (evalstr, globals())

It still does not work. The function runs without error but the modules are not available to me, say if I run script.py in which I do anyimport('from fun_abc import *') but nothing from fun_abc is there. Python will tell me "NameError: name 'fun_you_want' is not defined".

Would anyone be kind enough to point me in the right direction as to how to solve this problem?

Thank you for your attention and I really appreciate your help!

Attention:

In addition to @Noya's spot-on answer that one has to pass the scope to make the exec work, to avoid "ImportError", you need to add this line before running exec:

sys.path.append(os.path.join(os.path.dirname(__file__), pointdir))        
exec (evalstr, scope)

This is due to the reason that our modification of sys.path assumes the current working directory is always in main/. We need to add the parent directory to sys.path. See this Stack Overflow discussion "ImportError: No module named - Python" for more information on resolving this issue.

Could you please specify what "it still does not work" means and provide a [minimal reproducable example](http://stackoverflow.com/help/mcve). — Noya, Feb 12 '15 at 15:51
Dear @Noya, thank you for pointing me out. Now it should be clearer. — Harry MacDowel, Feb 12 '15 at 15:56
I think it's because the imported items still end up in the locals of the function, despite what the answer says to that linked question. If that's true -- you can check -- then there's a simple fix. — martineau, Feb 12 '15 at 15:57
Yes the imported items still stay inside the function. When I run the whole thing by defining the function right inside iPython Notebook, it runs correctly. — Harry MacDowel, Feb 12 '15 at 15:58
OK, try adding a `globals().update(locals())` after the `exec`. — martineau, Feb 12 '15 at 16:01
Dear @martineau, do you mean: exec (evalstr, globals()) ; globals().update(locals()) — Harry MacDowel, Feb 12 '15 at 16:04
I tried exec (evalstr, globals()) ; globals().update(locals()) but still no luck...thanks @martineau! — Harry MacDowel, Feb 12 '15 at 16:06
Yes, each on a separate line. This will update the globals of whatever module `anyimport` is in. If you want the globals of the user of the class to be updated, you'll have to pass the globals to be updated to the method as an argument. — martineau, Feb 12 '15 at 16:10
Maybe you can try the minified example below and report if it works for you. — Noya, Feb 12 '15 at 16:24
As commented on the question you cited: "why would you want to do this?" and "This is sign of bad design - avoid." — msw, Feb 12 '15 at 16:24

score 5 · Accepted Answer · edited May 23 '17 at 11:52

5

exec executes code in the current scope. Inside a function, this means the (function-) local scope.

You can tell exec to put variables in another scope by giving it a tuple (code, scope). For example, you can use globals() to make names available at the module level.

Be aware, that globals

is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).

Thus, in your example, you have to pass the desired scope to your utility function:

anyimport.py:

class anyimport(object):
    def __init__(self, importmodule, scope):
        exec (importmodule, scope)

test.py:

a = 42
b = 'foo'

main.py:

from anyimport import anyimport

if __name__ == '__main__':
    anyimport('from test import *', globals())
    # 42 foo
    print a, b

Test it with python main.py. Make sure all files are present in the current directory.

Alternative Solution

If you are not bound to use exec, a more elegant way would be to use the import utilities provided by Python.

The following, taken from https://stackoverflow.com/a/4526709/453074 , is equivalent to from some.package import *:

[...] it's more convenient to user importlib:

globals().update(importlib.import_module('some.package').__dict__)

.

edited May 23 '17 at 11:52

Community

1
1

answered Feb 12 '15 at 16:22

Noya

151
5

Thank you! I have tested your codes and they work correctly! Motivated, I alter my codes accordingly and now a new issue has come up: "ImportError: No module named fun_abc", and I did a `os.listdir(pointdir)` check, the .py file is there! What is going wrong?? – Harry MacDowel Feb 12 '15 at 16:46
I added `sys.path.append(os.path.join(os.path.dirname(__file__), pointdir))` and turns out I am back to square one, the error becomes "NameError: name 'getnearpos' is not defined" again. =.= – Harry MacDowel Feb 12 '15 at 16:52
To be of further assistance, one would need a [minimal example](http://stackoverflow.com/help/mcve) of your code (like in this answer), that provokes the error you describe. – Noya Feb 12 '15 at 16:57
I think I have reproduced the error with your codes. You may try to put `test.py` in another folder. Then create a `tryimport.py` with the codes under `if __name__ == '__main__'`. In `tryimport.py`, you would need to import the anyimport from the `anyimport.py` first. Then try to do `python tryimport.py`. I find the error `Traceback (most recent call last): File "stack.py", line 9, in ; print a, b; NameError: name 'a' is not defined` – Harry MacDowel Feb 12 '15 at 17:23
NICE! It works!! Thanks @Noya for the explicit answer and very kind follow-ups!! – Harry MacDowel Feb 13 '15 at 03:02

score 2 · Answer 2 · 2015-02-12T17:23:19.503

2

You might want to try something like this:

_globals = {}
code = """import math;"""
code += """import numpy;"""
code = compile(code, '<string>', 'exec')
exec code in _globals

It's safer than just doing an exec, and it should import correctly inside a function's local scope.

You can then update globals() with whatever modules (or functions) you import.

When using exec for functions, you can get a handle to globals with g = globals(), then do an update on g. For modules, you should do another step... you will want to also update the modules in sys.modules as well.

UPDATE: to be explicit:

>>> def foo(import_string):
...   _globals = {}
...   code = compile(import_string, '<string>', 'exec')
...   exec code in _globals
...   import sys
...   g = globals()
...   g.update(_globals)
...   sys.modules.update(_globals)
... 
>>> foo('import numpy as np')
>>> np.linspace
<function linspace at 0x1009fc848>

edited Feb 12 '15 at 17:23

answered Feb 12 '15 at 16:12

Dear @Wally, I tried to implement your code like this: `def trytryimport(in_string): _globals = {}; code = compile(in_string, '', 'exec'); exec code in _globals` and I call it from another script by doing the simple `import os; os.chdir("/home/General"); from trytryimport import trytryimport; trytryimport("""import numpy as np;"""); x = np.linspace(0,10,100)` yet the same error comes up "NameError: name 'np' is not defined" – Harry MacDowel Feb 12 '15 at 16:23
If you tried exactly what you typed above in your comment, then it looks like you forgot to update `np` from the function `locals` to `globals` and then update `sys.modules`. – Feb 12 '15 at 17:16
Thank you @Wally. I updated with the latest version of your code. I still have the "NameError: name 'np' is not defined". I think perhaps I should restart my computer ... – Harry MacDowel Feb 12 '15 at 17:36

How to make modules from exec("import xxx") in a Python function available?

Attention:

2 Answers2

Alternative Solution