259

I'm working on a documentation (personal) for nested matplotlib (MPL) library, which differs from MPL own provided, by interested submodule packages. I'm writing Python script which I hope will automate document generation from future MPL releases.

I selected interested submodules/packages and want to list their main classes from which I'll generate list and process it with pydoc.

The problem is that I can't find a way to instruct Python to load a submodule from a string. Here is an example of what I tried:

import matplotlib.text as text
x = dir(text)
i = __import__('matplotlib.text')
y = dir(i)
j = __import__('matplotlib')
z = dir(j)

And here is a 3-way comparison of above lists through pprint:

enter image description here

I don't understand what's loaded in y object - it's base matplotlib plus something else, but it lacks information that I wanted and that is main classes from matplotlib.text package. It's the top blue coloured part on screenshot (x list).

mkrieger1
  • 19,194
  • 5
  • 54
  • 65
theta
  • 24,593
  • 37
  • 119
  • 159
  • Can you explain why you need to use `__import__(str)` rather than the standard `import` statemetn? – thesamet Jan 03 '12 at 21:53
  • It's because I'll process lists which items are MPL submodules and get their methods paths – theta Jan 03 '12 at 22:00
  • 19
    @thesamet - c'mon - there are endless ideas where you'd want this functionality. When you have a textual configuration of libraries, you can load them by name, which wouldn't quite work with the `import` statement. Here's one example of use: https://djangosnippets.org/snippets/3048/ – Tomasz Gandor May 12 '14 at 04:55

7 Answers7

350

The __import__ function can be a bit hard to understand.

If you change

i = __import__('matplotlib.text')

to

i = __import__('matplotlib.text', fromlist=[''])

then i will refer to matplotlib.text.

In Python 3.1 or later, you can use importlib:

import importlib

i = importlib.import_module("matplotlib.text")

Some notes

  • If you're trying to import something from a sub-folder e.g. ./feature/email.py, the code will look like importlib.import_module("feature.email")

  • Before Python 3.3 you could not import anything if there was no __init__.py in the folder with file you were trying to import (see caveats before deciding if you want to keep the file for backward compatibility e.g. with pytest).

mirekphd
  • 4,799
  • 3
  • 38
  • 59
mzjn
  • 48,958
  • 13
  • 128
  • 248
  • 3
    `importlib` should be avaliable on pypi for < python2.7 – Jeffrey Jose Aug 19 '12 at 12:12
  • 63
    For anyone who's coming here from Google. It should be noted that if you're trying to import something from a sub-folder (for example, `./feature/email.py`) the code will look like `importlib.import_module("feature.email")` – Seanny123 Dec 06 '13 at 07:13
  • 14
    Finally, also remember that you can't import anything if there is no `__init__.py` in the folder with file you are trying to import. – Seanny123 Dec 06 '13 at 07:31
  • 3
    @mzjn This is for `import moduleName` where moduleName is string. How about `from moduleName import *` ? – Nam G VU May 30 '17 at 06:47
  • @NamGVU: What is the problem? The question title is "import module from string variable". Hasn't that been answered? – mzjn May 30 '17 at 07:26
  • You have perfectly answered the OP. I just expand the topic. Thank you any way! – Nam G VU May 30 '17 at 07:53
  • 2
    Just found answer for my question here in case any one needs it https://stackoverflow.com/a/31306598/248616 – Nam G VU May 30 '17 at 07:54
  • 1
    and what if I want to import something like `../../feature/email.py` ? – ioaniatr Aug 29 '18 at 23:08
  • 1
    I found the answer on how to import from parent directory [here](https://forum.glyphsapp.com/t/python-how-to-import-a-module-from-a-parent-path-for-ga/2210/5). – ioaniatr Aug 31 '18 at 16:51
  • 1
    Due to implicit namespace packages, it is possible to import from folders without an `__init__.py`. – MisterMiyagi Feb 03 '20 at 07:29
113

importlib.import_module is what you are looking for. It returns the imported module.

import importlib

# equiv. of your `import matplotlib.text as text`
text = importlib.import_module('matplotlib.text')

You can thereafter access anything in the module as text.myclass, text.myfunction, etc.

mirekphd
  • 4,799
  • 3
  • 38
  • 59
gecco
  • 17,969
  • 11
  • 51
  • 68
  • 2
    An alternative is [`imp.load_source(..)` using the `imp` module](https://docs.python.org/2/library/imp.html#imp.load_source) as suggested in this answer http://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path – Evgeni Sergeev Jun 08 '14 at 06:30
  • 13
    @gecco This is for `import moduleName` where moduleName is string. How about `from moduleName import *` ? – Nam G VU May 30 '17 at 06:57
  • 1
    Anyone figure out @NamGVU question? – Gunner Stone Mar 22 '22 at 23:05
6

spent some time trying to import modules from a list, and this is the thread that got me most of the way there - but I didnt grasp the use of ___import____ -

so here's how to import a module from a string, and get the same behavior as just import. And try/except the error case, too. :)

  pipmodules = ['pycurl', 'ansible', 'bad_module_no_beer']
  for module in pipmodules:
      try:
          # because we want to import using a variable, do it this way
          module_obj = __import__(module)
          # create a global object containging our module
          globals()[module] = module_obj
      except ImportError:
          sys.stderr.write("ERROR: missing python module: " + module + "\n")
          sys.exit(1)

and yes, for python 2.7> you have other options - but for 2.6<, this works.

keen
  • 816
  • 10
  • 11
4

Apart from using the importlib one can also use exec method to import a module from a string variable.

Here I am showing an example of importing the combinations method from itertools package using the exec method:

MODULES = [
    ['itertools','combinations'],
]

for ITEM in MODULES:
    import_str = "from {0} import {1}".format(ITEM[0],', '.join(str(i) for i in ITEM[1:]))
    exec(import_str)

ar = list(combinations([1, 2, 3, 4], 2))
for elements in ar:
    print(elements)

Output:

(1, 2)
(1, 3)
(1, 4)
(2, 3)
(2, 4)
(3, 4)
arshovon
  • 13,270
  • 9
  • 51
  • 69
2

You can also use exec built-in function that execute any string as a Python code.

In [1]: module = 'pandas'
   ...: function = 'DataFrame'
   ...: alias = 'DF'

In [2]: exec(f"from {module} import {function} as {alias}")

In [3]: DF
Out[3]: pandas.core.frame.DataFrame

For me this was the most readable way to solve my problem.

1

Module auto-install & import from list

Below script works fine with both submodules and pseudo submodules.

# PyPI imports
import pkg_resources, subprocess, sys

modules   = {'lxml.etree', 'pandas', 'screeninfo'}
required  = {m.split('.')[0] for m in modules}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing   = required - installed

if missing:
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--upgrade', 'pip'])
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])

for module in set.union(required, modules):
    globals()[module] = __import__(module)

Tests:

print(pandas.__version__)
print(lxml.etree.LXML_VERSION)
Serge Stroobandt
  • 28,495
  • 9
  • 107
  • 102
0

I developed these 3 useful functions:

def loadModule(moduleName):
    module = None
    try:
        import sys
        del sys.modules[moduleName]
    except BaseException as err:
        pass
    try:
        import importlib
        module = importlib.import_module(moduleName)
    except BaseException as err:
        serr = str(err)
        print("Error to load the module '" + moduleName + "': " + serr)
    return module

def reloadModule(moduleName):
    module = loadModule(moduleName)
    moduleName, modulePath = str(module).replace("' from '", "||").replace("<module '", '').replace("'>", '').split("||")
    if (modulePath.endswith(".pyc")):
        import os
        os.remove(modulePath)
        module = loadModule(moduleName)
    return module

def getInstance(moduleName, param1, param2, param3):
    module = reloadModule(moduleName)
    instance = eval("module." + moduleName + "(param1, param2, param3)")
    return instance

And everytime I want to reload a new instance I just have to call getInstance() like this:

myInstance = getInstance("MyModule", myParam1, myParam2, myParam3)

Finally I can call all the functions inside the new Instance:

myInstance.aFunction()

The only specificity here is to customize the params list (param1, param2, param3) of your instance.

prossblad
  • 688
  • 1
  • 7
  • 11