Python: Importing files based on which files the user wants

Question

I have the following directory structure

+ code
|
--+ plugins
  |
  -- __init__.py
  -- test_plugin.py (has a class TestPlugin)
  -- another_test_plugin.py (has a class AnotherTestPlugin)
--+ load.py
--+ __init__.py

In load.py, I want to be able to initialize only those classes that the user specifies. For example, lets say I do something like

$ python load.py -c test_plugin # Should only import test_plugin.py and initialize an object of the TestPlugin class

I am having trouble trying to use the "imp" module to do it. It keeps on saying "No such file or directory". My understanding is that it is somehow not understanding the path properly. Can someone help me out with this?

you say you're having trouble trying to use the 'imp' module, but what code did you try so far? what are you having trouble on? what part of the tutorials/documentation don't you understand? stackoverflow is not a place for other people to write code for you! — zmo, May 08 '13 at 10:08
did you have a look at http://stackoverflow.com/questions/10675054/how-to-import-the-module-in-python ? — zmo, May 08 '13 at 10:10
I am trying the following code `import imp` `import.load_source('TestPlugin', 'plugins/test_plugin.py')` — Nikhil Singh, May 08 '13 at 10:34
I have mistype `import.load_source`. It should be `imp.load_source` — Nikhil Singh, May 08 '13 at 10:42
... and how are we supposed to know what's wrong when you: 1) Didn't show the code you are using and 2) Didn't show the *full traceback* you are getting? — Bakuriu, May 08 '13 at 10:46
Sorry about missing out on the details. I was trying to figure out the issue and realized that the plugins directory was inside another directory 'lib'. So, I corrected the path and things are working fine now. — Nikhil Singh, May 08 '13 at 11:00

score 0 · Answer 1 · edited May 08 '13 at 11:13

0

ok, your problem is a path related problem. You expect that the script is being run in the same directory as where load.py is, where it is not the case.

what you have to do is something like:

import imp, os, plugins

path = os.path.dirname(plugins.__file__)
imp.load_source('TestPlugin', os.path.join(path, 'test_plugin.py')

where plugins is the module containing all your plugins (i.e. just the empty __init__.py), that will help you get the full path to your plugin modules' files.

edited May 08 '13 at 11:13

wim

338,267
99
616
750

answered May 08 '13 at 10:50

zmo

24,463
4
54
90

thanks, didn't notice I made that typo :-) – zmo May 08 '13 at 11:39

Markon · Answer 2 · 2013-05-08T13:06:01.443

Another solution, if you want a "plugins" discovery tool:

import imp, os
import glob

def load_plugins(path):
  """
  Assuming `path` is the only directory in which you store your plugins,
  and assuming each name follows the syntax:
  plugin_file.py -> PluginFile
  Please note that we don't import files starting with an underscore.

  """
  plugins = {}
  plugin_files = glob.glob(path + os.sep + r'[!_]*.py')
  for plugin_path in plugin_files:
    module_name, ext = os.path.splitext(plugin_path)
    module_name = os.path.basename(module_name)
    class_name = module_name.title().replace('_', '')
    loaded_module = imp.load_source(class_name, plugin_path) # we import the plugin
    plugins[module_name] = getattr(loaded_module, class_name)
  return plugins

plugins = load_plugins(your_path_here)
plugin_name = sys.argv[3]
plugin = plugins.get(plugin_name)
if not plugin:
    # manage a not existing plugin
else:
    plugin_instance = plugin() # creates an instance of your plugin

This way, you can also specify different names by changing your keys, e.g., 'test_plugins' => 'tp'. You don't have to initialize your plugins, but you can still run this function whenever you want to load your plugins at runtime.

What if the user wrote his own plug-in and would like to load it? The main program cannot hardcode its name, hence a dynamic discovery and *dynamic import* of the plug-in is necessary. — Bakuriu, May 08 '13 at 11:44

thavan · Answer 3 · 2013-05-11T04:07:22.070

-7

exec('import ' + sys.argv[2])
obj = test_plugin.TestPlugin()

Here sys.argv[2] is 'test_plugin' string from command line arguments.

EDIT: Another way to avoid using exec:

import importlib
mod = importlib.import_module(sys.argv[2])

edited May 11 '13 at 04:07

answered May 08 '13 at 10:13

thavan

2,409
24
32

5

What if `sys.argv[2]` is `os; os.system("evil command")`? – Bakuriu May 08 '13 at 10:43
1

`exec`, `eval` like functions are evil, and shall never be used. – zmo May 08 '13 at 10:47

Python: Importing files based on which files the user wants

3 Answers3