Dynamically import via variable a relative script in Python

Question

In my main Python script, a web API is called. The result of that call determines what 'protocol' needs to run. These protocols are individual python scripts inside a subfolder called protocols/.

So, from my main script, how do I dynamically call one of these scripts? The script name is a variable coming from the API, and it's in a subfolder of the main script - not the standard library location.

I could use subprocess.popen or os.system, but I want everything to stay as in-house as possible.

There's gotta be a way to accomplish this within one Python instance, right?

Answer 1

There is a Python built-in function that can handle calling scripts.

It is called execfile and you can learn more about it at:

https://docs.python.org/2/library/functions.html#execfile

If I have a file called testmeagain.py with the contents:

print "test print"

You can do this without importing additional libraries:

In [1]: execfile("testmeagain.py")
test print

Answer 2

Assuming you run your main Python program from the directory, which see protocols as subfolder, you shall add empty __init__.py into protocols directory to make it a package.

Then you shall be able importing your protocol specific modules from there, e.g.

from protocols.http import process_it

Here I assume, you would have there some function named process_it

There are also other methods, how to import package with name defined in a variable as described e.g. in this SO answer enter link description here.

In your case it would look somehow like:

def process_it(protocol="http"):
    mod = __import__("protocols." + protocol)
    mod.process_it()

Answer 3

Yes, there is a possibility to accomplish this within one Python instance. It is possible to load individual python scripts dynamically as the following:

1. Load only the module dynamically with importlib.import_module()
2. Load a module and class dynamically with getattr().

Alternative 1

Use importlib.import_module(), if you only want to load the module dynamically. Let’s assume you have the following protocols in the subfolder protocols:

protocols/protocol_1.py containing the following class:

class Protocol():
    # some protocol specific code

protocols/protocol_2.py containing the following class:

class Protocol():
    # some protocol specific code

First we define the dynamic variables with the protocol name that will be returned from the Web API:

module = 'protocols.protocol_1' 

Then we import the importlib and load the module dynamically. The Protocol class has in both files the same class name but protocol specific code. Afterwards, we instantiate the api_protocol with the dynamically loaded Protocol:

import importlib

Protocol = importlib.import_module(module).Protocol

api_protocol = Protocol()

Alternative 2

Use getattr, if you also want to load the class dynamically. Let’s assume you have the following protocols in the subfolder protocols:

protocols/protocol_1.py containing the following class:

class Protocol_1():
    # some protocol specific code

protocols/protocol_2.py containing the following class:

class Protocol_2():
    # some protocol specific code

First we define the dynamically variables with the protocol name that will be returned from the Web API:

module = 'protocols.protocol_1' 
class_name = 'Protocol_1'

Then we call the __import__ function in order to load dynamically the class of a dynamic module. Afterwards we can create the new class with getattr and instantiate the api_protocol with the dynamically loaded Protocol:

mod = __import__(module, fromlist=[class_name])
Protocol = getattr(mod, class_name)
api_protocol = Protocol()