1747

How do I load a Python module given its full path?

Note that the file can be anywhere in the filesystem where the user has access rights.


See also: How to import a module given its name as string?

Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
derfred
  • 18,881
  • 3
  • 23
  • 25
  • 144
    Nice and simple question - and useful answers but they make me wonder what happened with the python mantra "There is _one_ _obvious_ way" to do it.. It doesn't seem like anything like a single or a simple and obvious answer to it.. Seems ridiculously hacky and version-dependent for such a fundamental operation (and it looks and more bloated in newer versions..). – inger Dec 06 '19 at 12:08
  • 65
    @inger **what happened with the python mantra "There is one obvious way" to do it [...] [not] a single or a simple and obvious answer to it [...] ridiculously hacky[...] more bloated in newer versions** Welcome to the terrible world of python package management. Python's `import`, `virtualenv`, `pip`, `setuptools` whatnot should all be thrown out and replaced with working code. I just tried to grok `virtualenv` or was it `pipenv` and had to work thru the equivalent of a Jumbo Jet manual. How that contrivance is paraded as The Solution to dealing with deps totally escapes me. – John Frazer Jun 26 '20 at 20:59
  • 70
    relevant XKCD https://xkcd.com/1987/ – John Frazer Jun 26 '20 at 21:08
  • 5
    @JohnFrazer it's been made worse by constant nagging of people who couldn't be bothered to read 2 paragraphs of documentation. Your XKCD isn't really relevant, as it shows what these kinds of people can achieve when trying things until something works. Also, just because there's a new way doesn't mean there's now "two obvious ways". The old way is obvious for some cases, the new way introduces ease of use to other. That's what happens when you actually care about DevX. – Błażej Michalik Dec 02 '20 at 00:13
  • 9
    And think that Java or even PHP (these days) have clear and simple way of splitting things in packages/namespaces and reuse it. It's a shock to see such pain in Python which adopted simplicity in every other aspect. – Alex Jan 27 '21 at 07:46
  • There's [parsing - What's the best practice using a settings file in Python? - Stack Overflow](https://stackoverflow.com/questions/5055042/whats-the-best-practice-using-a-settings-file-in-python?noredirect=1&lq=1) -- but I don't find a good way to use Python configuration file. – user202729 Oct 24 '21 at 01:16
  • 2
    How I miss the obvious and intuitive semantics JS "require" and C/C++ "#include" when working with Python. – Mark K Cowan Mar 09 '22 at 12:23
  • 3
    @MarkKCowan you **miss** having a preprocessor literally copy and paste other source files into the same "translation unit" and praying the result makes sense? To say nothing of the arcane `#ifndef` idiom? It's absolutely mind-boggling to me that there are so many people out there who will complain about Python while praising absolutely bizarre alternatives. – Karl Knechtel Nov 30 '22 at 11:27
  • 1
    @KarlKnechtel I miss having "some way of referencing symbols in one file from another" that wasn't designed by a monkey on crack. I also miss being able to choose what symbols to expose to other files vs Python's "publish everything". – Mark K Cowan Nov 30 '22 at 23:15
  • 1
    Python has a way of referencing symbols in one file from another. It is called `import`, and it was designed by the Python dev team and it works very well. By default, symbols with a leading underscore are not exposed; you can modify the behaviour using `__all__`. You can also `del` unneeded symbols at the end of the top-level code. – Karl Knechtel Dec 01 '22 at 17:40
  • 2
    "and it works very well." - hence why we have questions like this one lol, and the dozens about relative imports, and why they seem to resolve differently depending on how the program is launched / imported As for the underscore-prefix for not exporting symbols - not very useful for preventing re-export of imports! It's amazing how Python manages to un-solve so many problems that are solved in other languages. – Mark K Cowan Dec 17 '22 at 20:10
  • 1
    The root of the module needs to be in `sys.path`. If its not in there, there are multiple ways to add it either dynamically or permanently depending on how you are launching your python code. The best way is to not do this, make it a python package and install it to your virtual environment. – Tom McLean Mar 22 '23 at 10:54

35 Answers35

1743

For Python 3.5+ use (docs):

import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py")
foo = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = foo
spec.loader.exec_module(foo)
foo.MyClass()

For Python 3.3 and 3.4 use:

from importlib.machinery import SourceFileLoader

foo = SourceFileLoader("module.name", "/path/to/file.py").load_module()
foo.MyClass()

(Although this has been deprecated in Python 3.4.)

For Python 2 use:

import imp

foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()

There are equivalent convenience functions for compiled Python files and DLLs.

See also http://bugs.python.org/issue21436.

jdehesa
  • 58,456
  • 7
  • 77
  • 121
Sebastian Rittau
  • 20,642
  • 3
  • 24
  • 21
  • 85
    If I knew the namespace - 'module.name' - I would already use `__import__`. – Sridhar Ratnakumar Aug 10 '09 at 21:54
  • 73
    @SridharRatnakumar the value of the first argument of `imp.load_source` only sets the `.__name__` of the returned module. it doesn't effect loading. – Dan D. Dec 14 '11 at 04:51
  • 23
    @DanD. — the first argument of `imp.load_source()` determines the key of the new entry created in the `sys.modules` dictionary, so the first argument does indeed affect loading. – Brandon Rhodes Apr 21 '13 at 16:32
  • 5
    Is the idea that this is replicating: ""import module.name as foo"", if your working directory was /path/to ?? Else, what is foo here?? – capybaralet Feb 27 '15 at 01:53
  • See new (for december 2015) message in bug discussion (http://bugs.python.org/issue21436#msg255901): there a third new three-lines-long way to load module in python 3.5! – d9k Dec 05 '15 at 18:39
  • I'm not seeing that `SourceFileLoader` is deprecated in the 3.4+ docs. – Ryne Everett Jan 09 '16 at 20:21
  • 3
    SourceFileLoader is not deprecated, but SourceFileLoader.load_module() is. – Sebastian Rittau Jan 18 '16 at 13:11
  • Despite the use of "import" in the question, you probably want to use `runpy.run_path` for execution of Python code as a configuration format, rather than dynamically manipulating the import system. – ncoghlan May 20 '16 at 06:56
  • 29
    @AXO and more to the point one wonders why something as simple and basic as this *has* to be so complicated. It isn't in many many other languages. – rocky May 22 '16 at 17:04
  • If you want a version of the code that works in all versions of Python check http://stackoverflow.com/a/37611448/99834 – sorin Jun 03 '16 at 10:05
  • 3
    I'm in Python 3.5.2 and I've found it **only** works if the extension of the file is .py – Paolo Celati Jul 09 '16 at 06:01
  • 2
    @Paolo Celati In Python 3.5+ You should use importlib.import_module(module_name). Some like this. sys,path.append(path_to_file) module = importlib.import_module(module_name) – ZigZag Sep 09 '16 at 09:06
  • 1
    may I know why `importlib.import_module` is not mentioned here? – Mahesha999 Sep 26 '16 at 13:13
  • 8
    @Mahesha999 Because importlib.import_module() does not allow you to import modules by filename, which is what the original question was about. – Sebastian Rittau Sep 28 '16 at 12:50
  • 6
    For Python 3.5+, if `/path/to/file.py` imports a sibling implicitly (e.g. `import bar` to import `/path/to/bar.py`), the solution yields `ModuleNotFoundError: No module named 'bar'`. Any way to fix this? – Pedro Cattori Jan 24 '17 at 18:22
  • 2
    ^I ended up asking this in a separate StackOverflow Question: http://stackoverflow.com/q/41861427/1490091 – Pedro Cattori Jan 25 '17 at 21:28
  • 2
    `importlib.util.spec_from_file_location` won't import files that don't end in `.py` =( – AlexLordThorsen Jun 07 '17 at 02:22
  • I'm running `python 3.6.3` on `Sierra 10.12.6` and using option 1 (for python 3.5+). The code works, but when I run the line `foo.MyClass()` I get the error `AttributeError: module 'myFileName' has no attribute 'MyClass'`, where `myFileName` is the name of the python file I pass to the first arg of `importlib.util.spec_from_file_location("module.name", "/path/to/file.py")`. Yet when I comment out the line `foo.MyClass()` the script executes the imported script without issue. Would someone please explain what `foo.MyClass()` is doing in the suggested code? – Cole Robertson Nov 23 '17 at 13:26
  • @ColeRobertson That line is just an example to show that you need to prefix any access of the module with `foo.` (or however you call that variable). – Sebastian Rittau Dec 11 '17 at 13:28
  • It's mentioned that for Python 2, importlib should be used instead of lib, yet I see no example of using importlib to import a module at a path. Anyone have such an example? – JacksonHaenchen Dec 15 '17 at 18:57
  • Is that top one safe? It seems a bit too close to just "exec"ing whatever the user tells you too. Wouldn't it be safer to add a directory to `sys.path`, and then ask for the specific module that you want? – setholopolus Jan 30 '18 at 18:56
  • 1
    @setholopolus None of those is safe, if you use untrusted user input. – Sebastian Rittau Feb 01 '18 at 15:31
  • @SebastianRittau You're right I guess, because even if you imported a specific module by name they could have replaced it by their own module with the same name. – setholopolus Feb 01 '18 at 19:26
  • @SebastianRittau Then what is currently the best way for python 3.4? – Nathan majicvr.com May 09 '18 at 02:35
  • 4
    This *almost* worked for me but when I'm importing a module structured as a directory with an \__init__.py I needed an additional line. See my answer below -- hope it helps somebody! – Sam Grondahl May 17 '18 at 15:24
  • 4
    As mentioned by @SamGrondahl, these fail if the module has relative imports. His [answer](https://stackoverflow.com/a/50395128/1088938) provides a solution for python 3.5+: Is there any similar solution for python 2.7? – mforbes Jul 16 '18 at 01:10
  • Should `spec.loader.exec_module(foo)` also ensure that the script is completely run? I have a set-up where I import a list of a variables (and purposely not a class), that all need to be valid and active within the script from which I'm calling it. In contrast to the regular `import` statement, however, my variables cannot be further recalled. – Lena Jun 18 '19 at 09:25
  • @Lena It should and in my short test it did. – Sebastian Rittau Jun 19 '19 at 10:24
  • 2
    Beware of periods in a file name, because the latest version of the python `importlib` module still could not handle module file names with periods: https://bugs.python.org/issue38000 – Andry Sep 07 '19 at 19:42
  • 2
    Interesting how the code gets LONGER and longer with every new version of Python. Not sure if this is really the "pythonic way" of developing a programming language :/ (same for other things, like print() etc.) – Marcin Wojnarski Oct 03 '19 at 16:33
  • Python 2.7: TypeError: load_module() takes exactly 4 arguments – Shai Alon Nov 10 '19 at 15:54
  • 1
    If the imported file contains any relative import these solutions fail ```ImportError: attempted relative import with no known parent package``` – Marco Castanho Dec 05 '19 at 16:18
  • 1
    `module.name` is the module from which I run? – Gulzar Jan 20 '20 at 10:03
  • 2
    Using the v3.5+ method described leads to pickling errors. An answer linked above by @mforbes [here](https://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path/50395128#50395128) adds an additional step that appears to fix this: `sys.modules[spec.name] = foo` – drootang Mar 03 '20 at 12:40
  • 1
    Are they out of their minds establishing 3 different ways for the same thing in these different language versions??? – Christoph90 Jul 03 '20 at 11:22
  • 4
    What is `module.name`?? I'm importing a file, or something from a file. It does not have a namespace. – Myridium Aug 27 '20 at 03:42
  • 1
    Isn't module.name always going to be the same as the name of the .py file? – Chris A Mar 24 '21 at 00:33
  • I just wanted a quick solution. So I changed the directory `cd "folder"` and then I was fine. Hope this helps someone. – YoussefDir Jun 28 '21 at 21:19
  • 2
    Since no one had mentioned `pydoc` yet -- here are my two cents: [pydoc::importfile()](https://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path/68361215#68361215) – ジョージ Jul 13 '21 at 11:05
  • The example in the docs is also quite clear: [importing-a-source-file-directly](https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly) – djvg Feb 18 '22 at 14:53
  • If you need to import `class MyClass` into the current namespace (like I did during a dynamic Py2 vs Py3 import), try this: `MyClass = getattr(foo, "MyClass")` Discovered here: https://stackoverflow.com/a/41678146/257299 – kevinarpe Mar 07 '22 at 05:24
  • To import a file which does not end .py: `import importlib` `NAME, PATH = 'coverage', '/usr/bin/coverage'` `loader = importlib.machinery.SourceFileLoader(NAME, PATH)` `spec = importlib.util.spec_from_file_location(NAME, loader=loader)` `module = importlib.util.module_from_spec(spec)` `spec.loader.exec_module(module)` Annoying as hell. – zbyszek Apr 14 '23 at 15:01
566

The advantage of adding a path to sys.path (over using imp) is that it simplifies things when importing more than one module from a single package. For example:

import sys
# the mock-0.3.1 dir contains testcase.py, testutils.py & mock.py
sys.path.append('/foo/bar/mock-0.3.1')

from testcase import TestCase
from testutils import RunTests
from mock import Mock, sentinel, patch
Daryl Spitzer
  • 143,156
  • 76
  • 154
  • 173
  • 22
    How do we use `sys.path.append` to point to a single python file instead of a directory? – Phani Jan 13 '14 at 17:46
  • @Daryl Spitzer: How do we do it for just one python file – Srivatsan Mar 06 '14 at 16:20
  • How do we use this for a single python file? And why have you been ignoring us for 7 years? – NPN328 Mar 05 '15 at 22:59
  • 35
    :-) Perhaps your question would be better suited as a StackOverflow question, not a comment on an answer. – Daryl Spitzer Mar 06 '15 at 00:12
  • 2
    To all people who were trying to include a file to their path... by definition "the shell path is a colon delimited list of directories". I'm relatively new to python, but the python path also follows the unix design principle from what I have seen. Please correct me if I am wrong. – m33k Apr 15 '15 at 05:37
  • 5
    The python path can contain zip archives, "eggs" (a complex kind of zip archives), etc. Modules can be imported out of them. So the path elements are indeed _containers_ of files, but they are not necessarily directories. – alexis Apr 30 '15 at 21:21
  • 22
    Beware of the fact that Python caches import statements. In the rare case that you have two different folders sharing a single class name (classX), the approach of adding a path to sys.path, importing classX, removing the path and repeating for the reamaining paths won't work. Python will always load the class from the first path from its cache. In my case I aimed at creating a plugin system where all plugins implement a specific classX. I ended up using [SourceFileLoader](http://stackoverflow.com/a/67692), note that its [deprecation is controversial](http://bugs.python.org/issue21436). – ComFreek Jul 03 '15 at 17:18
  • 2
    I prefer this solution to the accepted one because it still works on Python 3.6 and because it only requires 2 simple lines of code to allow any number of modules to be found within another site directory. Our specific situation was on a web server where only a few core packages are installed centrally and this way CGI scripts created by users can import from their user-site directory. Thanks! – ybull Mar 07 '18 at 22:07
  • In my case the imported file had other (transitive) relative-path imports; combining the accepted answer (`importlib`) with this worked for me. – Janaka Bandara Mar 29 '19 at 06:07
  • 4
    Note this approach allows the imported module to import other modules from the same dir, which modules often do, while the accepted answer's approach does not (at least on 3.7). `importlib.import_module(mod_name)` can be used instead of the explicit import here if the module name isn't known at runtime I would add a `sys.path.pop()` in the end, though, assuming the imported code doesn't try to import more modules as it is used. – Eli_B May 06 '19 at 21:45
  • This is an excellent solution. Note that you can use a relative path in your `sys.path.append`, so with this answer if your working directory was `/foo/bar/`, you could use `sys.path.append('./mock-0.3.1')` – mikey Jun 04 '21 at 19:45
134

To import your module, you need to add its directory to the environment variable, either temporarily or permanently.

Temporarily

import sys
sys.path.append("/path/to/my/modules/")
import my_module

Permanently

Adding the following line to your .bashrc (or alternative) file in Linux and excecute source ~/.bashrc (or alternative) in the terminal:

export PYTHONPATH="${PYTHONPATH}:/path/to/my/modules/"

Credit/Source: saarrrr, another Stack Exchange question

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Miladiouss
  • 4,270
  • 1
  • 27
  • 34
  • 15
    This "temp" solution is a great answer if you want to prod a project around in a jupyter notebook elsewhere. – fordy Nov 16 '18 at 17:20
  • But... it's dangerous tampering with the path – Shai Alon Nov 10 '19 at 15:56
  • 1
    @ShaiAlon You are adding paths, so no danger other than when you transfer codes from one computer to another, paths might get messed up. So, for package development, I only import local packages. Also, package names should be unique. If you are worried, use the temporary solution. – Miladiouss Nov 13 '19 at 00:11
  • I had difficulty getting my Unit Tests to import, and your temp worked fine. I modified it to import from same directory as Unit Test with: >>> import os >>> import sys >>> sys.path.append(os.getcwd()) – brewmanz Sep 19 '22 at 03:45
101

If your top-level module is not a file but is packaged as a directory with __init__.py, then the accepted solution almost works, but not quite. In Python 3.5+ the following code is needed (note the added line that begins with 'sys.modules'):

MODULE_PATH = "/path/to/your/module/__init__.py"
MODULE_NAME = "mymodule"
import importlib
import sys
spec = importlib.util.spec_from_file_location(MODULE_NAME, MODULE_PATH)
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module 
spec.loader.exec_module(module)

Without this line, when exec_module is executed, it tries to bind relative imports in your top level __init__.py to the top level module name -- in this case "mymodule". But "mymodule" isn't loaded yet so you'll get the error "SystemError: Parent module 'mymodule' not loaded, cannot perform relative import". So you need to bind the name before you load it. The reason for this is the fundamental invariant of the relative import system: "The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former" as discussed here.

K. Frank
  • 1,325
  • 1
  • 10
  • 19
Sam Grondahl
  • 2,397
  • 2
  • 20
  • 26
  • 2
    Thanks a lot! This method enables relative imports between submodules. Great! – tebanep Oct 10 '19 at 14:09
  • 1
    This answer matches the documentation here: https://docs.python.org/3/library/importlib.html#importing-programmatically. – Tim Ludwinski Dec 09 '19 at 16:08
  • 2
    but what is `mymodule`? – Gulzar Jan 20 '20 at 10:02
  • @Gulzar, it is whatever name you'd like to give your module, such that you can later do: "from mymodule import myclass" – Idodo Feb 16 '20 at 22:44
  • So... `/path/to/your/module/` is actually `/path/to/your/PACKAGE/`? and by `mymodule` you mean `myfile.py`? – Gulzar Feb 17 '20 at 07:03
  • What is `mymodule` in relation to `/path/to/your/module/__init__.py`? – IAbstract May 20 '20 at 16:24
  • 2
    Though unconventional, if your package entry point is something other than `__init__.py`, you can still import it as a package. Include `spec.submodule_search_locations = [os.path.dirname(MODULE_PATH)]` after creating the spec. You can also treat a `__init__.py` as a non-package (e.g. single module) by setting this value to `None` – Azmisov Nov 05 '20 at 20:32
  • note, for idiots like. You have to refer to **file** and not a folder in `MODULE_PATH`. Took me quite a while to figure out – CutePoison Aug 21 '23 at 10:12
51

It sounds like you don't want to specifically import the configuration file (which has a whole lot of side effects and additional complications involved). You just want to run it, and be able to access the resulting namespace. The standard library provides an API specifically for that in the form of runpy.run_path:

from runpy import run_path
settings = run_path("/path/to/file.py")

That interface is available in Python 2.7 and Python 3.2+.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
ncoghlan
  • 40,168
  • 10
  • 71
  • 80
  • I like this method but when I get the result of run_path its a dictionary which I cannot seem to access? – Stephen Ellwood Sep 11 '18 at 09:00
  • What do you mean by "cannot access"? You can't import from it (that's why this is only a good option when import-style access isn't actually required), but the contents should be available via the regular dict API (`result[name]`, `result.get('name', default_value)`, etc) – ncoghlan Sep 13 '18 at 03:16
  • This answer is way underrated. It's very short and simple! Even better, if you need a proper module namespace, you can do something like `from runpy import run_path; from argparse import Namespace; mod = Namespace(**run_path('path/to/file.py'))` – RuRo Mar 18 '20 at 21:26
  • Hi Nick. In [PEP 338](https://www.python.org/dev/peps/pep-0338/), you introduced only the function `runpy.run_module` and said that `python -m module_name` now delegates to it. 1. Did you write a similar PEP for the function `runpy.run_path`? 2. Does `python file_path.py` now delegate to `runpy.run_path`? – Géry Ogam Sep 15 '20 at 21:50
  • Also, I have just opened a public question [here](https://stackoverflow.com/q/63909243/2326961) about your [PEP 366](https://www.python.org/dev/peps/pep-0366/). I am very interested by your thought on this. – Géry Ogam Sep 15 '20 at 21:53
  • 1
    @Maggyero The command line never goes through `runpy.run_path`, but if a given path is a directory or zipfile, then it ends up delegating to `runpy.run_module` for the `__main__` execution. The duplicated logic for "Is it a script, directory, or zipfile?" isn't complicated enough to be worth delegating to Python code. – ncoghlan Sep 22 '20 at 07:39
  • Thanks, I was not aware of this. In CPython, does the delegation to `runpy.run_module` for a given module name (`python -m name`) happen exactly [here](https://github.com/python/cpython/blob/v3.8.5/Modules/main.c#L599) and for a given directory or zip file path (`python path`) happen exactly [here](https://github.com/python/cpython/blob/v3.8.5/Modules/main.c#L602)? – Géry Ogam Sep 22 '20 at 09:30
  • 1
    Also by looking at the [implementation](https://github.com/python/cpython/blob/v3.8.5/Modules/main.c#L280) of the C function `pymain_run_module`, it seems that CPython delegates to the Python function `runpy._run_module_as_main` instead of `runpy.run_module`—though if I understood correctly the only difference is that the first function executes the code in the built-in `__main__` environment (cf. [here](https://github.com/python/cpython/blob/v3.8.5/Lib/runpy.py#L191)) while the second function executes it in a new environment? – Géry Ogam Sep 22 '20 at 09:33
  • 1
    @Maggyero Yep, that's the only difference. Originally it used the public function, but that turned out to interact badly with the interpreter's `-i` option (which drops you into an interactive shell in the original `__main__` module, so `-m` running in a new module was inconvenient) – ncoghlan Oct 26 '20 at 07:10
25

You can also do something like this and add the directory that the configuration file is sitting in to the Python load path, and then just do a normal import, assuming you know the name of the file in advance, in this case "config".

Messy, but it works.

configfile = '~/config.py'

import os
import sys

sys.path.append(os.path.dirname(os.path.expanduser(configfile)))

import config
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Chris Cherry
  • 28,118
  • 6
  • 68
  • 71
  • That is not dynamically. – Shai Alon Nov 10 '19 at 16:10
  • I tried: config_file = 'setup-for-chats', setup_file = get_setup_file(config_file + ".py"), sys.path.append(os.path.dirname(os.path.expanduser(setup_file))), import config_file >> "ImportError: No module named config_file" – Shai Alon Nov 10 '19 at 16:17
21

I have come up with a slightly modified version of @SebastianRittau's wonderful answer (for Python > 3.4 I think), which will allow you to load a file with any extension as a module using spec_from_loader instead of spec_from_file_location:

from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader 

spec = spec_from_loader("module.name", SourceFileLoader("module.name", "/path/to/file.py"))
mod = module_from_spec(spec)
spec.loader.exec_module(mod)

The advantage of encoding the path in an explicit SourceFileLoader is that the machinery will not try to figure out the type of the file from the extension. This means that you can load something like a .txt file using this method, but you could not do it with spec_from_file_location without specifying the loader because .txt is not in importlib.machinery.SOURCE_SUFFIXES.

I've placed an implementation based on this, and @SamGrondahl's useful modification into my utility library, haggis. The function is called haggis.load.load_module. It adds a couple of neat tricks, like the ability to inject variables into the module namespace as it is loaded.

Mad Physicist
  • 107,652
  • 25
  • 181
  • 264
18

You can use the

load_source(module_name, path_to_file)

method from the imp module.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
zuber
  • 3,449
  • 2
  • 24
  • 19
17

Here is some code that works in all Python versions, from 2.7-3.5 and probably even others.

config_file = "/tmp/config.py"
with open(config_file) as f:
    code = compile(f.read(), config_file, 'exec')
    exec(code, globals(), locals())

I tested it. It may be ugly, but so far it is the only one that works in all versions.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
sorin
  • 161,544
  • 178
  • 535
  • 806
  • 1
    This answer worked for me where `load_source` did not because it imports the script and provides the script access to the modules and globals at the time of importing. – Klik Nov 22 '17 at 19:13
  • Note that the behavior of this answer is **different** from importing a module, as for a module (imported the normal way or not) the "global" scope of the code is *the module object*, while for this answer it's the globals scope of the called object. (although this answer can be modified to change the scope too, any dictionary can be passed in as `globals` and `locals`) – user202729 Oct 24 '21 at 01:09
17

Do you mean load or import?

You can manipulate the sys.path list specify the path to your module, and then import your module. For example, given a module at:

/foo/bar.py

You could do:

import sys
sys.path[0:0] = ['/foo'] # Puts the /foo directory at the start of your path
import bar
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Wheat
  • 845
  • 4
  • 12
15

To add to Sebastian Rittau's answer: At least for CPython, there's pydoc, and, while not officially declared, importing files is what it does:

from pydoc import importfile
module = importfile('/path/to/module.py')

PS. For the sake of completeness, there's a reference to the current implementation at the moment of writing: pydoc.py, and I'm pleased to say that in the vein of xkcd 1987 it uses neither of the implementations mentioned in issue 21436 -- at least, not verbatim.

ジョージ
  • 1,476
  • 1
  • 22
  • 29
  • 2
    This is arguably the simplest method and no dependencies are required. Tested under py3.8. – mirekphd Oct 14 '22 at 17:27
  • This is really good when writing disposable code, easy to remember. – Deadly Pointer Nov 09 '22 at 20:05
  • Looks great, but is it possible to turn it into a global import? I asked a followup question here https://stackoverflow.com/questions/76481173/python-dynamically-load-library-in-global-path – tobiasBora Jun 15 '23 at 09:40
  • 1
    @tobiasBora there are several ways to do that, added a comment to your question with one alternative; note that to fully imitate `from module import *` one would need to _filter_ "internal" module content -- like names [starting with an underscrore](https://docs.python.org/3/tutorial/modules.html#more-on-modules). – ジョージ Jun 15 '23 at 09:57
  • Great! Since my question got closed (not sure why since linked answers did not answer my question as far as I could tell), maybe you can update this answer? – tobiasBora Jun 15 '23 at 10:05
  • 1
    @tobiasBora I do not remember when - but I believe some time after 2010 moderators started to get overwhelmed by the amount of relatively low quality questions and answers (and I absolutely do not mean your own question here, but rather some next to "white noise" ones), and that led to a fairly draconian shift in general policy, in particular giving up on [dups](https://stackoverflow.blog/2010/11/16/dr-strangedupe-or-how-i-learned-to-stop-worrying-and-love-duplication/). I disagree, but have no time to argue. Besides, I have not been in their shoes. – ジョージ Jun 15 '23 at 10:18
  • Interesting. This does not seem to be [documented](https://docs.python.org/3/library/pydoc.html), I'm not sure it's intended to be a public API.. – wim Aug 06 '23 at 01:37
  • @wim May be not. But it's useful. And besides, there were formerly undocumented api endpoints in the past that became published later -- check e.g. [pipes.quote](https://docs.python.org/2.7/library/pipes.html#pipes.quote) story – ジョージ Aug 07 '23 at 03:28
14

You can do this using __import__ and chdir:

def import_file(full_path_to_module):
    try:
        import os
        module_dir, module_file = os.path.split(full_path_to_module)
        module_name, module_ext = os.path.splitext(module_file)
        save_cwd = os.getcwd()
        os.chdir(module_dir)
        module_obj = __import__(module_name)
        module_obj.__file__ = full_path_to_module
        globals()[module_name] = module_obj
        os.chdir(save_cwd)
    except Exception as e:
        raise ImportError(e)
    return module_obj


import_file('/home/somebody/somemodule.py')
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Chris Calloway
  • 1,418
  • 1
  • 10
  • 12
  • 44
    Why write 14 lines of buggy code when this is already addressed by the standard library? You haven't done error checking on format or content of full_path_to_module or the os.whatever operations; and using a catch-all `except:` clause is rarely a good idea. – Chris Johnson Jun 07 '13 at 19:17
  • You should use more "try-finally"s in here. E.g. `save_cwd = os.getcwd()` `try: …` `finally: os.chdir(save_cwd)` – Kijewski Sep 21 '14 at 01:33
  • 15
    @ChrisJohnson `this is already addressed by the standard library` yeah, but python has nasty habit of not being backward-compatible... as the checked answer says there're 2 different ways before and after 3.3. In that case I'd rather like to write my own universal function than check version on the fly. And yes, maybe this code isn't too well error-protected, but it shows an idea (which is os.chdir(), I haven't though about it), basing on which I can write a better code. Hence +1. – Sushi271 May 15 '15 at 10:27
  • 2
    It would be cool if this actually returned the module. – Pithikos Jul 30 '20 at 14:41
12

If we have scripts in the same project but in different directory means, we can solve this problem by the following method.

In this situation utils.py is in src/main/util/

import sys
sys.path.append('./')

import src.main.util.utils
#or
from src.main.util.utils import json_converter # json_converter is example method
Kumar KS
  • 873
  • 10
  • 21
9

I believe you can use imp.find_module() and imp.load_module() to load the specified module. You'll need to split the module name off of the path, i.e. if you wanted to load /home/mypath/mymodule.py you'd need to do:

imp.find_module('mymodule', '/home/mypath/')

...but that should get the job done.

Mathieu Rodic
  • 6,637
  • 2
  • 43
  • 49
Matt
  • 141
  • 1
  • 3
8

You can use the pkgutil module (specifically the walk_packages method) to get a list of the packages in the current directory. From there it's trivial to use the importlib machinery to import the modules you want:

import pkgutil
import importlib

packages = pkgutil.walk_packages(path='.')
for importer, name, is_package in packages:
    mod = importlib.import_module(name)
    # do whatever you want with module now, it's been imported!
Mathieu Rodic
  • 6,637
  • 2
  • 43
  • 49
bob_twinkles
  • 252
  • 2
  • 9
  • I'm running into this problem here that kind of relates - https://stackoverflow.com/questions/73329009/pkgutil-isnt-considering-an-absolute-path-properly – openCivilisation Aug 12 '22 at 03:49
7

Create Python module test.py:

import sys
sys.path.append("<project-path>/lib/")
from tes1 import Client1
from tes2 import Client2
import tes3

Create Python module test_check.py:

from test import Client1
from test import Client2
from test import test3

We can import the imported module from module.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
abhimanyu
  • 730
  • 1
  • 10
  • 23
7

There's a package that's dedicated to this specifically:

from thesmuggler import smuggle

# À la `import weapons`
weapons = smuggle('weapons.py')

# À la `from contraband import drugs, alcohol`
drugs, alcohol = smuggle('drugs', 'alcohol', source='contraband.py')

# À la `from contraband import drugs as dope, alcohol as booze`
dope, booze = smuggle('drugs', 'alcohol', source='contraband.py')

It's tested across Python versions (Jython and PyPy too), but it might be overkill depending on the size of your project.

fny
  • 31,255
  • 16
  • 96
  • 127
5

This area of Python 3.4 seems to be extremely tortuous to understand! However with a bit of hacking using the code from Chris Calloway as a start I managed to get something working. Here's the basic function.

def import_module_from_file(full_path_to_module):
    """
    Import a module given the full path/filename of the .py file

    Python 3.4

    """

    module = None

    try:

        # Get module name and path from full path
        module_dir, module_file = os.path.split(full_path_to_module)
        module_name, module_ext = os.path.splitext(module_file)

        # Get module "spec" from filename
        spec = importlib.util.spec_from_file_location(module_name,full_path_to_module)

        module = spec.loader.load_module()

    except Exception as ec:
        # Simple error printing
        # Insert "sophisticated" stuff here
        print(ec)

    finally:
        return module

This appears to use non-deprecated modules from Python 3.4. I don't pretend to understand why, but it seems to work from within a program. I found Chris' solution worked on the command line but not from inside a program.

Redlegjed
  • 51
  • 1
  • 1
4

I'm not saying that it is better, but for the sake of completeness, I wanted to suggest the exec function, available in both Python 2 and Python 3.

exec allows you to execute arbitrary code in either the global scope, or in an internal scope, provided as a dictionary.

For example, if you have a module stored in "/path/to/module" with the function foo(), you could run it by doing the following:

module = dict()
with open("/path/to/module") as f:
    exec(f.read(), module)
module['foo']()

This makes it a bit more explicit that you're loading code dynamically, and grants you some additional power, such as the ability to provide custom builtins.

And if having access through attributes, instead of keys is important to you, you can design a custom dict class for the globals, that provides such access, e.g.:

class MyModuleClass(dict):
    def __getattr__(self, name):
        return self.__getitem__(name)
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
yoniLavi
  • 2,624
  • 1
  • 24
  • 30
4

To import a module from a given filename, you can temporarily extend the path, and restore the system path in the finally block reference:

filename = "directory/module.py"

directory, module_name = os.path.split(filename)
module_name = os.path.splitext(module_name)[0]

path = list(sys.path)
sys.path.insert(0, directory)
try:
    module = __import__(module_name)
finally:
    sys.path[:] = path # restore
Peter Zhu
  • 1,154
  • 3
  • 15
  • 27
4

A simple solution using importlib instead of the imp package (tested for Python 2.7, although it should work for Python 3 too):

import importlib

dirname, basename = os.path.split(pyfilepath) # pyfilepath: '/my/path/mymodule.py'
sys.path.append(dirname) # only directories should be added to PYTHONPATH
module_name = os.path.splitext(basename)[0] # '/my/path/mymodule.py' --> 'mymodule'
module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")

Now you can directly use the namespace of the imported module, like this:

a = module.myvar
b = module.myfunc(a)

The advantage of this solution is that we don't even need to know the actual name of the module we would like to import, in order to use it in our code. This is useful, e.g. in case the path of the module is a configurable argument.

Ataxias
  • 1,085
  • 13
  • 23
  • This way you are modifying the `sys.path`, which does not fit every use case. – bgusach Jul 19 '18 at 14:26
  • @bgusach This may be true, but it is also desirable in some cases (adding a path to sys.path simplifies things when importing more than one module from a single package). At any rate, if this not desirable, one can immediately afterwards do `sys.path.pop()` – Ataxias Jul 20 '18 at 16:38
4

I made a package that uses imp for you. I call it import_file and this is how it's used:

>>>from import_file import import_file
>>>mylib = import_file('c:\\mylib.py')
>>>another = import_file('relative_subdir/another.py')

You can get it at:

http://pypi.python.org/pypi/import_file

or at

http://code.google.com/p/import-file/

ubershmekel
  • 11,864
  • 10
  • 72
  • 89
  • 1
    os.chdir ? (minimal characters to approve comment). – ychaouche Oct 14 '12 at 10:46
  • I've spent all day troubleshooting an import bug in a pyinstaller generated exe. In the end this is the only thing that worked for me. Thank you so much for making this! – Frak Nov 29 '18 at 22:12
3

I have written my own global and portable import function, based on importlib module, for:

  • Be able to import both modules as submodules and to import the content of a module to a parent module (or into a globals if has no parent module).
  • Be able to import modules with a period characters in a file name.
  • Be able to import modules with any extension.
  • Be able to use a standalone name for a submodule instead of a file name without extension which is by default.
  • Be able to define the import order based on previously imported module instead of dependent on sys.path or on a what ever search path storage.

The examples directory structure:

<root>
 |
 +- test.py
 |
 +- testlib.py
 |
 +- /std1
 |   |
 |   +- testlib.std1.py
 |
 +- /std2
 |   |
 |   +- testlib.std2.py
 |
 +- /std3
     |
     +- testlib.std3.py

Inclusion dependency and order:

test.py
  -> testlib.py
    -> testlib.std1.py
      -> testlib.std2.py
    -> testlib.std3.py

Implementation:

Latest changes store: https://github.com/andry81/tacklelib/blob/master/python/tacklelib/tacklelib.py

test.py:

import os, sys, inspect, copy

SOURCE_FILE = os.path.abspath(inspect.getsourcefile(lambda:0)).replace('\\','/')
SOURCE_DIR = os.path.dirname(SOURCE_FILE)

print("test::SOURCE_FILE: ", SOURCE_FILE)

# portable import to the global space
sys.path.append(TACKLELIB_ROOT) # TACKLELIB_ROOT - path to the library directory
import tacklelib as tkl

tkl.tkl_init(tkl)

# cleanup
del tkl # must be instead of `tkl = None`, otherwise the variable would be still persist
sys.path.pop()

tkl_import_module(SOURCE_DIR, 'testlib.py')

print(globals().keys())

testlib.base_test()
testlib.testlib_std1.std1_test()
testlib.testlib_std1.testlib_std2.std2_test()
#testlib.testlib.std3.std3_test()                             # does not reachable directly ...
getattr(globals()['testlib'], 'testlib.std3').std3_test()     # ... but reachable through the `globals` + `getattr`

tkl_import_module(SOURCE_DIR, 'testlib.py', '.')

print(globals().keys())

base_test()
testlib_std1.std1_test()
testlib_std1.testlib_std2.std2_test()
#testlib.std3.std3_test()                                     # does not reachable directly ...
globals()['testlib.std3'].std3_test()                         # ... but reachable through the `globals` + `getattr`

testlib.py:

# optional for 3.4.x and higher
#import os, inspect
#
#SOURCE_FILE = os.path.abspath(inspect.getsourcefile(lambda:0)).replace('\\','/')
#SOURCE_DIR = os.path.dirname(SOURCE_FILE)

print("1 testlib::SOURCE_FILE: ", SOURCE_FILE)

tkl_import_module(SOURCE_DIR + '/std1', 'testlib.std1.py', 'testlib_std1')

# SOURCE_DIR is restored here
print("2 testlib::SOURCE_FILE: ", SOURCE_FILE)

tkl_import_module(SOURCE_DIR + '/std3', 'testlib.std3.py')

print("3 testlib::SOURCE_FILE: ", SOURCE_FILE)

def base_test():
  print('base_test')

testlib.std1.py:

# optional for 3.4.x and higher
#import os, inspect
#
#SOURCE_FILE = os.path.abspath(inspect.getsourcefile(lambda:0)).replace('\\','/')
#SOURCE_DIR = os.path.dirname(SOURCE_FILE)

print("testlib.std1::SOURCE_FILE: ", SOURCE_FILE)

tkl_import_module(SOURCE_DIR + '/../std2', 'testlib.std2.py', 'testlib_std2')

def std1_test():
  print('std1_test')

testlib.std2.py:

# optional for 3.4.x and higher
#import os, inspect
#
#SOURCE_FILE = os.path.abspath(inspect.getsourcefile(lambda:0)).replace('\\','/')
#SOURCE_DIR = os.path.dirname(SOURCE_FILE)

print("testlib.std2::SOURCE_FILE: ", SOURCE_FILE)

def std2_test():
  print('std2_test')

testlib.std3.py:

# optional for 3.4.x and higher
#import os, inspect
#
#SOURCE_FILE = os.path.abspath(inspect.getsourcefile(lambda:0)).replace('\\','/')
#SOURCE_DIR = os.path.dirname(SOURCE_FILE)

print("testlib.std3::SOURCE_FILE: ", SOURCE_FILE)

def std3_test():
  print('std3_test')

Output (3.7.4):

test::SOURCE_FILE:  <root>/test01/test.py
import : <root>/test01/testlib.py as testlib -> []
1 testlib::SOURCE_FILE:  <root>/test01/testlib.py
import : <root>/test01/std1/testlib.std1.py as testlib_std1 -> ['testlib']
import : <root>/test01/std1/../std2/testlib.std2.py as testlib_std2 -> ['testlib', 'testlib_std1']
testlib.std2::SOURCE_FILE:  <root>/test01/std1/../std2/testlib.std2.py
2 testlib::SOURCE_FILE:  <root>/test01/testlib.py
import : <root>/test01/std3/testlib.std3.py as testlib.std3 -> ['testlib']
testlib.std3::SOURCE_FILE:  <root>/test01/std3/testlib.std3.py
3 testlib::SOURCE_FILE:  <root>/test01/testlib.py
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', '__file__', '__cached__', 'os', 'sys', 'inspect', 'copy', 'SOURCE_FILE', 'SOURCE_DIR', 'TackleGlobalImportModuleState', 'tkl_membercopy', 'tkl_merge_module', 'tkl_get_parent_imported_module_state', 'tkl_declare_global', 'tkl_import_module', 'TackleSourceModuleState', 'tkl_source_module', 'TackleLocalImportModuleState', 'testlib'])
base_test
std1_test
std2_test
std3_test
import : <root>/test01/testlib.py as . -> []
1 testlib::SOURCE_FILE:  <root>/test01/testlib.py
import : <root>/test01/std1/testlib.std1.py as testlib_std1 -> ['testlib']
import : <root>/test01/std1/../std2/testlib.std2.py as testlib_std2 -> ['testlib', 'testlib_std1']
testlib.std2::SOURCE_FILE:  <root>/test01/std1/../std2/testlib.std2.py
2 testlib::SOURCE_FILE:  <root>/test01/testlib.py
import : <root>/test01/std3/testlib.std3.py as testlib.std3 -> ['testlib']
testlib.std3::SOURCE_FILE:  <root>/test01/std3/testlib.std3.py
3 testlib::SOURCE_FILE:  <root>/test01/testlib.py
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', '__file__', '__cached__', 'os', 'sys', 'inspect', 'copy', 'SOURCE_FILE', 'SOURCE_DIR', 'TackleGlobalImportModuleState', 'tkl_membercopy', 'tkl_merge_module', 'tkl_get_parent_imported_module_state', 'tkl_declare_global', 'tkl_import_module', 'TackleSourceModuleState', 'tkl_source_module', 'TackleLocalImportModuleState', 'testlib', 'testlib_std1', 'testlib.std3', 'base_test'])
base_test
std1_test
std2_test
std3_test

Tested in Python 3.7.4, 3.2.5, 2.7.16

Pros:

  • Can import both module as a submodule and can import content of a module to a parent module (or into a globals if has no parent module).
  • Can import modules with periods in a file name.
  • Can import any extension module from any extension module.
  • Can use a standalone name for a submodule instead of a file name without extension which is by default (for example, testlib.std.py as testlib, testlib.blabla.py as testlib_blabla and so on).
  • Does not depend on a sys.path or on a what ever search path storage.
  • Does not require to save/restore global variables like SOURCE_FILE and SOURCE_DIR between calls to tkl_import_module.
  • [for 3.4.x and higher] Can mix the module namespaces in nested tkl_import_module calls (ex: named->local->named or local->named->local and so on).
  • [for 3.4.x and higher] Can auto export global variables/functions/classes from where being declared to all children modules imported through the tkl_import_module (through the tkl_declare_global function).

Cons:

  • Does not support complete import:
    • Ignores enumerations and subclasses.
    • Ignores builtins because each what type has to be copied exclusively.
    • Ignore not trivially copiable classes.
    • Avoids copying builtin modules including all packaged modules.
  • [for 3.3.x and lower] Require to declare tkl_import_module in all modules which calls to tkl_import_module (code duplication)

Update 1,2 (for 3.4.x and higher only):

In Python 3.4 and higher you can bypass the requirement to declare tkl_import_module in each module by declare tkl_import_module in a top level module and the function would inject itself to all children modules in a single call (it's a kind of self deploy import).

Update 3:

Added function tkl_source_module as analog to bash source with support execution guard upon import (implemented through the module merge instead of import).

Update 4:

Added function tkl_declare_global to auto export a module global variable to all children modules where a module global variable is not visible because is not a part of a child module.

Update 5:

All functions has moved into the tacklelib library, see the link above.

Andry
  • 2,273
  • 29
  • 28
3

Import package modules at runtime (Python recipe)

http://code.activestate.com/recipes/223972/

###################
##                #
## classloader.py #
##                #
###################

import sys, types

def _get_mod(modulePath):
    try:
        aMod = sys.modules[modulePath]
        if not isinstance(aMod, types.ModuleType):
            raise KeyError
    except KeyError:
        # The last [''] is very important!
        aMod = __import__(modulePath, globals(), locals(), [''])
        sys.modules[modulePath] = aMod
    return aMod

def _get_func(fullFuncName):
    """Retrieve a function object from a full dotted-package name."""

    # Parse out the path, module, and function
    lastDot = fullFuncName.rfind(u".")
    funcName = fullFuncName[lastDot + 1:]
    modPath = fullFuncName[:lastDot]

    aMod = _get_mod(modPath)
    aFunc = getattr(aMod, funcName)

    # Assert that the function is a *callable* attribute.
    assert callable(aFunc), u"%s is not callable." % fullFuncName

    # Return a reference to the function itself,
    # not the results of the function.
    return aFunc

def _get_class(fullClassName, parentClass=None):
    """Load a module and retrieve a class (NOT an instance).

    If the parentClass is supplied, className must be of parentClass
    or a subclass of parentClass (or None is returned).
    """
    aClass = _get_func(fullClassName)

    # Assert that the class is a subclass of parentClass.
    if parentClass is not None:
        if not issubclass(aClass, parentClass):
            raise TypeError(u"%s is not a subclass of %s" %
                            (fullClassName, parentClass))

    # Return a reference to the class itself, not an instantiated object.
    return aClass


######################
##       Usage      ##
######################

class StorageManager: pass
class StorageManagerMySQL(StorageManager): pass

def storage_object(aFullClassName, allOptions={}):
    aStoreClass = _get_class(aFullClassName, StorageManager)
    return aStoreClass(allOptions)
Eric Leschinski
  • 146,994
  • 96
  • 417
  • 335
user10370
  • 47
  • 1
3

This should work

path = os.path.join('./path/to/folder/with/py/files', '*.py')
for infile in glob.glob(path):
    basename = os.path.basename(infile)
    basename_without_extension = basename[:-3]

    # http://docs.python.org/library/imp.html?highlight=imp#module-imp
    imp.load_source(basename_without_extension, infile)
joran
  • 169,992
  • 32
  • 429
  • 468
Hengjie
  • 4,598
  • 2
  • 30
  • 35
  • 5
    A more general way to cut the extension out is: `name, ext = os.path.splitext(os.path.basename(infile))`. Your method works because the previous restriction to .py extension. Also, you should probably import the module to some variable/dictionary entry. – ReneSac Dec 06 '12 at 13:16
2

In Linux, adding a symbolic link in the directory your Python script is located works.

I.e.:

ln -s /absolute/path/to/module/module.py /absolute/path/to/script/module.py

The Python interpreter will create /absolute/path/to/script/module.pyc and will update it if you change the contents of /absolute/path/to/module/module.py.

Then include the following in file mypythonscript.py:

from module import *
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
  • 2
    This is the hack I used, and it has caused me some problems. One of the more painful ones was that IDEA has an issue where it doesn't pickup altered code from within the link, but yet attempts to save what it thinks is there. A race condition where the last to save is what sticks... I lost a decent amount of work because of this. – Gripp Jun 16 '15 at 23:23
  • @Gripp not sure if I am understanding your issue, but I frequently (almost exclusively) edit my scripts on a remote server from my desktop via SFTP with a client like CyberDuck, and in that case as well it is a bad idea to try and edit the symlinked file, instead its much safer to edit the original file. You can catch some of these issues by using `git` and checking your `git status` to verify that your changes to the script are actually making it back to the source document and not getting lost in the ether. – user5359531 Aug 01 '17 at 19:39
2

This will allow imports of compiled (pyd) Python modules in 3.4:

import sys
import importlib.machinery

def load_module(name, filename):
    # If the Loader finds the module name in this list it will use
    # module_name.__file__ instead so we need to delete it here
    if name in sys.modules:
        del sys.modules[name]
    loader = importlib.machinery.ExtensionFileLoader(name, filename)
    module = loader.load_module()
    locals()[name] = module
    globals()[name] = module

load_module('something', r'C:\Path\To\something.pyd')
something.do_something()
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
David
  • 692
  • 8
  • 21
2

A quite simple way: suppose you want import file with relative path ../../MyLibs/pyfunc.py

libPath = '../../MyLibs'
import sys
if not libPath in sys.path: sys.path.append(libPath)
import pyfunc as pf

But if you make it without a guard you can finally get a very long path.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
1

These are my two utility functions using only pathlib. It infers the module name from the path.

By default, it recursively loads all Python files from folders and replaces init.py by the parent folder name. But you can also give a Path and/or a glob to select some specific files.

from pathlib import Path
from importlib.util import spec_from_file_location, module_from_spec
from typing import Optional


def get_module_from_path(path: Path, relative_to: Optional[Path] = None):
    if not relative_to:
        relative_to = Path.cwd()

    abs_path = path.absolute()
    relative_path = abs_path.relative_to(relative_to.absolute())
    if relative_path.name == "__init__.py":
        relative_path = relative_path.parent
    module_name = ".".join(relative_path.with_suffix("").parts)
    mod = module_from_spec(spec_from_file_location(module_name, path))
    return mod


def get_modules_from_folder(folder: Optional[Path] = None, glob_str: str = "*/**/*.py"):
    if not folder:
        folder = Path(".")

    mod_list = []
    for file_path in sorted(folder.glob(glob_str)):
        mod_list.append(get_module_from_path(file_path))

    return mod_list
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Benos
  • 676
  • 7
  • 17
0

This answer is a supplement to Sebastian Rittau's answer responding to the comment: "but what if you don't have the module name?" This is a quick and dirty way of getting the likely Python module name given a filename -- it just goes up the tree until it finds a directory without an __init__.py file and then turns it back into a filename. For Python 3.4+ (uses pathlib), which makes sense since Python 2 people can use "imp" or other ways of doing relative imports:

import pathlib

def likely_python_module(filename):
    '''
    Given a filename or Path, return the "likely" python module name.  That is, iterate
    the parent directories until it doesn't contain an __init__.py file.

    :rtype: str
    '''
    p = pathlib.Path(filename).resolve()
    paths = []
    if p.name != '__init__.py':
        paths.append(p.stem)
    while True:
        p = p.parent
        if not p:
            break
        if not p.is_dir():
            break

        inits = [f for f in p.iterdir() if f.name == '__init__.py']
        if not inits:
            break

        paths.append(p.stem)

    return '.'.join(reversed(paths))

There are certainly possibilities for improvement, and the optional __init__.py files might necessitate other changes, but if you have __init__.py in general, this does the trick.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
-1

Here's a way of loading files sorta like C, etc.

from importlib.machinery import SourceFileLoader
import os

def LOAD (MODULE_PATH):
    if (MODULE_PATH [ 0 ] == "/"):
        FULL_PATH = MODULE_PATH;
    else:
        DIR_PATH = os.path.dirname (os.path.realpath (__file__))
        FULL_PATH = os.path.normpath (DIR_PATH + "/" + MODULE_PATH)

    return SourceFileLoader (FULL_PATH, FULL_PATH).load_module ()

Implementations Where:

Y = LOAD ("../Z.py")
A = LOAD ("./A.py")
D = LOAD ("./C/D.py")
A_ = LOAD ("/IMPORTS/A.py")

Y.DEF ();
A.DEF ();
D.DEF ();
A_.DEF ();

Where each of the files looks like this:

def DEF ():
    print ("A");
Bryan Grace
  • 1,826
  • 2
  • 16
  • 29
-1

You can use importfile from pydoc

from pydoc import importfile
module = importfile('/full/path/to/module/module.py')
name = module.myclass() # myclass is a class inside your python file
Jorge
  • 400
  • 1
  • 5
  • 15
  • does the upper code means the same as this `import module` i mean it is can compile variables, functions and classes ? – Shihab Feb 22 '22 at 09:25
  • It was already mentioned in another answer here https://stackoverflow.com/a/68361215/674039 – wim Aug 06 '23 at 01:36
-2

The best way, I think, is from the official documentation (29.1. imp — Access the import internals):

import imp
import sys

def __import__(name, globals=None, locals=None, fromlist=None):
    # Fast path: see if the module has already been imported.
    try:
        return sys.modules[name]
    except KeyError:
        pass

    # If any of the following calls raises an exception,
    # there's a problem we can't handle -- let the caller handle it.

    fp, pathname, description = imp.find_module(name)

    try:
        return imp.load_module(name, fp, pathname, description)
    finally:
        # Since we may exit via an exception, close fp explicitly.
        if fp:
            fp.close()
Zompa
  • 448
  • 4
  • 8
-3

I find this is a simple answer:

module = dict()

code = """
import json

def testhi() :
    return json.dumps({"key" : "value"}, indent = 4 )
"""

exec(code, module)
x = module['testhi']()
print(x)
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Mhadhbi issam
  • 197
  • 3
  • 6
-4

Something special is to import a module with absolute path with Exec(): (exec takes a code string or code object. While eval takes an expression.)

PYMODULE = 'C:\maXbox\mX47464\maxbox4\examples\histogram15.py';
Execstring(LoadStringJ(PYMODULE));

And then get values or object with eval():

println('get module data: '+evalStr('pyplot.hist(x)'));

Load a module with exec is like an import with wildcard namespace:

Execstring('sys.path.append(r'+'"'+PYMODULEPATH+'")');
Execstring('from histogram import *'); 
Max Kleiner
  • 1,442
  • 1
  • 13
  • 14