1

I am looking for a Python routine that loads and returns a module from a filename, even if a module with that name or from that filename was imported before.

More specifically, I want to write a function unconditional_import that passes the following tests:

import os

def make_module(folder, filename, version):
    try: os.mkdir(folder)
    except OSError: pass

    f = open(os.path.join(folder,filename), "w")
    f.write("version = %i" % version)
    f.close()

make_module("test","spam.py", 1)
assert( unconditional_import("test","spam.py").version == 1 )

make_module("test","more_spam.py", 2)
assert( unconditional_import("test","more_spam.py").version == 2 )

make_module("test","more_spam.py", 3)
assert( unconditional_import("test","more_spam.py").version == 3 )

make_module("test2","more_spam.py", 4)
assert( unconditional_import("test2","more_spam.py").version == 4 )

I found a solution for this (which I will post as an answer to keep things structured), but it is rather ugly and does not feel robust. I would expect there to be some easier, native way to do this, but I fail to find it.

Notes:

  • I am aware of How to import a module given the full path?, which addresses how to import a module from a path, but only works once.

  • I am aware that in most cases, one would just reload the interpreter, but that’s not an option for me.

Community
  • 1
  • 1
Wrzlprmft
  • 4,234
  • 1
  • 28
  • 54
  • 1
    Why do you need this; what are you actually trying to achieve? And why not just `reload` (see http://stackoverflow.com/questions/1254370/reimport-a-module-in-python-while-interactive)? – jonrsharpe May 01 '16 at 07:58
  • @jonrsharpe: `reload` alone does not suffice. See my answer: If I remove the file-removal routines (and thus just rely on `reload`), the third test fails. — As for why I need this: At the end of the day, I am trying to implement some just-in-time compilation (and no, Cython and similar are not an option), but explaining this would be beyond the scope of this question in my opinion. – Wrzlprmft May 01 '16 at 08:05
  • 1
    Without that sort of information, this is probably an [XY problem](http://meta.stackexchange.com/q/66377). – jonrsharpe May 01 '16 at 08:09
  • @jonrsharpe: I am aware of that possibility and I take that risk. Explaining the necessary background would probably need more characters than allowed and turn this into asking you to do my work for me. I have to ask you to trust me that I explored several alternatives to doing this. Also, I do not think that I am asking for something utterly bizarre here. – Wrzlprmft May 01 '16 at 08:20

1 Answers1

2

My best solution so far is this:

def unconditional_import(folder, filename):
    full_path = os.path.join(folder,filename)

    if os.path.isfile(full_path+"c"):
        os.remove(full_path+"c")

    cachefolder = os.path.join(folder,"__pycache__")
    if os.path.isdir(cachefolder):
        for f in os.listdir(cachefolder):
            if f.startswith(filename.rstrip(".py")) and f.endswith(".pyc"):
                os.remove(os.path.join(cachefolder,f))

    if sys.version_info < (3,0):
        from imp import load_source
        foo = load_source("foo", full_path)

    elif sys.version_info < (3,5):
        from importlib.machinery import SourceFileLoader
        foo_module = SourceFileLoader("foo", full_path)
        foo = foo_module.load_module()

    else:
        from importlib.util import spec_from_file_location, module_from_spec
        spec = spec_from_file_location("foo", full_path)
        foo = module_from_spec(spec)
        spec.loader.exec_module(foo)

    return foo

Problems and notes:

  • I could not test the variant for Python 3.5 and higher, but I see no reason why it shouldn’t work or why there would be an easier solution.

  • Removing the PYC files manually makes me cringe and I expect it to cause problems.

Wrzlprmft
  • 4,234
  • 1
  • 28
  • 54