How to customize a module import in Python?

Question

I would like to customize the behavior of my module when it is imported.

For example, let say I want my module to print an incremented number each time another file use import my_module. And when from my_module import some_string is used, it should print "some_string".

How could I do that?

---

I read several questions here and there but this does not seems to work.

# my_module.py
import sys

class MyImporter:

    def find_module(self, module_name, package_path):
        print(module_name, package_path)
        return self

    def load_module(self, module_name):
        print(module_name)
        return self

sys.meta_path.append(MyImporter())

# file.py
import my_module  # Nothing happens

Answer 1

What you're asking for is to have Python work not like Python. Whenever it imports a module it parses and executes the 'opened' code only once so it can pick up the definitions, functions, classes, etc. - every subsequent import of the module just references the cached & parsed first import.

That's why even if you put something like vars()["counter"] = vars().get("counter", 0) + 1 at your module's 'root', the counter will never go above 1 indicating that the module was indeed executed only once. You can force module reload using reload() (or importlib.reload() on Python 3.6+) but then you'd lose your counter if you keep it in the module itself.

Of course, you can have an external counter to be called when your module is imported, but that would have to be a contract with the users of your module at which point the question becomes - can't you just contract your users to call a function to increase your counter whenever they import your module instead of having to reload it for you to capture the count? Reloading a module will also make it have a potentially different state in every context it was reloaded which will make Python behave unexpectedly and should be avoided at any cost.

So, a short answer would be - no, you cannot do that and you should not attempt to do it. If you want something that doesn't work like Python - use something that isn't Python.

However... If you have a really, REALLY good reason to do this (and you don't!) and you don't mind hacking how Python fundamentally behaves (and you should mind) then you might attempt to do this by wrapping the built-in import and checking whenever it gets fired for your module. Something like:

your_module.py:

# HERE BE DRAGONS!!!

import sys
try:
    import builtins  # Python 3.4+
except ImportError:
    import __builtin__ as builtins  # Python 2.6+

__builtin_import__ = builtins.__import__  # store a reference to the built-in import
def __custom_import__(name, *args, **kwargs):
    # execute builtin first so that the import fails if badly requested
    ret = __builtin_import__(name, *args, **kwargs)
    if ret is sys.modules[__name__]:  # we're trying to load this module
        if len(args) > 1 and args[2]:  # using the `from your_module import whatever` form
            if "some_string" in args[2]:  # if some_string is amongst requested properties
                print("some_string")
        else:  # using the `import your_module` form...
            print_counter()  # increase and print the latest count
    return ret  # return back the actual import result
builtins.__import__ = __custom_import__  # override the built-in import with our method


counter = 0
# a convinience function, you can do all of this through the `__custom_import__` function
def print_counter():
    global counter
    counter += 1
    print(counter)
print_counter()  # call it immediately on the first import to print out the counter

some_string = "I'm holding some string value"  # since we want to import this

# HAVE I FORGOT TO TELL YOU NOT TO DO THIS? WELL, DON'T!!!

Keep in mind that this will not account for the first import (be it in the pure import your_module or in the from your_module import whatever form) as the import override won't exist until your module is loaded - that's why it calls print_counter() immediately in hope that the first import of the module was in the form of import your_module and not in the from..import form (if not it will wrongly print out the count instead of some_string the first time). To solve the first-import issue, you can move this 'ovverride' to the __init__.py in the same folder so that the override loads before your module starts and then delegate the counter change / some_string print to the module once loaded, just make sure you do your module name check properly in that case (you need to account for the package as well) and make sure it doesn't automatically execute the counter.

You also, technically, don't need the some_string property at all - by moving the execution of the built-in import around you can do your from..import check first, find the position of some_string in args[2] and pop it before calling the builtin import, then return None in the same position once executed. You can also do your printing and counter incrementing from within the overriden import function.

Again, for the love of all things fluffy and the poor soul who might have to rely on your code one day - please don't do this!

Answer 2

Actually, it does look like it's possible to do what you're looking for in python3.5. It's probably a bad idea, and I've carefully written my code to demonstrate the concept without being polished enough to use as-is, because I'd think carefully before doing something like this in a production project. If you need to look at a more-or-less production example of this, take a look at the SelfWrapper class in the sh module.

Meanwhile, you can override your own entry in sys.modules to be a subclass of Module. Then you can override getattribute and detect accesses to attributes. As best I can tell:

• Every subsiquent import of the module references spec so you could probably count accesses to spec to count total imports

• Each from foo import bar accesses bar as an attribute. I don't think you can distinguish between "from foo import bar" and "import foo; foo.bar"

  import sys, types
  class Wrapper(types.ModuleType):
  
      def __getattribute__(self, attr):
          print(attr)
          return super().__getattribute__(attr)
  
  
  test = "test"
  
  sys.modules[__name__].__class__ = Wrapper
  

Answer 3

Here is how you can dynamically import modules-

 from importlib import import_module

 def import_from(module, name):
     module = import_module(module, name)
     return getattr(module, name)

and use it like this-

 funcObj = import_from("<file_name>", "<method_name>")
 response = funcObj(arg1,arg2)