Get module instance given its vars dict

Question

Suppose I have the dict of a module (via vars(mod), or mod.__dict__, or globals()), e.g.:

import mod

d = vars(mod)

Given the dict d, how can I get back the module mod? I.e. I want to write a function get_mod_from_dict(d), which returns the module if the dict belongs to a module, or None:

>>> get_mod_from_dict(d)
<module 'mod'>

If get_mod_from_dict returns a module, I must have that this holds:

mod = get_mod_from_dict(d)
assert mod is None or mod.__dict__ is d

---

I actually can implement it like this:

def get_mod_from_dict(d):
    mods = {id(mod.__dict__): mod for (modname, mod) in sys.modules.items()
                                  if mod and modname != "__main__"}
    return mods.get(id(d), None)

However, this seems inefficient to me, to iterate through sys.modules.

Is there a better way?

---

Why do I need this?

• In some cases, you get access to the dict only. E.g. in the stack frames. And then, depending on what you want to do, maybe just for inspection/debugging purpose, it is helpful to get back the module.

• I wrote some extension to Pickler which can pickle methods, functions, etc. Some of these have references to the module, or the module dict. Wherever I have a dict which belongs to a module during pickling, I don't want to pickle the dict, but instead a reference to the module.

Answer 1

Every module has a __name__ attribute that uniquely identifies the module in the import system:

>>> import os
>>> os.__name__
'os'
>>> vars(os)['__name__']
'os'

Imported modules are also cached in sys.modules, which is a dict mapping module names to module instances. You can simply look up the module's name there:

import sys

def get_mod_from_dict(module_dict):
    module_name = module_dict['__name__']
    return sys.modules.get(module_name)

Some people have expressed concern that this might not work for (sub-)modules in packages, but it does:

>>> import urllib.request
>>> get_mod_from_dict(vars(urllib.request))
<module 'urllib.request' from '/usr/lib/python3.7/urllib/request.py'>

There is a very minor caveat, though: This will only work for modules that have been properly imported and cached by the import machinery. If a module has been imported with tricks like How to import a module given the full path?, it might not be cached in sys.modules and your function might then unexpectedly return None.

Answer 2

You can use importlib.import_module to import a module given it's name. Example for numpy

In [77]: import numpy 
    ...: import importlib                                                                                                                                                                               

In [78]: d = vars(numpy)                                                                                                                                                                                

In [79]: np = importlib.import_module(d['__name__'])                                                                                                                                                    

In [80]: np.array([1,2,3])                                                                                                                                                                              
Out[80]: array([1, 2, 3])

Answer 3

For completeness, another solution, via the gc module:

def get_mod_from_dict_3(d):
  """
  :param dict[str] d:
  :rtype: types.ModuleType|None
  """
  objects = gc.get_referrers(d)
  for obj in objects:
    if isinstance(obj, types.ModuleType) and vars(obj) is d:
      return obj
  return None

Using gc might be Python interpreter dependent, though. Not all Python interpreters might have a GC. And even if they have, I'm not sure it is guaranteed that the module has a reference to its dict (although, very likely, it does; it cannot really think of a good reason why it would not have).

So, I think the other solution via sys.modules[d['__name__']] is probably better.

Although I checked CPython and PyPy, and in both cases, this solution works. And also, this solution is more generic. It works (without the check for ModuleType) even for any arbitrary object.

Despite, thinking about different Python interpreters, I could imagine even a Python interpreter where vars(mod) will never return the same dict, where this will create a dict on-the-fly. Then such a function cannot be implemented at all. Not sure.

I collected all the given solutions, and some testing code, here.

Answer 4

You could eventually improve a little your solution by using a generator:

def get_mod_from_dict_2(d):
    return next((mod for modname, mod in sys.modules.items() if mod and modname != "__main__" and id(mod.__dict__) == id(d)), None)

But that won't help you to avoid the use of sys.modules...

Update: As said in the answer of @Devesh Kumar Singh, you could use the importlib module to retrieve an already imported module by name (or import it if it haven't been already). The module's dictionary holds the module's name and file as long as it is not the '__main__' module. From there, you can do:

import importlib
import some_module

d = vars(some_module)
print(d['__name__']) # >> 'some_module'

m = importlib.import_module(d['__name__'])
print(m)   # >> <module 'some_module' from '/path/to/some_module.py'>