6

I want to access some kind of import dependency tracking in Python, if there is any.

I decided to add to my module a __dependencies__ dict describing versions of all modules imported by the module.

I want to have an automated way of obtaining a list of modules imported by my module. Preferably in the last line of the module.

ModuleFinder (as suggested by How to list imports within a Python module? ) would not work since the inspection shall be performed for already loaded module.

Another problem with ModuleFinder is that it inspects a Python script (with if __name__ == '__main__' branch), not a module.

If we consider a toy script script.py:

if __name__ == '__main__':
    import foo

then the result is:

>>> mf = ModuleFinder
>>> mf.run_script('script.py')
>>> 'foo' in mf.modules
True

which should be False if script is imported as a module.

I do not want to list all imported modules - only modules imported by my module - so sys.modules (suggested by What is the best way of listing all imported modules in python?) would return too much.

I may compare snapshots of sys.modules from the beginning and the end of the module code. But that way I would miss all modules used by my module, but imported before by any other module.

It is important to list also modules from which the module imports objects.

If we consider a toy module example.py:

from foo import bar
import baz

then the result should be like:

>>> import example
>>> moduleImports(example)
{'foo': <module 'foo' from ... >,
 'baz': <module 'baz' from ...>}

(it may contain also modules imported recursively or foo.bar given bar is a module).

Use of globls() (according to How to list imported modules?) requires me to deal with non-module imports manually, like:

from foo import bar
import bar

How can I avoid that?

There is another issue with my solution so far. PyCharm tends to clean up my manual imports on refactoring, which makes it hard to keep it working.

wpercy
  • 9,636
  • 4
  • 33
  • 45
abukaj
  • 2,582
  • 1
  • 22
  • 45
  • It's not clear why you think `ModuleFinder` would not work. In fact, it surely must be the only thing that *could* possibly work. How else are you going to distinguish between modules that were already loaded and ones that were first imported by your module? And how would you identify imports that are not in the module-global scope (e.g. inside functions)? – ekhumoro Oct 19 '17 at 18:52
  • @ekhumoro I have no idea why SO has not informed me about your comment. See the edited question, then please let me know so I can remove [EDIT] tags. – abukaj Nov 02 '17 at 12:30
  • Your claim that `ModuleFinder` does not work with modules is obviously false - see the [example in the python docs](https://docs.python.org/2/library/modulefinder.html#example-usage-of-modulefinder). I don't see any other good reasons why you shouldn't use it. – ekhumoro Nov 02 '17 at 18:43
  • Can you give a better example? – TheChetan Nov 03 '17 at 09:30
  • @ekhumoro Please point the example importing the module, then passing the imported module (not a `str`) to the `ModuleFinder`. I can not see it. Also, please see the updated question - I added an example of `ModuleFinder` failure at "__main__" branch. – abukaj Nov 03 '17 at 09:46
  • @abukaj. It makes no sense at all to pass an already imported module. As I pointed out in my first comment, if you did that, there would be no way to detect imports that were already in `sys.modules`, or not in the module-global scope. Also, there is no failure in your example - `ModuleFinder` correctly finds all the imports in the given python code. The only issue is how to explicitly treat a file as a module. Unfortunately, `run_script` doesn't allow passing in a file object, which would provide an easy work-around. However, the undocumented `load_module` method does allow that. – ekhumoro Nov 03 '17 at 14:05
  • @ekhumoro That is great that `ModuleFinder` does what it is intended to do. However it is not what I want to do. See the first line of the question now. – abukaj Nov 03 '17 at 15:13
  • @abukaj. Your question remains unclear. The requirement that this must work for an "already loaded module" makes no sense, and is contradicted by the rest of your question. Do you really expect to obtain this information *after* all the importing has already been done? Your own current solution does not work that way. – ekhumoro Nov 03 '17 at 15:51
  • @ekhumoro Yes, I expect there is a some kind of import tracking in Python. Also as an author of the module I am fine with injecting into my module some of inspection code. – abukaj Nov 03 '17 at 17:10
  • @abukaj. No, there is no such import tracking. If there was, what would be the purpose of `ModuleFinder`? Putting inspection code into the module doesn't seem feasible - surely such code would have to perform its own imports? I think there must be very good reasons why `ModuleFinder` works the way it does. – ekhumoro Nov 03 '17 at 17:32

1 Answers1

7

You can use the inspect moudule

For example:

import inspect
import os
m = inspect.getmembers(os) # get module content
filter(lambda x: inspect.ismodule(x[1]), m) # filter dependant modules

Here you have a live example

If you want the local modules imported just use:

filter(lambda x: inspect.ismodule(x[1]), locals().items()) # filter dependant modules

Another live example

Netwave
  • 40,134
  • 6
  • 50
  • 93