0

I'm trying to get all the functions that are defined in a given module (as opposed to imported by it), which I do like this:

from inspect import getmembers, isfunction
import my_module

fns = [fn for fn in vars(my_module).values() if isfunction(fn) and fn.__module__ == my_module.__name__]

The problem with the above is that if it's possible for my_module to import a function from other_module where other_module.__name__ == my_module.__name__, then the above code will erroneously mark that function as defined in my_module when it was actually defined in other_module and imported by my_module. I've been trying to create such a scenario but I can't seem to do that. I also know there's a sys.modules which is a dictionary mapping strings to modules, which makes me think that maybe module names must be unique in the current scope. Is this possible?

Boris Verkhovskiy
  • 14,854
  • 11
  • 100
  • 103
  • I'm not sure what you are trying to achieve but maybe you should read about the [`__all__` variable](https://stackoverflow.com/questions/44834/can-someone-explain-all-in-python). – mkrieger1 Dec 12 '20 at 12:20
  • @mkrieger1 I have a module. That module imports functions and defines functions. I want to get a list of only the functions that it defines, not the ones it imports. You are right that one way is to make the author of the module explicitly list the names of all the defined functions in the `__all__` variable (or some other variable) in the module, but a) aint nobody got time for that (for my usecase) and b) it has a slightly different use, you could define functions that you don't list in `__all__`. – Boris Verkhovskiy Dec 12 '20 at 12:25
  • 1
    What do you need this list of functions that are defined, but not imported, for? I sense an [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). – mkrieger1 Dec 12 '20 at 12:29
  • 1
    Is this different from your earlier question of [Get all defined functions in Python module](https://stackoverflow.com/questions/65251833/get-all-defined-functions-in-python-module)? Does that solution not work here? – DarrylG Dec 12 '20 at 12:30
  • @DarrylG I'm asking if my solution is correct. – Boris Verkhovskiy Dec 12 '20 at 12:30
  • @mkrieger1 if you must know, I need to write lots of scripts that convert CSV files (which can be in arbitrary formats) to JSON dictionaries in my internal format. I want to do this by defining a module with a bunch of functions like `def internal_value_name(input_csv_column_name, input_other_csv_column_name)`. I'm thinking I might want to import `internal_value_name` from a different script to use in the logic of a separate function, plus I don't want to have to look at the imports to see what a script will do. – Boris Verkhovskiy Dec 12 '20 at 12:38

1 Answers1

1

TL;DR of my finding is that module files may have a shared name, but the actually saved in the Python's sys.modules is unique. Hence, it can be concluded that it is not possible for two or more modules to have the same name in Python.

To illustrate, we setup the following structures and explanation follows:

#add C:\superflous to the PYTHONPATH
C:\
    superfluous\
        mod.py
    
D:\
    root\
        mod.py

The question states:

if it's possible for my_module to import a function from other_module where other_module.__name__ == my_module.__name__

For two modules to have the same __name__, the module files name must be the same. But how to do it in Python context? We know we can't have two files with same name in a directory.

And, if we put in the ...sub\mod.py the following line:

#...sub\mod.py
from mod import *

#or

import mod

...sub\mod.py will just import itself and sys.modules will include mod as one of his dict keys. We would like to import a module having the same as mod, so let's import mod from the C:superfluous\mod.py. To do this we first put an __init__.py inside sub to turn it into a package directory and also to enable absolute import. In addition, let's add main.py to the sub directory. Our final structure:

D:\
    root\
        __init__.py
        main.py
        mod.py

We define main.py and both of the mod.py as follows:

#root\main.py

from . import mod
import sys
import types

print(f"{'mod:':<12}{{}}".format(sys.modules['mod']))
print(f"{'root.mod:':<12}{{}}".format(sys.modules['root.mod']))
print(f"{'-'*60}")

function_from_root_mod = (root_mod := sys.modules['root.mod'].__dict__).keys() - sys.modules['__main__'].__dict__

for i in function_from_root_mod:
    if isinstance((root_mod_values := root_mod[i]), types.FunctionType) and root_mod_values.__module__ == 'root.mod':
        print(f'func_name => {i} from modules "{root_mod_values.__module__}"')

#root\mod.py

from mod import *

def sub_mod_func(): pass

var_1 = 1

#C:\superfluous\mod.py

from collections import defaultdict
from itertools import permutations

def c_mod_func(): pass

We then execute main.py through command prompt:

PS D:\> python -m root.main

the following is the output:

mod:        <module 'mod' from 'C:\\superfluous\\mod.py'>
root.mod:   <module 'root.mod' from 'D:\\root\\mod.py'>
------------------------------------------------------------
func_name => sub_mod_func from modules "root.mod"

Important findings:

  • Relative import (.) in main.py translated into root.mod and becomes a key as in sys.modules['root.mod']
  • mod.py in C:\superfluous becomes sys.modules['mod']

Module files may have the same name, but the paths (directory that containing them, e.g., D:\root\mod.py) provide uniqueness for the sys.modules keys.

Henry Tjhia
  • 742
  • 1
  • 5
  • 11