2

I'm trying to get a list of all modules inside a python package. This is my directory structure:

foo/
   __init__.py
   bar.py
   dumb.py
   dog.py

I'm trying to list all modules like this:

import foo

dir(foo)  
# ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']

As you can see the dir function doesn't list the modules of the package. But if I do something like this look what happens:

import foo
import foo.bar

dir(foo)  
# ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'bar']

Can someone explain what just happened? There's a way I can list all modules names without importing them?

Felipe Netto
  • 57
  • 8
  • Maybe you're confusing Python's `dir` with Window's `dir`. The former really only lists you the attributes or members of an object. This kind of works to list files (modules) that are in a folder (package), but only if the modules are in fact loaded and therefore accessible as attributes of the package. – Jeronimo Aug 12 '19 at 13:09
  • Following on @Jeronimo: you could use `os.walk` to scan the directory of `foo` to find the files in there, if you *really* wanted to avoid importing everything. – Scott Hunter Aug 12 '19 at 13:14
  • Looks like a duplicate of https://stackoverflow.com/q/23829845/407651 – mzjn Aug 12 '19 at 13:17

3 Answers3

2

You should first import all the foo' submodules if you want python to recognise it. Just change your line import foo.bar for from foo import *. I think it should work :D. You can also import them in the __init__.py file. Then you can filter the output of dir(foo) with: 

list(map(lambda x: not x.startswith("_"), dir(foo)))
ivallesp
  • 2,018
  • 1
  • 14
  • 21
2

Python's import mechanism isn't a directory listing, when you import foo only foo is imported along with a file named __init__.py (if it exists in that directory).

When a module is imported its code is executed and stored within sys.modules, and a use of __init__.py is that you use it to from . import bar thus allowing import foo to import whatever it is from the directory that you want.

When you ran your second line import foo.bar thus you told the interpreter to import bar (be it a file named bar.py or a directory named bar (with or without an __init__.py)) and that this bar module is a child of the foo module (the . which connects them).

p.s. this is a small abstraction on the import system, but I hope this answers your question.

EDIT: To get your desired results, you can use the imported modules metadata to inspect the folder imported.

waving my hands over the internals, here is a snippet

[pyimport]
$: tree
.
├── mod1
│   ├── __init__.py
│   └── inner.py
└── mod2
    └── inner.py

[pyimport]
$: py --no-banner

In [1]: import mod1

In [2]: import mod2

In [3]: mod1.__file__
Out[3]: '/home/yhoffman/Sandbox/pyfiles/pyimport/mod1/__init__.py'

In [4]: mod1.__path__
Out[4]: ['/home/yhoffman/Sandbox/pyfiles/pyimport/mod1']

In [5]: mod2.__file__

In [6]: mod2.__path__
Out[6]: _NamespacePath(['/home/yhoffman/Sandbox/pyfiles/pyimport/mod2'])

In [7]: import os

In [8]: os.listdir(os.path.dirname(mod1.__file__))
Out[8]: ['__init__.py', 'inner.py', '__pycache__']

In [9]: os.listdir(mod1.__path__[0])
Out[9]: ['__init__.py', 'inner.py', '__pycache__']

In [11]: os.listdir(mod2.__path__._path[0])
Out[11]: ['inner.py']
Zubda
  • 943
  • 1
  • 6
  • 16
  • I understood, thanks. So, do you know any way to do what I'm trying to do? :P – Felipe Netto Aug 12 '19 at 14:05
  • For a deep dive in python's `import` mechanism, I would reccomend you checking out David Beazley's great course named **Modules live and let die** https://www.youtube.com/watch?v=0oTh1CXRaQ0 – Zubda Aug 12 '19 at 17:20
1

You could also use the pkgutil module

import pkgutil

[name for _, name, _ in pkgutil.iter_modules(['foo'])]

Note: the parameter of iter_modules is a list of paths to modules.

Copatek
  • 13
  • 2