0

I've been learning about creating Python packages and the Python import system because I'm attempting to make my own Python package for the first time, following "best practices" as much as I can.

In the process, I noticed that some well-known Python packages name themselves in their namespace, and some don't. A small sample from my testing in Python 3.8.10, with package versions:

>>> import numpy, matplotlib, scipy, tqdm, setuptools
>>> "numpy" in dir(numpy), "matplotlib" in dir(matplotlib), "scipy" in dir(scipy), "tqdm" in dir(tqdm), "setuptools" in dir(setuptools)
(False, False, False, True, True)
>>> numpy.__version__, matplotlib.__version__, scipy.__version__, tqdm.__version__, setuptools.__version__
('1.20.2', '3.4.2', '1.6.3', '4.60.0', '49.6.0.post20210108')

From this and other samples, it seems like at least a few big-name Python package do list themselves in their namespaces, but most big-name Python packages do not list themselves in their own namespaces.

I found that the package I'm creating does list itself in its own namespace, which makes the above observation relevant to me.

Consider a use-case where this might have practical consequences: a module is listed in its own namespace and a process that recursively searches for submodule names in dir(<module>) begins. The module name module will be returned as a submodule in an infinite loop because module is always in dir(<module>).

I'm wondering:

  1. Is there an intentional, established rationale for including or not including a package in its own namespace, and if so, what is that rationale?
  2. What do big-name Python packages do under the hood with their directory structure or their distribution/packaging files (pyproject.toml, setup.cfg, setup.py, etc.) to avoid including the package in its own namespace?

Thanks for informing a curious newbie package writer.

DoodleVib
  • 159
  • 13
  • 1
    `dir` doesn't exactly give you the namespace of an object, btw. – juanpa.arrivillaga Sep 04 '21 at 02:15
  • BTW, the reason `setuptools` has itself is because `setuptools.__init__` imports `setuptools.version` – juanpa.arrivillaga Sep 04 '21 at 02:24
  • Thanks for your point about `dir` @juanpa.arrivillaga. I just learned about the distinction between attributes and namespaces from [this post](https://stackoverflow.com/questions/32003472/difference-between-locals-and-globals-and-dir-in-python) and [this post](https://stackoverflow.com/questions/980249/difference-between-dir-and-vars-keys-in-python), and from the [docs here](https://docs.python.org/3/tutorial/classes.html#python-scopes-and-namespaces). – DoodleVib Sep 05 '21 at 05:26

1 Answers1

2

Too long for a comment; TLDR: try to avoid extra imports and show the code to get concrete suggestions.

  1. No, there is no established rationale for this. Do whatever makes sense. Having shorter imports is more user-friendly; cases when you have to introduce a no-op import layer on top are VERY RARE.

Note that from tqdm import tqdm actually imports object tqdm from module tqdm, so it is not really an empty import layer on top - the module contains a bunch of other objects.

  1. Python project are diverse. In fact, many high profile projects are written in other languages, like C/C++. They all have different folder structure, so there is no one size fits all.
Marat
  • 15,215
  • 2
  • 39
  • 48
  • I appreciate you sharing those thoughts @Marat . I'm glad to hear that there's no One-Size-Fits-All Namespace Standards Club that will put me on its naughty list if my package has itself in its namespace due to the structure and imports. I was mostly curious about accepted standards and the rationale behind them, not necessarily ways to modify my specific package. – DoodleVib Sep 05 '21 at 07:48
  • @DoodleVib I feel for you. Share your code and I can come up with specific suggestions on module structure. – Marat Sep 05 '21 at 15:29