Import symbols starting with underscore

Question

I'm writing a simple Python library in which I have several "private" functions starting with underscore:

def _a():
    pass

def _b():
    pass

def public_interface_call():
    _a()
    _b()

This way my library users can simply do from MyLib.Module import * and their namespace won't be cluttered with implementation detail.

However I'm also writing unit tests in which I'd love to test these functions separately and simple importing truly all symbols from my module would be very handy. Currently I'm doing from Mylib.Module import _a _b public_interface_call but I wonder if there's any better/quicker/cleaner way to achieve what I want?

Usage of `from … import *` is generally considered a pretty bad practice. What I use instead is usually `from … import public_interface_call` in actual code, and `import …` for unit tests. — Underyx, Jul 20 '15 at 15:06
From a TDD point of view, unit testing "private" methods has a tiny bit of code smell. This isn't a hard and fast rule, but you should really be testing the interfaces of an object. If you find yourself needing to test the non-interface methods, then you probably need to consider that maybe they shouldn't be part of the class at all. I realize these underscored functions aren't necessarily part of a class, but the same logic applies. — Joel Cornett, Mar 30 '16 at 05:11
@JoelCornett, notice that it is not about class-methods but about module functions. Also I consider testing private functions as absolutely necessary and not just the API. They are the smallest part of your code and testing should not run your full application and test every possibility but rather small pieces. Btw. what can happen? *exclusively* testing private functions is of course wrong, but as the OP writes, he wants to test all functions. — Mayou36, Jul 26 '17 at 14:24

score 15 · Accepted Answer · edited May 23 '17 at 12:33

15

I'm not sure if it was a blackout or something when I wrote that question but today I realized (inspired by Underyx's comment) that I can simply do this:

import MyLib.Module

MyLib.Module._a()
MyLib.Module._b()

Or even to shorten things a little (because I'm a lazy bastard):

import MyLib.Module as mm

mm._a()
mm._b()

edited May 23 '17 at 12:33

Community

1
1

answered Jul 21 '15 at 06:08

Michał Góral

1,439
2
16
30

It does not seem to be a *better/quicker/cleaner* solution for your problem. How is this better than `from Mylib.Module import _a, _b`? – Feb 18 '20 at 08:24
3

I think there's a benefit that I don't have a long chain of imports which must be additionally maintained. Suppose that I add function `_c`. Then I have to import it and only then I can call it, where with my solution I can use it right away. – Michał Góral Feb 18 '20 at 16:49

score 5 · Answer 2 · answered Mar 30 '16 at 04:44

5

According to docs,

There is even a variant to import all names that a module defines:
from fibo import *
...
This imports all names except those beginning with an underscore (_).

Not sure why this is the case however.

answered Mar 30 '16 at 04:44

Jet Blue

5,109
7
36
48

2

This is the case because modules with a leading underscore are considered private, e.g. should not be used outside of the package. If you really want, you can. But in other languages, these functions would be strictly private, meaning you can't even access them from outside. It's a great thing that import * does not import them. – Mayou36 Jul 26 '17 at 14:25
@Mayou36 When testing subroutines of a module, one may end up importing also the private objects for that test script. This is also the scenario in which I'd use the "import *" anti-pattern. – normanius Oct 21 '20 at 00:07
I still do not understand the reason for using * import. Why not explicitly Import them? If you want to test them, you will need to know their names anyway – Mayou36 Oct 22 '20 at 10:03

score 1 · Answer 3 · answered Oct 22 '19 at 13:46

The best and most common solution for your problem already has been given:

import MyLib.Module as mm

If one still wants to make use of the variant from MyLib.Module import *, there is the possibility to override its default behavior: Add a list __all__ to the module's source file MyLib/Module.py and declare which objects should be exported.

Note that you need to do this for every object you want to be visible. This does not affect the behavior of the other import mechanism above.

Example:

# MyLib/Module.py
__all__ = ["_a", "_b"]

def _a(): pass 
def _b(): pass
def  c(): pass

# ...

# Import
from MyLib.Module import *
# => this imports _a() and _b(), but not c()

To specify a package index __all__ can make sense to control which submodules should be loaded when importing a package. See here for more information, or this SO thread on a related question.

In addition, I found this useful article on [medium.com](https://medium.com/@s16h/importing-star-in-python-88fe9e8bd4d2) about `from module import *` — normanius, Oct 22 '19 at 14:16

Import symbols starting with underscore

3 Answers3