3

I have a module:

test
   __init__.py

In __init__.py I have single line:

import numpy as np

I want to use numpy across the package (modules and submodules) I want to create.

Problem is, somehow numpy is getting exposed. If I run IPython and do

import test

There will be a test.np accessible.

I looked at many packages on github and they often import a lot in the main __init__.py (sys, division, etc) and yet when I import those packages in IPython none of the external modules are exposed.

What are they doing differently?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
dreamwalker
  • 1,693
  • 3
  • 18
  • 23

2 Answers2

3

You can define a global name __all__, set to a list or tuple of names, to limit what is imported and what documentation tools generally will list:

__all__ = ['function1', 'ClassName2']

The __all__ name limits what from test import * will import, and is also used by documentation tools to limit what is listed as public API for a given module.

See the import statement documentation:

The public names defined by a module are determined by checking the module’s namespace for a variable named __all__; if defined, it must be a sequence of strings which are names defined or imported by that module. The names given in __all__ are all considered public and are required to exist. If __all__ is not defined, the set of public names includes all names found in the module’s namespace which do not begin with an underscore character ('_'). __all__ should contain the entire public API. It is intended to avoid accidentally exporting items that are not part of the API (such as library modules which were imported and used within the module).

The __init__ modules you inspected will almost certainly define __all__ sequences.

You can also delete names again from your module, provided your functions do not need access to the global names later:

del sys

The IPython autocompletion otherwise uses all names that do not start with an underscore; the autocompletion ignores the __all__ list, but will ignore names like _sys.

The numpy.__init__ module (before version 1.8.0) itself deletes names from the global namespace again:

if __NUMPY_SETUP__:
    import sys as _sys
    _sys.stderr.write('Running from numpy source directory.\n')
    del _sys

but here sys is bound as _sys and IPython would ignore that name even if it wasn't deleted. numpy also builds up an __all__ list in that module.

In numpy version 1.8.0 and newer, a import sys statement was added to that file and IPython offers it for autocompletion, because it is still part of the global namespace.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Using `__all__` and leaving out `np` won't make `test.np` inaccessible. – martineau Nov 14 '13 at 22:11
  • @martineau: No, but *that is not the question*. IPython shows auto-completion targets, and the OP is wondering why some projects seem to import all sorts of libraries but IPython is not listing them. – Martijn Pieters Nov 14 '13 at 22:17
  • I tested this and even if I have `__all__` = [] I still see in IPython everything I import in `__init__.py`. Regardless whether the import statement is import numpy or import numpy as np, import sys etc. – dreamwalker Nov 14 '13 at 22:20
  • @dreamwalker: Did you restart IPython or reload the module? The module is imported just once and not automatically re-loaded. – Martijn Pieters Nov 14 '13 at 22:21
  • I restart it every time I make changes to the module. – dreamwalker Nov 14 '13 at 22:22
  • @dreamwalker: can you name a library where you *don't* see everything in it listed? – Martijn Pieters Nov 14 '13 at 22:24
  • `__init__.py` for numpy imports sys and warnings for example. When I run IPython I don't see them and also I can't do numpy.sys ('module' object has no attribute 'sys') I can however do numpy.warnings – dreamwalker Nov 14 '13 at 22:25
  • @dreamwalker: I may be wrong about IPython heeding `__all__` for autocompletion; I'm testing this again now. It should certainly use `__all__` for documentation (`help(testing)`). `numpy` *deletes* names again from the `__init__`. – Martijn Pieters Nov 14 '13 at 22:34
  • @dreamwalker: see [this line](https://github.com/numpy/numpy/blob/master/numpy/__init__.py#L141); `sys` is imported as `_sys` as well, but it is also deleted again with `del _sys`. – Martijn Pieters Nov 14 '13 at 22:37
  • @alko: indeed it is. Intriguing. And curious why the file then reimports `sys` as `_sys` to do the warning. – Martijn Pieters Nov 14 '13 at 22:44
  • @alko: using 1.8.0 IPython *does* offer `sys` as an auto-completion target. – Martijn Pieters Nov 14 '13 at 22:47
  • @alko: So your point is that things numpy `__init__.py` imports that aren't deleted are accessible -- so the OP is wrong? – martineau Nov 14 '13 at 22:49
  • @martineau: The OP is using a version before 1.8.0. – Martijn Pieters Nov 14 '13 at 22:51
  • thank you very much for all your help, this sheds much more light on the situation, guess I need to read up a bit more on how to organize modules and also I seem to be wanting to break up things into too many files where I guess in the Python world it doesn't make sense... – dreamwalker Nov 14 '13 at 22:58
-1

You have to make sure you are only importing it into the local namespace somehow.

qwertynl
  • 3,912
  • 1
  • 21
  • 43
  • While this could be done by importing it into each module function that uses it, besides being tedious it would also be inefficient. – martineau Nov 14 '13 at 22:23