33

I wonder if is there any reliable and consistant way to get a Python package's "import name" / namespace. For example;

Package; django-haystack
Import name; haystack

or

Package; ipython
Import name; IPython

So far I know, PyPi doesn't store that information that I've checked with PyPiXmlRpc.

I also tried to automate to download the package, extract it and dig the .egg-info but some packages doesn't have that folder at all.

Any help will be appreciated and will be used for a good-manner gadget :)

kirpit
  • 4,419
  • 1
  • 30
  • 32

5 Answers5

10

Wheels

I know this is an old question, but wheel packages have since been invented! Since a wheel is simply a zip file that gets extracted into the lib/site-packages directory, an examination of the contents of the wheel archive can give you the top level imports.

>>> import zipfile
>>> zf = zipfile.ZipFile('setuptools-35.0.2-py2.py3-none-any.whl')
>>> top_level = set([x.split('/')[0] for x in zf.namelist()])
>>> # filter out the .dist-info directory
>>> top_level = [x for x in top_level if not x.endswith('.dist-info')]
>>> top_level 
['setuptools', 'pkg_resources', 'easy_install.py']

So setuptools actually gives you three top level imports!

pip download

pip now has a download command, so you can simply run pip download setuptools (or whatever package you like) and then examine it.

Reverse look up

Unfortunately I haven't found an easy way to go the other direction yet. That is, given the import name, what is the package name. This can be a problem if you are looking at some example code or maybe if you use Anaconda that comes with a bunch of packages pre-installed and you want to know the actual package name.

Toshinou Kyouko
  • 334
  • 9
  • 21
Brad Campbell
  • 2,969
  • 2
  • 23
  • 21
9

Note that what you call a package here is not a package but a distribution. A distribution can contain zero or more modules or packages. That means there is no one-to-one mapping of distributions to packages.

I'm not sure there is a way to detect what modules and packages will be installed by a distribution, other than actually installing it and introspect filesystem changes for newly added packages, modules and pth files.

iacob
  • 20,084
  • 6
  • 92
  • 119
Wichert Akkerman
  • 4,918
  • 2
  • 23
  • 30
  • I understand. Major problem is the relation between a distribution and a package/packages as I see. But there must be an answer to my question. I will find the most reliable and consistent one and let you guyz know for sure. – kirpit Sep 19 '11 at 08:55
  • after a long long researching period, i end up with your answer that any Python distribution may have one or more packages such as the package itself and let's say a test package next to as a simple example. Anyways, i was able to finish my experimental prototype (pydoc.net) and hopefully will provide some sort of API to cover this question consistently.. Thanks Wichert. – kirpit Nov 02 '11 at 07:08
5

In principal, everything you need to get that information is in the setup.py that is supposed to be in every such package. That information would roughly be the union of the packages, py_modules, ext_package and ext_modules of the Distribution object. In fact, here's a little script that mocks out distutils.core.setup just for the purpose of getting that information. 

import distutils.core
distutils.core._setup_stop_after = "config"
_real_setup = distutils.core.setup
def _fake_setup(*args, **kwargs):
    global dist
    dist = _real_setup(*args, **kwargs)

distutils.core.setup = _fake_setup

import sys
setup_file = sys.argv[1]
sys.argv[:] = sys.argv[1:]
import os.path
os.chdir(os.path.dirname(setup_file))

execfile(os.path.basename(setup_file))

cat = lambda *seq: sum((i for i in seq if i is not None), [])
pkgs = set(package.split('.')[0] for package
           in cat(dist.packages,
                  dist.py_modules,
                  [m.name for m in cat(dist.ext_modules)],
                  [m.name for m in cat(dist.ext_package)]))

print "\n".join(pkgs)

For many packages, this will work like a charm, but for a counterexample, see numpy, It breaks because numpy provides its own distutils, and I can see no obvious way around it.

SingleNegationElimination
  • 151,563
  • 33
  • 264
  • 304
  • I didn't have chance to try, will do as soon as possible and feedback. thanks in advance! – kirpit Aug 26 '11 at 00:34
  • yes it works for many package releases but as I understood it works in the modulefinder's logic that gives you all the modules had been used and you get the high-level namespace(s) out of it. Unfortunately, it gives more than one namespace if the release is using some other package such as test; check django-uni-form for example. I just modified your script a bit by the way; https://gist.github.com/1176645 – kirpit Aug 28 '11 at 13:12
  • 4
    And Python should make "import name" field required on PyPI! Can't be that painful... stucked. – kirpit Aug 28 '11 at 13:20
  • 2
    Seriously... I've been racking my brain for a while on how to get the distribution name from the import name. There's just not a way currently. I had hoped that the `provides` key in `setup.py`would provide this. I had hoped to be able to query that through https://wiki.python.org/moin/PyPIXmlRpc, but no luck ): – d0c_s4vage Sep 01 '16 at 02:30
2

My project johnnydep has this feature:

>>> from johnnydep import JohnnyDist
>>> dist = JohnnyDist("django-haystack")
>>> dist.import_names
['haystack']

Do note that a distribution may provide multiple import names:

>>> JohnnyDist("setuptools").import_names
['pkg_resources', 'setuptools']

or none at all:

>>> JohnnyDist("bs4").import_names
[]
wim
  • 338,267
  • 99
  • 616
  • 750
0

After searching lots of information, finally found the only thing which worked for me as expected. Example for python-dotenv package which has dotenv import name:

$ cat $(python -c "import pkg_resources; print(pkg_resources.get_distribution('python-dotenv').egg_info)")/top_level.txt
dotenv

(taken from this answer)

maoizm
  • 682
  • 13
  • 17