1

Some time ago, I installed dynet for Python and it worked fine. I installed it automatically with pip install git+https://github.com/clab/dynet#egg=dynet.

Then, I wanted to add MKL support. So I installed dynet again manually, using the instructions here: http://dynet.readthedocs.io/en/latest/python.html#manual-installation, passing a -DMKL parameter to cmake.

I want to make sure that I did everything correctly and that Python indeed takes the new installation of dynet. How can I check this?

In general, when I have a library that is built in C++ and linked from Python, how can I check, from within Python, the details of the installed library, and particularly, what binary file does Python link to?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Erel Segal-Halevi
  • 33,955
  • 36
  • 114
  • 183
  • *How* did you install dynet? Using the system package manager, pip, directly from source? – Martijn Pieters Jan 03 '18 at 09:04
  • 1
    I see the library has a `__version__` attribute; `import dynet; print(dynet.__version__)`. – Martijn Pieters Jan 03 '18 at 09:05
  • The library would have to expose, in some library-specific manner, what flags were used to build it. There is no standard for that. Python stores such metadata in the `sys` and [`sysconfig`](https://docs.python.org/3/library/sysconfig.html) modules, but that doesn't cover extension modules. – Martijn Pieters Jan 03 '18 at 09:07
  • @MartijnPieters I added explanation. Initially I installed automatically using pip, but later I wanted to add a flag so I installed manually from source. __version__ only gives the major version number (2.0), it gives no details about flags.. – Erel Segal-Halevi Jan 03 '18 at 09:08
  • @MartijnPieters Is there a way to see what binary file does python link to? Then, I can see whether the binary file is the one I just re-built. – Erel Segal-Halevi Jan 03 '18 at 09:09
  • Looking at the source the extension module is imported as `_dynet`, so `import _dynet; print(_dynet.__file__)`, I think. – Martijn Pieters Jan 03 '18 at 09:14
  • @MartijnPieters great, this solved my problem! Where in the source did you find it? – Erel Segal-Halevi Jan 03 '18 at 09:18
  • I found the [`dynet.py.in` file](https://github.com/clab/dynet/blob/master/python/dynet.py.in) on GitHub. – Martijn Pieters Jan 03 '18 at 09:27

3 Answers3

3

This is tricky. There is no standard way, you have to rely on information provided by the package (which many don't do) and on implementation details for the specific library.

Python libraries often expose their version number in a __version__ attribute. This attribute is not standardised, it is at most a convention. The dynet library does have this attribute but it doesn't list the patch-level version, only the major and minor versions:

>>> import dynet
>>> dynet.__version__
'2.0'

Because you installed the library either directly from the version control system (with git+https://...) or manually from source, you can't use pip freeze or pkg_resources.get_distribution() to inspect the version either:

$ pip freeze | grep -i dynet
dyNET==0.0.0

Compilation flags are not stored anywhere, normally. Python exposes it's own compilation-time info in structures in the sys module and the sysconfig module, but there is no such facility for extension modules. Unless the extension module explicitly includes such info in the API, you are out of luck there.

At best, you can try to locate the actual dynamic module loaded. Many projects use a wrapper Python module, which indirectly loads the actual extension module, confusing matters. In this case, importing dynet gives you the dynet.py file:

>>> dynet.__file__
'/.../lib/python3.6/site-packages/dynet.py'

This file was generated from the dynet.py.in file in the project source code. It imports the dynamic module as _dynet:

from _dynet import *

So you can still at least find the location of the dynamic object loaded with:

>>> import _dynet
>>> _dynet.__file__
'/.../lib/python3.6/site-packages/_dynet.cpython-36m-darwin.so'

You can further check what that dynamic library links to if you want to check they are the right versions; how you do this is platform dependent:

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • @ErelSegalHalevi: sorry, I had to roll back that edit. `ldd` is not a tool available on OS X, but you edited in my OS X path into the answer; you'd use `otool -L` instead. For Windows, you'd have to use a different tool altogether again. I'll happily link to instructions on how to list what dependencies are there on different platforms, but to put all that into this one probably is a little too verbose. – Martijn Pieters Jan 13 '18 at 16:56
1

3 ways from this source:

>>> import django
>>> django.__version__
'1.8'
>>> 
>>> import flask
>>> flask.__version__
'0.11.1'
>>> 
>>> import urllib2
>>> urllib2.__version__
'2.7'
>>> 
>>> import json
>>> json.__version__
'2.0.9'

Or by using pkg_resources

>>> import pkg_resources
>>> pkg_resources.get_distribution('flask').version

Or by using pip freeze

pip freeze | find /I "virtualenv"

Adelin
  • 7,809
  • 5
  • 37
  • 65
  • While `__version__` is more widespread, it is **still package specific**. If a package doesn't set it, it is not set. – Martijn Pieters Jan 03 '18 at 09:11
  • There won't be a perfect solution for it if author just didn't give any information about version. For example, as Martijn Pieters♦ has metioned, no `__version__` is set. – Sraw Jan 03 '18 at 09:13
  • These methods only give serial version number, but do not give information about flags used during the build. Is there maybe a way to see what binary file does python link to? – Erel Segal-Halevi Jan 03 '18 at 09:14
  • The version is either `2.0` or `0.0.0` according to each of these methods. All those are *wrong*. – Martijn Pieters Jan 03 '18 at 09:35
0

Not very clear with your question. But you can see all installed libraries by running the below command.

pip freeze

It will list all the libraries installed in your environment along with its version number.

Abhijit
  • 1,728
  • 1
  • 14
  • 25