Building a library with external dependencies -- pep8

Question

We keep a python library for various code at work. It has been a great way to share code, implement things only once, stay consistent, etc. Parts of it depends on external libraries, and a there have been a couple soulutions to this, none of which feel right.

One solution was to include the entire external library in our library. The argument was that the external library's version can be controlled, but this adds a bunch of code, is ugly and doesn't feel pythonic.

Another solution was a try import with a print error if the import fails. This feels good, is pep8, but our library has a large variety of modules and you'll often need one part of it, but not another. With this solution you'll see an error when doesn't affect you. There have been complaints.

try:
    import zmq
    import simplejson
except:
    print "Install zmq and simplejson to use the LiveFeedSubscription()."

I proposed putting try imports into the Class __init__ (solves the above problem), but it is not pep8; and arguably makes it harder to tell what the dependencies are as they are not being imported at the top of the module.

def __init__(self, msg_type='', msg_types=[], debug=False):
    try:
        import zmq
        import simplejson
    except:
        print "Install zmq and simplejson to use the LiveFeedSubscription()."

What is the python best practice for including external libraries in a library with such greatly divergent modules?

Martijn Pieters · Accepted Answer · 2014-12-01T19:33:05.063

6

Use a proper setup.py install script instead, which lists your dependencies. See the Python Packaging User Guide and the Setuptools documentation.

Tools such as easy_install, pip or zc.buildout will also install your dependencies as needed.

edited Dec 01 '14 at 19:33

answered Feb 13 '13 at 15:05

Martijn Pieters

1,048,767
296
4,058
3,343

This isn't ideal because there are many use cases where people do not need/want the external libraries. I would catch hell the first time a colleague needed to use the simple gps_date_converter(), but couldn't get an external dependency like matplotlib to compile on some old RHEL3 machine where he needed it. – jhau Feb 13 '13 at 16:10
@jhau: But the problem there is then not of dependency management but of older target machines not *supporting* those dependencies. That has nothing to do with bundling your dependencies in your package or having tools install them for you. – Martijn Pieters Feb 13 '13 at 16:12
The consensus here is it should not be a requirement to install all the deps to use an unrelated part of the library. – jhau Feb 13 '13 at 16:28
1

@jhau: That's fine, zc.buildout and pip support optional dependencies too. You asked for *best practices*, and best practices are to specify dependencies in standard metadata to be installable by tools. See [Optional dependencies in distutils / pip](http://stackoverflow.com/q/6237946) for an example. – Martijn Pieters Feb 13 '13 at 16:31
We are going to use Distribute in development mode with `extras_require`. This will allow us to keep everything in the repo, and itemize external packages with the features that require them in a sane way. Still going to put the imports in the classes `__init__` that need them though. Not having imports at the top of modules seems less bad since the external deps are clearly listed in `setup.py`. – jhau Feb 13 '13 at 18:09

score 2 · Answer 2 · answered Feb 13 '13 at 15:08

2

You could use a requirements file, which pip supports. This would look something like:

zmq
simplejson==1.2.3

You would then install the requirements (assuming the above was in requirements.txt using:

pip install -r requirements.txt

answered Feb 13 '13 at 15:08

Daniel Watkins

1,656
1
15
25

I like this. All the external libraries plus versions are listed and can be installed on request of the user. – jhau Feb 13 '13 at 16:24

Building a library with external dependencies -- pep8

2 Answers2