2

My package has a dependency on the latest version of the jsonpickle package. Older versions are installable via pip, however I need the latest version (i.e on Github) for it to work. Is it generally considered OK in this situation to bundle the latest version of jsonpickle in my code? Is there some other solution? I'd rather not ask my users not to clone from github.

I'm thinking of organising my package like this:

My package
    |
__init__.py
 file1.py
 file2.py
          \
        jsonpickle (latest)

i.e Doing what was done here: Python: importing a sub‑package or sub‑module

Community
  • 1
  • 1
Mike Vella
  • 10,187
  • 14
  • 59
  • 86
  • 1
    If you just want to avoid making people git clone and manually build and install, pip install git+https://github.com/foo/bar wraps all of that up transparently. Is that an acceptable temporary solution (until they update PyPI)? – abarnert Aug 17 '13 at 01:03
  • In the end I went with @abarnet's solution - it's just a temporary fix, and all my tests seem to pass. In the meantime I'll try and bug the jsonpickle guys to upload their latest version to pypi. – Mike Vella Aug 17 '13 at 01:41

2 Answers2

3

As kag says, this is generally not a good idea. It's not that it's "frowned upon" as being unfriendly to the other packages, it's that it causes maintenance burdens for you and your users. (Imagine that there's a bug that's fixed in jsonpickle that affects your users, but you haven't picked up the fix yet. If you'd done things normally, all they'd have to do is upgrade jsonpickle, but if you're using an internal copy, they have to download the jsonpickle source and yours, hack up your package, and install it all manually.)

Sometimes, it's still worth doing. For example, the very popular requests module includes its own copy of other packages like urllib3. And yes, it does face both of the costs described above. But it also means that each version of request can rely on an exact specific version of urllib3. Since requests makes heavy use of urllib3's rarely-used interfaces, and even has workarounds for some of its known bugs, that can be valuable.

In your case, that doesn't sound like the issue. You just need a bleeding-edge version of jsonpickle temporarily, until the upstream maintainers upload a new version to PyPI. The problem isn't that you don't want your users all having different versions; it's that you don't want to force them to clone the repo and figure out how to install it manually. Fortunately, pip takes care of that for you, by wrapping most of the difficulties up in one line:

pip install git+https://github.com/foo/bar

It's not a beautiful solution, but it's only temporary, right?

abarnert
  • 354,177
  • 51
  • 601
  • 671
2

It's generally not the best idea to bundle some dependency with your project. Some projects do it anyway, or bundle it as an alternative if there's no system package available. (This is mostly found in C projects, not Python.)

You didn't mention what the "latest" means exactly. Is this the latest in pypi?

The best way to make sure a specific version, or greater than a baseline version, of a package is installed is to properly specify the requirement in setup.py requires section. Read more about requires here [1]. This way pip can take care of resolving dependencies, and if it's available in pypi it will be automatic.

[1] http://docs.python.org/2/distutils/setupscript.html#relationships-between-distributions-and-packages

kag
  • 176
  • 6
  • I completely agree that it's generally best avoided, but in this situation I'm rather stuck - by latest I mean latest on github, the version on pypi is too old. I have contacted the developers about updating it but I need a solution in the meantime. – Mike Vella Aug 16 '13 at 22:19
  • 1
    Actually, it's not that uncommon with Python. For example, the very popular [`requests`](http://docs.python-requests.org/en/latest/) module includes [its own copy](https://github.com/kennethreitz/requests/tree/master/requests/packages) of other packages like [`urllib3`](http://urllib3.readthedocs.org/en/latest/). You're right that it's best to avoid it, but there are some caes where you can't. – abarnert Aug 17 '13 at 01:02
  • I agree that there could be some cases where it can't be avoided, but in general it's not usually done. Do you know why requests does this? Maybe they have a good reason (like it's a specially patched version), but I can't recall many python packages I've built from source as having done this. I think this can be attributed to python having a single de-facto repository. The reason it's found with C projects is that, well, there's no pypi-equivalent for the C community. – kag Aug 17 '13 at 03:22