61

When using the Anacoda Python distribution, what is the best way to install a PyPi package that isn't available directly through Anaconda? For now I'm using:

conda pipbuild [pypi_name]
conda install --use-local [package_spec]

But I'm unclear if this is the best way and if conda update --all will update these packages when updates are made available. I'm also unclear what the point of binstar is when PyPi already exists.

Cristian Em.
  • 103
  • 4
user1507844
  • 5,973
  • 10
  • 38
  • 55

3 Answers3

62

I will disagree with the accepted response and note that pip install [some-pypi-package] is often the best way to install PyPi packages in Conda environments.

While the packages won't be managed by the Conda package manager, they will still be managed by the Anaconda environment. It will download the correct version of the package for the active Python install and update it correctly using the pip package manager.

When using Anaconda, you should turn to conda before pip when you can, but you don't lose any of the replicability benefits of using Anaconda when you use pip.

Anaconda recently published a doc that supports this: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment

Chris Conlan
  • 2,774
  • 1
  • 19
  • 23
  • 15
    One issue to be aware of with this: If a `pip` package requires a different version of a dependency already installed by `conda`, `pip` will replace the installed package, and `conda` won't realize its package has been removed. For a one-shot, scripted environment setup, this is probably fine. For an environment that you install into by hand over time, you will likely end up confused after a while. – ws_e_c421 Mar 07 '18 at 21:22
  • 8
    One way to mitigate the issue I mentioned above: save the output of `pip freeze` to `constraints.txt` and then install with `pip` using `pip install -c constraints.txt `. This will prevent `pip` from removing `conda` packages. It will also pin `pip` packages. The `pip` packages could be filtered out of `constraints.txt` by looking at the output of `conda list` for the items marked as ``. – ws_e_c421 Mar 07 '18 at 21:36
  • For things like Django, I’ll set up a virtual environment using Anaconda then do installations purely through pip for reasons mentioned above. Pure conda works best for getting a scientific computing stack together that uses conda channels. – Chris Conlan Mar 07 '18 at 21:37
47

If you want to build conda packages for PyPI packages, the recommended way is to use conda skeleton pypi package and use conda build package on the recipe that it creates. To install the package, use conda install --use-local package (here and elsewhere, package is the name of the PyPI package you wish to install).

You will need to update the recipe each time the package is updated.

You can also use pip to install these packages. There are two disadvantages: firstly, these packages won't be managed by conda at all. Secondly, these packages will not work if your default python version is different from the python version you are using in conda.

asmeurer
  • 86,894
  • 26
  • 169
  • 240
  • 2
    Thanks! Dumb question: what does condo managing the packages get me? Dependency management mostly? Automatic updates? Also, why is the method you described preferable to what I described above? – user1507844 Mar 27 '15 at 20:32
  • 2
    conda managing things gets you the benefits of dependency management. It also makes it easier to use that package with conda environments. Updates won't happen "automatically" (in the sense that you do need to type `conda update` for that to happen). As to why it's preferable, `conda skeleton` is much more stable than `conda pipbuild`. – asmeurer Mar 27 '15 at 20:47
  • It doesn't say this in http://conda.pydata.org/docs/using/pkgs.html#install-non-conda-packages it just says `pip install x`. should those instructions be updated? – endolith Sep 21 '15 at 23:47
  • and is there a way to have conda manage binary packages like http://www.lfd.uci.edu/~gohlke/pythonlibs/#scikits.audiolab ? – endolith Sep 22 '15 at 00:05
  • `conda convert` can convert Gohlke's packages to conda packages http://conda.pydata.org/docs/commands/build/conda-convert.html – faph Oct 06 '15 at 19:12
  • 1
    Final command to install the package: `conda install --use-local package` – Nathan Lloyd May 31 '16 at 22:18
  • 8
    Note that you may need to run `conda install conda-build` to get the `conda skeleton` command (I'm using miniconda). – Bryce Guinta Mar 14 '17 at 19:00
  • @astabada, the package manager is called "conda" and the packages are called "conda packages". "Anaconda" is a distribution of conda packages, and not the only way to use conda. – asmeurer Feb 19 '20 at 21:38
  • I am having this error `pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out` when I tried doing `conda skeleton pypi package`. – li.SQ Apr 21 '20 at 23:20
  • OP doesn't want to build a conda package. He simply wants to install a package from pypi. Conda is a wrapper around pip, I don't understand why the simple channel cannot be added or something, so the whole installation wouldn't become a split personality between pip and conda... – Csaba Toth Sep 13 '22 at 03:57
36

Since version 4.6.0, Conda has improved its interoperability with pip:

Conda and pip have historically had difficulties getting along. Pip hasn’t respected Conda’s environment constraints, while Conda has been all too happy to clobber pip-installed software. It’s a mess. Conda 4.6.0 adds preview support for better interoperability. With this interoperability, Conda can use pip-installed packages to satisfy dependencies, and can even remove pip-installed software cleanly and replace them with Conda packages when appropriate. There’s still room for improvement before pip and Conda are hunky-dory BFFs, but we hope this is a good start. This feature is disabled by default right now because it can significantly impact Conda’s performance. If you’d like to try it, you can set this condarc setting:

conda config --set pip_interop_enabled True

So, the way to get PyPI packages into conda (at the time of writing this) seems to be:

pip install <package>

If you want conda to replace the PyPI packages with its own (where possible), just run:

conda update --all

Given that the above setting is made. Conda marks its own channels as higher priority than pip, thus packages will be replaced.

There is a caveat (thanks @alfalfasprout): Since conda did not install the pypi packages they are not included in conda export --from-history. You'd have to get the list (from conda or pip) and install these packages separately via pip, like you did originally. Thus, for people using the conda history functionality, there is at least a second step needed.

NichtJens
  • 1,709
  • 19
  • 27
  • 1
    CondaValueError: Key 'pip_interop_enabled' is not a known primitive parameter. – Himanshu Poddar Feb 11 '20 at 05:18
  • 3
    What's your conda version? You can check via `conda --version`. I would presume your version is simply older than 4.6.0... – NichtJens Feb 11 '20 at 09:54
  • There are limitations on pip interop though. Conda cannot install pypi packages using the `conda install` command and so packages installed via pypi won't be available when doing a `conda export --from-history`. – alfalfasprout Dec 15 '22 at 00:24
  • True, you have to install those via pip. Just like you did originally. I'll add this as a caveat to the answer. Thanks! – NichtJens Dec 15 '22 at 11:38