3

after using conda extensively for a while, I was asked to update it yesterday and now things look broken. I have to admit that I am not an expert of what goes on behind the scenes so bear with me :)

After installing conda I used pip to install the various packages. Yesterday I started working on some code from a tutorial on git, which suggested to create an ad-hoc environment to work with:

conda env create -f binder/environment.yml

At this step I was suggested to update conda, since I was using an old version. I can't find in my terminal that specific message (i.e. I can't find what version of conda I was using before), but here is what I get now when I check the version of conda:

(base) francesco ~$ conda list conda
# packages in environment at /Users/francesco/anaconda3:
#
# Name                    Version                   Build  Channel
_anaconda_depends         2019.03                  py36_0  
anaconda                  custom                   py36_1  
anaconda-client           1.7.2                    py36_0  
anaconda-navigator        1.8.7                    py36_0  
anaconda-project          0.8.4                      py_0  
conda                     4.8.3                    py36_0  
conda-build               3.10.5                   py36_0  
conda-env                 2.6.0                h36134e3_0  
conda-package-handling    1.6.0            py36h1de35cc_0  
conda-verify              3.4.2                      py_1

One of the things I noticed after the update is that in order to activate/deactivate the environment I had to use conda activate/deactivate <env> instead of source activate/deactivate <env>.

After that I worked with the code in the new environment without any problem.

Today I tried to activate the main environment I work with, but I was asked to "init" my shell first with:

conda init bash

After that I activated my "usual" environment:

conda activate testenv

and I tried to start Jupyter Lab, but I got this error:

(testenv) francesco ~$ jupyter lab
Error executing Jupyter command 'lab': [Errno 2] No such file or directory

What is happening? Why is Jupyter Lab not working anymore in my usual environment?

I checked the installation in testenv and things look ok:

(testenv) francesco ~$ conda list | grep jup
jupyter                   1.0.0                    py37_7  
jupyter_client            5.2.4                    py37_0  
jupyter_console           6.0.0                    py37_0  
jupyter_core              4.4.0                    py37_0  
(testenv) francesco ~$ pip list | grep jup
jupyter                  1.0.0   
jupyter-client           5.2.4   
jupyter-console          6.0.0   
jupyter-core             4.4.0   
(testenv) francesco ~$

Does anyone know what is going on? This is a huge problem for me as conda/jupyter lab are the main tools I use for work :\

FrancescoLS
  • 376
  • 1
  • 6
  • 17
  • 1
    Your `conda list ` output does not show jupyer lab. maybe the update removed it accidentally. try reinstalling it by `conda install jupyterlab`. – cel Mar 22 '20 at 09:41
  • your are right, I reinstalled it and it worked. Isn't it weird though? In principle it might be the case for all the packages I installed in whatever environment I have, right? – FrancescoLS Mar 22 '20 at 10:08
  • 1
    _After installing conda I used [pip] to install the various packages._ Be careful when doing that, see https://www.anaconda.com/using-pip-in-a-conda-environment/. – AMC Mar 22 '20 at 23:48
  • 2
    @cel "*accidentally removed it*" sort of doubt it. Looks to me more like they upgraded Conda above v4.4 for the first time, and this takes the *base* env off the `PATH` when other envs are activated. Because the new env didn't install Jupyter Lab, it isn't available; before the upgrade it would have loaded it from **base**. – merv Mar 23 '20 at 13:12

2 Answers2

3

The common practice is to only install Jupyter in a single Conda environment (typically your base env if an Anaconda user), and always launch Jupyter from there. To use other Python envs in Jupyter, you need to install ipykernel in those envs, e.g.,

conda install -n testenv ipykernel

And, to avoid having to register your additional envs, it is recommended to install nb_conda_kernels in the env with Jupyter, e.g., 

conda install -n base nb_conda_kernels

As a side note, installing things with Pip can make environments unstable. I strongly recommend learning and adhering to the documented best practices.

merv
  • 67,214
  • 13
  • 180
  • 245
  • I hadn't seen that part of the documentation about using Pip with Conda, thanks for sharing that! I usually mention https://www.anaconda.com/using-pip-in-a-conda-environment/ instead. – AMC Mar 22 '20 at 23:49
  • I didn't know I could use a single installation of a package in all my environments (e.g. what you suggest to do with JupyterLab). I usually have "complete" environments, meaning that they contain all that it's needed, and then create a kernel based on the environment to be able to use it from inside Jupyter – FrancescoLS Mar 23 '20 at 07:08
  • 1
    @FrancescoLS I should clarify that this does not involve any sharing of packages across environments. Rather, Jupyter is an application that is capable of running different environments within a given notebook. The environments are still isolated. – merv Feb 26 '21 at 15:35
1

@FrancescoLS It seems that you had perhaps an older version of Conda installed(?) as the CHANGELOG indicates that source activate was deprecated in favor of conda activate in Conda v4.4.0 (as is also noted in this "How to Get Ready for the Release of conda 4.4" post from Anaconda).

This really isn't an "answer" in any actionable way, but it seems that you are not alone in Conda update breaking people's environments.

I think it is safe to say that keeping virtual environments safely intact during an upgrade is hard to do, and that when doing a major upgrade across virtual environment maintainers (Conda) it is even harder. This is one of the reasons that I personally try to have all of my project virtual environments be maintained either in native Python 3 venv virtual environments with pip or through Poetry (as they are (or at least used to be) way faster to restore than having to go through Conda's slower solver) and then only resorting to using Conda when I need to bring in multiple external binary applications.

Were you able to make a new Conda environment for your work that effectively restores it (perhaps from your own environment.yml file for that environment)?

Matthew Feickert
  • 786
  • 6
  • 26