1

I like having my base conda environment extremely minimal. I accidentally installed a bunch of packages in this environment so I tried to remove them:

I ran the following command as suggested by conda: remove all installed packages from base/root environment: conda install --revision 0 -n base

However, when I ran the following script to reinstall the essentials: 

conda install -c conda-forge wget rsync pv ipython jupyter jupyterlab pandas numpy biopython r-recommended python=3 -y

conda install -c bioconda pullseq samtools bbmap hmmer ncbi-genome-download -y

It broke and I am unable to even access my conda

-bash-4.1$ which conda
/usr/local/devel/ANNOTATION/jespinoz/anaconda3/condabin/conda
-bash-4.1$ conda -h
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/bin/conda", line 12, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'

A few posts suggested it had something to do with my PYTHONPATH and PYTHONHOME environment variables but I don't have those set: 

-bash-4.1$ echo $PYTHONHOME

-bash-4.1$ echo $PYTHONPATH

I think it's because my default python version is now 3.8 which doesn't have conda installed. However, 3.7 does have conda installed. 

-bash-4.1$ ls -l /usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/  | grep "conda"
-bash-4.1$ ls -l /usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.7/site-packages/  | grep "conda"
drwxr-xr-x  2 jespinoz tigr     218 Dec  9 14:49 anaconda_client-1.7.2-py3.7.egg-info
drwxr-xr-x 10 jespinoz tigr     253 Jun 25 13:28 anaconda_navigator
drwxr-xr-x  2 jespinoz tigr     158 Jun 25 13:28 anaconda_navigator-1.9.7-py3.7.egg-info
drwxr-xr-x  6 jespinoz tigr     743 Dec  9 14:51 anaconda_project
drwxr-xr-x  2 jespinoz tigr     218 Dec  9 14:49 anaconda_project-0.8.2-py3.7.egg-info
drwxr-xr-x 11 jespinoz tigr     657 Dec  9 14:51 conda
drwxr-xr-x  2 jespinoz tigr     218 Dec  9 14:49 conda-4.6.11-py3.7.egg-info
drwxr-xr-x  7 jespinoz tigr    1012 Dec  9 14:51 conda_build
drwxr-xr-x  2 jespinoz tigr     218 Dec  9 14:49 conda_build-3.17.8-py3.7.egg-info
drwxr-xr-x  6 jespinoz tigr     295 Dec  9 14:51 conda_env
drwxr-xr-x  2 jespinoz tigr       0 Dec  9 14:59 conda_package_handling
drwxr-xr-x  3 jespinoz tigr     281 Dec  9 14:53 conda_verify
drwxr-xr-x  2 jespinoz tigr     188 Dec  9 14:49 conda_verify-3.1.1-py3.7.egg-info

Can I use this to my advantage to fix the broken setup?

I almost got it to work:

First symlink the conda binaries from the python3.7 site-packages to 3.8

-bash-4.1$ ln -s /usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.7/site-packages/cond* /usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/

When you try to restore python3.7 it doesn't work:

-bash-4.1$ conda install python=3.7
Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/common/serialize.py", line 19, in get_yaml
    import ruamel_yaml as yaml
ModuleNotFoundError: No module named 'ruamel_yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/common/serialize.py", line 22, in get_yaml
    import ruamel.yaml as yaml
ModuleNotFoundError: No module named 'ruamel'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/exceptions.py", line 1003, in __call__
    return func(*args, **kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/cli/main.py", line 74, in _main
    from ..base.context import context
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/base/context.py", line 28, in <module>
    from ..common.configuration import (Configuration, ConfigurationLoadError, MapParameter,
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/common/configuration.py", line 40, in <module>
    from .serialize import yaml_load
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/common/serialize.py", line 30, in <module>
    yaml = get_yaml()
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/_vendor/auxlib/decorators.py", line 59, in _memoized_func
    result = func(*args, **kwargs)
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/lib/python3.8/site-packages/conda/common/serialize.py", line 24, in get_yaml
    raise ImportError("No yaml library available.\n"
ImportError: No yaml library available.
To proceed, conda install ruamel_yaml

Of course, you can't really do conda install ruamel_yaml because that would be too simple. Instead use pip:

-bash-4.1$ pip install ruamel_yaml
Collecting ruamel_yaml
  Downloading https://files.pythonhosted.org/packages/fa/90/ecff85a2e9c497e2fa7142496e10233556b5137db5bd46f3f3b006935ca8/ruamel.yaml-0.16.5-py2.py3-none-any.whl (123kB)
     |████████████████████████████████| 133kB 8.8MB/s
ERROR: conda 4.6.11 requires pycosat>=0.6.3, which is not installed.
ERROR: conda 4.6.11 requires requests>=2.12.4, which is not installed.
ERROR: conda-build 3.17.8 requires beautifulsoup4, which is not installed.
ERROR: conda-build 3.17.8 requires chardet, which is not installed.
ERROR: conda-build 3.17.8 requires filelock, which is not installed.
ERROR: conda-build 3.17.8 requires libarchive-c, which is not installed.
ERROR: conda-build 3.17.8 requires lief, which is not installed.
ERROR: conda-build 3.17.8 requires pkginfo, which is not installed.
ERROR: conda-build 3.17.8 requires psutil, which is not installed.
ERROR: conda-build 3.17.8 requires pyyaml, which is not installed.
ERROR: conda-build 3.17.8 requires requests, which is not installed.
ERROR: conda-build 3.17.8 requires tqdm, which is not installed.
Installing collected packages: ruamel-yaml

Then I tried the following:

-bash-4.1$ conda install python=3.7 -n base
WARNING: The conda.compat module is deprecated and will be removed in a future release.
Collecting package metadata: done
Solving environment: - WARNING conda.common.logic:get_sat_solver_cls(278): Could not run SAT solver through interface 'pycosat'.
failed

CondaDependencyError: Cannot run solver. No functioning SAT implementations available.
O.rka
  • 29,847
  • 68
  • 194
  • 309
  • Yeah, conda is fun. Anyway, if your default Python executable is Python3.8, what about trying to add the Python3.7 to the path, so it's the one being executed? Something like `export PATH=/usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/py37/bin/python:$PATH`, or wherever the python37 binary is. – sammy Dec 09 '19 at 21:06
  • I couldn't get that to work either. – O.rka Dec 09 '19 at 21:11
  • 1
    Sounds like you have it fixed now, but I would have recommend following [these recovery instructions](https://stackoverflow.com/a/54563455/570918), specifically again doing a revision 0 install on **base**. The part that broke it for you was `conda install python=3` where you should have specifically used `python=3.7`. – merv Dec 09 '19 at 23:29
  • @merv that makes a lot of sense. I wonder why it didn't install Python 3.8 correctly? I'll make sure to specify it correctly in the future. Also, from that link you posted, there's this `::` syntax in `conda create -n tf36 anaconda::tensorflow-gpu python=3.6` . What does that do? I've never seen that before. – O.rka Dec 10 '19 at 19:37
  • That's part of [the MatchSpec syntax](https://stackoverflow.com/a/57734390/570918). It's a way to specify you want the package from the `anaconda` channel. As for Python 3.8, it's still a mystery to me why sometimes Conda will properly upgrade the `conda` package when Python changes and sometimes it doesn't. – merv Dec 10 '19 at 21:22

1 Answers1

0

I fixed it. I tried downloading the newest installation and then running -u to upgrade (which took forever) but it didn't work.

What I ended up doing was using my coworkers installation of conda:

source /path/to/their/conda/activate /path/to/my/anaconda3

Then I set up my ~/.condarc so it knew where my pkgs and environments should be located (we have a limited space in our home directory so it crashed if I didn't do this)

(/usr/local/devel/ANNOTATION/jespinoz/anaconda3) -bash-4.1$ cat ~/.condarc
channels:
  - bioconda
  - conda-forge
  - defaults
  - etetoolkit
  - r
channel_priority: true
report_errors: true

envs_dirs:
  - /usr/local/devel/ANNOTATION/jespinoz/anaconda3/envs/

pkgs_dirs:
  - /usr/local/devel/ANNOTATION/jespinoz/anaconda3/pkgs/

After that, I did the following: 

conda install python=3.7 -y

Though, I got the following error/warning: 

Executing transaction: / Traceback (most recent call last):
  File "/usr/local/devel/ANNOTATION/jespinoz/anaconda3/bin/jupyter-nbextension", line 7, in <module>
    from notebook.nbextensions import main
ModuleNotFoundError: No module named 'notebook'

WARNING conda.core.link:run_script(1098): pre-unlink script failed for package conda-forge::widgetsnbextension-3.5.1-py38_0
consider notifying the package maintainer
| Enabling notebook extension jupyter-js-widgets/extension...
      - Validating: OK

It still worked finally.

Now I'm running conda update --all because of all the package conflicts. Hopefully it doesn't break my environment again...

O.rka
  • 29,847
  • 68
  • 194
  • 309
  • 1
    I wouldn't run update all (just `conda update conda`), but if you insist, then consider [pinning](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-pkgs.html#preventing-packages-from-updating-pinning) your Python minor version to avoid future breakage. – merv Dec 09 '19 at 23:39
  • Thank you. I didn't know about pinning but that is an awesome feature. It would have saved me a lot of hassle. Do you know if the environment export propagates the pinning? – O.rka Dec 10 '19 at 19:34
  • No, I wouldn't expect it to. – merv Dec 10 '19 at 21:31