Can you help me in cloning my Anaconda base env?

Question

I already read this and checked the Official documentation, but none of them helped.

As first attempt I tried this:

conda create --name myclone --clone base

but I get many errors like the following:

ClobberError: This transaction has incompatible packages due to a shared path.
  packages: defaults/win-64::jupyterlab-3.4.4-py39haa95532_0, defaults/win-64::_ipyw_jlab_nb_ext_conf-0.1.0-py39haa95532_0
  path: 'scripts/jupyter-labhub-script.py'

Then, I tried the alternative route with

conda list --explicit > spec-file.txt
conda create --channel conda-forge --name myenv --file spec-file.txt
conda install --name myenv --file spec-file.txt

getting the following output:

   Collecting package metadata (current_repodata.json): done
    Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
    Collecting package metadata (repodata.json): done
    Solving environment: failed
    
    PackagesNotFoundError: The following packages are not available from current channels:
    
      -  / / c o n d a . a n a c o n d a . o r g / c o n d a - f o r g e / n o a r c h / s p h i n x - i n l i n e - t a b s - 2 0 2 2 . 1 . 2 b 1 1 - p y h d 8 e d 1 a b _ 0 . t a r . b z 2
      -  / / r e p o . a n a c o n d a . c o m / p k g s / m a i n / n o a r c h / s p h i n x c o n t r i b - h t m l h e l p - 2 . 0 . 0 - p y h d 3 e b 1 b 0 _ 0 . c o n d a
      -  / / r e p o . a n a c o n d a . c o m / p k g s / m a i n / n o a r c h / i t e m l o a d e r s - 1 . 0 . 4 - p y h d 3 e b 1 b 0 _ 1 . c o n d a
    ...
    
    Current channels:

  - https://conda.anaconda.org/conda-forge/win-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/win-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/win-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://repo.anaconda.com/pkgs/msys2/win-64
  - https://repo.anaconda.com/pkgs/msys2/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

As a novice, I was not aware of virtual environments, so I installed many packages in my base (many of them were only available on conda-forge). Now, I want to create a virtual environment based on my base env but I am failing and I would appreciate some help.

Running on Windows 10.

FYI I have never needed to clone the base environment. Every time I create a conda environment, I reinstall everything from scratch. — Tom McLean, Sep 10 '22 at 17:18
Yes, but that may be tedious if you have many packages. For this reason, I tried to export a list with `conda list --explicit > spec-file.txt` and tried to use that one. — Barzi2001, Sep 10 '22 at 17:26
Have you tried using mamba https://github.com/mamba-org/mamba to install your packages? Its a lot faster than conda — Tom McLean, Sep 10 '22 at 17:33
What's the point of cloning an environment that was put together rather randomly and for which you don't even know the dependencies? It would be better to set up specialized environments for each task and control the dependencies via yaml files. And of course you need meaningful test suites to verify the executability of the programs and their dependencies, but you certainly don't need faster packet managers. — Peter, Sep 10 '22 at 20:35
The point is that there is no environment that has not been put together randomly. Can you document your claim? I am aware only now that it was better to create a virtual environment immediately as you install Anaconda (that should be clearly stated imho during the installation process), but now I am in this situation and we don’t have any time machine. I am searching a solution given the current state. I know only now that “I should have done…”. — Barzi2001, Sep 11 '22 at 13:27

merv · Answer 1 · 2022-09-11T01:42:40.417

Heed the warnings

I would strongly encourage following the advice in the comments, i.e., don't clone your base, especially if you've been doing ad hoc installations on a whim. Moreover, if one needs packages from Conda Forge, it is almost always better to use only Conda Forge packages, since channel mixing can have issues. Take time to thoughtfully design the new environment.

A quick start to that can be to dump out a YAML representation of the environment and edit it to specify only the packages that matter.

conda env export --no-builds > env.yaml

Generally, you can remove most everything that you don't use directly (e.g., import numpy), and drop any versions that you don't care about. However, it is usually good practice to specify the python up through minor version (e.g., python=3.9). You shouldn't need any of the anaconda/conda-related packages in a non-base environment.

Problem: Package Clobbering

Nevertheless, we can still explain what is going wrong and suggest how to fix it. Specifically, the issue is package clobbering, which occurs when two packages want to install the same file(s), but do not declare that they should not be co-installed. This looks like an oversight by the Anaconda team, so I reported the issue. Please feel free to comment on that issue with any pertinent information, or even just to say you've encountered the issue.

Possible Resolution

It's difficult to give precise advice for a fix. Ultimately, you either want to remove the _ipyw_jlab_nb_ext_conf (since anaconda/win-64::jupyterlab=3.4.4 already provides it, or switch to a build of jupyterlab that does not have this.

Option 1: Switch to `conda-forge::jupyterlab`

The Conda Forge builds of jupyterlab do not include extension scripts like the anaconda version does. So, this might be the easiest solution. This may delete the clobbered script, so to be safe one may want to reinstall the _ipyw_jlab_nb_ext_conf package.

conda install -n base conda-forge::jupyterlab
conda install -n base --force-reinstall _ipyw_jlab_nb_ext_conf

Possible complication here is that the anaconda version includes additional extension scripts, not just the one that clobbers the scripts/jupyter-labhub-script.py file. Specifically,

jlpm-script.py
jupyter-lab-script.py
jupyter-labextension-script.py

So, you might lose functionality, but should be able to find packages to add back those extensions.

Option 2: Remove `_ipyw_jlab_nb_ext_conf`

If you want to keep the anaconda/win-64::jupyterlab, you could instead remove the _ipyw_jlab_nb_ext_conf package, since it is redundant. This operation may delete the clobbered file, so to be safe it should be followed by a forced reinstall of jupyterlab.

conda remove -n base _ipyw_jlab_nb_ext_conf
conda install -n base --force-reinstall anaconda::jupyterlab

Possible complication here is that other packages may have _ipyw_jlab_nb_ext_conf listed as a dependency, which would prevent removing it without also removing the depending package(s).

Option 3: Downgrade `jupyterlab`

The Anaconda team only added these additional extensions to the jupyterlab package in the last release. Downgrading would also eliminate the conflict.

conda install -n base anaconda::jupyterlab=3.3
conda install -n base --force-reinstall _ipyw_jlab_nb_ext_conf

Thanks for your effort in helping me finding a solution. By taking any of the three routes you suggested appear very time consuming and places too many uncertainties. Perhaps the easier, faster and more reliable is to uninstall/reinstall Anaconda, create a virtual environment and re-install of the packages one-by-one. — Barzi2001, Sep 11 '22 at 13:25

Can you help me in cloning my Anaconda base env?

1 Answers1

Heed the warnings

Problem: Package Clobbering

Possible Resolution

Option 1: Switch to conda-forge::jupyterlab

Option 2: Remove _ipyw_jlab_nb_ext_conf

Option 3: Downgrade jupyterlab

Option 1: Switch to `conda-forge::jupyterlab`

Option 2: Remove `_ipyw_jlab_nb_ext_conf`

Option 3: Downgrade `jupyterlab`