How can I install pandas 1.1.0 on Mac under anaconda and get past conda hanging on "Solving environment"

Question

When I try:

% conda update pandas
Collecting package metadata (current_repodata.json): done
Solving environment: | 

Updating pandas is constricted by 

anaconda -> requires pandas==1.0.5=py38h959d312_0

If you are sure you want an update of your package either try `conda update --all` or install a specific version of the package you want using `conda install <pkg>=<version>`

done

# All requested packages already installed.

This answer on stack overflow said to try:

conda install -c conda-forge pandas==1.1.0

but that just hangs:

% conda install -c conda-forge pandas==1.1.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: |

As you can see below, I can install the old 1.0.5 version of pandas using

conda install -c anaconda pandas

so I guess my environment seems to be functioning ok...I just can't install v 1.1.0 of Pandas from conda-forge

% conda install -c anaconda pandas
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/anaconda3

  added / updated specs:
    - pandas


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2020.6.24  |                0         132 KB  anaconda
    certifi-2020.6.20          |           py38_0         159 KB  anaconda
    conda-4.8.5                |           py38_0         3.1 MB  anaconda
    openssl-1.1.1g             |       h1de35cc_0         3.4 MB  anaconda
    pandas-1.0.5               |   py38h959d312_0         9.8 MB  anaconda
    ------------------------------------------------------------
                                           Total:        16.6 MB

The following packages will be SUPERSEDED by a higher-priority channel:

  ca-certificates                                 pkgs/main --> anaconda
  certifi                                         pkgs/main --> anaconda
  conda                                           pkgs/main --> anaconda
  openssl                                         pkgs/main --> anaconda
  pandas                                          pkgs/main --> anaconda


Proceed ([y]/n)? y


Downloading and Extracting Packages
openssl-1.1.1g       | 3.4 MB    | ################################################################################################################################################################################################ | 100% 
pandas-1.0.5         | 9.8 MB    | ################################################################################################################################################################################################ | 100% 
certifi-2020.6.20    | 159 KB    | ################################################################################################################################################################################################ | 100% 
conda-4.8.5          | 3.1 MB    | ################################################################################################################################################################################################ | 100% 
ca-certificates-2020 | 132 KB    | ################################################################################################################################################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

I tried Anaconda Navigator but when I select "pandas" to be upgraded and click on "Apply" a window appears saying the following packages will be modified, but the window is empty. The Apply button in that window is disabled so I don't think it is doing anything:

I was able to upgrade to pandas 1.1.3 via pip:

% pip install pandas --upgrade
Collecting pandas
  Downloading pandas-1.1.3-cp38-cp38-macosx_10_9_x86_64.whl (10.1 MB)
     |████████████████████████████████| 10.1 MB 2.1 MB/s 
Requirement already satisfied, skipping upgrade: python-dateutil>=2.7.3 in /opt/anaconda3/lib/python3.8/site-packages (from pandas) (2.8.1)
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /opt/anaconda3/lib/python3.8/site-packages (from pandas) (2020.1)
Requirement already satisfied, skipping upgrade: numpy>=1.15.4 in /opt/anaconda3/lib/python3.8/site-packages (from pandas) (1.18.5)
Requirement already satisfied, skipping upgrade: six>=1.5 in /opt/anaconda3/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)
Installing collected packages: pandas
  Attempting uninstall: pandas
    Found existing installation: pandas 1.0.5
    Uninstalling pandas-1.0.5:
      Successfully uninstalled pandas-1.0.5
Successfully installed pandas-1.1.3

I guess this is ok, not sure if my anaconda environment will now have lost its integrity in some way. I guess my question still stands, regarding the way to upgrade via anaconda/conda, or perhaps there is no difference and it is fine to mix anaconda/conda and pip commands. I really don't know.

SergiyKolesnikov · Answer 1 · 2020-10-13T13:38:23.517

First, some general points regarding conda-forge and pip:

This is because chances are high that you will end up with an environment in an inconsistent state without an ability to install or remove packages at all. If this happens to the conda's base environment then the only way is often to reinstall Anaconda completely.

The best practice in general is to use conda virtual environments, especially if you want/have to use conda-forge and pip. Then, if your environment gets into an inconsistent state you can just delete it and start anew.

In you case, it could look something like this:

Create a new conda environment pandas_project with pandas 1.1.0 installed from the official conda channel: conda create -n pandas_project pandas==1.1.0
Switch to the newly created environment conda activate pandas_project
Install as many packages as you can using conda and the official channel.
Setup conda-forge (for this environement only (!), note the --env flag): conda config --env --add channels conda-forge and conda config --env --set channel_priority strict
Install from conda-forge packages that are not available in the official channel
Save the state of the environment with conda list --explicit > pandas_project_env.txt.
Use pip to install packages that are not available neither from the official nor from the conda-forge channel

If the environment ever ends up in an inconsistent state, remove it conda env remove --name pandas_project and recreate it. Packages from the official and the conda-forge channels can be reinstalled quickly conda install --file pandas_project_env.txt

score 2 · Accepted Answer · answered Oct 12 '20 at 13:22

Anaconda comes with a whole bunch of packages pre-installed. As such, they of course have interdependencies that sometimes also restrict that not the newest version of some package can be used. So in your case, you can see that when trying

conda update pandas

it gave you

Updating pandas is constricted by 

anaconda -> requires pandas==1.0.5=py38h959d312_0

Basically telling you that the pre-installed anaconda package bundle requires that pandas is at version 1.0.5 to function properly

When you do

conda install -c conda-forge pandas=1.1.0

then conda tries to disentangle all the requirements that led to 1.0.5 being installed previously and tries to find a way to get the version you required working. Since the list of packages that are pre-installed in your base enviroment is long (check conda list), this takes a long time (what you described as hanging) and will probably fail eventually.

I was able to upgrade to pandas 1.1.3 via pip:

This is because essentially, pip will "not care" about all the interdependencies of libraries that where pre-installed with anaconda

not sure if my anaconda environment will now have lost its integrity

As a summary:

conda is convinced that at the current state 1.0.5 is the version needed for all packages to work properly
with pip install you forcefully installed a different version

so yes, in principle you have now a state of inconsisten package dependencies caused by using pip install to upgrade a package that was previously managed by conda, which is something you should never do, see the anaconda website for details.

To avoid complications with pre-installed packages in your base env, you can instead create a new environment and freely install your required version in there:

conda create -n <env-Name> pandas=1.1
conda activate <env-Name>

This creates a virtual environment where only pandas is installed. Then you can conda install other packages as needed

How can I install pandas 1.1.0 on Mac under anaconda and get past conda hanging on "Solving environment"

2 Answers2