Updating a specific module with Conda removes numerous packages

Question

I have recently started using the Anaconda Python distribution as it offers a lot of Data Analysis libraries out of the box. And using conda to create environments and install packages is also a breeze. But I have faced some serious issues when I want to update Python itself or any other module, I am informed beforehand that a LOT of my existing libraries will be removed.

For example, this is what I get when I use conda update [package_name]

$ conda update pandas
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: C:\Users\User\Anaconda3

  added / updated specs:
    - matplotlib

The following packages will be REMOVED:
[Almost half of my existing packages]

The following packages will be UPDATED:
[Some packages including my desired library, in this case, pandas]

I have searched the web on how to update packages and Python using conda and almost everywhere I saw that conda update [package name] was suggested. But why doesn't it work for me? I mean it will work but at the expense of tons of important libraries that I need.

So I have tried using the Anaconda Navigator to update the desired libraries (like matplotlib and pandas) hoping that the removal of existing libraries might be a command line issue on my computer. But I had seriously messed up my base (root) environment by updating pandas using Navigator. I didn't get any warnings that a lot of my modules will be removed so I thought I was doing fine. But after the update was done and I wrote some matplotlib code, I wasn't able to run it. I got errors that resembled something that indicated matplotlib was a "non-conda module". So I had to do conda install --revision n to go back to a state where I had my modules.

Right now, the only way for me to update any package or Python is to do this:

conda install pandas=[package_version_that_is_higher_than_mine]

But there's got to be a reason why I am facing this issue. Any help is absolutely appreciated.

EDIT: It turns out that the issue is mainly when I am trying to update using the base environment. When I use my other conda environments, the conda update [package_name] or conda update --all works fine.

Answer 1

Anaconda (as distinct from Conda) is designed to be used as a fixed set of package builds that have been vetted for compatibility (see "What's in a Name? Clarifying the Anaconda Metapackage). When you try to introduce new packages or package upgrades into that context, Conda can be rather unpredictable as to how it will solve that. I think it helps to keep in mind that commands like conda (install|upgrade|remove) mean requesting a distinct environment as a whole, and do not represent low-level commands to change a single package.

Conda does offer some options to get this more low-level behavior. One thing to try is the --freeze-installed flag, which would do what you're asking for. Recent versions of Conda do this by default in the first round of solves, and if it doesn't work then it attempts a full solve. There is also the more dangerous and brute force --no-dep flag, which won't do a solve at all and just install the package. The documentation for this literally says,

"This WILL lead to broken environments and inconsistent behavior. Use at your own risk."

Typically, if you want to use newer packages, it is better to create a new env (conda create -n my_env [pkg1 pkg2 ...]) because the fact is that you no longer want the Anaconda distribution, but instead a custom one with newer versions. My personal view is that most non-beginners should be using Miniconda and relegate their base env to only having conda, while being very liberal about creating envs for projects that have different package requirements. If you ever need a true Anaconda distribution, there's always the anaconda package for that.