138

I am using the conda package manager - a lot. By now I have quite a few environments and a lot of downloaded packages taking a lot of space on my SSD. An obvious path to free some of that space is to use the command

conda env export > environment.yml

from https://conda.io/docs/user-guide/tasks/manage-environments.html#exporting-the-environment-file to export which packages my old, inactive projects use(d) and then delete these environments. As far as I understand, this should free some of the space in anaconda2/envs/, but not in anaconda2/pkgs/. How do I get rid of these packages? Also, I suspect that there might be quite a few packages still sitting around, to which no environment is linking to - could that happen?

Questions:

  1. In general: What is the best way to reduce the space taken up by conda?
  2. How do I get rid of packages that no environment is using anymore? How do I prune my packages? I am searching for something like sudo apt-get autoremove from Ubuntu/Debian.
Mike Müller
  • 82,630
  • 20
  • 166
  • 161
Make42
  • 12,236
  • 24
  • 79
  • 155

2 Answers2

271

You can free some space with:

conda clean --all

clean Remove unused packages and caches.

Conda already use symlinks when possible for packages. So, not much to improve here, I guess.

Ok, thanks, but I would like to know "not for a specific environment, but in general" - for all environments.

You can list all packages in all envs with a few lines of Python:

import os
import subprocess
for env in os.listdir('/Users/me/miniconda3/envs'):
    subprocess.call(['conda', 'list', '-n', env])
Mike Müller
  • 82,630
  • 20
  • 166
  • 161
  • 16
    I tried it out with `conda clean --dry-run --all`. More than 3GB would be deleted - not too bad. However, it lists packages I recently installed and actively use. That confuses me: What does it mean when `.tar.bz2` are about to be deleted? Are those only the downloads and not the installs? Or maybe it is something old and I have a never version installed (and I forgot)? – Make42 Feb 10 '18 at 12:11
  • 7
    `.tar.bz2` is just the download that is cached for a potential second install. They are called tarballs. The installs stay as long as there are in use. – Mike Müller Feb 10 '18 at 12:21
  • How can I check which version I have installed - not for a specific environment, but in general. Or is there no such thing, because everything is always installed within an environment? (I think the environments are only linking to the installations, right?) – Make42 Feb 11 '18 at 18:20
  • `conda list -n my_env` lists all packages and shows their versions for this environment. – Mike Müller Feb 11 '18 at 19:37
  • 1
    Ok, thanks, but I would like to know "not for a specific environment, but in general" - for all environments. – Make42 Feb 12 '18 at 13:47
  • 2
    Add a solution to my answer. BTW, you can accept an answer if it solves your problem. ;) – Mike Müller Feb 17 '18 at 10:02
  • Thanks for your help so far, it has been a really good start, but my problem is not yet completely solved, yet. Your python code is a good idea, but the output is really long and difficult to read. I would require something more neatly arranged. I guess I have to program it myself. – Make42 Feb 19 '18 at 10:09
  • I also have the issue how to find out which environments are not in use by any PyCharm project. – Make42 Feb 19 '18 at 10:11
  • 1
    Was some change done to the command `conda clean --all`? These days I tried using it again and it reliably cleaned up all the packages that were not used by any packages. – Make42 Nov 12 '20 at 21:56
  • Just used this command to gain 7GB. Thanks. – Kumar May 02 '21 at 11:55
  • In my case it free much more when ran both commands: `conda clean -p` & `conda clean --all` – Mohith7548 Jun 29 '21 at 08:04
16

Finally I got around dealing with this issue. In the end it was a couple of days work:

  1. For all my Python projects I use PyCharm and with it I checked which project uses which environment. For all environments I used the conda env export > environment.yml to save the settings of the environment from https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#exporting-the-environment-file
  2. Check whether my projects still work with new environments created from the environment.yml.
  3. Use anaconda-clean from option B in https://docs.anaconda.com/anaconda/install/uninstall and put the created backup in a save place.
  4. Rename the old anaconda2 directory to anaconda2_backup.
  5. Install a new conda environment - miniconda3 in my case.
  6. Build new environments which are need for current projects from the environment.ymls and check whether these work.
  7. Delete the old anaconda backups.

Finally I also reduced my logical volume with https://blog.shadypixel.com/how-to-shrink-an-lvm-volume-safely/ but this is only for Linux users using LVMs.

This way I was able to free 20 to 30 GB of space.

Make42
  • 12,236
  • 24
  • 79
  • 155
  • I don't quite understand the point here. The `conda clean --all` suggested above does not work with individual environments, it works with all environments. Can you explain what you achieve here that is not achieved by `conda clean --all`. Not that it takes 30s instead of a few days! It does not automatically remove envs you don't need anymore but reading your question it was about removing packages not used by any env not about envs you are not using anymore. – A Roebel Apr 01 '21 at 10:54
  • 1
    @ARoebel: https://stackoverflow.com/questions/48706548/how-to-free-disk-space-taken-up-by-anaconda/54726607?noredirect=1#comment114591652_48706601 hints that at the time the `clean --all` did not work for me: Too little was cleaned then, while for current versions it cleans much more. That is why back then, I accepted my own answer as the solution and later changed that to Mike Müller's answer. Also, I had the issue that I had a lot of environments of dormant projects which configs I had to back up and remove. (Mybe that was the real problem, but I cannot reproduce this.) – Make42 Apr 01 '21 at 11:18