2

What are good strategies to take a requirements.txt file or take the result of pip freeze and convert it to a conda environment file? For example, the ideal end result would be to take the requirements.txt with the pinned pip versions and have conda compatible versions under dependencies rather than under pip.

name: myconverted_condaenv
channels:
  - conda-forge
dependencies:
   <all packages installed via pip>

Also, there are some packages that may not be available in conda that were installed via pip. The endgoal of this question is to understand how to help someone quickly convert their setup to use conda without a lot of manual adjustments.

Howard_Roark
  • 4,088
  • 1
  • 14
  • 24
  • Is this for building a package with conda? Or just recording the contents of an environment? Also, does it matter if the dependencies are installed via pip or conda? – Matt Thompson Jan 16 '21 at 20:38
  • Effectively it’s for recording the contents of an environment. For example, a person who has used pip to install packages for a year might have 250 packages on their desktop, and now if they want to move those to a conda environment I’m trying to make that easy without a lot of manual work. Ideally the dependencies would be under conda – Howard_Roark Jan 16 '21 at 21:23
  • Try `pip freeze > requirements.txt && conda install --file requirements.txt -c conda-forge` – Matt Thompson Jan 16 '21 at 22:14
  • The downside to that is the environment file wouldn't have a nice self documenting list of the dependencies right? or are you saying after running that command to generate an environment file and it will have them? (I am new-ish to conda) – Howard_Roark Jan 16 '21 at 22:55
  • I'm not really sure why you'd need to track dependencies; as a user I'm happy to tell a package manger what I want installed and let it go figure out what dependencies need to be installed. And here, most of your deps are either coming from PyPI or the `conda-forge channel` so I'm not sure how important it is to track those. We'd need more specifics on your use case to add any more detail - I'm generally unsure of the motivation to do this export at all, given that you can safely `pip install` whatever you want into a conda environment just like in virtual environments (or not in one at all). – Matt Thompson Jan 16 '21 at 23:42
  • Ya I see what you mean. I was tasked with moving people’s python environments from one Desktop to another with a preference of switching any users not currently using conda to use conda on their new Desktops. That’s what motivated the question. Ie- I can see their packages via pip freeze and was wondering the best way to create a conda environment yml file that contains those packages and put it in their git repos. For example, I like your idea and will probably use it but then I was wondering if I could then generate a conda file from it and put it in source control – Howard_Roark Jan 17 '21 at 00:13
  • 1
    If they're working in pure Python, and they don't really care about using Conda as the package manager for these old environments, then one could also simply throw the whole `requirements.txt` in a YAML file, [as suggested here](https://stackoverflow.com/a/59056234/570918). This would still let them switch between environments (use Conda as the virtual environment manager), and they could start using Conda for package management in new environments, just that they should stick to Pip in the transitioned one(s). – merv Jan 18 '21 at 03:02
  • I suppose I should explicitly note that, if you do follow that other answer, I do not recommend including the `anaconda` package - that was specific to the OP in that other answer. Simply a `python` (with version) and `pip`, then the requirements file. – merv Jan 19 '21 at 21:08

1 Answers1

0

So, it turns out there were already good answers to help me accomplish what I needed. This answer shows how to get the packages without any locked versions.

pip freeze | sed s/=.*// > packages.txt

This answer shows a great loop that will install everything via conda when it can and use pip otherwise.

FOR /F "delims=~" %f in (requirements.txt) DO conda install --yes "%f" || pip install "%f"

Then to generate the actual conda file that can be kept in source control so that new environments can be created from it, you can run

conda env export
Howard_Roark
  • 4,088
  • 1
  • 14
  • 24
  • 1
    Certainly a useful first approximation, but be aware that because Conda is a generic package manager, there can be naming collisions and the Python packages don't always win out. For example, I've seen other SO questions where users were confused why the Conda `wget` and `nodejs` packages weren't installing the Python versions of those packages (i.e., they install the native applications instead). Also, that loop is rather brutal: it runs a separate solve for every single package in the file. – merv Jan 18 '21 at 02:58
  • Oh interesting-- I did notice that the loop took a very long time to complete due to that. – Howard_Roark Jan 18 '21 at 03:09