9

I have an environment.yml file, but don't want to use Conda:

name: foo
channels:
  - defaults
dependencies:
  - matplotlib=2.2.2

Is it possible to have pip install the dependencies inside an environment.yml file as if it's a requirements.txt file?

I tried pip install -r environment.yml and it doesn't work with pip==22.1.2.

Intrastellar Explorer
  • 3,005
  • 9
  • 52
  • 119
  • 2
    No, but you can always parse the yaml yourself using Python. I'm not familiar with Conda, but it looks like it might be as simple as extracting the `dependencies` list and then either invoking pip directly using subprocess` or outputting each entry as one line in a `requirements.txt` file – Brian61354270 Jul 01 '22 at 04:52

4 Answers4

12

Based on Beni implementation, I just wanted to adjust the code since it has lots of errors;

import os
import yaml

with open("environment.yaml") as file_handle:
    environment_data = yaml.safe_load(file_handle)

for dependency in environment_data["dependencies"]:
    if isinstance(dependency, dict):
      for lib in dependency['pip']:
        os.system(f"pip install {lib}")
Ahmed
  • 796
  • 1
  • 5
  • 16
  • 3
    This answer solves the problem and should be marked as the solution! – Joel Sep 20 '22 at 14:20
  • This answer only solves the very limited case where there are `pip: ` dependencies declared the YAML. Any dependencies listed as Conda packages are completely ignored. – merv Jan 25 '23 at 21:16
  • First, maybe because the question is asking about how to install packages within the `.yaml` file using `pip`. I don't think this is limiting installation. In many cases, you can't use Conda or installing it is time expensive – we have Colab, for example, or a remote server where you want to do a specific task. – Ahmed Jan 25 '23 at 22:26
  • It is limited and take the OP YAML as an example: it lacks a `pip:` section, so this code would do nothing given it. On Colab, `condacolab` works fine directly - no need to convert. For other situations, Micromamba is standalone (no install) and fast. – merv Jan 26 '23 at 03:01
  • I have not known that before. I will check this out! – Ahmed Jan 26 '23 at 12:42
6

No, pip does not support this format. The format it expects for a requirements file is documented here. You'll have to convert the environment.yml file to a requirements.txt format either manually or via a script that automates this process. However, keep in mind that not all packages on Conda will be available on PyPI.

bgfvdu3w
  • 1,548
  • 1
  • 12
  • 17
6

I've implemented what Brian suggests in his comment.

This converts the environment.yaml to requirements.txt:

import yaml

with open("environment.yaml") as file_handle:
    environment_data = yaml.load(file_handle)

with open("requirements.txt", "w") as file_handle:
    for dependency in environment_data["dependencies"]:
        package_name, package_version = dependency.split("=")
        file_handle.write("{} == {}".format(package_name, package_version))

And this installs the dependencies directly with pip:

import os
import yaml

with open("environment.yaml") as file_handle:
    environment_data = yaml.load(file_handle)

for dependency in environment_data["dependencies"]:
    package_name, package_version = dependency.split("=")
    os.system("pip install {}=={}".format(package_name, package_version))

NOTE: I've omitted error handling and any other variations of package definitions (e.g., specification of a package version greater than or equal to a certain version) to keep it simple.

Beni Trainor
  • 346
  • 1
  • 11
5

The first answer makes important points: there is not direct conversion because Conda is a general package manager and so includes additional packages. Furthermore, Conda packages can often go by different names. None of the proposed parsing solutions cover this situation.

Personally, I think the most efficacious complete approach is to recreate the environment with Mamba, then use pip in the environment to dump out a legitimate requirements.txt.

# use mamba, not conda
mamba env create -n foo -f environment.yaml
mamba install -yn foo pip
mamba run -n foo pip list --format freeze > requirements.txt
mamba env remove -n foo

That is, don't overthink it and use the reliable tools at hand.

merv
  • 67,214
  • 13
  • 180
  • 245