5

I am in the process of migrating a number of environments from PC to linux.

On windows I run:

C:\foo> conda env export > environment.yml

And, later, on linux, I run:

$ conda env create -f environment.yml

But I get errors like:

ResolvePackageNotFound: 
  - icc_rt=2017.0.4
  - vc=14
  - vs2015_runtime=14.0.25123
  - wincertstore=0.2
  - qt==5.9.5=vc14he4a7d60_0

This must (I assume) be a solved problem.

May someone smarter than I on this topic please let me know how they would go about this?

Thanks!

jason m
  • 6,519
  • 20
  • 69
  • 122
  • Does this answer your question? [conda environment from windows to linux](https://stackoverflow.com/questions/51708668/conda-environment-from-windows-to-linux) – muxevola Feb 28 '21 at 20:08

2 Answers2

9

When exporting your environment, use the option --from-history. It will export just the libs you explicitly installed, and not the dependencies:

conda env export --from-history > environment.yml

No platform specific info will be exported. It will prevent a lot of headaches.

Usually some dependencies are platform specific. Also the default conda env export put platform specific info in the libs. This will make an environment.yml file created in Windows fail when trying to recreate it in Linux, and vice-versa.

Conda and Pip aren't really very good to perfectly recreate an environment in another machine, since they don't record all dependencies of dependencies. Usually you won't have trouble, but if you have, it will be hard to spot that a dependency of a dependency is a slightly different version.

Extra tip: always install a lib referencing its version number (e.g.: conda install pandas=1.2.1). Without the version, the command above will export the dependencies without a version, ruining the reproducibility of your environment.

But what if you created your env from a environment.yml file? Now --from-history will export the platform specific dependencies. Then grep is your friend. You will need to grep all of your import statements, see which of them are defined in your environment.yml file and just use them using the same version without platform specific info. Better to start doing it correctly using the --from-history or manually editing the file.

Extra extra tip: Python and Conda default tooling still aren't very good to create reproducible installs between platforms. Nowadays I've been successfully using Poetry for it.

neves
  • 33,186
  • 27
  • 159
  • 192
  • My venv was already created. and I wanted to only install the packages. So I used this approach to get the packages I installed on my local machine, manually edited the file to keep only the package names. and then used conda install --file envi.yml to install the packages – Sid Apr 19 '21 at 09:59
  • but this doesn't include packages installed via pip :/ – Kaoutar Dec 07 '22 at 21:09
3

Yes and no. Using conda export will enable someone else to exactly replicate your environment. This implicitly assumes you are on the same platform.

Unfortunately, when swapping platforms, you need to handle packages that are platform dependent. The easiest way is just remove them. Keep in mind that if you include a high level package with lots of dependencies, all of those dependencies are looked up/handled by conda.

For example, if you want to include pandas, you don't need to include numpy, qt, matplotlib, and dateutils in your environment spec. Just listing pandas is enough, conda takes care of the rest.

In this way, you may be better off just listing out the bare minimum of your environment requirements by hand in a text editor.

Alternatively, you can use conda export, but you may still need to remove a good number of the build numbers (i.e. =vc17gnad8qt6h) and packages that are Windows only (like wincertstore).

James
  • 32,991
  • 4
  • 47
  • 70
  • Interesting. That's crappy, but I assumed was the answer. – jason m Apr 29 '19 at 01:03
  • @jasonm How does `conda` know whether package `x` in version `y` is in your env because you specifically wanted that version, or because it's the version that was current when you asked for package `x`, or because it's a dependency of other package `z`? Only by you recording which packages (and versions if applicable) you actually asked it for, in other words *'listing out the bare minimum of your environment requirements by hand in a text editor'*. You could ask for a feature where each env records the package specs that were listed in its `conda create` and `conda install` commands, but… – nekomatic Apr 30 '19 at 10:30
  • …that still leaves questions, e.g. if you installed something then later `conda update`d it, is your bare minimum requirement still the original package, or did you `update` because you now *needed* the later version? – nekomatic Apr 30 '19 at 10:33