conda environment from windows to linux

Question

I have developed a project on windows using pycharm and I want to deploy in on an ubuntu server.

I am trying to create a requirements.txt using these commands:

conda list -e > requirements.txt
conda list > requirements.txt

Depending on the options the requirements.txt looks like any of these:

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
@EXPLICIT
https://repo.anaconda.com/pkgs/main/win-64/blas-1.0-mkl.tar.bz2
https://repo.anaconda.com/pkgs/main/win-64/ca-certificates-2018.03.07-0.tar.bz2
https://repo.anaconda.com/pkgs/main/win-64/icc_rt-2017.0.4-h97af966_0.tar.bz2
https://repo.anaconda.com/pkgs/main/win-64/intel-openmp-2018.0.3-0.tar.bz2

Or this

# packages in environment at C:\ProgramData\Anaconda2\envs\myenvs:
#
# Name                    Version                   Build  Channel
anyjson                   0.3.3            py36h1110a06_1  
arrow                     0.12.1                   py36_1  
asn1crypto                0.24.0                   py36_0  
babel                     2.6.0                    py36_0

Or this

name: myenv
channels:
  - defaults
dependencies:
  - anyjson=0.3.3=py36h1110a06_1
  - arrow=0.12.1=py36_1
  - asn1crypto=0.24.0=py36_0
  - babel=2.6.0=py36_0
  - blas=1.0=mkl

No matter how I try to do this I get errors on the ubuntu machine, in some cases because the package is for windows: (/win-64/)

https://repo.anaconda.com/pkgs/main/win-64/ca-certificates-2018.03.07-0.tar.bz2

I have read a lot of documentation but I seem not to be able to get what I want. Conda (Python) Virtual Environment is not Portable from Windows to Linux

Any solution?

It could work using `conda env export` output and removing packages only available to `win-64`. Some may require replacement by a linux equivalent. I think using an install of conda under WSL could solve some of your problems: doing so you can develop under windows with `linux-64` packages directly. Note that I was not able to fully test this configuration (yet). — FabienP, Nov 09 '18 at 13:23
Just maybe: If you do not need to pin all dependency versions, something like `pipreqs` or similar tools might only export the top dependencies that are generelly more plattform invariant, and plattform specific ones will be requested on demand. BUT this has many chances to fail, as some packages are not directly used in the source code and have to be manually installed. — E. Körner, Mar 31 '20 at 13:30

score 14 · Answer 1 · answered Feb 21 '19 at 07:26

14

By default, conda will export your environment with builds, but builds can be platform-specific.

A solution that worked for me is to use the --no-build flag:

$ conda env export --no-build > environment.yml

Hope this helps.

answered Feb 21 '19 at 07:26

LoicViennois

151
1
4

7

Unfortunately, this doesn't work... still some windows specific packages are left behind. – Chris Illusion Apr 30 '19 at 17:28
3

I'm trying to do the same thing and getting the same issues - got a dozen packages that are windows-specific.. – chemicalcrux May 01 '19 at 13:07
1

I'm trying to wrap my head around why this works for me and also why this isn't the default behavior of `export`. I mean who wants to export an env that by default is tightly coupled to the machine it was developed on? – Adam Hughes Feb 17 '22 at 17:50

score 5 · Answer 2 · answered Mar 15 '21 at 19:55

5

I was handling with the same problem, I found this on Anaconda documentation:

Conda does not check architecture or dependencies when installing from a spec file. To ensure that the packages work correctly, make sure that the file was created from a working environment, and use it on the same architecture, operating system, and platform, such as linux-64 or osx-64

Managing enviroments

answered Mar 15 '21 at 19:55

Franco Vega Maza

51
1
2

2

AKA - conda is basically worthless in this regard. – Adam Hughes Feb 17 '22 at 17:38

Adam Hughes · Answer 3 · 2022-02-17T19:33:43.790

Alternative

As an alternative, you export only the depenencies you added and not any transitive deps using conda env export --from-history which results in something like

name: email_scraper
channels:
  - https://conda.anaconda.org/conda-forge/
  - defaults
dependencies:
  - python=3.9
  - img2pdf
  - python-dotenv
  - google-cloud-storage
  - psycopg2
  - pypdf2
  - pytest
  - google-cloud-logging

Although this loses versions so you may need to manually add them by cross-referencing the other environmnet.yaml file (tedious).

Platform-related issues often occur in the underlying transitive dependencies. You are basically making a tradeoff between specificity and tranferability of env. The default export is a highly specific environment that runs on a single OS platform. The latter is more generic but may internally be different on different OS.

If you are here in 2022, you may want to have a look at Poetry as the package manager instead.

conda environment from windows to linux

3 Answers3

Alternative

Linked