How to create a Relocatable Conda Environment? Is it doable?

Question

I would like to make a relocatable environment. So I need to use relative paths in the package installations. For this I just create a Conda Environment like this:

conda create --prefix env python=3.6.5
activate .\env

And then I have installed the needed packages as usual with

pip install package_name

The problem comes when I want to install my own package. I have created a structure like this and I have followed this tutorial:

some_root_dir/
|-- setup.py
|-- python_files
|-- |-- runall.py
|-- |-- test0.py

And the content of the setup.py is this:

from setuptools import setup

setup(
    name='my_app',
    version='0.1',
    description='My app',
    keywords="app csv some other word",
    url='https://www.my_domain.com/',
    author='My name',
    author_email='email@email_domain.com',
    license='MIT',
    packages=['my_package'],
    zip_safe=False,
)

But after the installation with:

cd some_root_dir
pip install .

and moving it to another location, the paths that are appearing in the application are the ones where I did the pip install .

I have been looking for information here, but I did not find anything useful.

Main Steps I want to do

Create a conda environment and install some packages with pip or conda, my own python package included
Copy the environment folder to another computer
Run the application in this computer where conda and python are not installed. If I use the python.exe included in the folder python should know where the packages are installed and how to import them.

Questions

How can I use relative paths in the environment packages?
Is this doable? Or am I doing anything wrong?
Which are the best practices to achieve what I want?
Are the relocatable environments possible?

Note: I am using Windows 10 and Miniconda 3.

Virtualenv

The equivalent on virtualenv would be this:

virtualenv --relocatable env_folder

But it is an experimental feature

Update (August 7, 2018)

Actually what I want is what @interfect says in his comment, the issue is here. So relocatable environments on conda are not possible yet

1) Why do you install it two times? 2) "And move it to another location, the paths that are appearing in the application are" -- please give a sample output for both statements, it's not clear what you move where and what paths "appear" and where. — ivan_pozdeev, Aug 06 '18 at 19:50
So you want a Conda environment that can be set up in one place, and then still work when you move the whole environment to another place? It looks like you may want this uninplemented conda feature: https://github.com/conda/conda/issues/3097 — interfect, Aug 06 '18 at 20:02
@ivan_pozdeev @interfect Not exactly. What I want is to create an environment in my computer, let´say the path `C:\path\to\my\environment`, copy the folder to another computer with Windows and make it run in this other windows in a different path `C:\Users\MyUser\AppData\Roaming\MyApp`. This other windows shouldn´t have python or conda installed. So the environment should be runnable itself — ChesuCR, Aug 06 '18 at 23:21
Actually a standalone installer is the perfect solution, but I do not know if it is possible because The complete app is created with Nodejs, JavaScript and python. Is there a way to create a `exe` file to install all these contents? — ChesuCR, Aug 06 '18 at 23:25
@mike Yes, actually my custom package works well now with `virtualenv` and the installation with `pip install .` of my package (without links). I had to modify my setup arguments to make it work as well. My problem now is with some JavaScript resources of third-party packages. I need to clone the original source code, modify a couple of things and compile it. I think the only way to fix this is to modify their resources paths to convert them in relative paths manually. But that's another matter. — ChesuCR, Aug 08 '18 at 09:30
I have answered my own question. Read it and check if you want to add or correct anything. Thank you all. — ChesuCR, Aug 10 '18 at 11:26

ChesuCR · Accepted Answer · 2018-11-29T18:08:09.360

I think that relocatable environments depend on the installed packages. They should be implemented with relative paths and avoiding hardcoded paths. All the paths that are used in the source code of the package should be inside the own package. So if you install well done package you won´t have any problem to relocate the environment in other folder or computer.

As you will need to add all the folders inside the package you will need to modify the arguments of the setup. Add these two parameters in order to add folders to the final package. If you don´t do this the folders won´t copied to the site-packages folder within the environment (the final destination when you install the package with pip):

packages=[
    'main_folder',
    'main_folder.folder_with_python_files',
    'main_folder.other_folder_with_python_files',
],
package_data={
    'main_folder': [
        'static/css/*.*',
        'templates/*.*',
    ],
},

Environments, Package Manager and Paths

I have tried to build the environment on Windows with Virtualenv, but I had some problems building a basic environment:

There was a dll library missing: VCRUNTIME140.dll
The runpy module was missing as well. This is used to run commands with the -m parameter: python -m ...
Other packages dependencies were not installed when I used pip such as zipfile

So I came back to Conda Environments again, but I have built the environment with package manager pip, instead of conda, because the packages were much lighter in my case.

Therefore, my recommendation is to install the packages with pip. If any of them are giving problems after the relocation, we should check if there is any harcoded path and change it directly. Though the best solution would be to modify the original source code and install the customised package.

Some python scripts in the environment had the absolute path on the header with #!.

#!C:\absolute\path\to\python.exe

I just removed them because if I call any script with the python.exe that is currently inside the environment those headers are ignored

Update

Also conda-pack can be useful. I haven´t tried it yet

conda-pack is a command line tool for creating relocatable conda environments. This is useful for deploying code in a consistent environment, potentially in a location where python/conda isn’t already installed.

conda-pack works if the source env path and the destination env path are same. — Neeraj Gulia, Jun 20 '19 at 09:24
That's not true. The whole point of conda-pack is to make it relocatable, but you have to activate the environment and run `conda-unpack` to rewrite the prefixes. — Bradley Kreider, Jun 23 '20 at 00:49

score -1 · Answer 2 · answered May 07 '20 at 15:22

-1

If you turn your package into a conda-package (trivial if you are using pip already), you just conda install your packages on the new machine and everything will be relocated at install time.

That includes any compiled libraries let alone the paths in scripts. Conda will modify everything so it just works no matter where you install it.

answered May 07 '20 at 15:22

Bradley Kreider

1,115
10
16

1

I know. But that is not to relocate an environment, you are installing it from scratch. Also, if you check point 3 I have written in my question: "Run the application in this computer where conda and python are not installed". I need a final user without almost computer knowledge should know how to install it. Creating an `exe` installer for Windows, for instance, would be one solution. I am also working with Electron, so including a relocatable environment in the installer (with `electron-builder`) is the best solution I found. – ChesuCR May 07 '20 at 16:01