34

I have a virtualenv located at /home/user/virtualenvs/Environment. Now I need this environment at another PC. So I installed virtualenv-clone and used it to clone /Environment. Then I copied it to the other PC via USB. I can activate it with source activate, but when I try to start the python interpreter with sudo ./Environment/bin/python I get

./bin/python: 1: ./bin/python: Syntax Error: "(" unexpected

Executing it without sudo gives me an error telling me that there is an error in the binaries format. But how can this be? I just copied it. Or is there a better way to do this? I can not just use pip freeze because there are some packages in /Environment/lib/python2.7/site-packages/ which I wrote myself and I need to copy them, too. As I understand it pip freeze just creates a list of packages which pip then downloads and installs.

vicco
  • 1,049
  • 2
  • 14
  • 33
  • How did those packages get into site-packages in the first place? You must have installed them, and they must exist in version control somewhere, surely. – Daniel Roseman Jan 25 '16 at 13:20
  • I created a folder, added a `__init__.py` and put my modules in there, so I can access them from my project, which uses the virtualenv. – vicco Jan 25 '16 at 13:22
  • Well you shouldn't be doing that. How are you intending to deploy these libraries? You need to put them somewhere online they can be installed from, and you need to package them so that they can be installed by pip. Or, if they're simply part of your project, they should be inside your project itself. – Daniel Roseman Jan 25 '16 at 13:26
  • Okay, good to know that. They're not simply part of the project, they're actually libraries which I wrote for my project. I thought I should store them there so I could access them easily. So I should put it into a real package, add a `setup.py` and then I can simply install them to the virtualenv on the other PC? – vicco Jan 25 '16 at 13:30

5 Answers5

38

Do the following steps on the source machine:

  1. workon [environment_name]
  2. pip freeze > requirements.txt
  3. copy requirements.txt to other PC

On the other PC:

  1. create a virtual environment using mkvirtualenv [environment_name]
  2. workon [environment_name]
  3. pip install -r requirements.txt

You should be done.

Other Resources:

Trenton McKinney
  • 56,955
  • 33
  • 144
  • 158
Anuj Gupta
  • 6,328
  • 7
  • 36
  • 55
  • Does this also copy the packages in `site-packages` which I wrote on my own? – vicco Jan 25 '16 at 13:02
  • It will create a replica of the working environment on the source machine. It will download what all it need on its own – Anuj Gupta Jan 25 '16 at 13:03
  • I don't quite understand how it can recreate the packages I wrote myself from a text file? The requirements.txt only contains packages which are available to download. – vicco Jan 25 '16 at 13:08
  • 1
    @vicco it doesn't, since freeze only prints out packages installed via pip, and even if it printed yours, if it's not on pip the other computer obviously wouldn't be able to download it from pip. You'll likely need to copy those manually – Fabio Apr 17 '17 at 16:04
  • 1
    this doesn't work if the requirements are local repositories – Alexander McFarlane Oct 24 '18 at 12:32
  • On the target machine can direct mkvirtualenv using the requirement file like mkvirtualenv -r requirements_file ENVNAME – Kurumi Tokisaki May 03 '19 at 03:28
13

Pip Freeze Not Applicable For You?

Scenario: you have libraries installed on your current system that are very hard to migrate using pip freeze and am talking really hard, because you have to download and install the wheels manually such as gdal, fiona, rasterio, and then even doing so still causes the project to crash because possibly they were installed in the wrong order or the dependencies were wrong and so on.

This is the experience I had when I was brought on board a project.

For such a case, when you finally get the environment right you basically don't want to go through the same hell again when you move your project to a new machine. Which I did, multiple times. Until finally I found a solution.

Now, disclaimer before I move on:

I don't advocate for this method as the best, but it was the best for my case at the time.

I also cannot guarantee it will work when switching between different OSes as I have only tried it between Windows machine. In fact I don't expect it to work when you move from Windows to other OSs as the structure of the virtualenv folder from Unix-based OS is different from that of Windows.

Finally, the best way to do all of this is to use Docker. My plan is to eventually do so. I have just never used Docker for a non-web-app project before and I needed a quick fix as my computer broke down and the project could not be delayed. I will update this thread when I can once I apply Docker to the project.

THE HACK

So this is what I did:

  • Install the same base Python on your new machine. If you have 3.9 on the old, install 3.9 on the new one and so on. Keep note of where the executable can be located, usually something like C:\Users\User\Appdata\Local\Programs\Python\PythonXX

  • Compress your virtual env folder, copy it into the project directory inside your new machine. Extract all files there

  • Using text editor of your choice, or preferably IDE, use the 'Search in all files' feature to look for all occurrences of references to
    your old machine paths: C:\Users*your-old-username*

  • Replace these with your new references. For my case I had to do it in the following files inside the virtual env folder: pyvenv.cfg, Scripts/activate, Scripts/activate.bat, Scripts/activate.fish and Scripts/activate.nu.

And that's it!

Good luck everyone.

Imran Said
  • 215
  • 3
  • 13
2

I think what occurs is that you just copy the symbolic links in the source file to the target machine as binary files(no longer links). You should copy it using rsync -l to copy to keep those links.

Lerner Zhang
  • 6,184
  • 2
  • 49
  • 66
1

Usually I use virtualenv to create a new environment, then I go to the environment where I want to copy from, copy all the folders and paste it into the environment folder I just created, but most importantly when asking if you want to replace the Destination files, choose to skip these files. This way you keep your settings. At least for me, this has worked very well. I hope it works for you too.

Mueladavc
  • 73
  • 7
1

I share my experience. 

Suppose another PC does not install Python
Python version: 3.7.3 
Platform: Platform: Windows 10, 7 (64bit)

The following is working for me.

Step:

  1. download Windows embeddable zip file
  2. download get-pip.py (because the embeddable zip file does not provide pip)
  3. [Optional] install tkinter, see this article: Python embeddable zip: install Tkinter
  4. Choose a packaging method (I use NSIS: https://nsis.sourceforge.io/Download)

folder artictures:

- main.nsi
- InstallData
    - contains: Step1 & Step2
- YourSitePackages  # I mean that packages you do not intend to publish to PyPI.
    - LICENSE
    - MANIFEST.in
    - README.rst
    - ...
    - requirements.txt
    - setup.py

The abbreviated content about main.nsi is as follows:

!define InstallDirPath "$PROGRAMFILES\ENV_PYTHON37_X64"
!define EnvScriptsPath "${InstallDirPath}\Scripts"
...

CreateDirectory "${InstallDirPath}"  # Make sure the directory exists before the writing of Uninstaller. Otherwise, it may not write correctly!
SetOutPath "${InstallDirPath}"
SetOverwrite on
File /nonfatal /r "InstallData\*.*"

SetOutPath "${InstallDirPath}\temp"
SetOverwrite on
File /nonfatal /r "YourSitePackages\*.*"
nsExec::ExecToStack '"${InstallDirPath}\python.exe" "${InstallDirPath}\get-pip.py"'  # install pip
nsExec::ExecToStack '"${InstallDirPath}\Scripts\pip.exe" install "${InstallDirPath}\temp\."'  # install you library. same as: `pip install .`
RMDir /r "${InstallDirPath}\temp"  # remove source folder.
...

/*
Push ${EnvScriptsPath}  # Be Careful about the length of the HKLM.Path. it is recommended to write it to the HKCU.Path, it is difficult for the user path to exceed the length limit
Call AddToPath  # https://nsis.sourceforge.io/Path_Manipulation
*/

hope someone will benefit from this.

Carson
  • 6,105
  • 2
  • 37
  • 45