2

Our command line utility is written with Python.

On Linux/OS X this is usually not a problem since both come with Python 2.x pre installed. However on Windows, Python isn't installed by default.

Additional problem is that few of our dependencies require compiling which yet again is not a trivial problem for Windows users since it requires tinkering with MSVC/Cygwin/etc'.

Up until now we solved this issue by using Pyinstaller to create "frozen" Python package with pre-installed dependencies. This worked well, however made our utility non extendable - we cannot add additional Python modules by using utilities such as pip for example. Since our CLI depends on this ability in order to add additional usability, this limitation became a blocker for us and we would like to rethink our approach.

Searching around, I found how Rhodecode solve this. Basicly their installer brings Python and everything else (including pre-compiled dependencies).

This seem as a good idea for us, the only limitation I see here is that their installer actually installs Python from .msi which puts stuff in Windows Registry. So there can be only one Python of version X.Y installed on Windows (from .msi)

For a server application this might be reasonable since server application tends to act like it's the only thing installed on the PC, but for command line utility, this is completely unacceptable.

Looking around I found few projects that claim to make Python portable - for example Portable Python. However I'm not sure how "portable" it really is, especially after issues like this.

So questions are:

  1. Is it possible to install same Python version multiple times on Windows without creating collisions between the instances?

  2. Would you choose other workaround to solving this problem (please no "smart" solutions such as: drop Windows support/don't use Python)

Thanks!

m1keil
  • 4,515
  • 22
  • 26
  • Here is how to have `different` python versions simultaneously: http://stackoverflow.com/questions/4583367/how-to-run-multiple-python-version-on-windows Perhaps you could do the same for the similar versions? – ρss Jan 20 '15 at 12:51
  • @pss, the thing is, I cannot install two PythonX.Y.Z. This is because the installer put some info inside the registry. The next time you try to run installer, you'll get the options to Change/Repair/Uninstall. I also cannot predict which Python version might be preinstalled on users PC, so I cannot assume he won't have 2.6.9 and bring it. So what I really want is to be able to bring Python to PC, but in a "portable" way. – m1keil Jan 20 '15 at 13:04
  • Perhaps you could remove that info from the registry to over come the `Change/Repair/Uninstall` option using a python script. The path is located here in registry `HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\` And to know if python is installed you could try this http://stackoverflow.com/questions/8917885/which-version-of-python-do-i-have-installed – ρss Jan 20 '15 at 13:35
  • Right, but what do I do if Python is already installed? Obviously I cannot break user's Python. So I can decide to use the installed Python but do I really want to do it? (dependencies are shared so our CLI becomes more fragile / uninstall leaves dirt behind since I cannot remove Python / support becomes harder since now there are two possible paths for problems during and after the install). So ideally I still think that we need someway to bring Python without any registry trace and just use it. – m1keil Jan 20 '15 at 13:48

2 Answers2

0

Frankly I would stick with PyInstaller or something similar. That will always provide you with the correct version of Python whether or not the target machine has Python installed. It also protects you from clobbering a previously installed version of Python.

If you need to add plugins, then you should build that into your app. There are a few projects that might help you with that. Here are a couple of examples:

You might also take a look at how Django or Flask handles extensions. For example, you can add an extension to Flask to allow it to work with SQLAlchemy. You should be able to do something similar with your own application. The pip utility won't work with a frozen application after all. Alternatively, have you looked at conda? It might work for your purposes.

Mike Driscoll
  • 32,629
  • 8
  • 45
  • 88
  • We already have some kind of a plugin management mechanism. Our plugins are just additional Python modules that are being discovered by CLI. The problem is that these plugin can have their own Python dependencies (and the dependencies of their dependencies...), so "bundling" them and importing them into the frozen PyInstaller package becomes a real pain. I heard about conda before but not sure how it can help me, I'll take a deeper look. Thanks! – m1keil Jan 21 '15 at 08:55
0

This is an answer for my second question, sadly I still haven't figured out a better solution for number 1.

For now here's how we changed our approach for creating setup file:

  1. We package our code and its dependencies as set of pre-built Python wheels. It's relatively easy to create pre-built wheels on Windows since the release of Visual C++ compiler for Python2.7.
  2. We package Python setup MSI together with Pip, Setuptools and Virtualenv wheels.
  3. Before install starts we check whether Python, Pip and Virtualenv are already installed (by looking in the registry and \Scripts), if it's not, we install it from the packaged wheels. Pip wheel is installed by using get-pip.py script which we bundle as well.
  4. We create separate Virtualenv and install our wheels into it.
  5. Uninstall is done by removing the virtualenv folder. Python, Pip and Virtualenv install are left behind.

Installer created by Inno Setup.

I'm still not fully satisfied with this solution since some components are globally installed and might collide with what user had already installed before (older/newer version of python, pip, setuptools or virtualenv). This creates potential for unpredictable bugs during install or runtime. Or the possibility for user to upgrade one of the components in the future and somehow break the application.

Also, uninstall is dirty and leaves stuff behind.

m1keil
  • 4,515
  • 22
  • 26