146

What I'm trying to do is ship my code to a remote server, that may have different python version installed and/or may not have packages my app requires.

Right now to achieve such portability I have to build relocatable virtualenv with interpreter and code. That approach has some issues (for example, you have to manually copy a bunch of libraries into your virtualenv, since --always-copy doesn't work as expected) and generally slow.

There's (in theory) a way to build python itself statically.

I wonder if I could pack interpreter with my code into one binary and run my application as module. Something like that: ./mypython -m myapp run or ./mypython -m gunicorn -c ./gunicorn.conf myapp.wsgi:application.

Prakhar Agarwal
  • 2,724
  • 28
  • 31
roboslone
  • 2,847
  • 3
  • 20
  • 28
  • Do you mean something like cx_freeze? – TigerhawkT3 Oct 07 '16 at 09:47
  • @TigerhawkT3, I've never heard of cx_freeze until today. Looked through docs, seems like it's what I want, but it fails to build on Ubuntu 12.04.5 with python3.5.1. I'm looking for alternatives, thanks for kick in the right direction. – roboslone Oct 07 '16 at 11:30
  • 1
    Sadly, pyinstaller doesn't work with Django 1.9 and py2app/py2exe don't support linux :( – roboslone Oct 07 '16 at 11:39
  • 2
    if cx_Freeze and pyinstaller don't work, there is still hope. You can create package(s) for your target distribution(s) which will resolve all dependencies on installation. Or, maybe use something like docker to run your app. – KeatsPeeks Oct 11 '16 at 21:29
  • pyinstaller works perfect on Windows! – denfromufa Oct 11 '16 at 22:14
  • 1
    other options are nuitka and cython – denfromufa Oct 11 '16 at 22:14
  • If you have a spare hour, this is very good talk about shipping python - https://www.youtube.com/watch?v=5BqAeN-F9Qs – iScrE4m Oct 12 '16 at 08:24
  • I think cx_freeze is really what you want, so we might be able to help you more if you shared the error... Is it [this issue](https://bitbucket.org/anthony_tuininga/cx_freeze/issues/32/cant-compile-cx_freeze-in-ubuntu-1304) by any chance? – Peter Brittain Oct 15 '16 at 13:12

6 Answers6

187

There are two ways you could go about to solve your problem

  1. Use a static builder, like freeze, or pyinstaller, or py2exe
  2. Compile using cython

This answer explains how you can go about doing it using the second approach, since the first method is not cross platform and version, and has been explained in other answers. Also, using programs like pyinstaller typically results in huge file sizes, while using cython will result in a file that's much smaller

  1. First, install cython.

    sudo -H pip3 install cython

  2. Then, you can use cython to generate a C file out of the Python .py file (in reference to https://stackoverflow.com/a/22040484/5714445)

    cython example_file.py --embed

  3. Use GCC to compile it after getting your current python version (Note: The below assumes you are trying to compile it to Python3)

    PYTHONLIBVER=python$(python3 -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')$(python3-config --abiflags)
gcc -Os $(python3-config --includes) example_file.c -o output_bin_file $(python3-config --ldflags) -l$PYTHONLIBVER

You will now have a binary file output_bin_file, which is what you are looking for

Other things to note:

  1. Change example_file.py to whatever file you are actually trying to compile.
  2. Cython is used to use C-Type Variable definitions for static memory allocation to speed up Python programs. In your case however, you will still be using traditional Python definitions.
  3. If you are using additional libraries (like opencv, for example), you might have to provide the directory to them using -L and then specify the name of the library using -l in the GCC Flags. For more information on this, please refer to GCC flags
  4. The above method might not work for anaconda python, as you will likely have to install a version of gcc that is compatible with your conda-python.
R. S. Nikhil Krishna
  • 3,962
  • 1
  • 13
  • 27
  • 38
    Hi, what if my application spans across multiple files. This approach seems to only compile the current script and not all the Python dependencies. – RChat Dec 14 '16 at 13:19
  • 6
    no need to rename your python file (say test.py) into a pyx right now – iMath Jun 02 '19 at 04:15
  • 6
    This was of great help to me. To answer the question about multiple files, you could use cythonize: `cythonize -i file_0.py [...] file_n.py` for all your python modules, then `cython main_file.py --embed` for your main script. You'll end up with an executable and .so files (.dll, .dylib depending on your OS) that you can use together as a standalone solution. – afp Jun 16 '19 at 19:51
  • I am getting "No modules named encodings". I got to know I require certain files from python installation. However, my target system doesn't have python. Please help. – abhiarora Oct 18 '19 at 09:40
  • 14
    Warning: Cython requires dynamically loading stuff, and it DOES NOT produce self-contained binaries. Don't waste your time! – gonzo Feb 24 '21 at 00:24
  • Also note that you'll need to specify the Python directory during build or you will get fatal error: 'Python.h' file not found for the include line #include "Python.h" (which is what the posted command produces). – Network Effects May 07 '21 at 19:25
16

You might wish to investigate Nuitka. It takes python source code and converts it in to C++ API calls. Then it compiles into an executable binary (ELF on Linux). It has been around for a few years now and supports a wide range of Python versions.

You will probably also get a performance improvement if you use it. Recommended.

Prakhar Agarwal
  • 2,724
  • 28
  • 31
  • I couldn't find any information about using Nuitka with Django. Do you, by any chance, have such experience? – roboslone Oct 18 '16 at 12:42
  • 1
    @roboslone, Prakhar Agarwal Cython supports external libraries really well, when compared to Nuitka PyPy's static builder. For instance, from the link at the end of this comment, "people use Cython because it helps them solve a problem. Which is either that they want to connect to external non-Python libraries from Python code or that they want to be able to manually optimise their code, or both.", the link being http://blog.behnel.de/posts/indexp241.html – R. S. Nikhil Krishna Oct 18 '16 at 13:03
  • 3
    R. S. Nikhil Krishna, I'm not trying to make my code faster, I'm trying to ship it in one binary with python interpreter inside. – roboslone Oct 18 '16 at 18:06
  • Aah okay. I thought you wanted to bind it with some external library. My bad. – R. S. Nikhil Krishna Oct 18 '16 at 18:56
3

You're probably looking for something like Freeze, which is able to compile your Python application with all its libraries into a static binary:

PyPi page of Freeze

Python Wiki page of Freeze

Sourceforge page of Freeze

asc11
  • 419
  • 3
  • 7
2

If you are on a Mac you can use py2app to create a .app bundle, which starts your Django app when you double-click on it.

I described how to bundle Django and CherryPy into such a bundle at https://moosystems.com/articles/14-distribute-django-app-as-native-desktop-app-01.html

In the article I use pywebview to display your Django site in a local application window.

André Aulich
  • 190
  • 10
1

Freeze options:

However, your target server should have the environment you want -> you should be able to 'create' it. If it doesn't, you should build your software to match the environment.

I found this handy guide on how to install custom version of python to a virtualenv, assuming you have ssh access: https://stackoverflow.com/a/5507373/5616110

In virtualenv, you should be able to pip install anything and you shouldn't need to worry about sudo privileges. Of course, having those and access to package manager like apt makes everything a lot easier.

Community
  • 1
  • 1
iScrE4m
  • 882
  • 1
  • 12
  • 31
0

I have created a docker image that relies on Nuitka and a custom statically linked python3.10 to create a static binary.

Did not test it extensively, if you have the chance please let me know if it works for your use case.

You can check it at: https://github.com/joaompinto/docker-build-python-static-bin

João Pinto
  • 5,521
  • 5
  • 21
  • 35