32

This is a bit of a complex one, and may take some of your time.

The basic problem is, that on linux (Ubuntu in my test case) a cx-freeze'd version of my program (Omnitool) is not able to create subprocesses. It works on Windows 7, however. Or when running directly from source code. Unfortunately it's not as simple as forgetting freeze_support.

The Problem

Default behaviour of starting a subprocess, is that the X Server crashes. Specifically like this:

XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
      after 23 requests (23 known processed) with 0 events remaining.
[xcb]Unknown sequence number while processing queue 
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called 
[xcb] Aborting, sorry about that. 
Omnitool: ../../src/xcb_io.c:274: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed. 
Aborted (core dumped)

There is no python traceback. I tried to manually call XInitThreads with ctypes, it returns 0 for successfully set, but crashes regardless. Shorty before the crash, the pygame UI bugs out, so I expect that something there faults.

Now, setting multiprocessing.set_start_method() changes problems around: "forkserver" gives me this lovely traceback, that tells me nothing:

Pastebin Example

Setting it to spawn, instead, just makes it not do anything. The process starts, and goes through __main__, as I can prove with prints, but never enters the target function for the subprocess.

Trying yourself

Make sure you have Ubuntu or comparable Linux with python 3.4. Then to get all dependencies:

Download omnitool as zip or clone from git: https://github.com/Berserker66/omnitool requirements.txt in the following code is from Omnitool.

sudo apt-get update -qq
sudo apt-get install --fix-missing mercurial subversion python3-dev python3-numpy libav-tools libsdl-image1.2-dev libsdl-mixer1.2-dev libsdl-ttf2.0-dev libsmpeg-dev libsdl1.2-dev libportmidi-dev libswscale-dev libavformat-dev libavcodec-dev python3-pip
pip3 install -r requirements.txt
pip3 install cx_Freeze

To freeze, run omnisetup.py with the buildargument.

I've also tried freezing with pyinstallers python 3 fork, which fails very similar. The spec file isn't git tracked though.

Zac
  • 1,305
  • 3
  • 17
  • 28
Berserker
  • 1,112
  • 10
  • 26
  • 5
    Why do you want to freeze it for Linux? – Cyphase Aug 14 '15 at 21:33
  • 2
    The dependencies are hard-ish to come by, involve compiling and many of them are only used during compiling of pygame. Currently my linux users have to do several steps in console to get my program to work, while windows users just run a setup and click through. I'd like to provide the same convenience to all users. I'd also personally like to know why this is failing and how to fix, for curiosity's sake. – Berserker Aug 14 '15 at 21:39
  • Two of the dependency packages aren't on pip or apt-get. Pygame-3 and pgu. – Berserker Aug 14 '15 at 21:58
  • Hmm - seems like you are not alone in seeing the symptom at least - see https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1024202 and all the duplicate bugs. I'll try to have a look at it when I can get to an Ubuntu box. Out of interest have you tried it on a non-Debian based linux? – J Richard Snape Sep 20 '15 at 19:47
  • No, so far only ubuntu and xubuntu – Berserker Sep 21 '15 at 21:06
  • Some things about Snappy seem good, while others seem suspect, but it might solve your problem for Ubuntu >=16.04 at least. – Wodin May 13 '16 at 21:34
  • Am unable to install requirements as it breaks on Cloning hg http://bitbucket.org/pygame/pygame. The standard way to distribute native binaries of python libraries is with wheel or egg packages, however, depending on if the library contains native code (wheel) or not (egg). With setuptools it's `python setup.py bdist_wheel` and/or `python setup.py bdist_egg`. The resulting package is a binary that can be installed on the platform it was generated on. Pip as of version 8 fully supports egg and wheel packages. – danny Aug 12 '16 at 15:56
  • I've had several strange, unsolvable bugs with cx_freeze in the past. In the end, I just gave up on using it (or any other packer) and now just distribute my program along with a standalone python distribution that has everything needed and just works (as long as the target machine has the right OS/bitness/etc. of course). The 'downside' is that your package will be larger by maybe a few hundred megabytes, but I don't consider this an issue that is worth a single cx_freeze headache. – Daniel Sk Aug 26 '16 at 09:30
  • In this case the problem is almost certainly that cx_freeze and Xcb simply won't get along. Xcb in particular uses "static" data space variables that don't take well to being saved and reloaded. The thread init would be the way to paper over the problem, I think, but I'd have to stare at the code a lot to say more. – torek Dec 17 '16 at 00:23

3 Answers3

1

I used cx_freeze for a project at work. I'm not sure if this is your problem... but I was using the Anaconda distribution, and cx_freeze was not properly gathering the .dll's that I needed for my project.

The solution was to:

  1. Install a plane version of Python
  2. make an environment with the packages that I needed for that project
  3. Run cx_freeze.

Magically, all of the problems went away and everything compiled like it was supposed too.

Stephen Rauch
  • 47,830
  • 31
  • 106
  • 135
Clay
  • 577
  • 3
  • 8
  • 14
0

I know that probably this is not what you want to hear, but the reality is that you will never be able to succeed with this approach and here is why:

  • Linux does not have portably Python package binaries, meaning that a compiled numpy on one distro will probably not work on another. It can even happen with the same distribution and version, just because one system library received an update.
  • manylinux1 is supposed to solve this but is really hard to build packages for it and only few started using it.
  • cx-freeze project had latest release in December 2014, which on Python timescale puts it as an abandoned project.
sorin
  • 161,544
  • 178
  • 535
  • 806
  • There is no numpy involved. PyInstaller has the same problem, which is still actively developed. – Berserker May 31 '16 at 11:35
  • Will note that multiple builds of python libraries containing native code are needed for 32/64bit variants of all OSs - see https://pypi.python.org/pypi/gevent/1.1.2 for an example which has builds for all supported python versions for both 32bit and 64bit windows, as well as the same for OSX and 64 bit Linux. – danny Aug 12 '16 at 15:49
0

Well the problem you are facing may be due to the lack of dependencies. cx_Freeze have some dependencies. In order to distribute your work in python many lightweight packages are avalible. You can use:-

  1. Pyinstaller: it can be installed by

    pip install pyinstaller pyinstaller is really good for making executables with less size

  2. Py2exe

Namah Jain
  • 1