1

So this is my first time trying to build a python project (using pyinstaller in this case). I use pipenv for managing my packages, but it looks like there may be some fundamental issues between the distribution of python I'm using and pyinstaller.

I installed pyinstaller into my environment with $ pipenv install --dev pyinstaller.

Running $ pipenv run pyinstaller src/game.py then yields:

OSError: Python library not found: .Python, libpython3.8m.dylib, libpython3.8.dylib, Python
    This would mean your Python installation doesn't come with proper library files.
    This usually happens by missing development package, or unsuitable build parameters of Python installation.

    * On Debian/Ubuntu, you would need to install Python development packages
      * apt-get install python3-dev
      * apt-get install python-dev
    * If you're building Python by yourself, please rebuild your Python with `--enable-shared` (or, `--enable-framework` on Darwin)

The issue seems to be that I'm not using a "shared"/"framework" version of python.

How to use pyinstaller with pipenv / pyenv offers a potential solution, but I'm not understanding why I'd need to move from pipenv to pyenv for this case.

Is there a way to get pipenv to use a shared version of python? Is there a fundamental concept of pipenv that I'm misunderstanding that would make this incompatible with pyinstaller?

Justian Meyer
  • 3,623
  • 9
  • 35
  • 53
  • Did you ever solve this problem? I added a solution below but I'm curious if you found another way. – lortimer Jun 23 '20 at 03:53

1 Answers1

2

I just had the same problem using Pyinstaller with Pipenv. You don't need to switch to pyenv, I just got this working.

When Python is built from source, there is a flag you can set, --enable-shared. When set, it allows you to bundle up the Python Library with an application. The default is for this flag not to be set, which is a problem for Pyinstaller, which wants to bundle the Python interpreter with your application.

Did you build your own version of Python from source? If not, your Linux distro's default Python may be built without this flag. You can check by running the version of Python you're struggling with and running the following code:

from distutils.sysconfig import get_config_var
print get_config_var('CONFIG_ARGS') # Python 2
# print(get_config_var('CONFIG_ARGS')) # Python 3

If you don't see '--enable-shared' among the output, your version of Python was not built with this flag.

How I recommend to solve this problem

I did all of this in a virtual machine so I could blow it up and start over if something went weird.

I installed a special version of Python just for my application. That way, the Python version will never change if the system Python is upgraded.

  1. Install all the dependencies you'll need to install Python:
sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev
  1. Download the source code for the desired version of Python from python.org
  2. Extract the archive and cd into the extracted folder
  3. Run the configuration script for building the shared version of Python. Include all your other desired build flags. I recommend setting the prefix flag to include the name of your application to ensure that this version of Python is tied to your app, at least in name:
./configure --enable-shared --enable-optimizations --prefix=/usr/local/<name of project>
  1. run make followed by sudo make altinstall. We use altinstall to avoid overriding our system's version of Python. We use sudo because you need root permission to install into /usr/local/bin

At this point, you're now free to create a new Pipenv based on the newly installed version of Python. Running Pyinstaller as you did above should work if you use the new, shared Python.

Technically, the interpreter will be a bit slower than the static version when built this way. You can take one extra step to create a sort of hybrid version of Python that can be embedded in your app, but should benefit from being statically linked when running scripts locally.

I don't fully understand why you would want your interpreter to run faster during development than it does when running your built application, but I may have misunderstood the argument going on in this thread.

Anyway, if you want to use this hybrid approach, you can pick up here after step 5 above:

  1. Still in the Python source folder, clean up your previous configuration: make distclean
  2. Run the same configure again without --enable-shared:
./configure --enable-optimizations --prefix=/usr/local/<name of project>
  1. Run make
  2. Finally, run sudo make altbininstall

This will apparently install the static Python Library and an executable linked to it, but your installation will still have the shared library for embedding in your app. I tried this and it works fine.

I adapted this strategy from this article, which has a lot more information about static and shared Python. I did not use make install as the author recommends because I'm not doing any of this in a docker container, and I definitely don't want to mess with my system's default python.

lortimer
  • 701
  • 9
  • 14
  • This also seems to work for checking whether --python-shared was specified. https://stackoverflow.com/a/23202055/373196. Digging into this in my spare time. Unsure if I'll get all the way through with it to confirm. – Justian Meyer Sep 29 '20 at 02:38
  • For the install on mac I modified the above to be: `brew install openssl xz gdbm` (from the python.org docs) and `./configure --enable-shared --enable-optimizations --prefix=/usr/local/py-traffic-sim --with-openssl=$(brew --prefix openssl)` to properly setup openssl. – Justian Meyer Sep 29 '20 at 03:17
  • Reinstalled env with: `pipenv install --dev --python=/usr/local/py-traffic-sim/bin/python3.8` (pyinstaller is a dev package in this env). – Justian Meyer Sep 29 '20 at 03:50