1

I recently successfully embedded a python distribution with an application in Mac OS X using a homebrew installed python3.7 as per the methodology outlined in Joao Ventura's very useful two part series, provided here for reference (http://joaoventura.net/blog/2016/embeddable-python-osx/) and (http://joaoventura.net/blog/2016/embeddable-python-osx-from-src/).

The only remaining issue for me was to reduce the size of the python distribution size in the application by zip compressing the whole standard library minus lib-dynload, config-3.7m-darwin and site-packages.

My directory structures is as follows:

- python3.7/
  - include/
  - lib/
    - python3.7/
  - libpython3.7.dylib
  - python3.7 <executable>

The basic initial step is to move lib-dynload and config-3.7m-darwin from lib/python3.7, so that I can compress the sodlib source files into lib/python37.zip and then move lib-dynload and config-3.7m-darwin back into now empty lib/python3.7 to end up with the desired structure:

- python3.7/
  - include/
  - lib/
    - python3.7/
      - lib-dynload/
      - config-3.7m-darwin
    - python37.zip
  - libpython3.7.dylib
  - python3.7 <executable>

To test whether it worked or not, I would check sys.path from the executable and try to import a module and check its __file__ attribute to see if it came from the zip archive.

On this basis, I would cd into lib/python3.7 and try the following:

  1. Select all files and folders and zip using OS X's Finder's compress to generate python37.zip

  2. Using the python zipfile module:

python -m zipfile -c python37.zip lib/python3.7/*

  1. Using the zip method from How can you bundle all your python code into a single zip file?
cd lib/python3.7
zip -r9 ../python37.zip *

In all cases, I got it to work by setting PYTHONPATH to the zipped library, as in:

PYTHONPATH=lib/python37.zip ./python3.7`

Doing, I was able to successfully import from the zip archive and verify that the modules came from the zip archive. But without setting PYTHONPATH, it did not work.

Hence, I would very much appreciate some help to establish the correct and most straightforward way to zip the standard library such that it would be recognized automatically from sys.path (without any extra steps such as specifying the PYTHONPATH environment value which may not be possible on a user's machine).

Thanks in advance for any help provided.

S

shakfu
  • 41
  • 1
  • 5

1 Answers1

1

Finally figured it out through a long process of elimination.

The only module you have to keep in site packages is os.py.

Here's a bash script for the whole process which may or may not work. It assumes you have downloaded a python source distribution from python.org

Then cd into the resultant source folder and run this script in the root

#!/usr/bin/env bash

# build_python.sh
# NOTE: need os.py to remain in site-packages or it will fail



NAME=xpython
PWD=$(pwd)
PREFIX=${PWD}/${NAME}
VERSION=3.8
VER="${VERSION//./}"
LIB=${PREFIX}/lib/python${VERSION}
MAC_DEP_TARGET=10.13


remove() {
    echo "removing $1"
    rm -rf $1
}

rm_lib() {
    echo "removing $1"
    rm -rf ${LIB}/$1
}


clean() {
    echo "removing __pycache__ .pyc/o from $1"
    find $1 | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf
}

clean_tests() {
    echo "removing 'test' dirs from $1"
    find $1 | grep -E "(tests|test)" | xargs rm -rf
}

clean_site_packages() {
    echo "removing everything in $LIB/site-packages"
    rm -rf $LIB/site-packages/*
}

rm_ext() {
    echo "removing $LIB/lib-dynload/$1.cpython-${VER}-darwin.so"
    rm -rf $LIB/lib-dynload/$1.cpython-38-darwin.so
}

rm_bin() {
    echo "removing $PREFIX/bin/$1"
    rm -rf $PREFIX/bin/$1
}

./configure MACOSX_DEPLOYMENT_TARGET=${MAC_DEP_TARGET} \
  --prefix=$PREFIX \
  --enable-shared \
  --with-universal-archs=64-bit \
  --with-lto \
  --enable-optimizations

make altinstall


clean $PREFIX
clean_tests $LIB
clean_site_packages
remove ${LIB}/site-packages

# remove what you want here...
rm_lib config-${VERSION}-darwin
rm_lib idlelib
rm_lib lib2to3
rm_lib tkinter
rm_lib turtledemo
rm_lib turtle.py
rm_lib ensurepip
rm_lib venv

remove $LIB/distutils/command/*.exe
remove $PREFIX/lib/pkgconfig
remove $PREFIX/share

# remove what you want here...
rm_ext _tkinter


rm_bin 2to3-${VERSION}
rm_bin idle${VERSION}
rm_bin easy_install-${VERSION}
rm_bin pip${VERSION}



mv $LIB/lib-dynload $PREFIX
cp $LIB/os.py $PREFIX
clean $PREFIX
python -m zipfile -c $PREFIX/lib/python${VER}.zip $LIB/*
remove $LIB
mkdir -p $LIB
mv $PREFIX/lib-dynload $LIB
mv $PREFIX/os.py $LIB
mkdir $LIB/site-packages

This is for a mac user, can be easily adapted for other platforms. It's not very well tested, so post feedback if you encounter any issues.

shakfu
  • 41
  • 1
  • 5