py2exe: excluding parts of a package that wants to import all its parts

Question

I am trying to run py2exe and minimize the third-party dependencies. I'm trying to include only the small necessary part of a huge third-party package but can't figure out how to prevent the package's __init__.py, which imports a whole lot of stuff I don't want, from being called at runtime.

This summarizes the situation:

myscript.py:   from BigPackage.SmallSubset import TheOnlyFunctionIReallyNeed

BigPackage/__init__.py:            import SmallSubset, HugeUnwantedSubset
BigPackage/SmallSubset.py:         import AcceptableDependencies
BigPackage/HugeUnwantedSubset.py:  import UnacceptablyHugeDependencies

The problem is that, even if I successfully include some parts of BigPackage but not others, when import BigPackage.SmallSubset is called at runtime, BigPackage/__init__.py runs first, which then tries to import the excluded parts and hence raises an exception. It would work if I could persuade py2exe to include BigPackage/SmallSubset.py but not BigPackage/__init__.py, but I'm having no luck getting py2exe to understand that idea. I've tried the following in my setup.py:

import BigPackage     # let's try a Deny/Allow approach:
options[ 'py2exe' ][ 'excludes' ].append( 'BigPackage' )
options[ 'py2exe' ][ 'includes' ].append( 'BigPackage.SmallSubset' )
# nope, py2exe fails with 'ImportError: No module named BigPackage' in py2exe/mf.py

...and/or:

import BigPackage     # this seems less plausible, but worth a try:
options[ 'py2exe' ][ 'excludes' ].append( 'BigPackage.__init__' )
# nope, __init__.pyc still turns up in dist

...and/or:

import BigPackage     # really getting desperate now:
options[ 'py2exe' ][ 'excludes' ].append( BigPackage.__file__ )
# nope, __init__.pyc still turns up in dist

... with no luck. Anybody know how to work around this?

It feels hackish, but can you write a dummy `__init__.py` and put it in place temporarily while you compile? If that did end up working you could place the `.pyc` version of that one somewhere and swap it in/out of the archive file (I presume you're using bundle_files of some level). — g.d.d.c, Jul 15 '14 at 00:49
@g.d.d.c Thanks, I hadn't thought of that! I can't work out whether to downvote it for its sheer ugliness, or upvote it for the fact that it *would* actually work - at least assuming setup.py is run with permission to change the BigPackage directory, which I suppose it might not always. In the end I think I'd be reluctant to go this way, in case future maintainers try to run setup.py, get some other error midway, and end up with a crippled BigPackage. But I suppose if I used a `library.zip` and worked out how to programmatically alter it, I could just nix `__init__.pyc` out of it post-hoc. — jez, Jul 15 '14 at 01:04
Having tried simply removing `__init__.py` using http://stackoverflow.com/questions/513788 and getting the error that BigPackage is no longer considered a package, I now see what you mean by keeping the .pyc version around. Yes, I guess that would work *and* avoid some of the ugliness. — jez, Jul 15 '14 at 01:24
My solution for now (unless a more elegant solution turns up) is (1) exclude `UnacceptablyHugeDependencies` explicitly in the options (so `BigPackage.HugeUnwantedSubset` will fail if anything tries to import it); (2) use the `compress=True` option but comment out my preferred `zipfile=None` argument to `setup()`; (3) use `zipfile` code after `setup()` to replace `BigPackage/__init__.pyc` with a zero-byte `BigPackage/__init__.py` inside `dist/library.zip`. And that works. Thanks again @g.d.d.c — jez, Jul 15 '14 at 01:46

score 0 · Answer 1 · answered Jul 15 '14 at 06:52

0

What you want is not possible without patching the BigPackage library.

answered Jul 15 '14 at 06:52

theller

2,809
19
19

jez · Accepted Answer · 2014-07-15T23:20:07.447

All solutions seems hackish at best (see comments on question). But rather than post-fixing __init__.py I finally settled on the following strategy of going in and extricating the specific things I need, and copying them to temporary files:

import sys, os, shutil

# awkward kludge to include BigPackage.SmallSubset by hand without having to include the rest of BigPackage
src = [ os.path.join( x, 'BigPackage', 'SmallSubset.py' ) for x in sys.path ]
src = [ x for x in src if os.path.isfile( x ) ][ 0 ]
shutil.copyfile( src, 'BigPackageSmallSubset.py' )
import BigPackageSmallSubset


options = { 'py2exe': { 'excludes' : [ 'BigPackage' ], 'includes' : [ 'BigPackageSmallSubset' ], 'compressed' : True, }, }

# setup( ..., options=options, ... )

os.remove( 'BigPackageSmallSubset.py' )
os.remove( 'BigPackageSmallSubset.pyc' )

Then I make myscript.py sensitive to the possible difference:

try: from BigPackage.SmallSubset import TheOnlyFunctionIReallyNeed
except ImportError: from BigPackageSmallSubset import TheOnlyFunctionIReallyNeed

py2exe: excluding parts of a package that wants to import all its parts

2 Answers2