cx_Freeze - Preventing including unneeded packages

Question

I have coded a tiny python program using PyQt4. Now, I want to use cx_Freeze to create a standalone application. Everything works fine - cx_Freeze includes automatically all necessary modules; the resulting exe works.

The only problem is that cx_Freeze packs plenty of unneeded modules into the standalone. Even though I only use QtCore and QtGui, also modules like sqlite3, QtNetwork, or QtScript are included. Surprisingly, I find also PyQt5 dlls in the resulting folder. It seems to me as if cx_Freeze uses all PyQt packages that I have installed. The result is a 200Mb program - albeit I only wrote a tiny script.

How can I prevent this behaviour?

I use the following setup.py:

import sys
from cx_Freeze import setup, Executable

setup(
    name="MyProgram",
    version="0.1",
    description="MyDescription",
    executables=[Executable("MyProgram.py", base = "Win32GUI")],
)

I tried explicitely excluding some packages (although it is quite messy to exclude all unused Qt modules) adding this code:

build_exe_options = {"excludes": ["tkinter", "PyQt4.sqlite3",
                              "PyQt4.QtOpenGL4", "PyQt4.QtSql"]}

but the upper modules were still used. I also tried

build_exe_options = {"excludes": ["tkinter", "PyQt4.sqlite3",
                              "QtOpenGL4", "QtSql"]}

with the same result.

In addition to the nedless Qt packages I find also unneded folders with names like "imageformats", "tcl", and "tk". How can I include only needed files in order to keep the standalone folder and installer as small as possible?

I googled this problem for hours but only found this thread which did not help me.

I am running python 3.4.2 amd64 on windows 8.

I am happy about every solution that gives me the desired result "standalone" with a reasonable size. I tried also pyqtdeploy but ran into the error: Unknown module(s) in QT (but this is a different question).

Edit:

I am using two modules. One is the GUI class created by uic, "MyProgramGUIPreset". In this file there are the following import commands:

from PyQt4 import QtCore, QtGui
from matplotlibwidget import MatplotlibWidget

In the main module I do the following imports:

import MyProgramGUIPreset
import numpy as np
from PyQt4.QtGui import QApplication, QMainWindow, QMessageBox
import sys
from math import *

Maybe this helps to figuring out where the issue is.

In general, 'excludes' is the way to tell it you don't need things. I guess your script is using something like matplotlib which has backends for different GUIs - cx_Freeze can't tell which one will be used, so it tries to include them all. If you can work out what module is causing the problem, excluding that should cut out a lot of other modules too. — Thomas K, Dec 05 '14 at 21:20
@ThomasK: Thank you for the advise! Indeed I am using matplotlibwidget. I tried to exclude it: `build_exe_options = {"excludes": ["tkinter", "matplotlibwidget"]}` but nothing changed. Do you have a hint for me on how to find out which module causes the issue? I also wonder whether I am using the "excludes"-command correctly. Is it possible at all to exclude modules that are explecitely used? If the matplotlibwidget causes the problem: Can I solve the issue without leaving the widget out? What would be an appropriate solution? — Samufi, Dec 06 '14 at 04:07
You probably want to exclude all of the `matplotlib.backends.backend_foo` for backends you're not using (i.e. all of them apart from Qt4). It looks like you may not be passing build_exe_options into setup - see the example here: http://cx-freeze.readthedocs.org/en/latest/distutils.html — Thomas K, Dec 06 '14 at 21:01
Thank you, @ThomasK! Indeed, I forgot to pass the `build_exe_options` to the setup method. Now I added `options = {"build_exe": build_exe_options},` and excluding works fine. That is, I could pass a list of all modules that I want to exclude. But I am still wondering about how to exclude **all** unneeded modules at once. I tried `build_exe_options = {"excludes": ["matplotlib.backends.backend_foo"]}` which did not do anything. Have I understood your advice correctly, Thomas K? Or do I have to include `matplotlib.backends.backend_foo` in a different way in my code? — Samufi, Dec 08 '14 at 21:42
I am pretty sure I misinterpreted your `backend_foo` (in particular the word "foo") due to my limited english knowledge. Yes, I want to exclude all matplotlib backends except (!) `matplotlib.backends.backend_qt4agg`. How can I achieve this without listing all 26 possibilities? Can I use regular expressions? Excluding `matplotlib.backends` makes the file as small as desired. I tried to exclude the above and then to include `matplotlib.backends.backend_qt4agg` but this gives me an exception that there is no module wth this name... — Samufi, Dec 08 '14 at 22:20
As far as I know, you need to pass it a list of all the modules you want to exclude - it doesn't take regexes or glob patterns. You can of course generate that list using Python code. — Thomas K, Dec 09 '14 at 18:22

score 18 · Accepted Answer · answered Dec 12 '14 at 03:51

The reason for the not working "excludes" command was that I forgot to include the build options into the setup. After adding the respective line into the code excluding works:

from cx_Freeze import setup, Executable
import sys

# exclude unneeded packages. More could be added. Has to be changed for
# other programs.
build_exe_options = {"excludes": ["tkinter", "PyQt4.QtSql", "sqlite3", 
                                  "scipy.lib.lapack.flapack",
                                  "PyQt4.QtNetwork",
                                  "PyQt4.QtScript",
                                  "numpy.core._dotblas", 
                                  "PyQt5"],
                     "optimize": 2}

# Information about the program and build command. Has to be adjusted for
# other programs
setup(
    name="MyProgram",                           # Name of the program
    version="0.1",                              # Version number
    description="MyDescription",                # Description
    options = {"build_exe": build_exe_options}, # <-- the missing line
    executables=[Executable("MyProgram.py",     # Executable python file
                            base = ("Win32GUI" if sys.platform == "win32" 
                            else None))],
)

This decreased the program size from 230MB to 120MB. Nevertheless, I did not find a nice way of excluding all unneeded packages. By trial and error (deleting the biggest files in the build folder test-wise) I figured out which classes I can exclude.

I tried whether the matplotlib backends cause the problem and finally figured out that this is not the case. Nontheless, if anybody needs code to exclude all modules of a certain name scheme in a particular folder except some special ones, he may adjust the following to his needs:

mplBackendsPath = os.path.join(os.path.split(sys.executable)[0],
                        "Lib/site-packages/matplotlib/backends/backend_*")

fileList = glob.glob(mplBackendsPath)

moduleList = []

for mod in fileList:
    modules = os.path.splitext(os.path.basename(mod))[0]
    if not module == "backend_qt4agg":
        moduleList.append("matplotlib.backends." + modules)

build_exe_options = {"excludes": ["tkinter"] + moduleList, "optimize": 2}

I would be happy about more elegant solutions. Further ideas are still welcome. Nevertheless, I regard the problem as solved for me.

does `uic` needs `QtNetwork`? My app didn't start w/o this module and qtwebkit — Winand, Nov 27 '15 at 13:30
Do i nned to exclude all hundreds of modules ? any alternatives ? — ghost21blade, Sep 19 '22 at 10:40
@ghost21blade I am not aware of any, but I haven't worked on this for years. There may have been new developments. — Samufi, Sep 20 '22 at 12:31

score 1 · Answer 2 · answered Sep 07 '18 at 16:38

I was having a similar problem on a very simple PyQt4 Gui for a small database where the program was 58Mb for a small amount of code, the problem being that the entire PyQt4 folder was being included in the program.

The article here refers to using zip_include_packages in your options to exclude files or to compress them to reduce the file size.

I excluded the entire PyQt4 folder and then included the bits I needed as shown below and it reduced the whole package to 16Mb automatically

options = {
'build_exe': {
    'packages':packages,
    'zip_include_packages':'PyQt4',
    'includes':['PyQt4.QtCore','PyQt4.QtGui','sqlite3','sys','os'],
},

Not sure it is the right way to do it but seems to have no negative impact on my program as of yet

I tried doing this with PyQt5, and while it worked and did reduce the build size significantly, the application was visibly lower quality (almost like it stuck on a classic windows theme). Additionally, including the specific PyQt5 "bits" appeared to make no difference to the visual quality, nor the build size. — Rexovas, Dec 16 '20 at 06:14

Elijah · Answer 3 · 2020-02-07T18:20:44.567

This is how I optimized my executable to the minimum file size

from cx_Freeze import setup, Executable
import subprocess
import sys


NAME = 'EXE NAME'
VERSION = '1.0'
PACKAGES = ['pygame', ('import_name', 'package_name')]
# if names are same just have a string not a tuple
installed_packages = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze']).decode('utf-8')
installed_packages = installed_packages.split('\r\n')
EXCLUDES = {pkg.split('==')[0] for pkg in installed_packages if pkg != ''}
EXCLUDES.add('tkinter')
for pkg in PACKAGES:
    if type(pkg) == str: EXCLUDES.remove(pkg)
    else: EXCLUDES.remove(pkg[1])


executables = [Executable('main.py', base='Win32GUI', icon='Resources/Jungle Climb Icon.ico', targetName=NAME)]

setup(
    name=NAME,
    version=VERSION,
    description=f'{NAME} Copyright 2019 AUTHOR',
    options={'build_exe': {'packages': [pkg for pkg in PACKAGES if type(pkg) == str else pkg[0]],
                           'include_files': ['FOLDER'],
                           'excludes': EXCLUDES,
                           'optimize': 2}},
    executables=executables)

Hmmm... it looks like this is unreliable. tkinter doesn't show up in the package list (as you noted, manually adding that package in) and the package "serial" shows up as "pyserial" in the list. If I put 'pyserial' into PACKAGES, I get "ImportError: No module named 'pyserial' but if I put 'serial' into PACKAGES, i get KeyError: 'serial' . Seems like a novel approach, but is there a better method to get the actual package names that will be consistent in both statements? — Trashman, Feb 03 '20 at 19:28
@Trashman I have modified my code to handle dual named packages. I have not tested this though. — Elijah, Feb 04 '20 at 20:12

score 1 · Answer 4 · answered Jan 09 '22 at 06:30

You'll need to create a list that scan all package in python , then remove your package in use in the list, finally assign excludes option with that list: (setup.py)

APP_NAME    = "Hezzo";      ## < Your App's name
Python_File = "app.py";     ## < Main Python file to run
Icon_Path   = "./icon.ico"; ## < Icon
Import      = ["tkinter"];  ## < Your Imported modules (cv2,numpy,PIL,...)

################################### CX_FREEZE IGNITER ###################################
import sys, pkgutil;
from cx_Freeze import setup, Executable;

BasicPackages=["collections","encodings","importlib"] + Import;
def AllPackage(): return [i.name for i in list(pkgutil.iter_modules()) if i.ispkg]; # Return name of all package
def notFound(A,v): # Check if v outside A
    try: A.index(v); return False;
    except: return True;
build_exe_options = {
    "includes": BasicPackages,
    "excludes": [i for i in AllPackage() if notFound(BasicPackages,i)],
}
setup(  name = APP_NAME,
        options = {"build_exe": build_exe_options},
        executables = [Executable(Python_File, base='Win32GUI', icon=Icon_Path, targetName=APP_NAME)]
);

Example app.py:

import tkinter as tk;
windowTk = tk.Tk();
windowTk.mainloop();

In some cases like creating light weight portable app, you can remove extra tcl module file and gear in the lib folder after export. It help reduce some more mega bytes (my app from 200MB decrease to 16MB then 10MB).

And you can use WinRAR to create portable program (SFX archive with option). It's usually reduce 45% more.

cx_Freeze - Preventing including unneeded packages

Edit:

4 Answers4

Linked