Reducing size of pyinstaller exe

Question

I have a simple pandas pyinstaller exe which is over 40MB.

My exe example:

import collections
import csv
import selenium
import pandas

print('hi')

40MB+ for this seems a bit overkill.

How can I reduce this as much as possible?

One method:

pyinstaller --onefile --exclude matplotlib --exclude scipy --exclude pandas --exclude numpy.py

This however is not practical considering how big the exclusion list would be.

How do I select a folder for pyinstaller to get modules from and exclude everything else so I may have a small application?

Spec file:

a = Analysis(['123.py'],
             pathex=['C:\\Users\\AA\\ZZ'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='123',
          debug=False,
          strip=False,
          upx=True,
          runtime_tmpdir=None,
          console=True )

It's also worth mentioning. By default, Pyinstaller does not detect pandas.

Add:

hiddenimports = ['pandas._libs.tslibs.timedeltas']

To: C:\Users\<NAME>\AppData\Local\Programs\Python\Python36\Lib\site-packages\PyInstaller\hooks

A possible solution when using multiple executables, could be to link each executable to a separate folder or executable with all imports.

If you remove the dependencies, the exe is not going to be able to run your script, or am I missing something? — Florent B., Dec 14 '17 at 15:06
@FlorentB. Pyinstaller is packaging other stuff not needed. Also, 40mb for that exe? I tried to UPX it, but it's not changing much for me. Must be doing it wrong. — , Dec 14 '17 at 15:07
When I do `pyinstaller --onefile test.py` (test.py contains your example), I get an executable that is about 5mb. — John Anderson, Dec 17 '17 at 02:43
Try using the --onedir option instead of --onefile, and see what pyinstaller is putting in the dist folder. — John Anderson, Dec 17 '17 at 02:44
In general, check your dependency tree. For example, `numpy` may be using using MKL, `matplotlib` may be using Qt, both of which can lead to very large executables. See e.g. https://stackoverflow.com/a/70113449, https://stackoverflow.com/a/67954011 — djvg, Feb 14 '22 at 09:58

score 53 · Accepted Answer · answered Dec 14 '17 at 09:39

53

try setting up your environment with a virtualenv, and install in there only the required libraries

some details on working with virtual env are here: https://virtualenv.pypa.io/en/stable/

answered Dec 14 '17 at 09:39

Ophir Yoktan

8,149
7
58
106

2

I am hesitant to do this as I have recently had to reinstall my entire os because of file corruption issues and python. Is this safe? I will see how this goes in a virtual box. – Dec 14 '17 at 10:17
5

yes - it somewhat like a python virtualbox - all packages are installed in a separate virtual env, and don't effect the main python installation. removing it is just deleting a directory, and you cn have sevral virtualenvs with different combinations of libraries and versions – Ophir Yoktan Dec 14 '17 at 10:20
I'm not familiar with virtual environments, but I'll have to be. Using pyinstaller on a virtual box is causing issues. Maybe a virtual env will fix. – Dec 14 '17 at 13:51
1

This method saved 50% file size in my case. I was importing `os`, `pandas`, `sqlalchemy`. – mccbala Oct 30 '18 at 02:41
1

Hm I installed pytorch in my virtualenv and it has 1.4 GB!!! Now if I use not all of pytorch (just some libs) is it possible to decrease the exe file size in this way? – sqp_125 Mar 02 '20 at 21:16
Just use a different virtualenv – Ophir Yoktan Mar 03 '20 at 05:44
2

This method actually made my exe 10.4 MB from 155 MB. lol, thanks. – Amin Pial Jun 08 '21 at 08:17
1

I used a clean conda environment for my executable, and this brought the executable size down to 12 MB from 266 MB ;') – Siddhant Sadangi Aug 06 '22 at 10:29
This is perfectly correct BUT before spending time on creating another environment please check the answers below! Try UPX. It's supposed to be used by default but it seams that in many cases the directory is not correctly found by pyInstaller. AFTER using the above method i reduced my exe size from 750 MB to 250 MB. Just by using UPX it was reduced to 29 MB – Jakob Aug 08 '22 at 17:26

score 17 · Answer 2 · edited Nov 05 '19 at 14:54

For me, it is a simple case of using pandas that the exe is huge.

Though removing certain directories was helpful, as was UPXING that helped a great deal also.

I got it reduced a lot and it was not doing this by default.

That being said, the final and most import solution is talked about here: Importing Python modules from a select location . So there was a feature that did all this, but for now there is some manual handling involved because: multipackage-bundles is broken.

Now to the simple solution for lots of exe's

If you have many executables, I highly recommend this approach:

pyinstaller -F abc.py --onedir (Have all imports of both scripts)
pyinstaller -F abd.py --onedir (Have all imports of both scripts)

Now put abd.exe in the one directory of abc.py folder as well as any other external scripts. Be sure they are differently named or only one script will run.

This works really well because all the dependencies are in one folder. This is how it should be. So in this example say you had a 40mb one folder. For each additional exe afterwards, it will only be +5mb(or how big the exe is) rather than 40mb each.

score 5 · Answer 3 · edited Oct 23 '22 at 02:09

5

The python interpreter and all imported modules are included in the executable.

You can try adding modules you want to exclude to the excludes list under Analysis in your spec file.

You could also try compressing the executable using UPX. See A note on using UPX

edited Oct 23 '22 at 02:09

MattDMo

100,794
21
241
231

answered Dec 14 '17 at 09:51

2

Does it not upx by default? I have a large number of modules to exclude so probably not ideal. There is no exclude all modules except for a given directory? – Dec 14 '17 at 10:15
1

I thought it was upxed automatically, my mistake. Something like: pyinstaller -F --upx-dir C:\Users\DD\Downloads\upx394w\upx394w\123\upx308w\upx.exe zz.spec does not work though. – Dec 14 '17 at 13:50

score 4 · Answer 4 · answered Sep 04 '21 at 23:12

I had a similar problem and found a solution. I used Windows terminal preview. This program allows creation of various virtual environments like Windows Power Shell (btw. Linux Ubuntu too. Also, worth noting: you can have many terminals in this program installed and, even, open a few at once. Very cool stuff).

Inside Windows Power Shell in Windows terminal preview I installed all the necessary libraries, then I opened the path to my file and tried to use this command:

pyinstaller --onefile -w 'filename.py'

...but, the output exe didn't work. For some reason, the console said that there is a lack of one library (which I had installed earlier). I've found the solution in mimic the auto-py-to-exe library. The command used by this GUI is:

pyinstaller --noconfirm --onedir --console "C:/Users/something/filename.py"

And this one works well. I reduced the size of my output exe program from 911MB to 82,9MB !!!

BTW. 911MB was the size of output made by auto-py-to-exe.

I wonder how is it possible that no one yet has created a compressor that reads the code, checks what libraries are part of the code, then putting only them inside the compression. In my case, auto-py-to-exe probably loaded all libraries that I ever installed. That would explain the size of this compressed folder.

Some suggest using https://virtualenv.pypa.io/en/stable/ but in my opinion, this library is very difficult, at least for me.

score 0 · Answer 5 · answered Aug 20 '19 at 10:11

0

I use the Anaconda environment and so the virtualenv solution isn't an option. my way was to exclude unnecessary modules in my spec file, ex.:

in Analysis(...)

excludes=['pandas', 'numpy'],

(this are modules whose raise the size of the files extraordinarily)

For every build i'm using this adjusted spec file to create the exe.

pyinstaller "mySpec.spec" --distpath="<path>"

answered Aug 20 '19 at 10:11

droebi

872
1
14
27

1

You can create a clean new conda environment. This brought the executable size down to 12 MB from 266 MB – Siddhant Sadangi Aug 06 '22 at 10:30

Reducing size of pyinstaller exe

5 Answers5

Linked

Related