6

I wrote a short program main.py using numpy and Qt:

from PyQt5 import QtWidgets
import numpy as np
import sys

if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)
    w = QtWidgets.QTextEdit('hello')
    w.show()
    sys.exit(app.exec_())

When I use pyinstaller --windowed main.py the resulting dist folder has a size of 390MB.

If I remove import numpy as np the resulting dist folder is only 70MB.

Is there any way to reduce the size while still using numpy? Maybe by only including the dlls I really use?

Jonas
  • 1,838
  • 4
  • 19
  • 35
  • 1
    Basically even the most simple functionalities in numpy are based on some BLAS library (and some more complex: LAPACK). These are huge and some (MKL) even bigger than others (OpenBLAS). – sascha Dec 14 '17 at 02:57
  • 1
    So no chance to decrease the executable size and keeping it standalone? – Jonas Dec 14 '17 at 09:38
  • I have the same question, did you find any workarounds? - besides factoring out Numpy altogether in the codebase... – balletpiraat Oct 10 '18 at 10:04
  • @balletpiraat considering the views this seems to be a common problem, but I didnt find any solution for this, I had to live with a huge exe – Jonas Oct 10 '18 at 10:32
  • 2
    See [link](https://github.com/conda-forge/numpy-feedstock/issues/84). It turns out that the size of the executable depends mostly on the type of library that is used for numpy, as sachsa explained. Some people manage to get a <100mb executable by making sure OpenBLAS is used instead of e.g. MKL. – balletpiraat Oct 11 '18 at 09:55
  • I didnt dare to that for my project :D If you can briefly explain what the people did in the link you posted feel free to post an answer. As I said this is my most viewed question and it would be nice to have an acceptable answer! – Jonas Oct 11 '18 at 11:47
  • 1
    related: https://stackoverflow.com/a/67954011, https://stackoverflow.com/q/62262398 – djvg Feb 14 '22 at 10:01

2 Answers2

7

Thanks to the user balletpiraat I found a solution.

Install numpy with: conda install -c conda-forge numpy

To test this I created two anaconda environments. Environment "normalnumpy" was created with:

conda create -n normalnumpy python=3.7
activate normalnumpy
conda install numpy
pip install pyinstaller

resulting in: 

altgraph                  0.16.1                    <pip>
blas                      1.0                         mkl
certifi                   2018.10.15               py37_0
future                    0.16.0                    <pip>
icc_rt                    2017.0.4             h97af966_0
intel-openmp              2019.0                      118
macholib                  1.11                      <pip>
mkl                       2019.0                      118
mkl_fft                   1.0.6            py37hdbbee80_0
mkl_random                1.0.1            py37h77b88f5_1
numpy                     1.15.2           py37ha559c80_0
numpy-base                1.15.2           py37h8128ebf_0
pefile                    2018.8.8                  <pip>
pip                       10.0.1                   py37_0
PyInstaller               3.4                       <pip>
python                    3.7.0                hea74fb7_0
pywin32-ctypes            0.2.0                     <pip>
setuptools                40.4.3                   py37_0
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.15.26706          h3a45250_0
wheel                     0.32.1                   py37_0
wincertstore              0.2                      py37_0

Environment "extranumpy" was created with:

conda create -n extranumpy python=3.7
activate extranumpy
conda install -c conda-forge numpy
pip install pyinstaller

resulting in:

altgraph                  0.16.1                    <pip>
blas                      1.1                    openblas    conda-forge
certifi                   2018.10.15            py37_1000    conda-forge
future                    0.16.0                    <pip>
libflang                  5.0.0             vc14_20180208  [vc14]  conda-forge
llvm-meta                 5.0.0                         0    conda-forge
macholib                  1.11                      <pip>
numpy                     1.15.2          py37_blas_openblash8d851b4_1  [blas_openblas]  conda-forge
openblas                  0.2.20                   vc14_8  [vc14]  conda-forge
openmp                    5.0.0                    vc14_1  [vc14]  conda-forge
pefile                    2018.8.8                  <pip>
pip                       10.0.1                   py37_0
PyInstaller               3.4                       <pip>
python                    3.7.0                hea74fb7_0
pywin32-ctypes            0.2.0                     <pip>
setuptools                40.4.3                   py37_0
vc                        14                            0    conda-forge
vs2015_runtime            14.15.26706          h3a45250_0
wheel                     0.32.1                   py37_0
wincertstore              0.2                      py37_0

I tested both environments with "main.py":

import numpy as np
if __name__ == '__main__':
    test = np.array([42])
    print(test)

and 

pyinstaller --onefile main.py

the "normalnumpy" executable is 228MB, the "extranumpy" executable is 10MB.

Jonas
  • 1,838
  • 4
  • 19
  • 35
  • Will the exe file be stored in Dist directory like normal? – Miffy Apr 01 '19 at 08:49
  • I have created (base) environment and kind of stuck. How do I get out? – Miffy Apr 01 '19 at 09:04
  • @Miffy There are [many tutorials](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) how to create an environment. If you did that you have to activate it and run pyinstaller like normal. The files will be in a `dist` folder also like running it from your "normal" python installation. All commands and code are posted in my answer. I suggest you try the example and continue from there. – Jonas Apr 01 '19 at 10:41
  • I did try and I was into base environment. I tried few solutions on how to exit it. That messed up all the libraries installed in my computer. Now I get a quarantine error whenever I try to install pyinstaller – Miffy Apr 02 '19 at 03:55
  • 1
    @Miffy You should ask a separate question about that, its hard to resolve without the error message and in the comments. – Jonas Apr 02 '19 at 05:35
  • 3
    I had to add the constraint `blas=*=openblas` to prevent `mkl` from being installed. Found this recommendation [here](https://github.com/conda-forge/numpy-feedstock/issues/108#issuecomment-428749055). – Steven C. Howell Jun 12 '19 at 19:13
0

Have you tried excluding modules? This is a common problem. Also you might want to check out:

Reducing size of pyinstaller exe

Worth mention, are you using Anaconda? A fresh oracle box python system and less of a huge file.

I believe in there docs somewhere you can adjust where the directories are found.

https://pythonhosted.org/PyInstaller/spec-files.html

  • If I exclude modules the executable is no longer standalone, can I exclude the parts of eg numpy that I dont use? – Jonas Dec 14 '17 at 09:39
  • @Jonas You can exclude modules sure, but you cannot exclude parts of numpy. Maybe in theory you could.. no that'd break numpy. According to the post I linked, UPX and virtual environment and exclusion are the general methods to reduce size. –  Dec 14 '17 at 14:51