1

I'm trying to create a virtual environment to limit the size of an exe file I'm creating. However, when I create a virtual environment, it seems that it still adds packages that I haven't installed in the said environment.

I've tried to do a very small test file where I import numpy:

import numpy as np
A = np.array([0,0,1])
print(A)

When I create my virtual environment and run the script from cmd, it gets that numpy is not installed in the environment, but when I make the test script into an exe file using pyinstaller, somehow it gets numpy mixed into it all anyway... I have my entire cmd procedure here:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\MRCH>Virtualenv test_env
Using base prefix 'c:\\anaconda3'
New python executable in C:\Users\MRCH\test_env\Scripts\python.exe
Installing setuptools, pip, wheel...done.

C:\Users\MRCH>test_env\scripts\activate

(test_env) C:\Users\MRCH\dist>pip list
Package    Version
---------- -------
pip        10.0.1
setuptools 39.2.0
wheel      0.31.1

(test_env) C:\Users\MRCH>python test.py
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

(test_env) C:\Users\MRCH>pyinstaller test.py -y -F
454 INFO: PyInstaller: 3.3.1
...
132266 INFO: Appending archive to EXE C:\Users\MRCH\dist\test.exe
132473 INFO: Building EXE from out00-EXE.toc completed successfully.

(test_env) C:\Users\MRCH>cd dist

(test_env) C:\Users\MRCH\dist>test.exe
[0 0 1]

(test_env) C:\Users\MRCH\dist>

What am I doing wrong?

Martin
  • 353
  • 1
  • 6
  • 23

2 Answers2

1

From the pyinstaller docs pyinstaller analyses import statements to collect dependencies.

It seems your issues are coming from pyinstaller finding packages in the anaconda directory. You need to either exclude the packages explicitly like this: 

pyinstaller test.py -y -F --exclude numpy

or create a fresh virtual env separate from anaconda like this answer suggests.

ConorSheehan1
  • 1,640
  • 1
  • 18
  • 30
  • hm, the exe file I get out is about 210mb, and the whole point of using the virtual environment was to get this down by only freezing what I was actually using in the script. But if pyinstaller just throws everything in there anyway, this may not be the best way to reduce my file size? – Martin May 30 '18 at 09:18
  • You need to add the ```--exclude``` flag or edit the excludes list under Analysis in your spec file if you want to exclude some specific dependencies from the executable and rely on a virtual environment to provide them instead. Have you seen [this answer](https://stackoverflow.com/questions/47692213/reducing-size-of-pyinstaller-exe)? – ConorSheehan1 May 30 '18 at 09:37
  • The top answer in the link you provided suggests to use virtual environment and install only the required libraries, which I think is exactly what I'm doing? – Martin May 30 '18 at 10:00
  • The **accepted** answer suggests using a virtual env. The **top voted** answer suggests using the ```--one-dir``` flag with a folder full of bash scripts to handle dependencies, and the original question uses the ```--exclude``` flag to handle ignoring dependencies. Since what you're currently doing isn't working, maybe try one of the other options? – ConorSheehan1 May 30 '18 at 10:29
  • Also, although you're using virtualenv, it's using base prefix 'c:\\anaconda3', which could be causing issues if you have ever installed numpy with conda. See [this answer](https://stackoverflow.com/questions/48629486/how-can-i-create-the-minimum-size-executable-with-pyinstaller) (*not the accepted, the highest voted) – ConorSheehan1 May 30 '18 at 10:44
  • 1
    I found out that if I install pyinstaller in the virtual environment, it won't install all the other stuff, and the file is reduced significantly. I also tried to install default Python 32bit and run pyinstaller on my real file. It has reduced the file size significantly but I get a lot of ImportError: DLL load failed: errors. I will look into this and perhaps make another thread tomorrow if I don't manage to solve it. Thanks a lot for your help! – Martin May 30 '18 at 13:31
  • 1
    Turns out the DLL errors were due to UPX. By not running the script through the UPX everything works perfectly and my script has been reduced from aprox 270mb to 35. – Martin May 31 '18 at 06:38
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/172144/discussion-between-con-and-martin). – ConorSheehan1 May 31 '18 at 08:04
0

Here's a quick summary of the solution.

1) Pyinstaller in Virtual environment I found out that when I create a virtual environment, the first thing I should do is to install Pyinstaller. If I don't do this, it includes a ton of packages from outside the virtual environment. (I'm not sure if this is what actually happens, but it did solve my problem, so if anyone has any insights to this, feel free to contribute).

After I have installed Pyinstaller, I install the packages required for the script.

In this case, the .exe was still quite large, even though only numpy was used.

2) Anaconda VS clean Python I installed a clean Python 32bit and created a virtual environment independent from Anaconda. That way a lot of the packages from the conda environment was omitted and I managed to reduce the file size to approximately 1/10 of the original size

UPX When compressing the file through UPX with the clean python I encountered a lot of ImportError: DLL load failed errors, which I found out was due to UPX. When Running Pyinstaller without UPX, no errors occurs and everything runs smoothly.

Martin
  • 353
  • 1
  • 6
  • 23