I'm trying to create an exe-file that can be transferred to any computer and run without problems, even one that receiver doesn't have a python interpreter and access to user environment variables is closed. For this purpose, I use pyinstaller, because the final file takes up less space with this module.
I have a test file that includes importing modules for machine learning. To check whether the program is working properly, the script outputs module versions. When running this file in the python IDE, there are no problems.
import sys
print('Версия Python:',sys.version)
import asyncio, socket
from threading import Thread
import subprocess
import sqlite3
from datetime import datetime, date, time
import datetime as dt
import numpy as np
print("Версия numpy:",np.__version__)
from numpy import split
from numpy import sqrt
from numpy import array
from numpy import newaxis
import pandas as pd
print("Версия pandas:",pd.__version__)
from pandas import read_csv
import matplotlib
print("Версия matplotlib",matplotlib.__version__)
from matplotlib import pyplot
import sklearn
print("Версия scikit-learn",sklearn.__version__)
from sklearn.metrics import mean_squared_error
from sklearn.preprocessing import StandardScaler
import tensorflow as tf
from tensorflow import keras
print('Версия tensorflow:',tf.__version__)
import keras
print('Версия keras:',keras.__version__)
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Dropout
from keras.layers import RepeatVector
from keras.layers import TimeDistributed
print('Все библиотеки импортированы. Нажмите <Enter> чтобы продолжить')
input()
All actions are performed in the virtual environment where the library data exists. Current version of Python 3.6 Current version of libraries: - tensorflow==2.0.0 - matplotlib==3.1.1 - numpy==1.18.2 - Keras==2.3.1 - scikit-learn==0.22.1 - scipy==1.4.1
To build into a single exe file, I use the following command in an active virtual environment: (test) C:\Users\ilka\python-virtual-environments>pyinstaller --onefile test.py
'''
118 INFO: PyInstaller: 3.6
119 INFO: Python: 3.6.8
121 INFO: Platform: Windows-10-10.0.18362-SP0
122 INFO: wrote C:\Users\ilka\python-virtual-environments\test.spec
126 INFO: UPX is not available.
129 INFO: Extending PYTHONPATH with paths
['C:\\Users\\ilka\\python-virtual-environments',
'C:\\Users\\ilka\\python-virtual-environments']
130 INFO: checking Analysis
374 INFO: Building because C:\Users\ilka\python-virtual-environments\test.py changed
374 INFO: Initializing module dependency graph...
411 INFO: Caching module graph hooks...
527 INFO: Analyzing base_library.zip ...
12015 INFO: Caching module dependency graph...
12229 INFO: running Analysis Analysis-00.toc
12265 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
required by c:\users\ilka\python-virtual-environments\test\scripts\python.exe
12646 INFO: Analyzing C:\Users\ilka\python-virtual-environments\test.py
17800 INFO: Processing pre-find module path hook distutils
17847 INFO: distutils: retargeting to non-venv dir 'C:\\Users\\ilka\\AppData\\Local\\Programs\\Python\\Python36\\lib'
20470 INFO: Processing pre-safe import module hook setuptools.extern.six.moves
22617 INFO: Processing pre-find module path hook site
22657 INFO: site: retargeting to fake-dir 'c:\\users\\ilka\\python-virtual-environments\\test\\lib\\site-packages\\PyInstaller\\fake-modules'
42702 INFO: Processing pre-safe import module hook six.moves
115257 INFO: Processing pre-safe import module hook urllib3.packages.six.moves
121660 INFO: Processing module hooks...
121661 INFO: Loading module hook "hook-certifi.py"...
121713 INFO: Loading module hook "hook-distutils.py"...
121763 INFO: Loading module hook "hook-encodings.py"...
121933 INFO: Loading module hook "hook-h5py.py"...
121946 INFO: Loading module hook "hook-lib2to3.py"...
122010 INFO: Loading module hook "hook-matplotlib.backends.py"...
128209 INFO: Matplotlib backend "GTK3Agg": ignored
cairo backend requires that pycairo>=1.11.0 or cairocffiis installed
128943 INFO: Matplotlib backend "GTK3Cairo": ignored
cairo backend requires that pycairo>=1.11.0 or cairocffiis installed
131171 INFO: Matplotlib backend "MacOSX": ignored
cannot import name '_macosx'
131911 INFO: Matplotlib backend "nbAgg": ignored
No module named 'IPython'
132915 INFO: Matplotlib backend "Qt4Agg": ignored
Failed to import any qt binding
133602 INFO: Matplotlib backend "Qt4Cairo": ignored
cairo backend requires that pycairo>=1.11.0 or cairocffiis installed
134330 INFO: Matplotlib backend "Qt5Agg": ignored
Failed to import any qt binding
135012 INFO: Matplotlib backend "Qt5Cairo": ignored
cairo backend requires that pycairo>=1.11.0 or cairocffiis installed
136509 INFO: Matplotlib backend "TkAgg": added
137490 INFO: Matplotlib backend "TkCairo": ignored
cairo backend requires that pycairo>=1.11.0 or cairocffiis installed
140466 INFO: Matplotlib backend "WebAgg": added
141410 INFO: Matplotlib backend "WX": ignored
No module named 'wx'
142097 INFO: Matplotlib backend "WXAgg": ignored
No module named 'wx'
142777 INFO: Matplotlib backend "WXCairo": ignored
No module named 'wx'
143467 INFO: Matplotlib backend "agg": added
144147 INFO: Matplotlib backend "cairo": ignored
cairo backend requires that pycairo>=1.11.0 or cairocffiis installed
145271 INFO: Matplotlib backend "pdf": added
146229 INFO: Matplotlib backend "pgf": added
146943 INFO: Matplotlib backend "ps": added
147664 INFO: Matplotlib backend "svg": added
148561 INFO: Matplotlib backend "template": added
149728 INFO: Loading module hook "hook-matplotlib.py"...
150401 INFO: Loading module hook "hook-numpy.core.py"...
150738 INFO: Loading module hook "hook-numpy.py"...
150742 INFO: Loading module hook "hook-pandas.py"...
162410 INFO: Loading module hook "hook-pkg_resources.py"...
163122 INFO: Processing pre-safe import module hook win32com
164014 INFO: Excluding import '__main__'
164019 INFO: Removing import of __main__ from module pkg_resources
164020 INFO: Loading module hook "hook-pydoc.py"...
164060 INFO: Loading module hook "hook-pythoncom.py"...
164612 INFO: Loading module hook "hook-pytz.py"...
164733 INFO: Loading module hook "hook-pywintypes.py"...
165265 INFO: Loading module hook "hook-scipy.linalg.py"...
165270 INFO: Loading module hook "hook-scipy.py"...
165341 INFO: Loading module hook "hook-scipy.sparse.csgraph.py"...
165366 INFO: Loading module hook "hook-scipy.special._ellip_harm_2.py"...
165368 INFO: Loading module hook "hook-scipy.special._ufuncs.py"...
165371 INFO: Loading module hook "hook-setuptools.py"...
166688 INFO: Loading module hook "hook-sklearn.metrics.cluster.py"...
166722 WARNING: Hidden import "sklearn.utils.sparsetools._graph_validation" not found!
166723 WARNING: Hidden import "sklearn.utils.sparsetools._graph_tools" not found!
166725 WARNING: Hidden import "sklearn.utils.lgamma" not found!
166736 INFO: Loading module hook "hook-sqlite3.py"...
167009 INFO: Loading module hook "hook-sysconfig.py"...
167025 INFO: Loading module hook "hook-win32com.py"...
168086 INFO: Loading module hook "hook-xml.dom.domreg.py"...
168116 INFO: Loading module hook "hook-xml.py"...
168118 INFO: Loading module hook "hook-_tkinter.py"...
168809 INFO: checking Tree
169765 INFO: checking Tree
170260 INFO: Looking for ctypes DLLs
170358 INFO: Analyzing run-time hooks ...
170387 INFO: Including run-time hook 'pyi_rth_certifi.py'
170406 INFO: Including run-time hook 'pyi_rth_pkgres.py'
170418 INFO: Including run-time hook 'pyi_rth_win32comgenpy.py'
170429 INFO: Including run-time hook 'pyi_rth__tkinter.py'
170444 INFO: Including run-time hook 'pyi_rth_mplconfig.py'
170454 INFO: Including run-time hook 'pyi_rth_mpldata.py'
170463 INFO: Including run-time hook 'pyi_rth_multiprocessing.py'
170541 INFO: Looking for dynamic libraries
189309 INFO: Looking for eggs
189309 INFO: Using Python library c:\users\ilka\python-virtual-environments\test\scripts\python36.dll
189311 INFO: Found binding redirects:
[]
189356 INFO: Warnings written to C:\Users\ilka\python-virtual-environments\build\test\warn-test.txt
190159 INFO: Graph cross-reference written to C:\Users\ilka\python-virtual-environments\build\test\xref-test.html
190556 INFO: checking PYZ
190659 INFO: Building because toc changed
190660 INFO: Building PYZ (ZlibArchive) C:\Users\ilka\python-virtual-environments\build\test\PYZ-00.pyz
197584 INFO: Building PYZ (ZlibArchive) C:\Users\ilka\python-virtual-environments\build\test\PYZ-00.pyz completed successfully.
197753 INFO: checking PKG
197873 INFO: Building because toc changed
197874 INFO: Building PKG (CArchive) PKG-00.pkg
277841 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
277980 INFO: Bootloader c:\users\ilka\python-virtual-environments\test\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run.exe
277980 INFO: checking EXE
278103 INFO: Building because toc changed
278103 INFO: Building EXE from EXE-00.toc
278180 INFO: Appending archive to EXE C:\Users\ilka\python-virtual-environments\dist\test.exe
279340 INFO: Building EXE from EXE-00.toc completed successfully.
'''
As you can see from the log, there are some problems with hidden imports:
166722 WARNING: Hidden import "sklearn.utils.sparsetools._graph_validation" not found!
166723 WARNING: Hidden import "sklearn.utils.sparsetools._graph_tools" not found!
166725 WARNING: Hidden import "sklearn.utils.lgamma" not found!
When I try to run the generated .exe I get module import errors related to sklearn library
Версия Python: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)]
Версия numpy: 1.18.2
c:\users\ilka\python-virtual-environments\test\lib\site-packages\PyInstaller\loader\pyimod03_importers.py:623: MatplotlibDeprecationWarning:
The MATPLOTLIBDATA environment variable was deprecated in Matplotlib 3.1 and will be removed in 3.3.
exec(bytecode, module.__dict__)
Версия pandas: 0.25.3
Версия matplotlib 3.1.1
Traceback (most recent call last):
File "lib\site-packages\PyInstaller\loader\pyiboot01_bootstrap.py", line 174, in __init__
File "ctypes\__init__.py", line 348, in __init__
OSError: [WinError 126] Не найден указанный модуль
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test.py", line 28, in <module>
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "c:\users\ilka\python-virtual-environments\test\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
exec(bytecode, module.__dict__)
File "lib\site-packages\sklearn\__init__.py", line 80, in <module>
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "c:\users\ilka\python-virtual-environments\test\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
exec(bytecode, module.__dict__)
File "lib\site-packages\sklearn\_distributor_init.py", line 18, in <module>
File "lib\site-packages\PyInstaller\loader\pyiboot01_bootstrap.py", line 176, in __init__
__main__.PyInstallerImportError: Failed to load dynlib/dll 'C:\\Users\\ilka\\AppData\\Local\\Temp\\_MEI69042\\sklearn\\.libs\\vcomp140.dll'. Most probably this dynlib/dll was not found when the application was frozen.
[7056] Failed to execute script test
I tried to run this code in python 3.5, 3.6, and 3.7. The result is the same everywhere.
The function freeze_support() that described here https://github.com/pyinstaller/pyinstaller/issues/3907 does not solve this problem
I try work with hooks that describe in How do you resolve 'hidden imports not found!' warnings in pyinstaller for scipy?. But file with dependencies already exists in a virtual environment (hook-sklearn.metrics.cluster.py) and hidden imports on needed libraries already fixed
# Tested on Windows 7 64bit with scikit-learn 0.17 and Python 2.7
hiddenimports = ['sklearn.utils.sparsetools._graph_validation',
'sklearn.utils.sparsetools._graph_tools',
'sklearn.utils.lgamma',
'sklearn.utils.weight_vector']
Can you help me with this problem?
