Pyinstaller numpy "Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll"

Question

I'm new with python apps. I'm trying to build my python GUI app with pyinstaller. My app depends on the following packages: PyQt4, numpy, pyqtgraph, h5py. I'm working with WinPython-32bit-3.4.4.1.

I build the app with this command:

pyinstaller --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy VOGE.py

I launch my app with the exe file in the dist directory created by pyinstaller and it seems work fine until the program call numpy and crash with this error:

Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll

The mkl_intel_thread.dll is not present in the software directory; but with the file copied in the root dir of the program I got the same error

Thanks for your help

I face the similar problem. But the error says "Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.1.dll.". I've tried to find mkl_intel_thread.1.dll but there is no such a thing. There are only mkl_intel_thread.dll. Any idea to solve it? — RANGGAJAYA CIPTAWAN, Jun 11 '21 at 13:52
The simplest solution I found is that use pip to install package (mainly numpy) instead of using conda packages. This pip installation does not include mkl library. You can find the resolved issue In Pyinstaller official github. Ref: https://github.com/pyinstaller/pyinstaller/issues/2270 — ariharan vijayarangam, Jul 06 '21 at 06:05

score 12 · Answer 1 · answered Apr 03 '16 at 17:58

I had the same issue using Pyinstaller and Numpy. By default pyinstaller seems to not take into account numpy binaries so you have to specify it manually. You can add the files editing the ".spec" file "binaries" variable, but that will only work for your current program. If you want it working for all the programs your freeze you should make a "hook" and save it in C:\Python3*\Lib\site-packages\PyInstaller\hooks.

I had to adapt LeonidR's code to make the numpy-hook working. I rewrited it using a more modern, pythonic approach using list comprehensions:

from PyInstaller import log as logging 
from PyInstaller import compat
from os import listdir

mkldir = compat.base_prefix + "/Lib/site-packages/numpy/core" 
logger = logging.getLogger(__name__)
logger.info("MKL installed as part of numpy, importing that!")
binaries = [(mkldir + "/" + mkl, '') for mkl in listdir(mkldir) if mkl.startswith('mkl_')]

"Binaries" is a list of tuples. The second item of the tuple corresponds to the folder where you want to place the 'dlls'. In this case is empty so it copies them directly in the main folder where your '.exe' is.

Can you please elaborate it ? I am facing the same problem but I don't understand that If I create hook.py, and save it to C:\Python3*\Lib\site-packages\PyInstaller\hooks how will I call it in my code so that it reads file as well ? — muazfaiz, Nov 17 '16 at 18:53
This almost worked for me but as others noted that libiomp5md.dll is also needed, I modified the last line to: binaries = [(mkldir + "/" + mkl, '') for mkl in listdir(mkldir) if mkl.startswith('mkl_') or mkl.startswith('libio')] — Raj, Feb 09 '18 at 00:50

score 3 · Answer 2 · edited May 23 '17 at 11:54

3

I just ran into the same problem. As a workaround I copied the DLL's manually, as described in https://stackoverflow.com/a/34893933/4089081

I'm trying to find a better solution though.

edited May 23 '17 at 11:54

Community

1
1

answered Feb 25 '16 at 14:14

durox

41
1
2

Oh, this is why my .exe suddenly weighted 200MB less when I switched computers. It doesn't find the binaries properly on this one. – Guimoute Nov 05 '19 at 10:12

score 1 · Accepted Answer · answered Mar 07 '16 at 20:06

1

I created a hook-numpy.py to deal with this problem:

from PyInstaller import log as logging 
from PyInstaller import compat
from os import listdir

libdir = compat.base_prefix + "/lib"
mkllib = filter(lambda x : x.startswith('libmkl_'), listdir(libdir))
if mkllib <> []: 
   logger = logging.getLogger(__name__)
   logger.info("MKL installed as part of numpy, importing that!")
   binaries = map(lambda l: (libdir + "/" + l, ''), mkllib)

In my case, conda is installing the mkl libraries to speed up numpy and scipy.

answered Mar 07 '16 at 20:06

LeonidR

117
8

Thanks. Your fix has solved my problem. But now I realized that when my app call function related to pyqtgraph it crash with the same error: Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll – f_ciriolo Mar 09 '16 at 19:08
Is this a different app? it is crashing with the same error? – LeonidR Mar 09 '16 at 20:59
No, the same app include also pyqtgraph. Now when I execute the the exe genereted by pyinstaller it runs well until a function that use pyqtgraph is called after that it crash with the Intel MKL FATAL ERROR. Before your fix the crash occurs erlier – f_ciriolo Mar 09 '16 at 21:17
Perplexing, my intuition is that you're using different Python package management schemes for `pyqtgraph` and `numpy`. I'd recommend that you build the one-dir app and then run it with a trace tracking which libraries get loaded. – LeonidR Mar 09 '16 at 21:56
I'm building with one-dir (default). Sorry but I'm new in python what do you exactly mean with "run it with a trace tracking"? Have I to track the library loading on the executable or on the original .py? – f_ciriolo Mar 11 '16 at 10:56

Jarad · Answer 4 · 2016-12-23T05:38:36.333

j4n7's answer was very helpful, however, it may or may not be buggy. compat.base_prefix uses backslashes (at least for me) but they then concatenate with "/Lib/site-packages/numpy/core" (forward slashes).

>>> from PyInstaller import compat
>>> compat.base_prefix
'C:\\Python34'
>>> mkldir = compat.base_prefix + "/Lib/site-packages/numpy/core"
>>> mkldir
'C:\\Python34/Lib/site-packages/numpy/core'

As you can see, it produces both forward and backward slashes in a path.

Here are my steps that allowed me to bundle the numpy mkl files into onefile. Note that my particular app uses matplotlib and the problem I was experiencing was everytime I clicked a button (tkinter) to execute the plot, the app crashed.

Steps

First: Make a build of your app using:

pyinstaller --onefile --windowed yourpythonappnamehere.py

Second: Open the .spec file and add this to it. Obviously make sure the below files actually exist first. You may not have Python34 so just a friendly warning not to copy blindly.

mkl_dlls = [('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_avx512.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_core.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_def.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_intel_thread.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_mc.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_mc3.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_rt.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_sequential.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_tbb_thread.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_avx512.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_cmpt.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_def.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc2.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\mkl_vml_mc3.dll', ''),
 ('C:\\Python34\\Lib\\site-packages\\numpy\\core\\libiomp5md.dll', '')]

Third: where it says binaries=None, change to binaries=mkl_dlls.

a = Analysis(['yourpythonappnamehere.py'],
             pathex=['C:\\Users\\...\\Documents\\...'],
             binaries=mkl_dlls,
             datas=None,
             ....

Fourth: Re-run the first step. When your app is built, go into the dist folder and launch your app. I hope it works for you!

UPDATE: If you get Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll but you can clearly see that mkl_intel_thread.dll IS IN your program directory, go to numpy/core and literally copy all the files with .dll extensions that you don't have and paste them into your program's directory and re-run. If it works, great, but you might want to delete one at a time to figure out which ones you need and which ones you don't.

score 1 · Answer 5 · answered Apr 17 '18 at 00:12

1

I just update numpy+mkl to the latest version, you can download numpy+mkl from here

answered Apr 17 '18 at 00:12

Hailin FU

492
4
14

This is my solution too. – Regi Mathew Oct 09 '18 at 07:30
I did a conda update numpy and that did not work for me. – doublefelix Oct 13 '22 at 14:45

score 1 · Answer 6 · edited Nov 13 '21 at 02:05

Just tested with Anaconda Navigator 2.1.0, pyinstaller 3.6, python 3.8.8, Windows 10. As documented above, pyinstaller doesn't know to install mkl_intel_thread.1.dll so there's a fatal error when you run the new program saying the DLL can't be found.

It works to tell pyinstaller either on the command line or in the .spec script to install from a specific location. You almost certainly have the DLL if you've got conda installed numpy. Pyinstaller is fussy about the specifications so here are two examples:

.spec script:

binaries=[('~\\anaconda3\\pkgs\\mkl-2021.4.0-haa95532_640\\Library\\bin\\mkl_intel_thread.1.dll', '.')]

command line:

pyinstaller  --add-binary '~\anaconda3\pkgs\mkl-2021.4.0-haa95532_640\Library\bin\mkl_intel_thread.1.dll;.' yourmodule.py

Note the single quotes on the command line and the ;. because this is really a tuple.

maggie · Answer 7 · 2017-03-09T07:50:47.767

I updated the snippet to get it working with my anaconda installation (the path is different):

from PyInstaller import log as logging
from PyInstaller import compat
from os import listdir
from os.path import join

mkldir = join(compat.base_prefix, "Library", "bin")
binaries = [(join(mkldir, mkl), '') for mkl in listdir(mkldir) if mkl.startswith('mkl_')]

Update: This is just working for windows. I also saw that there is a PR on github with a more accurate fix. See this Commit

score 0 · Answer 8 · answered Jan 03 '18 at 07:45

I had 2 versions of Python installed. One basic version of Python and one installed by Anaconda. When running my machine learning script it threw the same error. When I uninstalled one of the two versions (in my case I removed Anaconda) everything works fine again. I use deeplearning4j as library and apparently it has difficulties to resolve this dll, cause there were multiple Python installations installed.

score 0 · Answer 9 · answered Nov 03 '21 at 18:33

I used another solution. I simply replaced the hok-numpy.core.py with the following, which basically gets the mkl dll paths manually after everything else.

#-----------------------------------------------------------------------------
# Copyright (c) 2013-2020, PyInstaller Development Team.
#
# Distributed under the terms of the GNU General Public License (version 2
# or later) with exception for distributing the bootloader.
#
# The full license is in the file COPYING.txt, distributed with this software.
#
# SPDX-License-Identifier: (GPL-2.0-or-later WITH Bootloader-exception)
#-----------------------------------------------------------------------------
# If numpy is built with MKL support it depends on a set of libraries loaded
# at runtime. Since PyInstaller's static analysis can't find them they must be
# included manually.
#
# See
# https://github.com/pyinstaller/pyinstaller/issues/1881
# https://github.com/pyinstaller/pyinstaller/issues/1969
# for more information
import os
import os.path
import re
from PyInstaller.utils.hooks import get_package_paths
from PyInstaller import log as logging 
from PyInstaller import compat

binaries = []

# look for libraries in numpy package path
pkg_base, pkg_dir = get_package_paths('numpy.core')
re_anylib = re.compile(r'\w+\.(?:dll|so|dylib)', re.IGNORECASE)
dlls_pkg = [f for f in os.listdir(pkg_dir) if re_anylib.match(f)]
binaries += [(os.path.join(pkg_dir, f), '.') for f in dlls_pkg]

# look for MKL libraries in pythons lib directory
# TODO: check numpy.__config__ if numpy is actually depending on MKL
# TODO: determine which directories are searched by the os linker
if compat.is_win:
    lib_dir = os.path.join(compat.base_prefix, "Library", "bin")
else:
    lib_dir = os.path.join(compat.base_prefix, "lib")
if os.path.isdir(lib_dir):
    re_mkllib = re.compile(r'^(?:lib)?mkl\w+\.(?:dll|so|dylib)', re.IGNORECASE)
    dlls_mkl = [f for f in os.listdir(lib_dir) if re_mkllib.match(f)]
    if dlls_mkl:
        logger = logging.getLogger(__name__)
        logger.info("MKL libraries found when importing numpy. Adding MKL to binaries")
        binaries += [(os.path.join(lib_dir, f), '.') for f in dlls_mkl]
        logger.info(lib_dir)

libdir = 'C:/Users/bcubrich/.conda/envs/pyInstPermit/Library/bin' #<--- path to mkl dlls
for path in os.listdir(libdir):
    if path.startswith('mkl'):
        mkl=libdir+'/'+path
        final_path=os.path.normpath(mkl.replace('/','\\'))
        binaries.append((final_path,'.'))
        logger = logging.getLogger(__name__)    
#logger.info(binaries)
#logger.info(final_path)

Pyinstaller numpy "Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll"

9 Answers9

Linked