-1

I fixed a super-annoying case of "ImportError: DLL load failed while importing" in a way that generally applies to Windows, so let me share it with the group. Here is the question:

I installed FINUFFT via pip install finufft. When I import finufft, I get this error:

ImportError: DLL load failed while importing _finufft: The specified module could not be found.

How do I fix it?

dimitsev
  • 127
  • 1
  • 10
  • Why is this question so unpopular? xD It's useful for non-full-time programmers that want to get `finufft` working. – dimitsev Sep 05 '21 at 13:08

1 Answers1

-1

Read to the end before doing anything.

The error means that a DLL cannot find another DLL that it was linked with. But which other DLL?

  1. Download Dependencies.
  2. Locate your problematic DLL.
    In this specific case: Locate the folder ...\Lib\site-packages\finufft\ of the FINUFFT installation that you want to fix. ...\ is the path of your standard python installation or of your python virtual environment.
  3. Start DependenciesGui.exe and use it to open the problematic DLL, e.g. ...\finufft\_finufft.cp38-win_amd64.pyd. (A .pyd is a regular DLL with some specific entry points for python.)
  4. On the left, you will see a complete list of the problematic DLL's direct dependencies, whose dependencies you can in turn unfold by mouse click. Apart from typical Windows-DLLs, like kernel32.dll and MSVCRT.dll, and apart from the FFTW-DLLs, which should already be in the FINUFFT-folder, there will also be some - possibly missing - Linux-DLLs. For me, it was libgcc_s_seh-1.dll, libgomp-1.dll and libstdc++-6.dll. By checking their direct dependencies, I also discovered libwinpthread-1.dll as missing.
  5. [See EDIT below!!!] I found those DLLs in Anaconda (...\Anaconda3\Library\mingw-w64\bin\), but you can probably also get them from cygwin (...\cygwin64\bin\), git (...\Git\mingw64\bin\) or anything else that downloads mingw64 and its packages on Windows.
  6. To solve the problem, copy the respective DLLs into ...\Lib\site-packages\finufft\ and give them the exact filenames that the FINUFFT-DLL is expecting according to Dependencies. This works because Windows and because of the Windows DLL search order.

Now, import finufft should work in the specific python environment whose FINUFFT installation you fixed. Clearly, this method can be applied anytime DLL dependencies are missing.

EDIT - correction of my answer by @CristiFati: If possible, DLLs and similar things should always be built with the same toolchain. So if you don't compile them yourself, get them from as few different places as possible, i.e. don't mix regular python, Anaconda, cygwin, etc. - if possible. Of course, Windows DLLs will have a different origin from Linux DLLs.

dimitsev
  • 127
  • 1
  • 10
  • I would be grateful for other answers / comments on how to manage DLLs more elegantly, i.e. without copying them every time. – dimitsev Jul 29 '21 at 21:26
  • 1
    It seems that your environment is a total mess: Mixing *Anaconda* with *Cygwin* (and probably with "regular" *Python* installation). is a baaad idea. Choose your work environment and stick with it, don't search for solutions outside (as they might work but only till some point and afterwards you'd get strange errors). – CristiFati Jul 30 '21 at 00:35
  • [\[SO\]: PyWin32 and Python 3.8.0 (@CristiFati's answer)](https://stackoverflow.com/questions/58631512/pywin32-and-python-3-8-0/58632354#58632354). Also check (the bullets at the end): [\[SO\]: Python Ctypes - loading dll throws OSError: \[WinError 193\] %1 is not a valid Win32 application (@CristiFati's answer)](https://stackoverflow.com/questions/57187566/python-ctypes-loading-dll-throws-oserror-winerror-193-1-is-not-a-valid-win/57297745#57297745) – CristiFati Jul 30 '21 at 00:42
  • @cristifati: 1. Your solutions can be summarized as "manually import the missing DLLs in python before you import the problematic package" (finufft in my case). Thank you, noted (despite your problems with virtual environments: https://stackoverflow.com/questions/58805040/pywin32-226-and-virtual-environments/58805300#58805300) 2. My work environment is a virtual environment by my research group inside a standard python installation. Anaconda, etc., I just researched to find my missing DLLs. 3. Where is the more elegant solution that doesn't cost 50 years to implement for a regular user? – dimitsev Jul 31 '21 at 16:08
  • The *URL*s that I provided were to clarify certain aspects (not to provide a resolution). Maybe I didn't send the message that I wanted: when searching for dependencies try to avoid searching them in different environments as they are often built with different toolchains, and that may cause problems. For example if you mention *Cygwin*, try to have all dependencies from *Cygwin* provided packages. – CristiFati Aug 01 '21 at 08:37
  • @CristiFati, sorry for snapping at you. So it doesn't matter if I take my DLLs from Anaconda or cygwin or anything else as long as they are all from one place, because then they will have been built by the same toolchain. Did I understand that correctly? Are there any other mistakes in my answer above? – dimitsev Aug 03 '21 at 08:52
  • Not quite :). For example regular *Win* *Python* is built with *VStudio*, while *Cygwin* packages are typically built with *GCC*. Sometimes (not always - in many cases works fine) mixing such libraries, might be problematic. – CristiFati Aug 03 '21 at 08:55
  • @CristiFati, oh okay. But https://github.com/flatironinstitute/finufft requires both Windows libraries, such as `kernel32.dll`, `user32.dll`, etc., and linux libraries, such as `libgcc_s_seh-1.dll` (see above answer). Is there a way to get everything from one source? And is a linux-compiled python installation available on Windows? – dimitsev Aug 03 '21 at 08:59
  • True, those are system *Win* *.dll*s. A problem that may arise, is having 2 *C* runtimes (one from each toolchain) in the same process. Combined with jumping over *.dll* boundaries, it might have disastrous effects. But if it does the trick for you, it's probably fine. – CristiFati Aug 03 '21 at 09:03
  • @ChristiFati, thank you, I learned something and also corrected my answer above. In this case, the authors of github.com/flatironinstitute/finufft did not make it clear which dependencies are needed, but https://github.com/lucasg/Dependencies says both Windows- and Linux-DLLs. Basically, the authors of FINUFFT did not bother to compile their stuff for Windows as well and I am not about to do it for them (since it already works). – dimitsev Aug 04 '21 at 10:48